2024-09-19T06:21:37.581 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-19T06:21:37.607 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T06:21:37.700 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7911743' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1747 sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 sleep_before_teardown: 0 subset: 176/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 6fad5dd73c93545e52f57f681e140a9c3529c11e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi037.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoJlE8Ky5ibqLt2jJfDHHrvI8BMP2hKpTpzxyqmYeZx7Uv9WgikjO0eXe2YG39VoNrkoOQSyhbppejdO5/tlN0= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHoRLvvd7Fz37X/JuqRdRvYFZc0FFHB1ZVxVmP1h/jRkLONnJkf8sAvjV9BNiLfsiY0yXC4gbu3OcRlgABG98jY= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDAFlDP7V2DbDbrK1yhTUh+DKe2a/f0iDuw9+kZqYmE735CfWVlwHuqVW2lcX4vzlzEl/jrVFGEdRe2YeeC9ptM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/dbench.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_21:24:17 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-19T06:21:37.700 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa; will attempt to use it 2024-09-19T06:21:37.701 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks 2024-09-19T06:21:37.701 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-19T06:21:37.703 INFO:teuthology.task.internal:Checking packages... 2024-09-19T06:21:37.726 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2f599a46ad10e37c929e5c43da2f6907ddcd1961' 2024-09-19T06:21:37.726 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-19T06:21:37.726 INFO:teuthology.packaging:ref: None 2024-09-19T06:21:37.726 INFO:teuthology.packaging:tag: None 2024-09-19T06:21:37.726 INFO:teuthology.packaging:branch: squid 2024-09-19T06:21:37.726 INFO:teuthology.packaging:sha1: 2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:21:37.726 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-19T06:21:37.960 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-363.g6fad5dd7 2024-09-19T06:21:37.962 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-19T06:21:37.969 INFO:teuthology.task.internal:no buildpackages task found 2024-09-19T06:21:37.969 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-19T06:21:38.058 INFO:teuthology.task.internal:Saving configuration 2024-09-19T06:21:38.078 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-19T06:21:38.085 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-19T06:21:38.104 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi037.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 06:16:19.167420', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIoJlE8Ky5ibqLt2jJfDHHrvI8BMP2hKpTpzxyqmYeZx7Uv9WgikjO0eXe2YG39VoNrkoOQSyhbppejdO5/tlN0='} 2024-09-19T06:21:38.126 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 06:16:19.166038', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHoRLvvd7Fz37X/JuqRdRvYFZc0FFHB1ZVxVmP1h/jRkLONnJkf8sAvjV9BNiLfsiY0yXC4gbu3OcRlgABG98jY='} 2024-09-19T06:21:38.142 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-19 06:16:19.164391', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDAFlDP7V2DbDbrK1yhTUh+DKe2a/f0iDuw9+kZqYmE735CfWVlwHuqVW2lcX4vzlzEl/jrVFGEdRe2YeeC9ptM='} 2024-09-19T06:21:38.143 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-19T06:21:38.149 INFO:teuthology.task.internal:roles: ubuntu@smithi037.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-19T06:21:38.149 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-19T06:21:38.149 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-19T06:21:38.149 INFO:teuthology.run_tasks:Running task console_log... 2024-09-19T06:21:38.222 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9b093abbe0>, signals=[15]) 2024-09-19T06:21:38.222 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-19T06:21:38.228 INFO:teuthology.task.internal:Opening connections... 2024-09-19T06:21:38.228 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:21:38.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:21:38.307 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-09-19T06:21:38.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:21:38.382 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-09-19T06:21:38.383 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:21:38.458 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-19T06:21:38.465 DEBUG:teuthology.orchestra.run.smithi037:> uname -m 2024-09-19T06:21:38.484 INFO:teuthology.orchestra.run.smithi037.stdout:x86_64 2024-09-19T06:21:38.484 DEBUG:teuthology.orchestra.run.smithi037:> cat /etc/os-release 2024-09-19T06:21:38.541 INFO:teuthology.orchestra.run.smithi037.stdout:NAME="CentOS Stream" 2024-09-19T06:21:38.541 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION="9" 2024-09-19T06:21:38.541 INFO:teuthology.orchestra.run.smithi037.stdout:ID="centos" 2024-09-19T06:21:38.541 INFO:teuthology.orchestra.run.smithi037.stdout:ID_LIKE="rhel fedora" 2024-09-19T06:21:38.541 INFO:teuthology.orchestra.run.smithi037.stdout:VERSION_ID="9" 2024-09-19T06:21:38.541 INFO:teuthology.orchestra.run.smithi037.stdout:PLATFORM_ID="platform:el9" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:ANSI_COLOR="0;31" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:LOGO="fedora-logo-icon" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:HOME_URL="https://centos.org/" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T06:21:38.542 INFO:teuthology.orchestra.run.smithi037.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T06:21:38.543 INFO:teuthology.lock.ops:Updating smithi037.front.sepia.ceph.com on lock server 2024-09-19T06:21:38.567 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-09-19T06:21:38.585 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-09-19T06:21:38.586 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-09-19T06:21:38.644 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-09-19T06:21:38.644 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-09-19T06:21:38.644 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-09-19T06:21:38.644 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-09-19T06:21:38.644 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-09-19T06:21:38.644 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T06:21:38.645 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T06:21:38.645 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-09-19T06:21:38.667 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-09-19T06:21:38.685 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-09-19T06:21:38.685 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="CentOS Stream" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="9" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:ID="centos" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE="rhel fedora" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="9" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:PLATFORM_ID="platform:el9" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:ANSI_COLOR="0;31" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:LOGO="fedora-logo-icon" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-19T06:21:38.743 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://centos.org/" 2024-09-19T06:21:38.744 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-19T06:21:38.744 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-19T06:21:38.744 INFO:teuthology.orchestra.run.smithi151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-19T06:21:38.744 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-09-19T06:21:38.766 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-19T06:21:38.778 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-19T06:21:38.784 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-19T06:21:38.785 DEBUG:teuthology.orchestra.run.smithi037:> test '!' -e /home/ubuntu/cephtest 2024-09-19T06:21:38.787 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-09-19T06:21:38.790 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-09-19T06:21:38.806 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-19T06:21:38.812 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-19T06:21:38.812 DEBUG:teuthology.orchestra.run.smithi037:> test -z $(ls -A /var/lib/ceph) 2024-09-19T06:21:38.845 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-09-19T06:21:38.847 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-09-19T06:21:38.943 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-19T06:21:39.019 INFO:teuthology.run_tasks:Running task kernel... 2024-09-19T06:21:39.032 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-19T06:21:39.032 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-19T06:21:39.033 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-19T06:21:39.033 DEBUG:teuthology.orchestra.run.smithi037:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T06:21:39.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:21:39.050 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-19T06:21:39.108 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-503.el9.x86_64 2024-09-19T06:21:39.108 INFO:teuthology.task.kernel:Running kernel on smithi037: 5.14.0-503.el9.x86_64 2024-09-19T06:21:39.108 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum install -y kernel 2024-09-19T06:21:41.628 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.3 MB 00:00 2024-09-19T06:21:49.062 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 3.2 MB/s | 20 MB 00:06 2024-09-19T06:21:53.954 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 6.5 MB 00:01 2024-09-19T06:21:55.755 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-19T06:21:56.651 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 29 MB/s | 23 MB 00:00 2024-09-19T06:22:02.323 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-19T06:22:04.314 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-19T06:22:04.314 INFO:teuthology.orchestra.run.smithi037.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-19T06:22:04.418 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-19T06:22:04.430 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: Package Architecture Version Repository Size 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout:================================================================================ 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout:Install 4 Packages 2024-09-19T06:22:04.431 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:22:04.432 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 79 M 2024-09-19T06:22:04.432 INFO:teuthology.orchestra.run.smithi037.stdout:Installed size: 126 M 2024-09-19T06:22:04.432 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-19T06:22:04.550 INFO:teuthology.orchestra.run.smithi037.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 366 kB/s | 36 kB 00:00 2024-09-19T06:22:04.950 INFO:teuthology.orchestra.run.smithi037.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-09-19T06:22:05.400 INFO:teuthology.orchestra.run.smithi037.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 36 MB/s | 35 MB 00:00 2024-09-19T06:22:05.701 INFO:teuthology.orchestra.run.smithi037.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-19T06:22:05.701 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:22:05.701 INFO:teuthology.orchestra.run.smithi037.stdout:Total 62 MB/s | 79 MB 00:01 2024-09-19T06:22:06.413 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-19T06:22:06.794 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-19T06:22:06.794 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-19T06:22:07.753 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-19T06:22:07.754 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-19T06:22:10.292 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-19T06:22:10.972 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-19T06:22:11.051 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:22:13.796 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:22:13.921 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:22:19.046 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:22:19.097 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:22:24.236 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:23:03.145 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:23:03.186 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:23:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:23:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-19T06:23:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-19T06:23:03.549 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-19T06:23:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-19T06:23:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:23:04.420 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-19T06:23:04.420 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-19T06:23:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-19T06:23:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-19T06:23:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-19T06:23:04.421 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:23:04.421 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-19T06:23:04.804 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:23:04.898 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:23:04.898 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:23:04.899 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-19T06:23:04.899 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-19T06:23:04.899 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-19T06:23:04.899 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-19T06:23:04.899 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi037.front.sepia.ceph.com, path=None, version=distro) 2024-09-19T06:23:04.899 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q kernel | sort -rV | head -n 1 2024-09-19T06:23:04.958 INFO:teuthology.orchestra.run.smithi037.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-19T06:23:04.958 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-19T06:23:04.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo rpm -qi grub2-tools 2024-09-19T06:23:05.000 INFO:teuthology.orchestra.run.smithi037.stdout:Name : grub2-tools 2024-09-19T06:23:05.000 INFO:teuthology.orchestra.run.smithi037.stdout:Epoch : 1 2024-09-19T06:23:05.000 INFO:teuthology.orchestra.run.smithi037.stdout:Version : 2.06 2024-09-19T06:23:05.000 INFO:teuthology.orchestra.run.smithi037.stdout:Release : 82.el9 2024-09-19T06:23:05.000 INFO:teuthology.orchestra.run.smithi037.stdout:Architecture: x86_64 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Group : Unspecified 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Size : 8274670 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:License : GPLv3+ 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Packager : builder@centos.org 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Vendor : CentOS 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Summary : Support tools for GRUB. 2024-09-19T06:23:05.001 INFO:teuthology.orchestra.run.smithi037.stdout:Description : 2024-09-19T06:23:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:23:05.002 INFO:teuthology.orchestra.run.smithi037.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-19T06:23:05.002 INFO:teuthology.orchestra.run.smithi037.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-19T06:23:05.002 INFO:teuthology.orchestra.run.smithi037.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-19T06:23:05.002 INFO:teuthology.orchestra.run.smithi037.stdout:hardware devices. 2024-09-19T06:23:05.002 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:23:05.002 INFO:teuthology.orchestra.run.smithi037.stdout:This subpackage provides tools for support of all platforms. 2024-09-19T06:23:05.003 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-19T06:23:05.003 INFO:teuthology.task.kernel:Updating grub on smithi037 to boot 5.14.0-508.el9.x86_64 2024-09-19T06:23:05.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-19T06:23:05.691 INFO:teuthology.orchestra.run.smithi037.stderr:Generating grub configuration file ... 2024-09-19T06:23:07.017 INFO:teuthology.orchestra.run.smithi037.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-19T06:23:07.046 INFO:teuthology.orchestra.run.smithi037.stderr:done 2024-09-19T06:23:07.048 DEBUG:teuthology.orchestra.run.smithi037:> mktemp 2024-09-19T06:23:07.064 INFO:teuthology.orchestra.run.smithi037.stdout:/tmp/tmp.PDoTo4idfc 2024-09-19T06:23:07.064 DEBUG:teuthology.orchestra.run.smithi037:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.PDoTo4idfc 2024-09-19T06:23:07.137 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0666 /tmp/tmp.PDoTo4idfc 2024-09-19T06:23:07.273 DEBUG:teuthology.orchestra.remote:smithi037:/tmp/tmp.PDoTo4idfc is 6KB 2024-09-19T06:23:07.321 DEBUG:teuthology.orchestra.run.smithi037:> rm -fr /tmp/tmp.PDoTo4idfc 2024-09-19T06:23:07.335 DEBUG:teuthology.orchestra.run.smithi037:> sudo /bin/ls /boot/loader/entries || true 2024-09-19T06:23:07.398 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-19T06:23:07.398 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-19T06:23:07.398 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-19T06:23:07.398 INFO:teuthology.orchestra.run.smithi037.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-19T06:23:07.399 DEBUG:teuthology.orchestra.run.smithi037:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-19T06:23:07.565 DEBUG:teuthology.orchestra.run.smithi037:> sudo shutdown -r now 2024-09-19T06:23:37.568 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-19T06:23:37.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-19T06:23:37.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:23:55.969 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-09-19T06:24:04.970 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-19T06:24:04.971 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:24:23.357 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.37 2024-09-19T06:24:35.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-19T06:24:35.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:24:38.128 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-19T06:24:38.562 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-19T06:24:38.563 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-19T06:24:38.563 DEBUG:teuthology.orchestra.run.smithi037:> uname -r 2024-09-19T06:24:38.578 INFO:teuthology.orchestra.run.smithi037.stdout:5.14.0-508.el9.x86_64 2024-09-19T06:24:38.578 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-19T06:24:38.578 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-19T06:24:38.578 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-19T06:24:39.580 DEBUG:teuthology.parallel:result is None 2024-09-19T06:24:39.580 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-19T06:24:39.587 INFO:teuthology.task.internal:Creating test directory... 2024-09-19T06:24:39.587 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:24:39.590 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:24:39.592 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-19T06:24:39.611 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-19T06:24:39.620 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-19T06:24:39.628 INFO:teuthology.task.internal:Creating archive directory... 2024-09-19T06:24:39.628 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:24:39.645 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:24:39.652 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-19T06:24:39.769 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-19T06:24:39.776 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-19T06:24:39.776 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T06:24:39.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:24:39.794 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-19T06:24:39.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:24:39.811 DEBUG:teuthology.orchestra.run.smithi037:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:24:39.813 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:24:39.837 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-19T06:24:39.875 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:24:39.890 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:24:39.895 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:24:39.914 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:24:39.930 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:24:39.951 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-19T06:24:39.953 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-19T06:24:39.961 INFO:teuthology.task.internal:Configuring sudo... 2024-09-19T06:24:39.961 DEBUG:teuthology.orchestra.run.smithi037:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:24:39.963 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:24:39.965 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-19T06:24:39.995 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-19T06:24:40.004 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-19T06:24:40.004 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:24:40.029 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:24:40.032 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-19T06:24:40.053 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:24:40.128 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:24:40.170 DEBUG:teuthology.orchestra.run.smithi037:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:24:40.255 DEBUG:teuthology.orchestra.run.smithi037:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:24:40.294 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:24:40.294 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:24:40.388 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:24:40.441 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:24:40.495 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:24:40.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:24:40.627 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:24:40.627 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:24:40.696 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:24:40.741 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-19T06:24:40.792 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:24:40.875 DEBUG:teuthology.orchestra.run.smithi151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-19T06:24:40.919 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:24:40.919 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-19T06:24:40.985 DEBUG:teuthology.orchestra.run.smithi037:> sudo service rsyslog restart 2024-09-19T06:24:40.988 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-09-19T06:24:40.990 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-09-19T06:24:41.031 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:24:41.063 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:24:41.146 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T06:24:41.451 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-19T06:24:41.459 INFO:teuthology.task.internal:Starting timer... 2024-09-19T06:24:41.459 INFO:teuthology.run_tasks:Running task pcp... 2024-09-19T06:24:41.548 INFO:teuthology.run_tasks:Running task selinux... 2024-09-19T06:24:41.556 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-19T06:24:41.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo service auditd rotate 2024-09-19T06:24:41.619 INFO:teuthology.orchestra.run.smithi037.stdout:Rotating logs: 2024-09-19T06:24:41.621 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-09-19T06:24:41.665 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-09-19T06:24:41.667 DEBUG:teuthology.orchestra.run.smithi151:> sudo service auditd rotate 2024-09-19T06:24:41.728 INFO:teuthology.orchestra.run.smithi151.stdout:Rotating logs: 2024-09-19T06:24:41.730 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-19T06:24:41.730 DEBUG:teuthology.orchestra.run.smithi037:> /usr/sbin/getenforce 2024-09-19T06:24:41.758 INFO:teuthology.orchestra.run.smithi037.stdout:Permissive 2024-09-19T06:24:41.758 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-09-19T06:24:41.788 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-09-19T06:24:41.788 DEBUG:teuthology.orchestra.run.smithi151:> /usr/sbin/getenforce 2024-09-19T06:24:41.810 INFO:teuthology.orchestra.run.smithi151.stdout:Permissive 2024-09-19T06:24:41.810 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi037.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi151.front.sepia.ceph.com': 'permissive'} 2024-09-19T06:24:41.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:24:41.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:24:41.836 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:24:41.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:24:41.867 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T06:24:41.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:24:41.894 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-19T06:24:41.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:24:41.928 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:24:41.968 DEBUG:teuthology.orchestra.run.smithi151:> sudo /usr/sbin/setenforce permissive 2024-09-19T06:24:42.006 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-19T06:24:42.016 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-19T06:24:42.030 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-19T06:24:42.030 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi037.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-19T06:28:46.083 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi037.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-09-19T06:28:46.085 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-19T06:28:46.086 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi037.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:28:46.168 DEBUG:teuthology.orchestra.run.smithi037:> true 2024-09-19T06:28:46.246 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi037.front.sepia.ceph.com' 2024-09-19T06:28:46.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-19T06:28:46.247 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:28:46.321 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-09-19T06:28:46.404 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-09-19T06:28:46.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-19T06:28:46.405 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-19T06:28:46.483 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-09-19T06:28:46.566 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-09-19T06:28:46.566 INFO:teuthology.run_tasks:Running task clock... 2024-09-19T06:28:46.579 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-19T06:28:46.579 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:28:46.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:28:46.582 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:28:46.582 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:28:46.584 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-19T06:28:46.585 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T06:28:46.612 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:28:46.615 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:28:46.627 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:28:46.632 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:28:46.639 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-19T06:28:46.652 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-09-19T06:28:46.654 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-19T06:28:46.659 INFO:teuthology.orchestra.run.smithi037.stderr:sudo: ntpd: command not found 2024-09-19T06:28:46.663 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-09-19T06:28:46.672 INFO:teuthology.orchestra.run.smithi037.stdout:506 Cannot talk to daemon 2024-09-19T06:28:46.679 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:28:46.679 INFO:teuthology.orchestra.run.smithi151.stderr:sudo: ntpd: command not found 2024-09-19T06:28:46.688 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:28:46.691 INFO:teuthology.orchestra.run.smithi151.stdout:506 Cannot talk to daemon 2024-09-19T06:28:46.693 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:28:46.704 INFO:teuthology.orchestra.run.smithi037.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:28:46.705 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-19T06:28:46.719 INFO:teuthology.orchestra.run.smithi151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-19T06:28:46.755 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:28:46.758 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:28:46.758 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-19T06:28:46.758 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.758 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.758 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.758 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.763 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:28:46.766 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:28:46.766 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-19T06:28:46.766 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.766 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.766 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.766 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.776 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-19T06:28:46.778 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T06:28:46.778 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-19T06:28:46.779 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.779 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.779 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.779 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T06:28:46.780 INFO:teuthology.run_tasks:Running task install... 2024-09-19T06:28:46.799 DEBUG:teuthology.task.install:project ceph 2024-09-19T06:28:46.799 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'}} 2024-09-19T06:28:46.799 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961'} 2024-09-19T06:28:46.800 INFO:teuthology.task.install:Using flavor: default 2024-09-19T06:28:46.811 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-19T06:28:46.811 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-19T06:28:46.812 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:28:46.813 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:28:46.816 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:28:46.816 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:28:46.818 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'tag': None, 'wait_for_package': False} 2024-09-19T06:28:46.818 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:28:46.967 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:28:46.967 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:28:46.972 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:28:46.972 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:28:47.009 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/ 2024-09-19T06:28:47.009 INFO:teuthology.task.install.rpm:Package version is 19.1.1-361.g2f599a46 2024-09-19T06:28:47.087 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:28:47.087 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:28:47.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:28:47.092 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:28:47.092 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:28:47.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:28:47.119 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T06:28:47.120 DEBUG:teuthology.orchestra.run.smithi151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:28:47.123 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-19T06:28:47.123 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:28:47.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-19T06:28:47.124 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T06:28:47.125 DEBUG:teuthology.orchestra.run.smithi037:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:28:47.150 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-19T06:28:47.150 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f599a46ad10e37c929e5c43da2f6907ddcd1961/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-19T06:28:47.185 DEBUG:teuthology.orchestra.run.smithi151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:28:47.194 DEBUG:teuthology.orchestra.run.smithi037:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:28:47.216 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-19T06:28:47.262 DEBUG:teuthology.orchestra.run.smithi151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:28:47.272 DEBUG:teuthology.orchestra.run.smithi037:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:28:47.291 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-19T06:28:47.326 INFO:teuthology.orchestra.run.smithi151.stdout:check_obsoletes = 1 2024-09-19T06:28:47.328 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum clean all 2024-09-19T06:28:47.338 INFO:teuthology.orchestra.run.smithi037.stdout:check_obsoletes = 1 2024-09-19T06:28:47.340 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum clean all 2024-09-19T06:28:47.355 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-09-19T06:28:47.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-09-19T06:28:47.553 INFO:teuthology.orchestra.run.smithi151.stdout:45 files removed 2024-09-19T06:28:47.580 DEBUG:teuthology.orchestra.run.smithi151:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T06:28:47.581 INFO:teuthology.orchestra.run.smithi096.stdout:45 files removed 2024-09-19T06:28:47.584 INFO:teuthology.orchestra.run.smithi037.stdout:45 files removed 2024-09-19T06:28:47.608 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T06:28:47.610 DEBUG:teuthology.orchestra.run.smithi037:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-19T06:28:48.133 INFO:teuthology.orchestra.run.smithi151.stdout:ceph packages for x86_64 306 kB/s | 84 kB 00:00 2024-09-19T06:28:48.149 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 315 kB/s | 84 kB 00:00 2024-09-19T06:28:48.191 INFO:teuthology.orchestra.run.smithi037.stdout:ceph packages for x86_64 298 kB/s | 84 kB 00:00 2024-09-19T06:28:48.416 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-19T06:28:48.417 INFO:teuthology.orchestra.run.smithi151.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-19T06:28:48.483 INFO:teuthology.orchestra.run.smithi037.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-19T06:28:48.684 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-19T06:28:48.692 INFO:teuthology.orchestra.run.smithi151.stdout:ceph source packages 7.9 kB/s | 1.9 kB 00:00 2024-09-19T06:28:48.751 INFO:teuthology.orchestra.run.smithi037.stdout:ceph source packages 8.3 kB/s | 1.9 kB 00:00 2024-09-19T06:28:49.143 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.3 MB 00:00 2024-09-19T06:28:50.875 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.3 MB 00:02 2024-09-19T06:28:51.467 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.3 MB 00:02 2024-09-19T06:28:57.368 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - AppStream 4.3 MB/s | 20 MB 00:04 2024-09-19T06:28:58.661 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - AppStream 2.4 MB/s | 20 MB 00:08 2024-09-19T06:29:01.725 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 2.1 MB/s | 20 MB 00:09 2024-09-19T06:29:03.053 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-19T06:29:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-19T06:29:05.732 INFO:teuthology.orchestra.run.smithi037.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-19T06:29:06.275 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.5 MB 00:00 2024-09-19T06:29:07.751 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 71 kB/s | 19 kB 00:00 2024-09-19T06:29:08.639 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-19T06:29:11.638 INFO:teuthology.orchestra.run.smithi037.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-19T06:29:13.901 INFO:teuthology.orchestra.run.smithi037.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:29:13.902 INFO:teuthology.orchestra.run.smithi037.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:29:13.916 INFO:teuthology.orchestra.run.smithi037.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T06:29:13.917 INFO:teuthology.orchestra.run.smithi037.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T06:29:13.918 INFO:teuthology.orchestra.run.smithi037.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T06:29:13.919 INFO:teuthology.orchestra.run.smithi037.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T06:29:13.921 INFO:teuthology.orchestra.run.smithi037.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T06:29:13.924 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T06:29:13.925 INFO:teuthology.orchestra.run.smithi037.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T06:29:13.994 INFO:teuthology.orchestra.run.smithi037.stdout:Dependencies resolved. 2024-09-19T06:29:14.011 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout:====================================================================================== 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: Package Arch Version Repository Size 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout:====================================================================================== 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout:Installing: 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:29:14.013 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T06:29:14.014 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T06:29:14.015 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrading: 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T06:29:14.016 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout:Installing dependencies: 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:29:14.017 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:29:14.018 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T06:29:14.019 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T06:29:14.020 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:29:14.021 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:29:14.022 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:29:14.023 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:29:14.024 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:29:14.025 INFO:teuthology.orchestra.run.smithi037.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout:Installing weak dependencies: 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction Summary 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout:====================================================================================== 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout:Install 148 Packages 2024-09-19T06:29:14.026 INFO:teuthology.orchestra.run.smithi037.stdout:Upgrade 8 Packages 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi037.stdout:Total download size: 226 M 2024-09-19T06:29:14.027 INFO:teuthology.orchestra.run.smithi037.stdout:Downloading Packages: 2024-09-19T06:29:14.473 INFO:teuthology.orchestra.run.smithi037.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 53 kB/s | 6.6 kB 00:00 2024-09-19T06:29:14.673 INFO:teuthology.orchestra.run.smithi037.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 4.4 MB/s | 896 kB 00:00 2024-09-19T06:29:14.765 INFO:teuthology.orchestra.run.smithi037.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 1.5 MB/s | 145 kB 00:00 2024-09-19T06:29:14.940 INFO:teuthology.orchestra.run.smithi037.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 9.2 MB/s | 5.4 MB 00:00 2024-09-19T06:29:15.082 INFO:teuthology.orchestra.run.smithi037.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 7.5 MB/s | 2.4 MB 00:00 2024-09-19T06:29:15.124 INFO:teuthology.orchestra.run.smithi037.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 8.3 MB/s | 1.5 MB 00:00 2024-09-19T06:29:15.641 INFO:teuthology.orchestra.run.smithi037.stdout:(7/156): ceph-common-19.1.1-361.g2f599a46.el9.x 16 MB/s | 21 MB 00:01 2024-09-19T06:29:15.717 INFO:teuthology.orchestra.run.smithi037.stdout:(8/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 7.4 MB/s | 4.7 MB 00:00 2024-09-19T06:29:15.783 INFO:teuthology.orchestra.run.smithi037.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 380 kB/s | 25 kB 00:00 2024-09-19T06:29:16.200 INFO:teuthology.orchestra.run.smithi037.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 19 MB/s | 11 MB 00:00 2024-09-19T06:29:16.238 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:29:16.239 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:29:16.250 INFO:teuthology.orchestra.run.smithi037.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T06:29:16.250 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T06:29:16.251 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T06:29:16.252 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T06:29:16.253 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T06:29:16.254 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T06:29:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T06:29:16.257 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T06:29:16.320 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-09-19T06:29:16.334 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-19T06:29:16.334 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-09-19T06:29:16.334 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-19T06:29:16.334 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-09-19T06:29:16.334 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T06:29:16.334 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:29:16.335 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T06:29:16.336 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-09-19T06:29:16.337 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:29:16.338 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:29:16.339 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:29:16.340 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:29:16.341 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T06:29:16.342 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:29:16.343 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:29:16.344 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:29:16.345 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:29:16.346 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:29:16.347 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:29:16.348 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-09-19T06:29:16.348 INFO:teuthology.orchestra.run.smithi096.stdout:====================================================================================== 2024-09-19T06:29:16.348 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-09-19T06:29:16.348 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 8 Packages 2024-09-19T06:29:16.348 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:29:16.348 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 226 M 2024-09-19T06:29:16.349 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-09-19T06:29:16.434 INFO:teuthology.orchestra.run.smithi037.stdout:(12/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 13 MB/s | 17 MB 00:01 2024-09-19T06:29:16.459 INFO:teuthology.orchestra.run.smithi037.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 130 kB/s | 27 kB 00:00 2024-09-19T06:29:16.484 INFO:teuthology.orchestra.run.smithi037.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 635 kB/s | 32 kB 00:00 2024-09-19T06:29:16.535 INFO:teuthology.orchestra.run.smithi037.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 9.8 MB/s | 751 kB 00:00 2024-09-19T06:29:16.560 INFO:teuthology.orchestra.run.smithi037.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T06:29:16.585 INFO:teuthology.orchestra.run.smithi037.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T06:29:16.619 INFO:teuthology.orchestra.run.smithi037.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 8.4 MB/s | 506 kB 00:00 2024-09-19T06:29:16.686 INFO:teuthology.orchestra.run.smithi037.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 678 kB/s | 45 kB 00:00 2024-09-19T06:29:16.853 INFO:teuthology.orchestra.run.smithi037.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 817 kB/s | 136 kB 00:00 2024-09-19T06:29:16.904 INFO:teuthology.orchestra.run.smithi096.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 51 kB/s | 6.6 kB 00:00 2024-09-19T06:29:16.928 INFO:teuthology.orchestra.run.smithi037.stdout:(21/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 16 MB/s | 5.3 MB 00:00 2024-09-19T06:29:16.953 INFO:teuthology.orchestra.run.smithi037.stdout:(22/156): python3-cephfs-19.1.1-361.g2f599a46.e 1.6 MB/s | 161 kB 00:00 2024-09-19T06:29:16.978 INFO:teuthology.orchestra.run.smithi037.stdout:(23/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T06:29:17.020 INFO:teuthology.orchestra.run.smithi037.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 4.4 MB/s | 300 kB 00:00 2024-09-19T06:29:17.046 INFO:teuthology.orchestra.run.smithi037.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.4 MB/s | 99 kB 00:00 2024-09-19T06:29:17.088 INFO:teuthology.orchestra.run.smithi037.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-19T06:29:17.180 INFO:teuthology.orchestra.run.smithi037.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.8 MB/s | 171 kB 00:00 2024-09-19T06:29:17.254 INFO:teuthology.orchestra.run.smithi096.stdout:(2/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.5 MB/s | 896 kB 00:00 2024-09-19T06:29:17.297 INFO:teuthology.orchestra.run.smithi037.stdout:(28/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-19T06:29:17.313 INFO:teuthology.orchestra.run.smithi096.stdout:(3/156): ceph-immutable-object-cache-19.1.1-361 2.4 MB/s | 145 kB 00:00 2024-09-19T06:29:17.314 INFO:teuthology.orchestra.run.smithi037.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 238 kB/s | 32 kB 00:00 2024-09-19T06:29:17.347 INFO:teuthology.orchestra.run.smithi037.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.9 MB/s | 146 kB 00:00 2024-09-19T06:29:17.714 INFO:teuthology.orchestra.run.smithi037.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 8.9 MB/s | 3.6 MB 00:00 2024-09-19T06:29:17.788 INFO:teuthology.orchestra.run.smithi096.stdout:(4/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 5.3 MB/s | 5.4 MB 00:01 2024-09-19T06:29:17.831 INFO:teuthology.orchestra.run.smithi037.stdout:(32/156): ceph-mgr-diskprediction-local-19.1.1- 15 MB/s | 7.4 MB 00:00 2024-09-19T06:29:17.857 INFO:teuthology.orchestra.run.smithi037.stdout:(33/156): ceph-mgr-modules-core-19.1.1-361.g2f5 1.7 MB/s | 249 kB 00:00 2024-09-19T06:29:17.890 INFO:teuthology.orchestra.run.smithi037.stdout:(34/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 846 kB/s | 49 kB 00:00 2024-09-19T06:29:17.915 INFO:teuthology.orchestra.run.smithi037.stdout:(35/156): ceph-prometheus-alerts-19.1.1-361.g2f 282 kB/s | 16 kB 00:00 2024-09-19T06:29:17.949 INFO:teuthology.orchestra.run.smithi037.stdout:(36/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T06:29:17.955 INFO:teuthology.orchestra.run.smithi096.stdout:(5/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.7 MB/s | 2.4 MB 00:00 2024-09-19T06:29:17.997 INFO:teuthology.orchestra.run.smithi096.stdout:(6/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 7.3 MB/s | 1.5 MB 00:00 2024-09-19T06:29:18.007 INFO:teuthology.orchestra.run.smithi037.stdout:(37/156): cephadm-19.1.1-361.g2f599a46.el9.noar 8.1 MB/s | 765 kB 00:00 2024-09-19T06:29:18.033 INFO:teuthology.orchestra.run.smithi037.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 218 kB/s | 18 kB 00:00 2024-09-19T06:29:18.150 INFO:teuthology.orchestra.run.smithi037.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-19T06:29:18.217 INFO:teuthology.orchestra.run.smithi037.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 4.3 MB/s | 801 kB 00:00 2024-09-19T06:29:18.242 INFO:teuthology.orchestra.run.smithi037.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-19T06:29:18.276 INFO:teuthology.orchestra.run.smithi037.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-09-19T06:29:18.317 INFO:teuthology.orchestra.run.smithi037.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.5 MB/s | 106 kB 00:00 2024-09-19T06:29:18.360 INFO:teuthology.orchestra.run.smithi037.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-09-19T06:29:18.385 INFO:teuthology.orchestra.run.smithi037.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-19T06:29:18.410 INFO:teuthology.orchestra.run.smithi037.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-19T06:29:18.435 INFO:teuthology.orchestra.run.smithi037.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-19T06:29:18.619 INFO:teuthology.orchestra.run.smithi037.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.2 MB/s | 689 kB 00:00 2024-09-19T06:29:18.661 INFO:teuthology.orchestra.run.smithi037.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-19T06:29:18.686 INFO:teuthology.orchestra.run.smithi037.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-19T06:29:18.711 INFO:teuthology.orchestra.run.smithi037.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 434 kB/s | 22 kB 00:00 2024-09-19T06:29:18.737 INFO:teuthology.orchestra.run.smithi037.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 6.1 MB/s | 313 kB 00:00 2024-09-19T06:29:18.762 INFO:teuthology.orchestra.run.smithi037.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-09-19T06:29:18.804 INFO:teuthology.orchestra.run.smithi037.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 437 kB/s | 18 kB 00:00 2024-09-19T06:29:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-09-19T06:29:18.872 INFO:teuthology.orchestra.run.smithi096.stdout:(7/156): ceph-common-19.1.1-361.g2f599a46.el9.x 10 MB/s | 21 MB 00:02 2024-09-19T06:29:18.921 INFO:teuthology.orchestra.run.smithi037.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-09-19T06:29:18.947 INFO:teuthology.orchestra.run.smithi037.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-19T06:29:18.972 INFO:teuthology.orchestra.run.smithi037.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-19T06:29:19.005 INFO:teuthology.orchestra.run.smithi037.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-19T06:29:19.006 INFO:teuthology.orchestra.run.smithi096.stdout:(8/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 4.5 MB/s | 4.7 MB 00:01 2024-09-19T06:29:19.039 INFO:teuthology.orchestra.run.smithi037.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-19T06:29:19.064 INFO:teuthology.orchestra.run.smithi037.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 418 kB/s | 24 kB 00:00 2024-09-19T06:29:19.090 INFO:teuthology.orchestra.run.smithi037.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 950 kB/s | 47 kB 00:00 2024-09-19T06:29:19.115 INFO:teuthology.orchestra.run.smithi037.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-19T06:29:19.132 INFO:teuthology.orchestra.run.smithi096.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 199 kB/s | 25 kB 00:00 2024-09-19T06:29:19.140 INFO:teuthology.orchestra.run.smithi037.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-19T06:29:19.165 INFO:teuthology.orchestra.run.smithi037.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-19T06:29:19.274 INFO:teuthology.orchestra.run.smithi037.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 37 MB/s | 4.9 MB 00:00 2024-09-19T06:29:19.350 INFO:teuthology.orchestra.run.smithi037.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-19T06:29:19.607 INFO:teuthology.orchestra.run.smithi096.stdout:(10/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 10 MB/s | 17 MB 00:01 2024-09-19T06:29:19.740 INFO:teuthology.orchestra.run.smithi096.stdout:(11/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 12 MB/s | 11 MB 00:00 2024-09-19T06:29:19.765 INFO:teuthology.orchestra.run.smithi096.stdout:(12/156): cephfs-mirror-19.1.1-361.g2f599a46.el 1.4 MB/s | 228 kB 00:00 2024-09-19T06:29:19.775 INFO:teuthology.orchestra.run.smithi037.stdout:(68/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 12 MB/s | 49 MB 00:03 2024-09-19T06:29:19.790 INFO:teuthology.orchestra.run.smithi096.stdout:(13/156): cephfs-shell-19.1.1-361.g2f599a46.el9 541 kB/s | 27 kB 00:00 2024-09-19T06:29:19.816 INFO:teuthology.orchestra.run.smithi096.stdout:(14/156): libcephfs-devel-19.1.1-361.g2f599a46. 632 kB/s | 32 kB 00:00 2024-09-19T06:29:19.833 INFO:teuthology.orchestra.run.smithi037.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 1.8 MB/s | 910 kB 00:00 2024-09-19T06:29:19.866 INFO:teuthology.orchestra.run.smithi096.stdout:(15/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 9.7 MB/s | 751 kB 00:00 2024-09-19T06:29:19.875 INFO:teuthology.orchestra.run.smithi037.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 2.6 MB/s | 1.6 MB 00:00 2024-09-19T06:29:19.891 INFO:teuthology.orchestra.run.smithi096.stdout:(16/156): libcephsqlite-19.1.1-361.g2f599a46.el 2.1 MB/s | 163 kB 00:00 2024-09-19T06:29:19.917 INFO:teuthology.orchestra.run.smithi096.stdout:(17/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T06:29:19.934 INFO:teuthology.orchestra.run.smithi037.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-19T06:29:19.959 INFO:teuthology.orchestra.run.smithi037.stdout:(72/156): postgresql-private-libs-13.16-1.el9.x 748 kB/s | 137 kB 00:00 2024-09-19T06:29:19.975 INFO:teuthology.orchestra.run.smithi096.stdout:(18/156): libradosstriper1-19.1.1-361.g2f599a46 5.9 MB/s | 506 kB 00:00 2024-09-19T06:29:19.984 INFO:teuthology.orchestra.run.smithi037.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-09-19T06:29:20.034 INFO:teuthology.orchestra.run.smithi096.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 774 kB/s | 45 kB 00:00 2024-09-19T06:29:20.068 INFO:teuthology.orchestra.run.smithi037.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 25 MB/s | 5.8 MB 00:00 2024-09-19T06:29:20.084 INFO:teuthology.orchestra.run.smithi096.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.7 MB/s | 136 kB 00:00 2024-09-19T06:29:20.093 INFO:teuthology.orchestra.run.smithi037.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 564 kB/s | 61 kB 00:00 2024-09-19T06:29:20.118 INFO:teuthology.orchestra.run.smithi037.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-19T06:29:20.135 INFO:teuthology.orchestra.run.smithi096.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.2 MB/s | 161 kB 00:00 2024-09-19T06:29:20.144 INFO:teuthology.orchestra.run.smithi037.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-19T06:29:20.169 INFO:teuthology.orchestra.run.smithi037.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-19T06:29:20.194 INFO:teuthology.orchestra.run.smithi037.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-19T06:29:20.202 INFO:teuthology.orchestra.run.smithi096.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 4.7 MB/s | 320 kB 00:00 2024-09-19T06:29:20.220 INFO:teuthology.orchestra.run.smithi037.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-19T06:29:20.245 INFO:teuthology.orchestra.run.smithi037.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-19T06:29:20.277 INFO:teuthology.orchestra.run.smithi096.stdout:(23/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-09-19T06:29:20.295 INFO:teuthology.orchestra.run.smithi037.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-19T06:29:20.302 INFO:teuthology.orchestra.run.smithi096.stdout:(24/156): python3-rbd-19.1.1-361.g2f599a46.el9. 2.9 MB/s | 300 kB 00:00 2024-09-19T06:29:20.328 INFO:teuthology.orchestra.run.smithi096.stdout:(25/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T06:29:20.362 INFO:teuthology.orchestra.run.smithi037.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-19T06:29:20.369 INFO:teuthology.orchestra.run.smithi096.stdout:(26/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.3 MB/s | 85 kB 00:00 2024-09-19T06:29:20.396 INFO:teuthology.orchestra.run.smithi037.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-19T06:29:20.421 INFO:teuthology.orchestra.run.smithi037.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-19T06:29:20.424 INFO:teuthology.orchestra.run.smithi096.stdout:(27/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 3.1 MB/s | 171 kB 00:00 2024-09-19T06:29:20.446 INFO:teuthology.orchestra.run.smithi037.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 842 kB/s | 42 kB 00:00 2024-09-19T06:29:20.474 INFO:teuthology.orchestra.run.smithi096.stdout:(28/156): ceph-grafana-dashboards-19.1.1-361.g2 639 kB/s | 32 kB 00:00 2024-09-19T06:29:20.480 INFO:teuthology.orchestra.run.smithi037.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-19T06:29:20.556 INFO:teuthology.orchestra.run.smithi037.stdout:(88/156): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-09-19T06:29:20.558 INFO:teuthology.orchestra.run.smithi096.stdout:(29/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 14 MB/s | 3.1 MB 00:00 2024-09-19T06:29:20.580 INFO:teuthology.orchestra.run.smithi037.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-19T06:29:20.583 INFO:teuthology.orchestra.run.smithi096.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 1.3 MB/s | 146 kB 00:00 2024-09-19T06:29:20.606 INFO:teuthology.orchestra.run.smithi037.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-19T06:29:20.631 INFO:teuthology.orchestra.run.smithi037.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-19T06:29:20.657 INFO:teuthology.orchestra.run.smithi037.stdout:(92/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.5 MB/s | 279 kB 00:00 2024-09-19T06:29:20.682 INFO:teuthology.orchestra.run.smithi037.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 829 kB/s | 42 kB 00:00 2024-09-19T06:29:20.707 INFO:teuthology.orchestra.run.smithi037.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 881 kB/s | 44 kB 00:00 2024-09-19T06:29:20.732 INFO:teuthology.orchestra.run.smithi037.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-19T06:29:20.758 INFO:teuthology.orchestra.run.smithi037.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-09-19T06:29:20.775 INFO:teuthology.orchestra.run.smithi096.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 16 MB/s | 3.6 MB 00:00 2024-09-19T06:29:20.783 INFO:teuthology.orchestra.run.smithi037.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T06:29:20.808 INFO:teuthology.orchestra.run.smithi037.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-19T06:29:20.825 INFO:teuthology.orchestra.run.smithi096.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.9 MB/s | 249 kB 00:00 2024-09-19T06:29:20.833 INFO:teuthology.orchestra.run.smithi037.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-19T06:29:20.858 INFO:teuthology.orchestra.run.smithi037.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T06:29:20.875 INFO:teuthology.orchestra.run.smithi096.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 986 kB/s | 49 kB 00:00 2024-09-19T06:29:20.884 INFO:teuthology.orchestra.run.smithi037.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T06:29:20.909 INFO:teuthology.orchestra.run.smithi037.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-19T06:29:20.926 INFO:teuthology.orchestra.run.smithi096.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 329 kB/s | 16 kB 00:00 2024-09-19T06:29:20.934 INFO:teuthology.orchestra.run.smithi037.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-09-19T06:29:20.977 INFO:teuthology.orchestra.run.smithi096.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 5.6 MB/s | 285 kB 00:00 2024-09-19T06:29:20.977 INFO:teuthology.orchestra.run.smithi037.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 13 MB/s | 863 kB 00:00 2024-09-19T06:29:21.027 INFO:teuthology.orchestra.run.smithi037.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-19T06:29:21.043 INFO:teuthology.orchestra.run.smithi037.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-19T06:29:21.060 INFO:teuthology.orchestra.run.smithi096.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 9.0 MB/s | 765 kB 00:00 2024-09-19T06:29:21.068 INFO:teuthology.orchestra.run.smithi037.stdout:(107/156): gperftools-libs-2.9.1-3.el9.x86_64.r 3.3 MB/s | 308 kB 00:00 2024-09-19T06:29:21.094 INFO:teuthology.orchestra.run.smithi037.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T06:29:21.119 INFO:teuthology.orchestra.run.smithi037.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T06:29:21.136 INFO:teuthology.orchestra.run.smithi037.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 3.9 MB/s | 67 kB 00:00 2024-09-19T06:29:21.144 INFO:teuthology.orchestra.run.smithi096.stdout:(37/156): ceph-mgr-diskprediction-local-19.1.1- 13 MB/s | 7.4 MB 00:00 2024-09-19T06:29:21.169 INFO:teuthology.orchestra.run.smithi096.stdout:(38/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 167 kB/s | 18 kB 00:00 2024-09-19T06:29:21.170 INFO:teuthology.orchestra.run.smithi037.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-19T06:29:21.228 INFO:teuthology.orchestra.run.smithi037.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 14 MB/s | 839 kB 00:00 2024-09-19T06:29:21.271 INFO:teuthology.orchestra.run.smithi037.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T06:29:21.295 INFO:teuthology.orchestra.run.smithi037.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T06:29:21.321 INFO:teuthology.orchestra.run.smithi037.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T06:29:21.346 INFO:teuthology.orchestra.run.smithi037.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 572 kB/s | 14 kB 00:00 2024-09-19T06:29:21.371 INFO:teuthology.orchestra.run.smithi037.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.8 MB/s | 172 kB 00:00 2024-09-19T06:29:21.405 INFO:teuthology.orchestra.run.smithi037.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 10 MB/s | 358 kB 00:00 2024-09-19T06:29:21.472 INFO:teuthology.orchestra.run.smithi037.stdout:(119/156): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-09-19T06:29:21.514 INFO:teuthology.orchestra.run.smithi037.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-09-19T06:29:21.539 INFO:teuthology.orchestra.run.smithi037.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.2 MB/s | 220 kB 00:00 2024-09-19T06:29:21.565 INFO:teuthology.orchestra.run.smithi037.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T06:29:21.590 INFO:teuthology.orchestra.run.smithi037.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-19T06:29:21.615 INFO:teuthology.orchestra.run.smithi037.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 711 kB/s | 18 kB 00:00 2024-09-19T06:29:21.619 INFO:teuthology.orchestra.run.smithi096.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-09-19T06:29:21.641 INFO:teuthology.orchestra.run.smithi037.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 931 kB/s | 23 kB 00:00 2024-09-19T06:29:21.666 INFO:teuthology.orchestra.run.smithi037.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-19T06:29:21.699 INFO:teuthology.orchestra.run.smithi037.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 592 kB/s | 20 kB 00:00 2024-09-19T06:29:21.836 INFO:teuthology.orchestra.run.smithi096.stdout:(40/156): libgfortran-11.5.0-2.el9.x86_64.rpm 1.2 MB/s | 801 kB 00:00 2024-09-19T06:29:21.866 INFO:teuthology.orchestra.run.smithi037.stdout:(128/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-09-19T06:29:21.917 INFO:teuthology.orchestra.run.smithi037.stdout:(129/156): python3-kubernetes-26.1.0-2.el9.noar 4.7 MB/s | 1.0 MB 00:00 2024-09-19T06:29:21.967 INFO:teuthology.orchestra.run.smithi037.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-19T06:29:21.992 INFO:teuthology.orchestra.run.smithi037.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 370 kB/s | 46 kB 00:00 2024-09-19T06:29:22.017 INFO:teuthology.orchestra.run.smithi037.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 803 kB/s | 79 kB 00:00 2024-09-19T06:29:22.042 INFO:teuthology.orchestra.run.smithi037.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 767 kB/s | 58 kB 00:00 2024-09-19T06:29:22.075 INFO:teuthology.orchestra.run.smithi037.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 284 kB/s | 16 kB 00:00 2024-09-19T06:29:22.101 INFO:teuthology.orchestra.run.smithi037.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T06:29:22.112 INFO:teuthology.orchestra.run.smithi096.stdout:(41/156): libquadmath-11.5.0-2.el9.x86_64.rpm 388 kB/s | 191 kB 00:00 2024-09-19T06:29:22.126 INFO:teuthology.orchestra.run.smithi037.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-09-19T06:29:22.151 INFO:teuthology.orchestra.run.smithi037.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 616 kB/s | 31 kB 00:00 2024-09-19T06:29:22.177 INFO:teuthology.orchestra.run.smithi037.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T06:29:22.202 INFO:teuthology.orchestra.run.smithi037.stdout:(139/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-09-19T06:29:22.235 INFO:teuthology.orchestra.run.smithi037.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 613 kB/s | 36 kB 00:00 2024-09-19T06:29:22.269 INFO:teuthology.orchestra.run.smithi037.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-19T06:29:22.294 INFO:teuthology.orchestra.run.smithi037.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.9 MB/s | 90 kB 00:00 2024-09-19T06:29:22.319 INFO:teuthology.orchestra.run.smithi037.stdout:(143/156): python3-typing-extensions-4.12.2-2.e 639 kB/s | 74 kB 00:00 2024-09-19T06:29:22.337 INFO:teuthology.orchestra.run.smithi096.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 506 kB/s | 253 kB 00:00 2024-09-19T06:29:22.345 INFO:teuthology.orchestra.run.smithi037.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 804 kB/s | 20 kB 00:00 2024-09-19T06:29:22.370 INFO:teuthology.orchestra.run.smithi037.stdout:(145/156): python3-webob-1.8.8-2.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-09-19T06:29:22.404 INFO:teuthology.orchestra.run.smithi037.stdout:(146/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-19T06:29:22.429 INFO:teuthology.orchestra.run.smithi037.stdout:(147/156): re2-20211101-20.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-19T06:29:22.479 INFO:teuthology.orchestra.run.smithi037.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-19T06:29:22.663 INFO:teuthology.orchestra.run.smithi037.stdout:(149/156): NetworkManager-1.51.0-1.el9.x86_64.r 13 MB/s | 2.3 MB 00:00 2024-09-19T06:29:22.713 INFO:teuthology.orchestra.run.smithi037.stdout:(150/156): librados2-19.1.1-361.g2f599a46.el9.x 11 MB/s | 3.4 MB 00:00 2024-09-19T06:29:22.754 INFO:teuthology.orchestra.run.smithi096.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 256 kB/s | 106 kB 00:00 2024-09-19T06:29:22.763 INFO:teuthology.orchestra.run.smithi037.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 18 MB/s | 1.8 MB 00:00 2024-09-19T06:29:22.780 INFO:teuthology.orchestra.run.smithi037.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 613 kB/s | 41 kB 00:00 2024-09-19T06:29:22.814 INFO:teuthology.orchestra.run.smithi037.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 4.8 MB/s | 249 kB 00:00 2024-09-19T06:29:22.864 INFO:teuthology.orchestra.run.smithi037.stdout:(154/156): librbd1-19.1.1-361.g2f599a46.el9.x86 7.3 MB/s | 3.2 MB 00:00 2024-09-19T06:29:22.921 INFO:teuthology.orchestra.run.smithi096.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.5 MB/s | 1.2 MB 00:00 2024-09-19T06:29:22.956 INFO:teuthology.orchestra.run.smithi037.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-19T06:29:22.998 INFO:teuthology.orchestra.run.smithi037.stdout:(156/156): openssl-3.2.2-6.el9.x86_64.rpm 6.2 MB/s | 1.4 MB 00:00 2024-09-19T06:29:23.004 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:29:23.004 INFO:teuthology.orchestra.run.smithi037.stdout:Total 25 MB/s | 226 MB 00:08 2024-09-19T06:29:23.252 INFO:teuthology.orchestra.run.smithi096.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 271 kB/s | 135 kB 00:00 2024-09-19T06:29:23.421 INFO:teuthology.orchestra.run.smithi096.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 250 kB/s | 125 kB 00:00 2024-09-19T06:29:23.605 INFO:teuthology.orchestra.run.smithi096.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 3.7 MB/s | 689 kB 00:00 2024-09-19T06:29:23.676 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - CRB 315 kB/s | 6.5 MB 00:21 2024-09-19T06:29:23.680 INFO:teuthology.orchestra.run.smithi096.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 12 MB/s | 943 kB 00:00 2024-09-19T06:29:23.732 INFO:teuthology.orchestra.run.smithi096.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-19T06:29:23.756 INFO:teuthology.orchestra.run.smithi096.stdout:(50/156): python3-urllib3-1.26.5-6.el9.noarch.r 426 kB/s | 215 kB 00:00 2024-09-19T06:29:23.773 INFO:teuthology.orchestra.run.smithi096.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 517 kB/s | 22 kB 00:00 2024-09-19T06:29:23.807 INFO:teuthology.orchestra.run.smithi096.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 978 kB/s | 33 kB 00:00 2024-09-19T06:29:23.899 INFO:teuthology.orchestra.run.smithi096.stdout:(53/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 33 MB/s | 3.0 MB 00:00 2024-09-19T06:29:23.940 INFO:teuthology.orchestra.run.smithi096.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 1.7 MB/s | 313 kB 00:00 2024-09-19T06:29:23.966 INFO:teuthology.orchestra.run.smithi096.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-09-19T06:29:23.991 INFO:teuthology.orchestra.run.smithi096.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 221 kB/s | 11 kB 00:00 2024-09-19T06:29:24.016 INFO:teuthology.orchestra.run.smithi096.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-19T06:29:24.042 INFO:teuthology.orchestra.run.smithi096.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 900 kB/s | 45 kB 00:00 2024-09-19T06:29:24.084 INFO:teuthology.orchestra.run.smithi096.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-09-19T06:29:24.148 INFO:teuthology.orchestra.run.smithi096.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 5.4 MB/s | 581 kB 00:00 2024-09-19T06:29:24.176 INFO:teuthology.orchestra.run.smithi096.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 266 kB/s | 24 kB 00:00 2024-09-19T06:29:24.269 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction check 2024-09-19T06:29:24.276 INFO:teuthology.orchestra.run.smithi096.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 370 kB/s | 47 kB 00:00 2024-09-19T06:29:24.310 INFO:teuthology.orchestra.run.smithi096.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-09-19T06:29:24.344 INFO:teuthology.orchestra.run.smithi096.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-09-19T06:29:24.369 INFO:teuthology.orchestra.run.smithi096.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 658 kB/s | 38 kB 00:00 2024-09-19T06:29:24.437 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction check succeeded. 2024-09-19T06:29:24.437 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction test 2024-09-19T06:29:24.828 INFO:teuthology.orchestra.run.smithi096.stdout:(66/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 8.5 MB/s | 49 MB 00:05 2024-09-19T06:29:24.903 INFO:teuthology.orchestra.run.smithi096.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 8.8 MB/s | 4.9 MB 00:00 2024-09-19T06:29:24.970 INFO:teuthology.orchestra.run.smithi096.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 7.4 MB/s | 4.4 MB 00:00 2024-09-19T06:29:25.013 INFO:teuthology.orchestra.run.smithi096.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.2 MB/s | 910 kB 00:00 2024-09-19T06:29:25.037 INFO:teuthology.orchestra.run.smithi096.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-19T06:29:25.096 INFO:teuthology.orchestra.run.smithi096.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-19T06:29:25.171 INFO:teuthology.orchestra.run.smithi096.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 36 MB/s | 5.8 MB 00:00 2024-09-19T06:29:25.197 INFO:teuthology.orchestra.run.smithi096.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-09-19T06:29:25.239 INFO:teuthology.orchestra.run.smithi096.stdout:(74/156): postgresql-13.16-1.el9.x86_64.rpm 3.9 MB/s | 1.6 MB 00:00 2024-09-19T06:29:25.264 INFO:teuthology.orchestra.run.smithi096.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 917 kB/s | 61 kB 00:00 2024-09-19T06:29:25.289 INFO:teuthology.orchestra.run.smithi096.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-19T06:29:25.314 INFO:teuthology.orchestra.run.smithi096.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-19T06:29:25.340 INFO:teuthology.orchestra.run.smithi096.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-19T06:29:25.343 INFO:teuthology.orchestra.run.smithi151.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-19T06:29:25.365 INFO:teuthology.orchestra.run.smithi096.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 381 kB/s | 19 kB 00:00 2024-09-19T06:29:25.390 INFO:teuthology.orchestra.run.smithi096.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-19T06:29:25.407 INFO:teuthology.orchestra.run.smithi096.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 827 kB/s | 35 kB 00:00 2024-09-19T06:29:25.449 INFO:teuthology.orchestra.run.smithi096.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 5.9 MB/s | 250 kB 00:00 2024-09-19T06:29:25.525 INFO:teuthology.orchestra.run.smithi096.stdout:(83/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-19T06:29:25.550 INFO:teuthology.orchestra.run.smithi096.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-19T06:29:25.576 INFO:teuthology.orchestra.run.smithi096.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T06:29:25.601 INFO:teuthology.orchestra.run.smithi096.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 840 kB/s | 42 kB 00:00 2024-09-19T06:29:25.626 INFO:teuthology.orchestra.run.smithi096.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.3 MB/s | 269 kB 00:00 2024-09-19T06:29:25.651 INFO:teuthology.orchestra.run.smithi096.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-19T06:29:25.685 INFO:teuthology.orchestra.run.smithi096.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-19T06:29:25.719 INFO:teuthology.orchestra.run.smithi096.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 758 kB/s | 51 kB 00:00 2024-09-19T06:29:25.744 INFO:teuthology.orchestra.run.smithi096.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 914 kB/s | 54 kB 00:00 2024-09-19T06:29:25.819 INFO:teuthology.orchestra.run.smithi096.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-19T06:29:25.845 INFO:teuthology.orchestra.run.smithi096.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-09-19T06:29:25.870 INFO:teuthology.orchestra.run.smithi096.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 883 kB/s | 44 kB 00:00 2024-09-19T06:29:25.904 INFO:teuthology.orchestra.run.smithi096.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-19T06:29:25.929 INFO:teuthology.orchestra.run.smithi096.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 613 kB/s | 36 kB 00:00 2024-09-19T06:29:25.954 INFO:teuthology.orchestra.run.smithi096.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-09-19T06:29:25.988 INFO:teuthology.orchestra.run.smithi096.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-19T06:29:26.013 INFO:teuthology.orchestra.run.smithi096.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 986 kB/s | 58 kB 00:00 2024-09-19T06:29:26.038 INFO:teuthology.orchestra.run.smithi096.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-19T06:29:26.063 INFO:teuthology.orchestra.run.smithi096.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-19T06:29:26.089 INFO:teuthology.orchestra.run.smithi096.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 893 kB/s | 45 kB 00:00 2024-09-19T06:29:26.314 INFO:teuthology.orchestra.run.smithi096.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-19T06:29:26.406 INFO:teuthology.orchestra.run.smithi096.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 5.9 MB/s | 548 kB 00:00 2024-09-19T06:29:26.431 INFO:teuthology.orchestra.run.smithi096.stdout:(105/156): lua-devel-5.4.4-4.el9.x86_64.rpm 61 kB/s | 22 kB 00:00 2024-09-19T06:29:26.481 INFO:teuthology.orchestra.run.smithi096.stdout:(106/156): gperftools-libs-2.9.1-3.el9.x86_64.r 4.0 MB/s | 308 kB 00:00 2024-09-19T06:29:26.507 INFO:teuthology.orchestra.run.smithi096.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 258 kB/s | 19 kB 00:00 2024-09-19T06:29:26.532 INFO:teuthology.orchestra.run.smithi096.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-19T06:29:26.566 INFO:teuthology.orchestra.run.smithi096.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2024-09-19T06:29:26.591 INFO:teuthology.orchestra.run.smithi096.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T06:29:26.625 INFO:teuthology.orchestra.run.smithi096.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 4.5 MB/s | 151 kB 00:00 2024-09-19T06:29:26.673 INFO:teuthology.orchestra.run.smithi151.stdout:Extra Packages for Enterprise Linux 19 MB/s | 23 MB 00:01 2024-09-19T06:29:26.691 INFO:teuthology.orchestra.run.smithi096.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-19T06:29:26.734 INFO:teuthology.orchestra.run.smithi096.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T06:29:26.759 INFO:teuthology.orchestra.run.smithi096.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T06:29:26.781 INFO:teuthology.orchestra.run.smithi037.stdout:Transaction test succeeded. 2024-09-19T06:29:26.782 INFO:teuthology.orchestra.run.smithi037.stdout:Running transaction 2024-09-19T06:29:26.784 INFO:teuthology.orchestra.run.smithi096.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T06:29:26.810 INFO:teuthology.orchestra.run.smithi096.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-19T06:29:26.868 INFO:teuthology.orchestra.run.smithi096.stdout:(117/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-19T06:29:26.894 INFO:teuthology.orchestra.run.smithi096.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-09-19T06:29:26.927 INFO:teuthology.orchestra.run.smithi096.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.9 MB/s | 358 kB 00:00 2024-09-19T06:29:26.953 INFO:teuthology.orchestra.run.smithi096.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-19T06:29:26.986 INFO:teuthology.orchestra.run.smithi096.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 3.7 MB/s | 220 kB 00:00 2024-09-19T06:29:27.020 INFO:teuthology.orchestra.run.smithi096.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 4.2 MB/s | 144 kB 00:00 2024-09-19T06:29:27.045 INFO:teuthology.orchestra.run.smithi096.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-19T06:29:27.070 INFO:teuthology.orchestra.run.smithi096.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 710 kB/s | 18 kB 00:00 2024-09-19T06:29:27.096 INFO:teuthology.orchestra.run.smithi096.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 929 kB/s | 23 kB 00:00 2024-09-19T06:29:27.121 INFO:teuthology.orchestra.run.smithi096.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 778 kB/s | 19 kB 00:00 2024-09-19T06:29:27.146 INFO:teuthology.orchestra.run.smithi096.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 789 kB/s | 20 kB 00:00 2024-09-19T06:29:27.205 INFO:teuthology.orchestra.run.smithi096.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.1 MB/s | 2.0 MB 00:00 2024-09-19T06:29:27.230 INFO:teuthology.orchestra.run.smithi096.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T06:29:27.256 INFO:teuthology.orchestra.run.smithi096.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T06:29:27.289 INFO:teuthology.orchestra.run.smithi096.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-19T06:29:27.315 INFO:teuthology.orchestra.run.smithi096.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-09-19T06:29:27.340 INFO:teuthology.orchestra.run.smithi096.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 662 kB/s | 16 kB 00:00 2024-09-19T06:29:27.365 INFO:teuthology.orchestra.run.smithi096.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-19T06:29:27.391 INFO:teuthology.orchestra.run.smithi096.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-09-19T06:29:27.415 INFO:teuthology.orchestra.run.smithi096.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 562 kB/s | 28 kB 00:00 2024-09-19T06:29:27.441 INFO:teuthology.orchestra.run.smithi096.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-19T06:29:27.466 INFO:teuthology.orchestra.run.smithi096.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T06:29:27.491 INFO:teuthology.orchestra.run.smithi096.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T06:29:27.517 INFO:teuthology.orchestra.run.smithi096.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-19T06:29:27.542 INFO:teuthology.orchestra.run.smithi096.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.4 MB/s | 74 kB 00:00 2024-09-19T06:29:27.567 INFO:teuthology.orchestra.run.smithi096.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-19T06:29:27.592 INFO:teuthology.orchestra.run.smithi096.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-19T06:29:27.618 INFO:teuthology.orchestra.run.smithi096.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 799 kB/s | 20 kB 00:00 2024-09-19T06:29:27.652 INFO:teuthology.orchestra.run.smithi096.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-09-19T06:29:27.677 INFO:teuthology.orchestra.run.smithi096.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-19T06:29:27.736 INFO:teuthology.orchestra.run.smithi096.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-19T06:29:28.036 INFO:teuthology.orchestra.run.smithi096.stdout:(148/156): librbd1-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 3.2 MB 00:00 2024-09-19T06:29:28.087 INFO:teuthology.orchestra.run.smithi096.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 8.4 MB/s | 3.4 MB 00:00 2024-09-19T06:29:28.795 INFO:teuthology.orchestra.run.smithi096.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 3.1 MB/s | 2.3 MB 00:00 2024-09-19T06:29:28.845 INFO:teuthology.orchestra.run.smithi096.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 2.4 MB/s | 1.8 MB 00:00 2024-09-19T06:29:29.054 INFO:teuthology.orchestra.run.smithi096.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 158 kB/s | 41 kB 00:00 2024-09-19T06:29:29.396 INFO:teuthology.orchestra.run.smithi096.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 452 kB/s | 249 kB 00:00 2024-09-19T06:29:29.804 INFO:teuthology.orchestra.run.smithi096.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 1.8 MB/s | 1.4 MB 00:00 2024-09-19T06:29:30.230 INFO:teuthology.orchestra.run.smithi096.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.9 MB/s | 2.4 MB 00:00 2024-09-19T06:29:30.646 INFO:teuthology.orchestra.run.smithi096.stdout:(156/156): protobuf-compiler-3.14.0-13.el9.x86_ 189 kB/s | 863 kB 00:04 2024-09-19T06:29:30.651 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:29:30.651 INFO:teuthology.orchestra.run.smithi096.stdout:Total 16 MB/s | 226 MB 00:14 2024-09-19T06:29:31.881 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-09-19T06:29:32.032 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-09-19T06:29:32.032 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-09-19T06:29:32.121 INFO:teuthology.orchestra.run.smithi151.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-09-19T06:29:33.359 INFO:teuthology.orchestra.run.smithi037.stdout: Preparing : 1/1 2024-09-19T06:29:33.469 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T06:29:33.508 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T06:29:33.562 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T06:29:33.603 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T06:29:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T06:29:33.679 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T06:29:33.839 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-09-19T06:29:33.840 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-09-19T06:29:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T06:29:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:29:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:29:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:29:34.306 INFO:teuthology.orchestra.run.smithi151.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:29:34.307 INFO:teuthology.orchestra.run.smithi151.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-19T06:29:34.319 INFO:teuthology.orchestra.run.smithi151.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-19T06:29:34.320 INFO:teuthology.orchestra.run.smithi151.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-19T06:29:34.320 INFO:teuthology.orchestra.run.smithi151.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-19T06:29:34.321 INFO:teuthology.orchestra.run.smithi151.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-19T06:29:34.323 INFO:teuthology.orchestra.run.smithi151.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-19T06:29:34.325 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-19T06:29:34.326 INFO:teuthology.orchestra.run.smithi151.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-19T06:29:34.389 INFO:teuthology.orchestra.run.smithi151.stdout:Dependencies resolved. 2024-09-19T06:29:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: Package Arch Version Repository Size 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout:Installing: 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: ceph x86_64 2:19.1.1-361.g2f599a46.el9 ceph 6.6 k 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.4 M 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 896 k 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-361.g2f599a46.el9 ceph 145 k 2024-09-19T06:29:34.409 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr x86_64 2:19.1.1-361.g2f599a46.el9 ceph 1.5 M 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 146 k 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 3.6 M 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 7.4 M 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 49 k 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 11 M 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test x86_64 2:19.1.1-361.g2f599a46.el9 ceph 49 M 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 285 k 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 765 k 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 228 k 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell x86_64 2:19.1.1-361.g2f599a46.el9 ceph 27 k 2024-09-19T06:29:34.410 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 18 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 32 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 751 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel x86_64 2:19.1.1-361.g2f599a46.el9 ceph 127 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-19T06:29:34.411 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs x86_64 2:19.1.1-361.g2f599a46.el9 ceph 161 k 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados x86_64 2:19.1.1-361.g2f599a46.el9 ceph 320 k 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 300 k 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw x86_64 2:19.1.1-361.g2f599a46.el9 ceph 99 k 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 85 k 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.1 M 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 171 k 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrading: 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-19T06:29:34.412 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: librados2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.4 M 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 3.2 M 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout:Installing dependencies: 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 21 M 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 32 k 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds x86_64 2:19.1.1-361.g2f599a46.el9 ceph 2.4 M 2024-09-19T06:29:34.413 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 249 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon x86_64 2:19.1.1-361.g2f599a46.el9 ceph 4.7 M 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd x86_64 2:19.1.1-361.g2f599a46.el9 ceph 17 M 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts noarch 2:19.1.1-361.g2f599a46.el9 ceph-noarch 16 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux x86_64 2:19.1.1-361.g2f599a46.el9 ceph 25 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-19T06:29:34.414 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite x86_64 2:19.1.1-361.g2f599a46.el9 ceph 163 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 506 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2 x86_64 2:19.1.1-361.g2f599a46.el9 ceph 5.3 M 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-19T06:29:34.415 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-19T06:29:34.416 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse x86_64 2:19.1.1-361.g2f599a46.el9 ceph 45 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common x86_64 2:19.1.1-361.g2f599a46.el9 ceph 136 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-19T06:29:34.417 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-19T06:29:34.418 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-19T06:29:34.418 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-19T06:29:34.418 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-19T06:29:34.419 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-19T06:29:34.420 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-19T06:29:34.421 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-19T06:29:34.422 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout:Installing weak dependencies: 2024-09-19T06:29:34.423 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-19T06:29:34.424 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:29:34.424 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction Summary 2024-09-19T06:29:34.424 INFO:teuthology.orchestra.run.smithi151.stdout:====================================================================================== 2024-09-19T06:29:34.424 INFO:teuthology.orchestra.run.smithi151.stdout:Install 148 Packages 2024-09-19T06:29:34.424 INFO:teuthology.orchestra.run.smithi151.stdout:Upgrade 8 Packages 2024-09-19T06:29:34.424 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:29:34.424 INFO:teuthology.orchestra.run.smithi151.stdout:Total download size: 226 M 2024-09-19T06:29:34.425 INFO:teuthology.orchestra.run.smithi151.stdout:Downloading Packages: 2024-09-19T06:29:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:29:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T06:29:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T06:29:34.700 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:29:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:29:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:29:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:29:34.915 INFO:teuthology.orchestra.run.smithi151.stdout:(1/156): ceph-19.1.1-361.g2f599a46.el9.x86_64.r 46 kB/s | 6.6 kB 00:00 2024-09-19T06:29:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:29:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:29:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:29:35.265 INFO:teuthology.orchestra.run.smithi151.stdout:(2/156): ceph-base-19.1.1-361.g2f599a46.el9.x86 11 MB/s | 5.4 MB 00:00 2024-09-19T06:29:35.299 INFO:teuthology.orchestra.run.smithi151.stdout:(3/156): ceph-fuse-19.1.1-361.g2f599a46.el9.x86 2.3 MB/s | 896 kB 00:00 2024-09-19T06:29:35.332 INFO:teuthology.orchestra.run.smithi151.stdout:(4/156): ceph-immutable-object-cache-19.1.1-361 2.1 MB/s | 145 kB 00:00 2024-09-19T06:29:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:29:35.424 INFO:teuthology.orchestra.run.smithi151.stdout:(5/156): ceph-mgr-19.1.1-361.g2f599a46.el9.x86_ 17 MB/s | 1.5 MB 00:00 2024-09-19T06:29:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T06:29:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:29:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:29:35.750 INFO:teuthology.orchestra.run.smithi151.stdout:(6/156): ceph-mon-19.1.1-361.g2f599a46.el9.x86_ 14 MB/s | 4.7 MB 00:00 2024-09-19T06:29:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:29:35.975 INFO:teuthology.orchestra.run.smithi151.stdout:(7/156): ceph-common-19.1.1-361.g2f599a46.el9.x 18 MB/s | 21 MB 00:01 2024-09-19T06:29:36.005 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T06:29:36.042 INFO:teuthology.orchestra.run.smithi151.stdout:(8/156): ceph-mds-19.1.1-361.g2f599a46.el9.x86_ 3.2 MB/s | 2.4 MB 00:00 2024-09-19T06:29:36.100 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T06:29:36.109 INFO:teuthology.orchestra.run.smithi151.stdout:(9/156): ceph-selinux-19.1.1-361.g2f599a46.el9. 377 kB/s | 25 kB 00:00 2024-09-19T06:29:36.140 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T06:29:36.175 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T06:29:36.252 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T06:29:36.303 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T06:29:36.342 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T06:29:36.359 INFO:teuthology.orchestra.run.smithi151.stdout:(10/156): ceph-radosgw-19.1.1-361.g2f599a46.el9 28 MB/s | 11 MB 00:00 2024-09-19T06:29:36.385 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T06:29:36.410 INFO:teuthology.orchestra.run.smithi151.stdout:(11/156): cephfs-mirror-19.1.1-361.g2f599a46.el 4.5 MB/s | 228 kB 00:00 2024-09-19T06:29:36.418 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T06:29:36.445 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T06:29:36.451 INFO:teuthology.orchestra.run.smithi151.stdout:(12/156): cephfs-shell-19.1.1-361.g2f599a46.el9 652 kB/s | 27 kB 00:00 2024-09-19T06:29:36.502 INFO:teuthology.orchestra.run.smithi151.stdout:(13/156): libcephfs-devel-19.1.1-361.g2f599a46. 637 kB/s | 32 kB 00:00 2024-09-19T06:29:36.518 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T06:29:36.561 INFO:teuthology.orchestra.run.smithi151.stdout:(14/156): libcephfs2-19.1.1-361.g2f599a46.el9.x 13 MB/s | 751 kB 00:00 2024-09-19T06:29:36.569 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T06:29:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T06:29:36.611 INFO:teuthology.orchestra.run.smithi151.stdout:(15/156): libcephsqlite-19.1.1-361.g2f599a46.el 3.2 MB/s | 163 kB 00:00 2024-09-19T06:29:36.642 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T06:29:36.661 INFO:teuthology.orchestra.run.smithi151.stdout:(16/156): librados-devel-19.1.1-361.g2f599a46.e 2.5 MB/s | 127 kB 00:00 2024-09-19T06:29:36.720 INFO:teuthology.orchestra.run.smithi151.stdout:(17/156): libradosstriper1-19.1.1-361.g2f599a46 8.5 MB/s | 506 kB 00:00 2024-09-19T06:29:36.721 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T06:29:36.752 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T06:29:36.812 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T06:29:36.840 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T06:29:36.915 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T06:29:36.945 INFO:teuthology.orchestra.run.smithi151.stdout:(18/156): librgw2-19.1.1-361.g2f599a46.el9.x86_ 24 MB/s | 5.3 MB 00:00 2024-09-19T06:29:36.964 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T06:29:36.996 INFO:teuthology.orchestra.run.smithi151.stdout:(19/156): python3-ceph-argparse-19.1.1-361.g2f5 904 kB/s | 45 kB 00:00 2024-09-19T06:29:37.043 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T06:29:37.046 INFO:teuthology.orchestra.run.smithi151.stdout:(20/156): python3-ceph-common-19.1.1-361.g2f599 2.7 MB/s | 136 kB 00:00 2024-09-19T06:29:37.097 INFO:teuthology.orchestra.run.smithi151.stdout:(21/156): python3-cephfs-19.1.1-361.g2f599a46.e 3.1 MB/s | 161 kB 00:00 2024-09-19T06:29:37.147 INFO:teuthology.orchestra.run.smithi151.stdout:(22/156): python3-rados-19.1.1-361.g2f599a46.el 6.2 MB/s | 320 kB 00:00 2024-09-19T06:29:37.190 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T06:29:37.197 INFO:teuthology.orchestra.run.smithi151.stdout:(23/156): python3-rbd-19.1.1-361.g2f599a46.el9. 5.9 MB/s | 300 kB 00:00 2024-09-19T06:29:37.243 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T06:29:37.247 INFO:teuthology.orchestra.run.smithi151.stdout:(24/156): python3-rgw-19.1.1-361.g2f599a46.el9. 1.9 MB/s | 99 kB 00:00 2024-09-19T06:29:37.298 INFO:teuthology.orchestra.run.smithi151.stdout:(25/156): rbd-fuse-19.1.1-361.g2f599a46.el9.x86 1.7 MB/s | 85 kB 00:00 2024-09-19T06:29:37.465 INFO:teuthology.orchestra.run.smithi151.stdout:(26/156): ceph-osd-19.1.1-361.g2f599a46.el9.x86 9.8 MB/s | 17 MB 00:01 2024-09-19T06:29:37.532 INFO:teuthology.orchestra.run.smithi151.stdout:(27/156): rbd-mirror-19.1.1-361.g2f599a46.el9.x 13 MB/s | 3.1 MB 00:00 2024-09-19T06:29:37.557 INFO:teuthology.orchestra.run.smithi151.stdout:(28/156): rbd-nbd-19.1.1-361.g2f599a46.el9.x86_ 1.8 MB/s | 171 kB 00:00 2024-09-19T06:29:37.574 INFO:teuthology.orchestra.run.smithi151.stdout:(29/156): ceph-grafana-dashboards-19.1.1-361.g2 761 kB/s | 32 kB 00:00 2024-09-19T06:29:37.608 INFO:teuthology.orchestra.run.smithi151.stdout:(30/156): ceph-mgr-cephadm-19.1.1-361.g2f599a46 2.8 MB/s | 146 kB 00:00 2024-09-19T06:29:37.717 INFO:teuthology.orchestra.run.smithi151.stdout:(31/156): ceph-mgr-dashboard-19.1.1-361.g2f599a 25 MB/s | 3.6 MB 00:00 2024-09-19T06:29:37.767 INFO:teuthology.orchestra.run.smithi151.stdout:(32/156): ceph-mgr-modules-core-19.1.1-361.g2f5 4.9 MB/s | 249 kB 00:00 2024-09-19T06:29:37.812 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T06:29:37.817 INFO:teuthology.orchestra.run.smithi151.stdout:(33/156): ceph-mgr-rook-19.1.1-361.g2f599a46.el 988 kB/s | 49 kB 00:00 2024-09-19T06:29:37.868 INFO:teuthology.orchestra.run.smithi151.stdout:(34/156): ceph-prometheus-alerts-19.1.1-361.g2f 328 kB/s | 16 kB 00:00 2024-09-19T06:29:37.907 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T06:29:37.926 INFO:teuthology.orchestra.run.smithi151.stdout:(35/156): ceph-volume-19.1.1-361.g2f599a46.el9. 4.8 MB/s | 285 kB 00:00 2024-09-19T06:29:37.940 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T06:29:37.983 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T06:29:37.994 INFO:teuthology.orchestra.run.smithi151.stdout:(36/156): cephadm-19.1.1-361.g2f599a46.el9.noar 11 MB/s | 765 kB 00:00 2024-09-19T06:29:38.021 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T06:29:38.044 INFO:teuthology.orchestra.run.smithi151.stdout:(37/156): cephfs-top-19.1.1-361.g2f599a46.el9.n 365 kB/s | 18 kB 00:00 2024-09-19T06:29:38.064 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T06:29:38.136 INFO:teuthology.orchestra.run.smithi151.stdout:(38/156): ceph-mgr-diskprediction-local-19.1.1- 14 MB/s | 7.4 MB 00:00 2024-09-19T06:29:38.245 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T06:29:38.316 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T06:29:38.373 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T06:29:38.420 INFO:teuthology.orchestra.run.smithi151.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 213 kB/s | 80 kB 00:00 2024-09-19T06:29:38.447 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T06:29:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T06:29:38.520 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T06:29:38.568 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T06:29:38.602 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T06:29:38.620 INFO:teuthology.orchestra.run.smithi151.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 956 kB/s | 191 kB 00:00 2024-09-19T06:29:38.646 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T06:29:38.688 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T06:29:38.829 INFO:teuthology.orchestra.run.smithi151.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-09-19T06:29:38.833 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T06:29:38.872 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T06:29:38.914 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T06:29:39.023 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T06:29:39.064 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T06:29:39.099 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T06:29:39.129 INFO:teuthology.orchestra.run.smithi151.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 807 kB/s | 801 kB 00:00 2024-09-19T06:29:39.131 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T06:29:39.156 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T06:29:39.230 INFO:teuthology.orchestra.run.smithi151.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-09-19T06:29:39.247 INFO:teuthology.orchestra.run.smithi151.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 909 kB/s | 106 kB 00:00 2024-09-19T06:29:39.335 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T06:29:39.339 INFO:teuthology.orchestra.run.smithi151.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.2 MB/s | 135 kB 00:00 2024-09-19T06:29:39.364 INFO:teuthology.orchestra.run.smithi151.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-19T06:29:39.456 INFO:teuthology.orchestra.run.smithi151.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-09-19T06:29:39.540 INFO:teuthology.orchestra.run.smithi151.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 3.8 MB/s | 689 kB 00:00 2024-09-19T06:29:39.582 INFO:teuthology.orchestra.run.smithi151.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 2.5 MB/s | 107 kB 00:00 2024-09-19T06:29:39.591 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T06:29:39.616 INFO:teuthology.orchestra.run.smithi151.stdout:(50/156): elfutils-libelf-devel-0.191-4.el9.x86 649 kB/s | 22 kB 00:00 2024-09-19T06:29:39.649 INFO:teuthology.orchestra.run.smithi151.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 4.8 MB/s | 943 kB 00:00 2024-09-19T06:29:39.672 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T06:29:39.675 INFO:teuthology.orchestra.run.smithi151.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-19T06:29:39.700 INFO:teuthology.orchestra.run.smithi151.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 648 kB/s | 33 kB 00:00 2024-09-19T06:29:39.701 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T06:29:39.734 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T06:29:39.742 INFO:teuthology.orchestra.run.smithi151.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-19T06:29:39.774 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T06:29:39.784 INFO:teuthology.orchestra.run.smithi151.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 267 kB/s | 11 kB 00:00 2024-09-19T06:29:39.814 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T06:29:39.843 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T06:29:39.851 INFO:teuthology.orchestra.run.smithi151.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.4 MB/s | 160 kB 00:00 2024-09-19T06:29:39.878 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T06:29:39.893 INFO:teuthology.orchestra.run.smithi151.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-19T06:29:39.911 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T06:29:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T06:29:39.987 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T06:29:40.062 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-09-19T06:29:40.079 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T06:29:40.102 INFO:teuthology.orchestra.run.smithi151.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.1 MB/s | 662 kB 00:00 2024-09-19T06:29:40.126 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T06:29:40.165 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T06:29:40.188 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T06:29:40.212 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T06:29:40.266 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T06:29:40.286 INFO:teuthology.orchestra.run.smithi151.stdout:(59/156): libtool-2.4.6-46.el9.x86_64.rpm 3.1 MB/s | 581 kB 00:00 2024-09-19T06:29:40.293 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T06:29:40.299 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T06:29:40.325 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T06:29:40.328 INFO:teuthology.orchestra.run.smithi151.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 589 kB/s | 24 kB 00:00 2024-09-19T06:29:40.351 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T06:29:40.358 INFO:teuthology.orchestra.run.smithi151.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.5 MB/s | 47 kB 00:00 2024-09-19T06:29:40.368 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T06:29:40.459 INFO:teuthology.orchestra.run.smithi151.stdout:(62/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-09-19T06:29:40.534 INFO:teuthology.orchestra.run.smithi151.stdout:(63/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-09-19T06:29:40.568 INFO:teuthology.orchestra.run.smithi151.stdout:(64/156): openblas-0.3.26-2.el9.x86_64.rpm 1.1 MB/s | 38 kB 00:00 2024-09-19T06:29:40.668 INFO:teuthology.orchestra.run.smithi151.stdout:(65/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.0 MB/s | 3.0 MB 00:00 2024-09-19T06:29:40.776 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T06:29:40.877 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:29:40.991 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:29:41.019 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:29:41.094 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:29:41.123 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:29:41.125 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T06:29:41.180 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T06:29:41.213 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T06:29:41.281 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T06:29:41.318 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T06:29:41.345 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T06:29:41.374 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:29:41.377 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T06:29:41.410 INFO:teuthology.orchestra.run.smithi151.stdout:(66/156): ceph-test-19.1.1-361.g2f599a46.el9.x8 9.2 MB/s | 49 MB 00:05 2024-09-19T06:29:41.423 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T06:29:41.477 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:29:41.500 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:29:41.515 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T06:29:41.549 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T06:29:41.564 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:29:41.584 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:29:41.605 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:29:41.744 INFO:teuthology.orchestra.run.smithi151.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 4.8 MB/s | 1.6 MB 00:00 2024-09-19T06:29:41.744 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T06:29:41.771 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T06:29:41.794 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:29:41.852 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T06:29:41.929 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:29:42.011 INFO:teuthology.orchestra.run.smithi151.stdout:(68/156): postgresql-contrib-13.16-1.el9.x86_64 3.3 MB/s | 910 kB 00:00 2024-09-19T06:29:42.066 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T06:29:42.094 INFO:teuthology.orchestra.run.smithi151.stdout:(69/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 3.1 MB/s | 4.4 MB 00:01 2024-09-19T06:29:42.120 INFO:teuthology.orchestra.run.smithi151.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-19T06:29:42.166 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:29:42.220 INFO:teuthology.orchestra.run.smithi151.stdout:(71/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.0 MB/s | 4.9 MB 00:01 2024-09-19T06:29:42.235 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:29:42.279 INFO:teuthology.orchestra.run.smithi151.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-19T06:29:42.468 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:29:42.479 INFO:teuthology.orchestra.run.smithi151.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 2.8 MB/s | 1.0 MB 00:00 2024-09-19T06:29:42.521 INFO:teuthology.orchestra.run.smithi151.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-19T06:29:42.560 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T06:29:42.596 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T06:29:42.613 INFO:teuthology.orchestra.run.smithi151.stdout:(75/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-19T06:29:42.682 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T06:29:42.687 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T06:29:42.697 INFO:teuthology.orchestra.run.smithi151.stdout:(76/156): python3-jinja2-2.11.3-6.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-09-19T06:29:42.718 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T06:29:42.739 INFO:teuthology.orchestra.run.smithi151.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 618 kB/s | 26 kB 00:00 2024-09-19T06:29:42.769 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T06:29:42.781 INFO:teuthology.orchestra.run.smithi151.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 460 kB/s | 19 kB 00:00 2024-09-19T06:29:42.842 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T06:29:42.848 INFO:teuthology.orchestra.run.smithi151.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-19T06:29:42.890 INFO:teuthology.orchestra.run.smithi151.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 832 kB/s | 35 kB 00:00 2024-09-19T06:29:42.895 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T06:29:42.927 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T06:29:42.971 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T06:29:43.004 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T06:29:43.049 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T06:29:43.116 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T06:29:43.171 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T06:29:43.202 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T06:29:43.254 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T06:29:43.329 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T06:29:43.365 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T06:29:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T06:29:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T06:29:43.507 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T06:29:43.551 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T06:29:43.607 INFO:teuthology.orchestra.run.smithi151.stdout:(81/156): postgresql-server-13.16-1.el9.x86_64. 3.8 MB/s | 5.8 MB 00:01 2024-09-19T06:29:43.626 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T06:29:43.708 INFO:teuthology.orchestra.run.smithi151.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.5 MB/s | 250 kB 00:00 2024-09-19T06:29:43.759 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T06:29:43.799 INFO:teuthology.orchestra.run.smithi151.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-19T06:29:43.821 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T06:29:43.850 INFO:teuthology.orchestra.run.smithi151.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-19T06:29:43.883 INFO:teuthology.orchestra.run.smithi151.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 1.2 MB/s | 42 kB 00:00 2024-09-19T06:29:43.984 INFO:teuthology.orchestra.run.smithi151.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-09-19T06:29:44.033 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T06:29:44.059 INFO:teuthology.orchestra.run.smithi151.stdout:(87/156): python3-babel-2.9.1-2.el9.noarch.rpm 3.4 MB/s | 6.0 MB 00:01 2024-09-19T06:29:44.101 INFO:teuthology.orchestra.run.smithi151.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-19T06:29:44.118 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T06:29:44.135 INFO:teuthology.orchestra.run.smithi151.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-19T06:29:44.146 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T06:29:44.177 INFO:teuthology.orchestra.run.smithi151.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-19T06:29:44.178 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T06:29:44.202 INFO:teuthology.orchestra.run.smithi151.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 802 kB/s | 54 kB 00:00 2024-09-19T06:29:44.244 INFO:teuthology.orchestra.run.smithi151.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2024-09-19T06:29:44.286 INFO:teuthology.orchestra.run.smithi151.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-19T06:29:44.347 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T06:29:44.387 INFO:teuthology.orchestra.run.smithi151.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.2 MB/s | 221 kB 00:00 2024-09-19T06:29:44.424 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T06:29:44.454 INFO:teuthology.orchestra.run.smithi151.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.3 MB/s | 5.1 MB 00:01 2024-09-19T06:29:44.454 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T06:29:44.479 INFO:teuthology.orchestra.run.smithi151.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 390 kB/s | 36 kB 00:00 2024-09-19T06:29:44.497 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T06:29:44.504 INFO:teuthology.orchestra.run.smithi151.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-09-19T06:29:44.506 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T06:29:44.541 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T06:29:44.546 INFO:teuthology.orchestra.run.smithi151.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-19T06:29:44.550 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:29:44.583 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T06:29:44.605 INFO:teuthology.orchestra.run.smithi151.stdout:(99/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rpm 1.6 MB/s | 96 kB 00:00 2024-09-19T06:29:44.630 INFO:teuthology.orchestra.run.smithi151.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 2.0 MB/s | 304 kB 00:00 2024-09-19T06:29:44.650 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:29:44.672 INFO:teuthology.orchestra.run.smithi151.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 949 kB/s | 64 kB 00:00 2024-09-19T06:29:44.680 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T06:29:44.698 INFO:teuthology.orchestra.run.smithi151.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 669 kB/s | 45 kB 00:00 2024-09-19T06:29:44.725 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T06:29:44.748 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T06:29:44.790 INFO:teuthology.orchestra.run.smithi151.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 190 kB/s | 22 kB 00:00 2024-09-19T06:29:44.819 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T06:29:44.849 INFO:teuthology.orchestra.run.smithi151.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-19T06:29:44.874 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T06:29:44.882 INFO:teuthology.orchestra.run.smithi151.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-19T06:29:44.908 INFO:teuthology.orchestra.run.smithi151.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 779 kB/s | 19 kB 00:00 2024-09-19T06:29:44.947 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T06:29:44.966 INFO:teuthology.orchestra.run.smithi151.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.1 MB/s | 863 kB 00:00 2024-09-19T06:29:44.987 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T06:29:45.017 INFO:teuthology.orchestra.run.smithi151.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 519 kB/s | 26 kB 00:00 2024-09-19T06:29:45.023 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T06:29:45.042 INFO:teuthology.orchestra.run.smithi151.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-19T06:29:45.068 INFO:teuthology.orchestra.run.smithi151.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-19T06:29:45.069 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T06:29:45.093 INFO:teuthology.orchestra.run.smithi151.stdout:(111/156): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-19T06:29:45.106 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T06:29:45.148 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T06:29:45.160 INFO:teuthology.orchestra.run.smithi151.stdout:(112/156): libarrow-9.0.0-11.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-19T06:29:45.191 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T06:29:45.202 INFO:teuthology.orchestra.run.smithi151.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-19T06:29:45.227 INFO:teuthology.orchestra.run.smithi151.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-19T06:29:45.235 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T06:29:45.253 INFO:teuthology.orchestra.run.smithi151.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-19T06:29:45.261 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:29:45.278 INFO:teuthology.orchestra.run.smithi151.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-19T06:29:45.312 INFO:teuthology.orchestra.run.smithi151.stdout:(117/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.7 MB/s | 839 kB 00:00 2024-09-19T06:29:45.326 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T06:29:45.337 INFO:teuthology.orchestra.run.smithi151.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.8 MB/s | 172 kB 00:00 2024-09-19T06:29:45.342 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:29:45.368 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T06:29:45.371 INFO:teuthology.orchestra.run.smithi151.stdout:(119/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-19T06:29:45.371 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T06:29:45.396 INFO:teuthology.orchestra.run.smithi151.stdout:(120/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-19T06:29:45.409 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T06:29:45.422 INFO:teuthology.orchestra.run.smithi151.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-19T06:29:45.447 INFO:teuthology.orchestra.run.smithi151.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-19T06:29:45.472 INFO:teuthology.orchestra.run.smithi151.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 428 kB/s | 11 kB 00:00 2024-09-19T06:29:45.498 INFO:teuthology.orchestra.run.smithi151.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 703 kB/s | 18 kB 00:00 2024-09-19T06:29:45.511 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T06:29:45.523 INFO:teuthology.orchestra.run.smithi151.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 922 kB/s | 23 kB 00:00 2024-09-19T06:29:45.549 INFO:teuthology.orchestra.run.smithi151.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 780 kB/s | 19 kB 00:00 2024-09-19T06:29:45.552 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T06:29:45.574 INFO:teuthology.orchestra.run.smithi151.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 787 kB/s | 20 kB 00:00 2024-09-19T06:29:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T06:29:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T06:29:45.641 INFO:teuthology.orchestra.run.smithi151.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 8.3 MB/s | 2.0 MB 00:00 2024-09-19T06:29:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T06:29:45.667 INFO:teuthology.orchestra.run.smithi151.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-19T06:29:45.692 INFO:teuthology.orchestra.run.smithi151.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-19T06:29:45.718 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T06:29:45.734 INFO:teuthology.orchestra.run.smithi151.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-19T06:29:45.759 INFO:teuthology.orchestra.run.smithi151.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 861 kB/s | 58 kB 00:00 2024-09-19T06:29:45.785 INFO:teuthology.orchestra.run.smithi151.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 5.3 MB/s | 272 kB 00:00 2024-09-19T06:29:45.810 INFO:teuthology.orchestra.run.smithi151.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 325 kB/s | 16 kB 00:00 2024-09-19T06:29:45.835 INFO:teuthology.orchestra.run.smithi151.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-19T06:29:45.861 INFO:teuthology.orchestra.run.smithi151.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-09-19T06:29:45.886 INFO:teuthology.orchestra.run.smithi151.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-09-19T06:29:45.911 INFO:teuthology.orchestra.run.smithi151.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-19T06:29:45.937 INFO:teuthology.orchestra.run.smithi151.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-19T06:29:45.979 INFO:teuthology.orchestra.run.smithi151.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 533 kB/s | 36 kB 00:00 2024-09-19T06:29:45.996 INFO:teuthology.orchestra.run.smithi151.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.2 MB/s | 74 kB 00:00 2024-09-19T06:29:46.004 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T06:29:46.021 INFO:teuthology.orchestra.run.smithi151.stdout:(142/156): python3-webob-1.8.8-2.el9.noarch.rpm 5.4 MB/s | 230 kB 00:00 2024-09-19T06:29:46.044 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T06:29:46.046 INFO:teuthology.orchestra.run.smithi151.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-19T06:29:46.064 INFO:teuthology.orchestra.run.smithi151.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-19T06:29:46.078 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T06:29:46.097 INFO:teuthology.orchestra.run.smithi151.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.5 MB/s | 427 kB 00:00 2024-09-19T06:29:46.112 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T06:29:46.131 INFO:teuthology.orchestra.run.smithi151.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-19T06:29:46.151 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T06:29:46.192 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T06:29:46.217 INFO:teuthology.orchestra.run.smithi151.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-19T06:29:46.230 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T06:29:46.265 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T06:29:46.298 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T06:29:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T06:29:46.374 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T06:29:46.460 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T06:29:46.505 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T06:29:46.536 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T06:29:46.638 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T06:29:46.695 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T06:29:46.782 INFO:teuthology.orchestra.run.smithi151.stdout:(148/156): librbd1-19.1.1-361.g2f599a46.el9.x86 5.6 MB/s | 3.2 MB 00:00 2024-09-19T06:29:46.891 INFO:teuthology.orchestra.run.smithi151.stdout:(149/156): librados2-19.1.1-361.g2f599a46.el9.x 4.5 MB/s | 3.4 MB 00:00 2024-09-19T06:29:47.410 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T06:29:47.451 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T06:29:47.485 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T06:29:47.585 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T06:29:47.640 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T06:29:47.693 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T06:29:47.780 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T06:29:47.813 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T06:29:47.841 INFO:teuthology.orchestra.run.smithi151.stdout:(150/156): NetworkManager-1.51.0-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-09-19T06:29:47.868 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:29:47.883 INFO:teuthology.orchestra.run.smithi151.stdout:(151/156): NetworkManager-libnm-1.51.0-1.el9.x8 1.8 MB/s | 1.8 MB 00:00 2024-09-19T06:29:47.909 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:29:47.950 INFO:teuthology.orchestra.run.smithi151.stdout:(152/156): NetworkManager-team-1.51.0-1.el9.x86 376 kB/s | 41 kB 00:00 2024-09-19T06:29:47.995 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T06:29:48.001 INFO:teuthology.orchestra.run.smithi151.stdout:(153/156): NetworkManager-tui-1.51.0-1.el9.x86_ 2.1 MB/s | 249 kB 00:00 2024-09-19T06:29:48.027 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T06:29:48.104 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T06:29:48.176 INFO:teuthology.orchestra.run.smithi151.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 6.0 MB/s | 1.4 MB 00:00 2024-09-19T06:29:48.243 INFO:teuthology.orchestra.run.smithi151.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.9 MB/s | 2.4 MB 00:00 2024-09-19T06:29:48.490 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:29:48.529 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:29:48.621 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:29:48.621 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:29:48.622 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:29:48.622 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:29:48.655 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:29:48.743 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T06:29:48.850 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T06:29:49.543 INFO:teuthology.orchestra.run.smithi151.stdout:(156/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 3.0 MB/s | 16 MB 00:05 2024-09-19T06:29:49.549 INFO:teuthology.orchestra.run.smithi151.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:29:49.549 INFO:teuthology.orchestra.run.smithi151.stdout:Total 15 MB/s | 226 MB 00:15 2024-09-19T06:29:50.056 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T06:29:50.148 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T06:29:50.177 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T06:29:50.210 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T06:29:50.520 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T06:29:50.593 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:29:50.689 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:29:50.728 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T06:29:50.772 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T06:29:50.784 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction check 2024-09-19T06:29:50.936 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction check succeeded. 2024-09-19T06:29:50.936 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction test 2024-09-19T06:29:51.251 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T06:29:51.284 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:29:51.363 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:29:51.394 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T06:29:52.732 INFO:teuthology.orchestra.run.smithi151.stdout:Transaction test succeeded. 2024-09-19T06:29:52.732 INFO:teuthology.orchestra.run.smithi151.stdout:Running transaction 2024-09-19T06:29:53.703 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:29:53.741 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:29:54.284 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:29:54.344 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:29:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:29:54.438 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:29:54.438 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:29:54.438 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:29:54.471 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:29:59.116 INFO:teuthology.orchestra.run.smithi151.stdout: Preparing : 1/1 2024-09-19T06:29:59.231 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-19T06:29:59.269 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-19T06:29:59.323 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-19T06:29:59.358 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-19T06:29:59.383 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-19T06:29:59.426 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-19T06:29:59.749 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-19T06:29:59.860 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:29:59.981 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:30:00.011 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:30:00.103 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:30:00.132 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:30:00.280 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-19T06:30:00.338 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-19T06:30:00.366 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:30:00.461 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:30:00.484 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:30:00.538 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 14/164 2024-09-19T06:30:00.560 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:30:00.593 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-19T06:30:00.735 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:30:00.735 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /sys 2024-09-19T06:30:00.735 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /proc 2024-09-19T06:30:00.735 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /mnt 2024-09-19T06:30:00.735 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /var/tmp 2024-09-19T06:30:00.735 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /home 2024-09-19T06:30:00.735 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /root 2024-09-19T06:30:00.736 INFO:teuthology.orchestra.run.smithi037.stdout:skipping the directory /tmp 2024-09-19T06:30:00.736 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:00.788 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:30:00.915 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:30:00.915 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-19T06:30:00.932 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:30:00.932 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:00.932 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:30:00.932 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:01.035 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 17/164 2024-09-19T06:30:01.144 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:30:01.237 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:30:01.464 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 18/164 2024-09-19T06:30:01.567 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-19T06:30:01.663 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-19T06:30:01.709 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-19T06:30:01.744 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-19T06:30:01.810 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-19T06:30:01.862 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-19T06:30:01.895 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-19T06:30:01.939 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-19T06:30:01.971 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-19T06:30:02.000 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-19T06:30:02.068 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-19T06:30:02.138 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-19T06:30:02.170 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-19T06:30:02.213 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-19T06:30:02.289 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-19T06:30:02.325 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 34/164 2024-09-19T06:30:02.359 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 35/164 2024-09-19T06:30:02.389 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 36/164 2024-09-19T06:30:02.465 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-19T06:30:02.468 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:30:02.496 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:30:02.497 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:02.497 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:30:02.497 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:30:02.497 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:30:02.497 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:02.511 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-19T06:30:02.584 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-19T06:30:02.718 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-19T06:30:02.736 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:30:02.761 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:30:02.762 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:02.762 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:30:02.762 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:30:02.762 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:30:02.762 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:02.788 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-19T06:30:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:30:03.246 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:30:03.246 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:03.246 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:30:03.247 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:30:03.247 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:30:03.247 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:03.317 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-19T06:30:03.351 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T06:30:03.393 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-19T06:30:03.409 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T06:30:03.422 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-19T06:30:03.466 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-19T06:30:03.473 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T06:30:03.503 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 46/164 2024-09-19T06:30:03.545 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-19T06:30:03.671 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T06:30:03.704 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-19T06:30:03.722 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T06:30:03.766 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-19T06:30:03.803 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T06:30:03.822 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-19T06:30:03.894 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-19T06:30:03.901 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T06:30:03.926 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-19T06:30:03.962 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-19T06:30:04.009 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-19T06:30:04.045 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-19T06:30:04.053 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T06:30:04.088 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-19T06:30:04.106 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T06:30:04.130 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-19T06:30:04.145 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T06:30:04.172 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T06:30:04.267 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-19T06:30:04.322 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-19T06:30:04.364 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-19T06:30:04.466 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-19T06:30:04.498 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-19T06:30:04.538 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-19T06:30:04.564 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-19T06:30:04.588 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-19T06:30:04.612 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:30:04.629 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:30:04.661 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T06:30:04.662 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-19T06:30:04.726 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T06:30:04.752 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:30:04.790 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-19T06:30:05.099 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-19T06:30:05.160 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-19T06:30:05.194 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-19T06:30:05.235 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-19T06:30:05.274 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-19T06:30:05.303 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-19T06:30:05.338 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-19T06:30:05.363 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-19T06:30:05.406 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-19T06:30:05.440 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-19T06:30:05.531 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-19T06:30:05.579 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-19T06:30:05.610 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-19T06:30:05.713 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-19T06:30:05.769 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-19T06:30:05.872 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:30:05.929 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:30:06.110 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-09-19T06:30:06.111 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:06.285 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:30:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:30:06.301 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:06.301 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:30:06.301 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:06.490 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-19T06:30:06.543 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-19T06:30:06.577 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-19T06:30:06.676 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-19T06:30:06.732 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-19T06:30:06.777 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-19T06:30:06.876 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-19T06:30:06.947 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-19T06:30:06.952 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:30:06.979 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:30:07.090 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:30:07.124 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-19T06:30:07.153 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-19T06:30:07.231 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-19T06:30:07.257 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T06:30:07.281 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:30:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:30:07.307 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:07.307 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:30:07.307 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:30:07.307 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:30:07.307 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:07.379 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:30:07.391 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:30:07.496 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T06:30:07.552 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T06:30:07.788 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:30:07.816 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:30:07.816 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:07.816 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:30:07.816 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:30:07.816 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:30:07.816 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:07.878 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-19T06:30:07.994 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-19T06:30:08.038 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:30:08.062 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:30:08.062 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:08.063 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:30:08.063 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:30:08.063 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:30:08.063 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:08.506 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:30:08.530 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:30:08.530 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:08.530 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:30:08.530 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:30:08.530 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:30:08.530 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:08.536 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T06:30:08.636 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T06:30:08.684 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:30:08.701 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T06:30:08.709 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:30:08.710 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:08.710 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:30:08.710 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:30:08.710 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:30:08.710 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:08.760 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:30:08.765 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T06:30:08.785 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:30:08.785 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:08.785 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:30:08.785 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:08.835 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:30:08.861 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:30:08.861 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:08.861 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T06:30:08.861 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:30:08.861 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:30:08.861 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:08.951 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T06:30:09.007 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T06:30:09.094 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T06:30:09.178 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:30:09.191 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T06:30:09.203 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:30:09.203 INFO:teuthology.orchestra.run.smithi037.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:09.203 INFO:teuthology.orchestra.run.smithi037.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:30:09.203 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:30:09.203 INFO:teuthology.orchestra.run.smithi037.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:30:09.203 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:09.220 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-19T06:30:09.296 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-19T06:30:09.324 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-19T06:30:09.332 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T06:30:09.357 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-19T06:30:09.374 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T06:30:09.406 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T06:30:09.449 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T06:30:09.667 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-19T06:30:09.712 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:30:09.803 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 101/164 2024-09-19T06:30:09.834 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 102/164 2024-09-19T06:30:09.845 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:30:09.862 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-19T06:30:09.874 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:30:09.905 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T06:30:09.968 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T06:30:09.997 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:30:10.343 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-19T06:30:10.383 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:30:10.446 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 105/164 2024-09-19T06:30:10.476 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 106/164 2024-09-19T06:30:11.046 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:30:11.082 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:30:12.042 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:30:12.066 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:30:12.169 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:30:12.329 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T06:30:12.368 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:30:12.393 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:30:12.393 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:12.393 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:30:12.393 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:30:12.393 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:30:12.394 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:12.444 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:30:12.456 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:30:12.553 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T06:30:12.609 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T06:30:12.746 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:30:12.782 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:30:13.324 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 107/164 2024-09-19T06:30:13.362 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:30:13.445 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 108/164 2024-09-19T06:30:13.445 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-19T06:30:13.445 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-19T06:30:13.445 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:13.480 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:30:13.532 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T06:30:13.564 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:30:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:30:13.588 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:13.588 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:30:13.588 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:30:13.588 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:30:13.588 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:13.631 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:30:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:30:13.654 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:13.654 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:30:13.654 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:13.698 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:30:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:30:13.722 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:13.722 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T06:30:13.722 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:30:13.722 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:30:13.722 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:14.041 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:30:14.057 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T06:30:14.064 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:30:14.064 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:14.065 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:30:14.065 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:30:14.065 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:30:14.065 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:14.099 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T06:30:14.139 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T06:30:14.179 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T06:30:14.206 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T06:30:14.413 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T06:30:14.461 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T06:30:14.491 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T06:30:14.521 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T06:30:14.559 INFO:teuthology.orchestra.run.smithi037.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T06:30:15.211 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:15.256 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:15.278 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:16.210 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T06:30:16.310 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T06:30:16.360 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T06:30:16.461 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T06:30:16.497 INFO:teuthology.orchestra.run.smithi037.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T06:30:16.522 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T06:30:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T06:30:16.557 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:30:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:30:16.649 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:30:16.733 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:30:16.758 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T06:30:16.763 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:16.764 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:16.960 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:30:17.044 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:30:17.113 INFO:teuthology.orchestra.run.smithi037.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:18.595 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T06:30:18.637 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T06:30:18.677 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T06:30:18.709 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T06:30:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T06:30:18.921 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T06:30:18.975 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T06:30:19.005 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T06:30:19.034 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T06:30:19.089 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T06:30:19.686 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:19.729 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:19.750 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:20.584 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T06:30:20.676 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T06:30:20.717 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T06:30:20.799 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T06:30:20.838 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T06:30:20.863 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T06:30:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T06:30:20.889 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:30:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:30:20.965 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:30:21.033 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:30:21.057 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T06:30:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:21.063 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:21.251 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:30:21.344 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:30:21.407 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:22.880 INFO:teuthology.orchestra.run.smithi037.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:22.880 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T06:30:22.880 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T06:30:22.880 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T06:30:22.880 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:30:22.881 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T06:30:22.882 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T06:30:22.883 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T06:30:22.883 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T06:30:22.883 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T06:30:22.883 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T06:30:22.883 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T06:30:22.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T06:30:22.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T06:30:22.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T06:30:22.895 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T06:30:22.896 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T06:30:22.897 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T06:30:22.898 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T06:30:22.899 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T06:30:22.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T06:30:22.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T06:30:22.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T06:30:22.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T06:30:22.900 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T06:30:22.901 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T06:30:22.901 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T06:30:22.901 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T06:30:22.901 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T06:30:22.901 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T06:30:22.901 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T06:30:22.902 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T06:30:22.903 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T06:30:22.904 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T06:30:22.905 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T06:30:22.906 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T06:30:22.907 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T06:30:22.908 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T06:30:22.909 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T06:30:22.909 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T06:30:22.909 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T06:30:22.909 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T06:30:22.909 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T06:30:25.413 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 109/164 2024-09-19T06:30:25.413 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /sys 2024-09-19T06:30:25.413 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /proc 2024-09-19T06:30:25.413 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /mnt 2024-09-19T06:30:25.413 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /var/tmp 2024-09-19T06:30:25.414 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /home 2024-09-19T06:30:25.414 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /root 2024-09-19T06:30:25.414 INFO:teuthology.orchestra.run.smithi151.stdout:skipping the directory /tmp 2024-09-19T06:30:25.414 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout:Upgraded: 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.492 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout:Installed: 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.493 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.494 INFO:teuthology.orchestra.run.smithi037.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:30:25.495 INFO:teuthology.orchestra.run.smithi037.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:30:25.496 INFO:teuthology.orchestra.run.smithi037.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:30:25.497 INFO:teuthology.orchestra.run.smithi037.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.498 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:30:25.499 INFO:teuthology.orchestra.run.smithi037.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:30:25.500 INFO:teuthology.orchestra.run.smithi037.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:30:25.501 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:30:25.502 INFO:teuthology.orchestra.run.smithi037.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:30:25.503 INFO:teuthology.orchestra.run.smithi037.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:30:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:30:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T06:30:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T06:30:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:30:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T06:30:25.504 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:30:25.504 INFO:teuthology.orchestra.run.smithi037.stdout:Complete! 2024-09-19T06:30:25.594 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:30:25.612 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 110/164 2024-09-19T06:30:25.612 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:25.612 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-19T06:30:25.612 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:26.015 DEBUG:teuthology.parallel:result is None 2024-09-19T06:30:27.075 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T06:30:27.085 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T06:30:27.086 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T06:30:27.087 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T06:30:27.088 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T06:30:27.089 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T06:30:27.090 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T06:30:27.091 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T06:30:27.092 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T06:30:27.093 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T06:30:27.094 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T06:30:27.095 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T06:30:27.096 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T06:30:27.097 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T06:30:27.098 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T06:30:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 111/164 2024-09-19T06:30:27.101 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:27.101 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-19T06:30:27.101 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:30:27.101 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-19T06:30:27.101 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:27.333 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:30:27.356 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 112/164 2024-09-19T06:30:27.356 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:27.357 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-19T06:30:27.357 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:30:27.357 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-19T06:30:27.357 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:27.800 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:30:27.824 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 113/164 2024-09-19T06:30:27.824 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:27.824 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-19T06:30:27.824 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:30:27.824 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-19T06:30:27.824 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:27.930 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-19T06:30:27.979 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-19T06:30:28.034 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-19T06:30:28.219 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-19T06:30:28.269 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-19T06:30:28.338 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-19T06:30:28.426 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-19T06:30:28.557 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-19T06:30:28.603 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-19T06:30:28.635 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-19T06:30:28.664 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-19T06:30:29.093 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:30:29.121 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 125/164 2024-09-19T06:30:29.153 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 126/164 2024-09-19T06:30:29.215 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 127/164 2024-09-19T06:30:29.245 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:29.970 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.971 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.972 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:30:29.973 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:30:29.974 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:30:29.975 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:30:29.976 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:30:29.977 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:30:29.978 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:30:29.979 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.980 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T06:30:29.981 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:29.982 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:30:29.983 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:30:29.983 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:30:29.983 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T06:30:29.983 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T06:30:29.983 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:30:29.983 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T06:30:29.983 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:30:29.984 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-09-19T06:30:30.289 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 128/164 2024-09-19T06:30:30.331 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:30:30.422 DEBUG:teuthology.parallel:result is None 2024-09-19T06:30:31.290 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 129/164 2024-09-19T06:30:31.323 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:30:31.424 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 130/164 2024-09-19T06:30:31.585 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 131/164 2024-09-19T06:30:31.609 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:30:31.633 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 132/164 2024-09-19T06:30:31.633 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:31.633 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-19T06:30:31.633 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:30:31.633 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-19T06:30:31.634 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:31.685 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:30:31.697 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 133/164 2024-09-19T06:30:31.802 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 134/164 2024-09-19T06:30:31.858 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 135/164 2024-09-19T06:30:32.779 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-19T06:30:32.822 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:30:32.846 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 137/164 2024-09-19T06:30:32.846 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:32.846 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-19T06:30:32.846 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:30:32.846 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-19T06:30:32.846 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:32.889 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:30:32.912 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-361.g2f599a 138/164 2024-09-19T06:30:32.912 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:32.913 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-19T06:30:32.913 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:32.957 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:30:32.981 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 139/164 2024-09-19T06:30:32.981 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:32.981 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-19T06:30:32.981 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:30:32.981 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-19T06:30:32.981 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:33.283 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:30:33.306 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 140/164 2024-09-19T06:30:33.306 INFO:teuthology.orchestra.run.smithi151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-19T06:30:33.306 INFO:teuthology.orchestra.run.smithi151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-19T06:30:33.307 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:30:33.307 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-19T06:30:33.307 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:37.863 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 141/164 2024-09-19T06:30:37.904 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 142/164 2024-09-19T06:30:37.945 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 143/164 2024-09-19T06:30:37.977 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-19T06:30:38.005 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-19T06:30:38.212 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 146/164 2024-09-19T06:30:38.261 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-19T06:30:38.292 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 148/164 2024-09-19T06:30:38.322 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 149/164 2024-09-19T06:30:38.377 INFO:teuthology.orchestra.run.smithi151.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 150/164 2024-09-19T06:30:38.999 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:39.051 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:39.072 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-19T06:30:39.918 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 152/164 2024-09-19T06:30:40.018 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-19T06:30:40.059 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-19T06:30:40.140 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-19T06:30:40.172 INFO:teuthology.orchestra.run.smithi151.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-19T06:30:40.197 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-19T06:30:40.232 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-19T06:30:40.232 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:30:40.325 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-19T06:30:40.325 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:30:40.409 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-19T06:30:40.434 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-19T06:30:40.440 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:40.440 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:40.619 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-19T06:30:40.620 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:30:40.712 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-19T06:30:40.778 INFO:teuthology.orchestra.run.smithi151.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:46.684 INFO:teuthology.orchestra.run.smithi151.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:46.684 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-2:19.1.1-361.g2f599a46.el9.x86_64 1/164 2024-09-19T06:30:46.684 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2/164 2024-09-19T06:30:46.684 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 3/164 2024-09-19T06:30:46.684 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 4/164 2024-09-19T06:30:46.684 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-361.g2f599a 5/164 2024-09-19T06:30:46.684 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 6/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 7/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 8/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 9/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 10/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 11/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 12/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 13/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 14/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_6 15/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 16/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 17/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 18/164 2024-09-19T06:30:46.685 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_ 19/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 20/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9 21/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x 22/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 23/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 24/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 25/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 26/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 27/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 28/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 29/164 2024-09-19T06:30:46.686 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.e 30/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noar 31/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.no 32/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-361.g2f59 33/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9 34/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 35/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el 36/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 37/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephadm-2:19.1.1-361.g2f599a46.el9.noarch 38/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 39/164 2024-09-19T06:30:46.699 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-19T06:30:46.700 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-19T06:30:46.701 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-19T06:30:46.702 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-19T06:30:46.703 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-19T06:30:46.704 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-19T06:30:46.705 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-19T06:30:46.706 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-19T06:30:46.707 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:19.1.1-361.g2f599a46.el9.x86_64 149/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 151/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 153/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-19T06:30:46.708 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 155/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 157/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 159/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-19T06:30:46.709 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-19T06:30:49.272 INFO:teuthology.orchestra.run.smithi151.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-19T06:30:49.272 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:49.272 INFO:teuthology.orchestra.run.smithi151.stdout:Upgraded: 2024-09-19T06:30:49.272 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: librados2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: librbd1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout:Installed: 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: automake-1.16.2-8.el9.noarch 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: bison-3.7.4-5.el9.x86_64 2024-09-19T06:30:49.273 INFO:teuthology.orchestra.run.smithi151.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-grafana-dashboards-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-immutable-object-cache-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mds-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.274 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-diskprediction-local-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-rook-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mon-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-osd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-prometheus-alerts-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-radosgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-selinux-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-test-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-volume-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.275 INFO:teuthology.orchestra.run.smithi151.stdout: cephadm-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-shell-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: cephfs-top-2:19.1.1-361.g2f599a46.el9.noarch 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: flex-2.6.4-9.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-19T06:30:49.276 INFO:teuthology.orchestra.run.smithi151.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libcephsqlite-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: librados-devel-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.277 INFO:teuthology.orchestra.run.smithi151.stdout: libradosstriper1-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: librgw2-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: lua-5.4.4-4.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-19T06:30:49.278 INFO:teuthology.orchestra.run.smithi151.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-13.16-1.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-19T06:30:49.279 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cephfs-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-19T06:30:49.280 INFO:teuthology.orchestra.run.smithi151.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-19T06:30:49.281 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ply-3.11-14.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-19T06:30:49.282 INFO:teuthology.orchestra.run.smithi151.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rados-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rgw-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.283 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-19T06:30:49.284 INFO:teuthology.orchestra.run.smithi151.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-fuse-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-mirror-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: rbd-nbd-2:19.1.1-361.g2f599a46.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: re2-1:20211101-20.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-19T06:30:49.285 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-19T06:30:49.286 INFO:teuthology.orchestra.run.smithi151.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-19T06:30:49.286 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:30:49.286 INFO:teuthology.orchestra.run.smithi151.stdout:Complete! 2024-09-19T06:30:49.701 DEBUG:teuthology.parallel:result is None 2024-09-19T06:30:49.702 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:30:49.857 DEBUG:teuthology.orchestra.run.smithi037:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:30:49.885 INFO:teuthology.orchestra.run.smithi037.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:30:49.886 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:30:49.886 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:30:49.888 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:30:50.030 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:30:50.057 INFO:teuthology.orchestra.run.smithi096.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:30:50.057 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:30:50.057 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:30:50.059 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:30:50.204 DEBUG:teuthology.orchestra.run.smithi151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-19T06:30:50.229 INFO:teuthology.orchestra.run.smithi151.stdout:19.1.1-361.g2f599a46.el9 2024-09-19T06:30:50.229 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-361.g2f599a46.el9 2024-09-19T06:30:50.229 INFO:teuthology.task.install:The correct ceph version 19.1.1-361.g2f599a46 is installed. 2024-09-19T06:30:50.231 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-19T06:30:50.231 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:30:50.231 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:30:50.265 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:30:50.265 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:30:50.297 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:30:50.297 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-19T06:30:50.327 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-19T06:30:50.327 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:30:50.328 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:30:50.358 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:30:50.428 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:30:50.428 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:30:50.458 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:30:50.531 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:30:50.532 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-09-19T06:30:50.563 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-19T06:30:50.634 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-19T06:30:50.635 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:30:50.635 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:30:50.663 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:30:50.730 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:30:50.730 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:30:50.758 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:30:50.823 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:30:50.823 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-19T06:30:50.850 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-19T06:30:50.916 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-19T06:30:50.917 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:30:50.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:30:50.942 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:30:51.010 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:30:51.010 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:30:51.035 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:30:51.100 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:30:51.100 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-09-19T06:30:51.126 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-19T06:30:51.193 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-19T06:30:51.289 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'subvols': {'create': 2}} 2024-09-19T06:30:51.289 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:30:51.290 INFO:tasks.cephadm:Cluster fsid is b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:30:51.290 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-19T06:30:51.290 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.37', 'mon.b': '172.21.15.96', 'mon.c': '172.21.15.151'} 2024-09-19T06:30:51.290 INFO:tasks.cephadm:First mon is mon.a on smithi037 2024-09-19T06:30:51.290 INFO:tasks.cephadm:First mgr is x 2024-09-19T06:30:51.290 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-19T06:30:51.290 DEBUG:teuthology.orchestra.run.smithi037:> sudo hostname $(hostname -s) 2024-09-19T06:30:51.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-09-19T06:30:51.344 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-09-19T06:30:51.372 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-19T06:30:51.372 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:30:51.517 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2f599a46ad10e37c929e5c43da2f6907ddcd1961', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56106/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-361-g2f599a46', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-361.g2f599a46'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-18 17:52:24.287651', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-19T06:30:51.667 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 2f599a46ad10e37c929e5c43da2f6907ddcd1961 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:30:51.669 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T06:30:51.669 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm 2024-09-19T06:30:51.669 DEBUG:teuthology.orchestra.run.smithi037:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.045 INFO:teuthology.orchestra.run.smithi037.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:30 /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.045 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.439 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:30 /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.439 DEBUG:teuthology.orchestra.run.smithi151:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/2f599a46ad10e37c929e5c43da2f6907ddcd1961/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.820 INFO:teuthology.orchestra.run.smithi151.stdout:-rw-r--r--. 1 ubuntu ubuntu 784313 Sep 19 06:30 /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.821 DEBUG:teuthology.orchestra.run.smithi037:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.841 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.859 DEBUG:teuthology.orchestra.run.smithi151:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-19T06:30:52.894 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 on all hosts... 2024-09-19T06:30:52.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:30:52.896 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:30:52.901 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 pull 2024-09-19T06:30:54.422 INFO:teuthology.orchestra.run.smithi037.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:30:54.455 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:30:54.465 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:31:27.286 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T06:31:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:31:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:31:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-09-19T06:31:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:31:27.287 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-09-19T06:31:27.287 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T06:31:52.020 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:31:52.020 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:31:52.020 INFO:teuthology.orchestra.run.smithi037.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:31:52.020 INFO:teuthology.orchestra.run.smithi037.stdout: "repo_digests": [ 2024-09-19T06:31:52.020 INFO:teuthology.orchestra.run.smithi037.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:31:52.020 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:31:52.021 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:31:52.037 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T06:31:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable)", 2024-09-19T06:31:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d", 2024-09-19T06:31:52.037 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-09-19T06:31:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e" 2024-09-19T06:31:52.038 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-09-19T06:31:52.038 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T06:31:52.062 DEBUG:teuthology.orchestra.run.smithi037:> sudo mkdir -p /etc/ceph 2024-09-19T06:31:52.090 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-09-19T06:31:52.123 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-09-19T06:31:52.151 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 777 /etc/ceph 2024-09-19T06:31:52.177 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-09-19T06:31:52.204 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-09-19T06:31:52.229 INFO:tasks.cephadm:Writing seed config... 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-19T06:31:52.231 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-19T06:31:52.232 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-19T06:31:52.233 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-19T06:31:52.234 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-09-19T06:31:52.235 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:31:52.235 DEBUG:teuthology.orchestra.run.smithi037:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-19T06:31:52.251 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = b715dee2-7650-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-19T06:31:52.251 DEBUG:teuthology.orchestra.run.smithi037:mon.a> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service 2024-09-19T06:31:52.294 DEBUG:teuthology.orchestra.run.smithi037:mgr.x> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x.service 2024-09-19T06:31:52.337 INFO:tasks.cephadm:Bootstrapping... 2024-09-19T06:31:52.338 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 -v bootstrap --fsid b715dee2-7650-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.37 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:31:52.546 INFO:teuthology.orchestra.run.smithi037.stdout:-------------------------------------------------------------------------------- 2024-09-19T06:31:52.547 INFO:teuthology.orchestra.run.smithi037.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961', '-v', 'bootstrap', '--fsid', 'b715dee2-7650-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.37', '--skip-admin-label'] 2024-09-19T06:31:52.547 INFO:teuthology.orchestra.run.smithi037.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-19T06:31:52.547 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying podman|docker is present... 2024-09-19T06:31:52.571 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 5.2.2 2024-09-19T06:31:52.571 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying lvm2 is present... 2024-09-19T06:31:52.571 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying time synchronization is in place... 2024-09-19T06:31:52.579 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T06:31:52.579 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T06:31:52.586 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T06:31:52.586 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-09-19T06:31:52.593 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-09-19T06:31:52.599 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-09-19T06:31:52.599 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-09-19T06:31:52.599 INFO:teuthology.orchestra.run.smithi037.stdout:Repeating the final host check... 2024-09-19T06:31:52.621 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout 5.2.2 2024-09-19T06:31:52.621 INFO:teuthology.orchestra.run.smithi037.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-19T06:31:52.621 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl is present 2024-09-19T06:31:52.621 INFO:teuthology.orchestra.run.smithi037.stdout:lvcreate is present 2024-09-19T06:31:52.628 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-19T06:31:52.628 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-19T06:31:52.635 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-19T06:31:52.635 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-09-19T06:31:52.641 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout enabled 2024-09-19T06:31:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout active 2024-09-19T06:31:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:Unit chronyd.service is enabled and running 2024-09-19T06:31:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:Host looks OK 2024-09-19T06:31:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:Cluster fsid: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:31:52.648 INFO:teuthology.orchestra.run.smithi037.stdout:Acquiring lock 140342021846016 on /run/cephadm/b715dee2-7650-11ef-bceb-c7b262605968.lock 2024-09-19T06:31:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:Lock 140342021846016 acquired on /run/cephadm/b715dee2-7650-11ef-bceb-c7b262605968.lock 2024-09-19T06:31:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 3300 ... 2024-09-19T06:31:52.649 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying IP 172.21.15.37 port 6789 ... 2024-09-19T06:31:52.650 INFO:teuthology.orchestra.run.smithi037.stdout:Base mon IP(s) is [172.21.15.37:3300, 172.21.15.37:6789], mon addrv is [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-09-19T06:31:52.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.37 metric 100 2024-09-19T06:31:52.653 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.37 metric 100 2024-09-19T06:31:52.655 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-19T06:31:52.655 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-19T06:31:52.655 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1789sec hoplimit 64 pref medium 2024-09-19T06:31:52.658 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-19T06:31:52.658 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-19T06:31:52.658 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T06:31:52.658 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-19T06:31:52.658 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-19T06:31:52.658 INFO:teuthology.orchestra.run.smithi037.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-19T06:31:52.659 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-09-19T06:31:52.659 INFO:teuthology.orchestra.run.smithi037.stdout:Mon IP `172.21.15.37` is in CIDR network `172.21.0.0/20` 2024-09-19T06:31:52.659 INFO:teuthology.orchestra.run.smithi037.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-19T06:31:52.659 INFO:teuthology.orchestra.run.smithi037.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-19T06:31:52.660 INFO:teuthology.orchestra.run.smithi037.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:31:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stdout d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T06:31:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961... 2024-09-19T06:31:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Getting image source signatures 2024-09-19T06:31:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:c4221751d16d26d2a4b7031ebd71ecffffff13c8cf7401cf95ce80dab3b6d67a 2024-09-19T06:31:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-19T06:31:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Copying config sha256:d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d 2024-09-19T06:31:53.090 INFO:teuthology.orchestra.run.smithi037.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-19T06:31:53.952 INFO:teuthology.orchestra.run.smithi037.stdout:ceph: stdout ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T06:31:53.952 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph version: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable) 2024-09-19T06:31:53.952 INFO:teuthology.orchestra.run.smithi037.stdout:Extracting ceph user uid/gid from container image... 2024-09-19T06:31:54.799 INFO:teuthology.orchestra.run.smithi037.stdout:stat: stdout 167 167 2024-09-19T06:31:54.799 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial keys... 2024-09-19T06:31:55.698 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBbxetmeT71BBAAAVHivwP7Z65t1q726660dQ== 2024-09-19T06:31:56.778 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBcxetmwq7hAxAAqXcBSlb7j0a9+DX7U3t/uw== 2024-09-19T06:31:57.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph-authtool: stdout AQBdxetmGGI+BRAATVPjiTVajp3Y4EtFvteOsg== 2024-09-19T06:31:57.660 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial monmap... 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:monmaptool for a [v2:172.21.15.37:3300,v1:172.21.15.37:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:setting min_mon_release = quincy 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: set fsid to b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-19T06:31:58.584 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:31:58.585 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mon... 2024-09-19T06:31:59.464 INFO:teuthology.orchestra.run.smithi037.stdout:create mon.a on 2024-09-19T06:31:59.835 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-19T06:32:00.085 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-19T06:32:00.611 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-b715dee2-7650-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-b715dee2-7650-11ef-bceb-c7b262605968.target. 2024-09-19T06:32:00.611 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-b715dee2-7650-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-b715dee2-7650-11ef-bceb-c7b262605968.target. 2024-09-19T06:32:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a 2024-09-19T06:32:01.347 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-19T06:32:01.678 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b715dee2-7650-11ef-bceb-c7b262605968.target.wants/ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-b715dee2-7650-11ef-bceb-c7b262605968@.service. 2024-09-19T06:32:02.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:01 smithi037 podman[30827]: 2024-09-19 06:32:01.817707413 +0000 UTC m=+0.016507815 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:32:03.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:02 smithi037 podman[30827]: 2024-09-19 06:32:02.88962183 +0000 UTC m=+1.088422261 container create c63842f43d2032504df6137c3bdfb7c4559eab17a23f33f490ccc9a33669bc18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T06:32:05.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:04 smithi037 podman[30827]: 2024-09-19 06:32:04.871491577 +0000 UTC m=+3.070291966 container init c63842f43d2032504df6137c3bdfb7c4559eab17a23f33f490ccc9a33669bc18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T06:32:05.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:04 smithi037 podman[30827]: 2024-09-19 06:32:04.875854674 +0000 UTC m=+3.074655064 container start c63842f43d2032504df6137c3bdfb7c4559eab17a23f33f490ccc9a33669bc18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:32:05.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:04 smithi037 ceph-mon[30846]: mkfs b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:05.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:04 smithi037 ceph-mon[30846]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T06:32:05.690 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:32:05.690 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-09-19T06:32:05.699 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:32:05.699 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-09-19T06:32:05.699 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-09-19T06:32:05.699 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T06:32:05.699 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon to start... 2024-09-19T06:32:05.699 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mon... 2024-09-19T06:32:05.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 bash[30827]: c63842f43d2032504df6137c3bdfb7c4559eab17a23f33f490ccc9a33669bc18 2024-09-19T06:32:05.926 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 systemd[1]: Started Ceph mon.a for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:32:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T06:32:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: monmap epoch 1 2024-09-19T06:32:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: last_changed 2024-09-19T06:31:58.000017+0000 2024-09-19T06:32:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:32:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: min_mon_release 19 (squid) 2024-09-19T06:32:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: election_strategy: 1 2024-09-19T06:32:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:32:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: fsmap 2024-09-19T06:32:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T06:32:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:05 smithi037 ceph-mon[30846]: mgrmap e1: no daemons active 2024-09-19T06:32:09.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:08 smithi037 ceph-mon[30846]: from='client.? 172.21.15.37:0/1228972008' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-19T06:32:10.724 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout cluster: 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout id: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout services: 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 3s) 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:10.731 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout data: 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout pgs: 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.668+0000 7f5d033f7640 1 Processor -- start 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.668+0000 7f5d033f7640 1 -- start start 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.668+0000 7f5d033f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc07cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.668+0000 7f5d033f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc07d170 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.669+0000 7f5d01193640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc07cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.669+0000 7f5d01193640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc07cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40530/0 (socket says 172.21.15.37:40530) 2024-09-19T06:32:10.732 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.669+0000 7f5d01193640 1 -- 172.21.15.37:0/1317848803 learned_addr learned my addr 172.21.15.37:0/1317848803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.669+0000 7f5d01193640 1 -- 172.21.15.37:0/1317848803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc07d2f0 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.671+0000 7f5d01193640 1 --2- 172.21.15.37:0/1317848803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc07cba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009820 tx=0x7f5cf802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d703de31a1d4fbd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.671+0000 7f5cf3fff640 1 -- 172.21.15.37:0/1317848803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf8037070 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.671+0000 7f5cf3fff640 1 -- 172.21.15.37:0/1317848803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf8032440 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.671+0000 7f5cf3fff640 1 -- 172.21.15.37:0/1317848803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf8032740 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.672+0000 7f5d033f7640 1 -- 172.21.15.37:0/1317848803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 msgr2=0x7f5cfc07cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.672+0000 7f5d033f7640 1 --2- 172.21.15.37:0/1317848803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc07cba0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009820 tx=0x7f5cf802a270 comp rx=0 tx=0).stop 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.672+0000 7f5d033f7640 1 -- 172.21.15.37:0/1317848803 shutdown_connections 2024-09-19T06:32:10.733 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.672+0000 7f5d033f7640 1 --2- 172.21.15.37:0/1317848803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc07cba0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.672+0000 7f5d033f7640 1 -- 172.21.15.37:0/1317848803 >> 172.21.15.37:0/1317848803 conn(0x7f5cfc102fd0 msgr2=0x7f5cfc1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.672+0000 7f5d033f7640 1 -- 172.21.15.37:0/1317848803 shutdown_connections 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.672+0000 7f5d033f7640 1 -- 172.21.15.37:0/1317848803 wait complete. 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.673+0000 7f5d033f7640 1 Processor -- start 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.673+0000 7f5d033f7640 1 -- start start 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.673+0000 7f5d033f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc1a1510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.673+0000 7f5d033f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5cfc07d830 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.673+0000 7f5d01193640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc1a1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.673+0000 7f5d01193640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc1a1510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40540/0 (socket says 172.21.15.37:40540) 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.673+0000 7f5d01193640 1 -- 172.21.15.37:0/1228972008 learned_addr learned my addr 172.21.15.37:0/1228972008 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5d01193640 1 -- 172.21.15.37:0/1228972008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cfc1a1a50 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5d01193640 1 --2- 172.21.15.37:0/1228972008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc1a1510 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80090b0 tx=0x7f5cf80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:10.734 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5cf27fc640 1 -- 172.21.15.37:0/1228972008 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf8041070 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5cf27fc640 1 -- 172.21.15.37:0/1228972008 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5cf803c7a0 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cfc1a1ce0 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5cf27fc640 1 -- 172.21.15.37:0/1228972008 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf8037040 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cfc1a49d0 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5cf27fc640 1 -- 172.21.15.37:0/1228972008 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5cf8044070 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.674+0000 7f5cf27fc640 1 -- 172.21.15.37:0/1228972008 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5cf803d3f0 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.675+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cfc07cc20 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.678+0000 7f5cf27fc640 1 -- 172.21.15.37:0/1228972008 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5cf8046450 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.740+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f5cfc02d910 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.740+0000 7f5cf27fc640 1 -- 172.21.15.37:0/1228972008 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+310 (secure 0 0 0) 0x7f5cf8046630 con 0x7f5cfc07e6f0 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.741+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 msgr2=0x7f5cfc1a1510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.741+0000 7f5d033f7640 1 --2- 172.21.15.37:0/1228972008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc1a1510 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5cf80090b0 tx=0x7f5cf80046c0 comp rx=0 tx=0).stop 2024-09-19T06:32:10.735 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.741+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 shutdown_connections 2024-09-19T06:32:10.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.741+0000 7f5d033f7640 1 --2- 172.21.15.37:0/1228972008 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5cfc07e6f0 0x7f5cfc1a1510 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:10.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.741+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 >> 172.21.15.37:0/1228972008 conn(0x7f5cfc102fd0 msgr2=0x7f5cfc10a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:10.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.741+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 shutdown_connections 2024-09-19T06:32:10.736 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:08.741+0000 7f5d033f7640 1 -- 172.21.15.37:0/1228972008 wait complete. 2024-09-19T06:32:10.736 INFO:teuthology.orchestra.run.smithi037.stdout:mon is available 2024-09-19T06:32:10.736 INFO:teuthology.orchestra.run.smithi037.stdout:Assimilating anything we can from ceph.conf... 2024-09-19T06:32:11.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:11 smithi037 ceph-mon[30846]: from='client.? 172.21.15.37:0/195835912' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T06:32:11.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:11 smithi037 ceph-mon[30846]: from='client.? 172.21.15.37:0/195835912' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T06:32:11.768 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.168+0000 7f5ffe34b640 1 Processor -- start 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.168+0000 7f5ffe34b640 1 -- start start 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.169+0000 7f5ffe34b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff8106400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:11.769 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.169+0000 7f5ffe34b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff807bdd0 con 0x7f5ff8106000 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.169+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff8106400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.169+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff8106400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43678/0 (socket says 172.21.15.37:43678) 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.169+0000 7f5ff7fff640 1 -- 172.21.15.37:0/2297832521 learned_addr learned my addr 172.21.15.37:0/2297832521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.169+0000 7f5ff7fff640 1 -- 172.21.15.37:0/2297832521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff8106940 con 0x7f5ff8106000 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.170+0000 7f5ff7fff640 1 --2- 172.21.15.37:0/2297832521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff8106400 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009820 tx=0x7f5fe802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1539e146c230a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.170+0000 7f5ff6ffd640 1 -- 172.21.15.37:0/2297832521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fe8037070 con 0x7f5ff8106000 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.170+0000 7f5ff6ffd640 1 -- 172.21.15.37:0/2297832521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe8032440 con 0x7f5ff8106000 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.170+0000 7f5ff6ffd640 1 -- 172.21.15.37:0/2297832521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fe8032740 con 0x7f5ff8106000 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 -- 172.21.15.37:0/2297832521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 msgr2=0x7f5ff8106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 --2- 172.21.15.37:0/2297832521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff8106400 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5fe8009820 tx=0x7f5fe802a270 comp rx=0 tx=0).stop 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 -- 172.21.15.37:0/2297832521 shutdown_connections 2024-09-19T06:32:11.770 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 --2- 172.21.15.37:0/2297832521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff8106400 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 -- 172.21.15.37:0/2297832521 >> 172.21.15.37:0/2297832521 conn(0x7f5ff8101db0 msgr2=0x7f5ff8104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 -- 172.21.15.37:0/2297832521 shutdown_connections 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 -- 172.21.15.37:0/2297832521 wait complete. 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 Processor -- start 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.171+0000 7f5ffe34b640 1 -- start start 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ffe34b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff819d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ffe34b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff8106ee0 con 0x7f5ff8106000 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff819d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff819d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43686/0 (socket says 172.21.15.37:43686) 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff7fff640 1 -- 172.21.15.37:0/195835912 learned_addr learned my addr 172.21.15.37:0/195835912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff7fff640 1 -- 172.21.15.37:0/195835912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff819d610 con 0x7f5ff8106000 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff7fff640 1 --2- 172.21.15.37:0/195835912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff819d0d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80090b0 tx=0x7f5fe80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fe8041070 con 0x7f5ff8106000 2024-09-19T06:32:11.771 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5fe803c7a0 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff819d8a0 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.172+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fe8037040 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.173+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff81a0590 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.173+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5fe8044070 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.173+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5fe803d3f0 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.174+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff8106400 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.176+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f5fe8046450 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.235+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5ff81a1c70 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.238+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f5fe8046630 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ff57fa640 1 -- 172.21.15.37:0/195835912 <== mon.0 v2:172.21.15.37:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5fe8056070 con 0x7f5ff8106000 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 msgr2=0x7f5ff819d0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ffe34b640 1 --2- 172.21.15.37:0/195835912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff819d0d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80090b0 tx=0x7f5fe80046c0 comp rx=0 tx=0).stop 2024-09-19T06:32:11.772 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 shutdown_connections 2024-09-19T06:32:11.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ffe34b640 1 --2- 172.21.15.37:0/195835912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ff8106000 0x7f5ff819d0d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:11.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 >> 172.21.15.37:0/195835912 conn(0x7f5ff8101db0 msgr2=0x7f5ff81936e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:11.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 shutdown_connections 2024-09-19T06:32:11.773 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:11.239+0000 7f5ffe34b640 1 -- 172.21.15.37:0/195835912 wait complete. 2024-09-19T06:32:11.773 INFO:teuthology.orchestra.run.smithi037.stdout:Generating new minimal ceph.conf... 2024-09-19T06:32:12.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:12 smithi037 ceph-mon[30846]: from='client.? 172.21.15.37:0/2333303214' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:32:12.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.183+0000 7f146806a640 1 Processor -- start 2024-09-19T06:32:12.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.183+0000 7f146806a640 1 -- start start 2024-09-19T06:32:12.814 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.184+0000 7f146806a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146010a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.184+0000 7f146806a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f146007c710 con 0x7f14601081d0 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.184+0000 7f1465e06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146010a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.184+0000 7f1465e06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146010a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43700/0 (socket says 172.21.15.37:43700) 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.184+0000 7f1465e06640 1 -- 172.21.15.37:0/2315971761 learned_addr learned my addr 172.21.15.37:0/2315971761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.184+0000 7f1465e06640 1 -- 172.21.15.37:0/2315971761 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146010ab00 con 0x7f14601081d0 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.185+0000 7f1465e06640 1 --2- 172.21.15.37:0/2315971761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146010a5c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1458009820 tx=0x7f145802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ced7f1264ad922c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.185+0000 7f1464e04640 1 -- 172.21.15.37:0/2315971761 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1458037070 con 0x7f14601081d0 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.185+0000 7f1464e04640 1 -- 172.21.15.37:0/2315971761 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1458032900 con 0x7f14601081d0 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.185+0000 7f1464e04640 1 -- 172.21.15.37:0/2315971761 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1458030850 con 0x7f14601081d0 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.185+0000 7f146806a640 1 -- 172.21.15.37:0/2315971761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 msgr2=0x7f146010a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:12.815 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.185+0000 7f146806a640 1 --2- 172.21.15.37:0/2315971761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146010a5c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1458009820 tx=0x7f145802a270 comp rx=0 tx=0).stop 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 -- 172.21.15.37:0/2315971761 shutdown_connections 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 --2- 172.21.15.37:0/2315971761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146010a5c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 -- 172.21.15.37:0/2315971761 >> 172.21.15.37:0/2315971761 conn(0x7f1460101e50 msgr2=0x7f1460104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 -- 172.21.15.37:0/2315971761 shutdown_connections 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 -- 172.21.15.37:0/2315971761 wait complete. 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 Processor -- start 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 -- start start 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146019b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.186+0000 7f146806a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f146010bba0 con 0x7f14601081d0 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f1465e06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146019b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f1465e06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146019b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43710/0 (socket says 172.21.15.37:43710) 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f1465e06640 1 -- 172.21.15.37:0/2333303214 learned_addr learned my addr 172.21.15.37:0/2333303214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:12.816 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f1465e06640 1 -- 172.21.15.37:0/2333303214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146019bcd0 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f1465e06640 1 --2- 172.21.15.37:0/2333303214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146019b790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f14580097f0 tx=0x7f1458030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f145effd640 1 -- 172.21.15.37:0/2333303214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1458037040 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f145effd640 1 -- 172.21.15.37:0/2333303214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1458040070 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f145effd640 1 -- 172.21.15.37:0/2333303214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f145803a390 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146019bf60 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.187+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146019ec50 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.188+0000 7f145effd640 1 -- 172.21.15.37:0/2333303214 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f145803a630 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.188+0000 7f145effd640 1 -- 172.21.15.37:0/2333303214 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f14580459a0 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.188+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1460108630 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.190+0000 7f145effd640 1 -- 172.21.15.37:0/2333303214 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1458045460 con 0x7f14601081d0 2024-09-19T06:32:12.817 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.250+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f146019ee20 con 0x7f14601081d0 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.250+0000 7f145effd640 1 -- 172.21.15.37:0/2333303214 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f145803b430 con 0x7f14601081d0 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.251+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 msgr2=0x7f146019b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.251+0000 7f146806a640 1 --2- 172.21.15.37:0/2333303214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146019b790 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f14580097f0 tx=0x7f1458030d80 comp rx=0 tx=0).stop 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.251+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 shutdown_connections 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.251+0000 7f146806a640 1 --2- 172.21.15.37:0/2333303214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f14601081d0 0x7f146019b790 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.251+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 >> 172.21.15.37:0/2333303214 conn(0x7f1460101e50 msgr2=0x7f1460102a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.251+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 shutdown_connections 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:12.251+0000 7f146806a640 1 -- 172.21.15.37:0/2333303214 wait complete. 2024-09-19T06:32:12.818 INFO:teuthology.orchestra.run.smithi037.stdout:Restarting the monitor... 2024-09-19T06:32:13.129 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:12 smithi037 systemd[1]: Stopping Ceph mon.a for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:32:13.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a[30842]: 2024-09-19T06:32:13.127+0000 7efc4fcd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:32:13.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a[30842]: 2024-09-19T06:32:13.127+0000 7efc4fcd7640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-19T06:32:13.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:13 smithi037 podman[31087]: 2024-09-19 06:32:13.27091258 +0000 UTC m=+0.289861903 container died c63842f43d2032504df6137c3bdfb7c4559eab17a23f33f490ccc9a33669bc18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-19T06:32:13.811 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:13 smithi037 podman[31087]: 2024-09-19 06:32:13.601257425 +0000 UTC m=+0.620206745 container cleanup c63842f43d2032504df6137c3bdfb7c4559eab17a23f33f490ccc9a33669bc18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:32:13.811 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:13 smithi037 bash[31087]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a 2024-09-19T06:32:14.228 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:13 smithi037 podman[31102]: 2024-09-19 06:32:13.810586436 +0000 UTC m=+0.538775141 container remove c63842f43d2032504df6137c3bdfb7c4559eab17a23f33f490ccc9a33669bc18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T06:32:14.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T06:32:14.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 systemd[1]: Stopped Ceph mon.a for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:32:14.229 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 systemd[1]: Starting Ceph mon.a for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:32:14.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 podman[31192]: 2024-09-19 06:32:14.228063538 +0000 UTC m=+0.110829638 container create 98ade29977a1e0ece0b32d797a5175e96eaf2a9230222862612c0cb0f52b0a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T06:32:14.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 podman[31192]: 2024-09-19 06:32:14.133634063 +0000 UTC m=+0.016400171 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 podman[31192]: 2024-09-19 06:32:14.430352623 +0000 UTC m=+0.313118726 container init 98ade29977a1e0ece0b32d797a5175e96eaf2a9230222862612c0cb0f52b0a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 podman[31192]: 2024-09-19 06:32:14.433369102 +0000 UTC m=+0.316135189 container start 98ade29977a1e0ece0b32d797a5175e96eaf2a9230222862612c0cb0f52b0a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: set uid:gid to 167:167 (ceph:ceph) 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: ceph version 19.1.1-361-g2f599a46 (2f599a46ad10e37c929e5c43da2f6907ddcd1961) squid (stable), process ceph-mon, pid 2 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: pidfile_write: ignore empty --pid-file 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: load: jerasure load: lrc 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: RocksDB version: 7.9.2 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Git sha 0 2024-09-19T06:32:14.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Compile date 2024-09-18 16:29:48 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: DB SUMMARY 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: DB Session ID: M9BW5ENL5VNWZ8DFVHPQ 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: CURRENT file: CURRENT 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: IDENTITY file: IDENTITY 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 106707 ; 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.error_if_exists: 0 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.create_if_missing: 0 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.paranoid_checks: 1 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-19T06:32:14.508 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.env: 0x556d102f1940 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.fs: PosixFileSystem 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.info_log: 0x556d10ffd6e0 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_file_opening_threads: 16 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.statistics: (nil) 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.use_fsync: 0 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_log_file_size: 0 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.keep_log_file_num: 1000 2024-09-19T06:32:14.509 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.recycle_log_file_num: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.allow_fallocate: 1 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.allow_mmap_reads: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.allow_mmap_writes: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.use_direct_reads: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.create_missing_column_families: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.db_log_dir: 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.wal_dir: 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-19T06:32:14.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.advise_random_on_open: 1 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.db_write_buffer_size: 0 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.write_buffer_manager: 0x556d1100cdc0 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.rate_limiter: (nil) 2024-09-19T06:32:14.511 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-19T06:32:14.512 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.wal_recovery_mode: 2 2024-09-19T06:32:14.512 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.enable_thread_tracking: 0 2024-09-19T06:32:14.512 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.enable_pipelined_write: 0 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.unordered_write: 0 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.row_cache: None 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.wal_filter: None 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.allow_ingest_behind: 0 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.two_write_queues: 0 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.manual_wal_flush: 0 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.wal_compression: 0 2024-09-19T06:32:14.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.atomic_flush: 0 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.log_readahead_size: 0 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.best_efforts_recovery: 0 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.allow_data_in_errors: 0 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.db_host_id: __hostname__ 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_background_jobs: 2 2024-09-19T06:32:14.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_background_compactions: -1 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_subcompactions: 1 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_total_wal_size: 0 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_open_files: -1 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bytes_per_sync: 0 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_readahead_size: 0 2024-09-19T06:32:14.515 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_background_flushes: -1 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Compression algorithms supported: 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kZSTD supported: 0 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kXpressCompression supported: 0 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kBZip2Compression supported: 0 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kLZ4Compression supported: 1 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kZlibCompression supported: 1 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kLZ4HCCompression supported: 1 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: kSnappyCompression supported: 1 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-19T06:32:14.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.merge_operator: 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_filter: None 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_filter_factory: None 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.sst_partitioner_factory: None 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556d10ffd320) 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks: 1 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout: pin_top_level_index_and_filter: 1 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_type: 0 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_index_type: 0 2024-09-19T06:32:14.517 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_shortening: 1 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: checksum: 4 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: no_block_cache: 0 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache: 0x556d10ff11f0 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_name: BinnedLRUCache 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_options: 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: capacity : 536870912 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_shard_bits : 4 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: strict_capacity_limit : 0 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: high_pri_pool_ratio: 0.000 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_cache_compressed: (nil) 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: persistent_cache: (nil) 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size: 4096 2024-09-19T06:32:14.518 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_size_deviation: 10 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_restart_interval: 16 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: index_block_restart_interval: 1 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: metadata_block_size: 4096 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: partition_filters: 0 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: use_delta_encoding: 1 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: filter_policy: bloomfilter 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: whole_key_filtering: 1 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: verify_compression: 0 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: read_amp_bytes_per_bit: 0 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: format_version: 5 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_index_compression: 1 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: block_align: 0 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: max_auto_readahead_size: 262144 2024-09-19T06:32:14.519 INFO:journalctl@ceph.mon.a.smithi037.stdout: prepopulate_block_cache: 0 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout: initial_auto_readahead_size: 8192 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout: num_file_reads_for_auto_readahead: 2 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.write_buffer_size: 33554432 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_write_buffer_number: 2 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression: NoCompression 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression: Disabled 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.prefix_extractor: nullptr 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.num_levels: 7 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-19T06:32:14.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.level: 32767 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.strategy: 0 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-19T06:32:14.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.enabled: false 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.target_file_size_base: 67108864 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-19T06:32:14.522 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.arena_block_size: 1048576 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.disable_auto_compactions: 0 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-19T06:32:14.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.inplace_update_support: 0 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-19T06:32:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.bloom_locality: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.max_successive_merges: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.paranoid_file_checks: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.force_consistency_checks: 1 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.report_bg_io_stats: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.ttl: 2592000 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.enable_blob_files: false 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.min_blob_size: 0 2024-09-19T06:32:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.blob_file_size: 268435456 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.blob_file_starting_level: 0 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 808af4af-fb35-4cb2-a3d7-e1a65cd72f01 2024-09-19T06:32:14.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726727534460690, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726727534461803, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 101579, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 338, "table_properties": {"data_size": 99455, "index_size": 316, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14594, "raw_average_key_size": 49, "raw_value_size": 91504, "raw_average_value_size": 312, "num_data_blocks": 12, "num_entries": 293, "num_filter_entries": 293, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726727534, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "808af4af-fb35-4cb2-a3d7-e1a65cd72f01", "db_session_id": "M9BW5ENL5VNWZ8DFVHPQ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726727534461903, "job": 1, "event": "recovery_finished"} 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x556d1101ee00 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: DB pointer 0x556d1111a000 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** DB Stats ** 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-19T06:32:14.528 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: L0 2/0 101.06 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Sum 2/0 101.06 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** Compaction Stats [default] ** 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-19T06:32:14.529 INFO:journalctl@ceph.mon.a.smithi037.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 138.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: Cumulative compaction: 0.00 GB write, 28.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: Interval compaction: 0.00 GB write, 28.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-19T06:32:14.530 INFO:journalctl@ceph.mon.a.smithi037.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache BinnedLRUCache@0x556d10ff11f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: starting mon.a rank 0 at public addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] at bind addrs [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???) e1 preinit fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???).mds e1 new map 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???).mds e1 print_map 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: e1 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: btime 2024-09-19T06:32:04:919506+0000 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:32:14.531 INFO:journalctl@ceph.mon.a.smithi037.stdout: legacy client fscid: -1 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout: 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout: No filesystems configured 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 bash[31192]: 98ade29977a1e0ece0b32d797a5175e96eaf2a9230222862612c0cb0f52b0a0c 2024-09-19T06:32:14.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 systemd[1]: Started Ceph mon.a for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:32:14.533 INFO:teuthology.orchestra.run.smithi037.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-19T06:32:14.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: monmap epoch 1 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: last_changed 2024-09-19T06:31:58.000017+0000 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: election_strategy: 1 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: fsmap 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: osdmap e1: 0 total, 0 up, 0 in 2024-09-19T06:32:14.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:14 smithi037 ceph-mon[31211]: mgrmap e1: no daemons active 2024-09-19T06:32:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:15 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3475736014' entity='client.admin' 2024-09-19T06:32:16.755 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.908+0000 7f2756143640 1 Processor -- start 2024-09-19T06:32:16.759 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.908+0000 7f2756143640 1 -- start start 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.909+0000 7f2756143640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f2750106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.909+0000 7f2756143640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2750107010 con 0x7f2750106640 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.909+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f2750106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.909+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f2750106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43726/0 (socket says 172.21.15.37:43726) 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.909+0000 7f274f7fe640 1 -- 172.21.15.37:0/2438542464 learned_addr learned my addr 172.21.15.37:0/2438542464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.909+0000 7f274f7fe640 1 -- 172.21.15.37:0/2438542464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27501077f0 con 0x7f2750106640 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.911+0000 7f274f7fe640 1 --2- 172.21.15.37:0/2438542464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f2750106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f274000d180 tx=0x7f274002c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5fafa10089ddec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.911+0000 7f274e7fc640 1 -- 172.21.15.37:0/2438542464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f274000b410 con 0x7f2750106640 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.911+0000 7f274e7fc640 1 -- 172.21.15.37:0/2438542464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2740009ac0 con 0x7f2750106640 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.911+0000 7f274e7fc640 1 -- 172.21.15.37:0/2438542464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2740033be0 con 0x7f2750106640 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.912+0000 7f2756143640 1 -- 172.21.15.37:0/2438542464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 msgr2=0x7f2750106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:16.760 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.912+0000 7f2756143640 1 --2- 172.21.15.37:0/2438542464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f2750106a40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f274000d180 tx=0x7f274002c9a0 comp rx=0 tx=0).stop 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.912+0000 7f2756143640 1 -- 172.21.15.37:0/2438542464 shutdown_connections 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.912+0000 7f2756143640 1 --2- 172.21.15.37:0/2438542464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f2750106a40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.912+0000 7f2756143640 1 -- 172.21.15.37:0/2438542464 >> 172.21.15.37:0/2438542464 conn(0x7f2750101db0 msgr2=0x7f2750104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.912+0000 7f2756143640 1 -- 172.21.15.37:0/2438542464 shutdown_connections 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.912+0000 7f2756143640 1 -- 172.21.15.37:0/2438542464 wait complete. 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f2756143640 1 Processor -- start 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f2756143640 1 -- start start 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f2756143640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f27501995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f2756143640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f275007b120 con 0x7f2750106640 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f27501995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f274f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f27501995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43738/0 (socket says 172.21.15.37:43738) 2024-09-19T06:32:16.761 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f274f7fe640 1 -- 172.21.15.37:0/3475736014 learned_addr learned my addr 172.21.15.37:0/3475736014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.913+0000 7f274f7fe640 1 -- 172.21.15.37:0/3475736014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2750199b00 con 0x7f2750106640 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f274f7fe640 1 --2- 172.21.15.37:0/3475736014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f27501995c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2740033be0 tx=0x7f2740033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f274cff9640 1 -- 172.21.15.37:0/3475736014 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2740043070 con 0x7f2750106640 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f274cff9640 1 -- 172.21.15.37:0/3475736014 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f274003d900 con 0x7f2750106640 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2750199d90 con 0x7f2750106640 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f274cff9640 1 -- 172.21.15.37:0/3475736014 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f274003c7b0 con 0x7f2750106640 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275019ca80 con 0x7f2750106640 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f274cff9640 1 -- 172.21.15.37:0/3475736014 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2740046070 con 0x7f2750106640 2024-09-19T06:32:16.762 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.914+0000 7f274cff9640 1 -- 172.21.15.37:0/3475736014 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2740030050 con 0x7f2750106640 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.915+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2750106a40 con 0x7f2750106640 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.917+0000 7f274cff9640 1 -- 172.21.15.37:0/3475736014 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f2740048450 con 0x7f2750106640 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.977+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f275019cc50 con 0x7f2750106640 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.981+0000 7f274cff9640 1 -- 172.21.15.37:0/3475736014 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f274003dab0 con 0x7f2750106640 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.982+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 msgr2=0x7f27501995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.982+0000 7f2756143640 1 --2- 172.21.15.37:0/3475736014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f27501995c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f2740033be0 tx=0x7f2740033c10 comp rx=0 tx=0).stop 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.982+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 shutdown_connections 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.982+0000 7f2756143640 1 --2- 172.21.15.37:0/3475736014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2750106640 0x7f27501995c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.982+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 >> 172.21.15.37:0/3475736014 conn(0x7f2750101db0 msgr2=0x7f275018fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.982+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 shutdown_connections 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:14.982+0000 7f2756143640 1 -- 172.21.15.37:0/3475736014 wait complete. 2024-09-19T06:32:16.763 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-19T06:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout:Creating mgr... 2024-09-19T06:32:16.802 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-19T06:32:16.803 INFO:teuthology.orchestra.run.smithi037.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-19T06:32:17.078 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x 2024-09-19T06:32:17.078 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Failed to reset failed state of unit ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-19T06:32:17.239 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-b715dee2-7650-11ef-bceb-c7b262605968.target.wants/ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-b715dee2-7650-11ef-bceb-c7b262605968@.service. 2024-09-19T06:32:17.471 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 systemd[1]: Starting Ceph mgr.x for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:32:17.740 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 podman[31420]: 2024-09-19 06:32:17.373340933 +0000 UTC m=+0.016181803 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 2024-09-19T06:32:17.740 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 podman[31420]: 2024-09-19 06:32:17.479060066 +0000 UTC m=+0.121900933 container create 0d11568288e00937fa6984046f34a92244bc472c99ed39bf59c050ce1d88751d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T06:32:17.740 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 podman[31420]: 2024-09-19 06:32:17.664194724 +0000 UTC m=+0.307035592 container init 0d11568288e00937fa6984046f34a92244bc472c99ed39bf59c050ce1d88751d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T06:32:17.740 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 podman[31420]: 2024-09-19 06:32:17.666945796 +0000 UTC m=+0.309786664 container start 0d11568288e00937fa6984046f34a92244bc472c99ed39bf59c050ce1d88751d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0) 2024-09-19T06:32:17.740 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 bash[31420]: 0d11568288e00937fa6984046f34a92244bc472c99ed39bf59c050ce1d88751d 2024-09-19T06:32:17.754 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:32:17.754 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-09-19T06:32:17.762 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:32:17.762 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-09-19T06:32:17.762 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-09-19T06:32:17.762 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to enable service . firewalld.service is not available 2024-09-19T06:32:17.771 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:32:17.771 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-09-19T06:32:17.779 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:32:17.779 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-09-19T06:32:17.779 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-09-19T06:32:17.779 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-19T06:32:17.780 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr to start... 2024-09-19T06:32:17.780 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr... 2024-09-19T06:32:17.990 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 systemd[1]: Started Ceph mgr.x for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:32:17.990 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:17.792+0000 7ff33ebeb040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:32:17.990 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:17 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:17.857+0000 7ff33ebeb040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:32:18.248 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:18.018+0000 7ff33ebeb040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:32:18.248 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:18.087+0000 7ff33ebeb040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:32:18.248 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:18.246+0000 7ff33ebeb040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:32:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2870231454' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:32:18.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:18.396+0000 7ff33ebeb040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:32:18.830 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:18.731+0000 7ff33ebeb040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:32:18.830 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:18.795+0000 7ff33ebeb040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:32:18.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:18.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:32:18.834 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:18.835 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:32:18.836 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:32:18.843 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:32:18.844 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:32:04:919506+0000", 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:32:18.845 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:32:04.920291+0000", 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:32:18.846 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.180+0000 7f12b10e1640 1 Processor -- start 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.180+0000 7f12b10e1640 1 -- start start 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.180+0000 7f12b10e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.180+0000 7f12b10e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac107010 con 0x7f12ac106640 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.181+0000 7f12aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.181+0000 7f12aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43750/0 (socket says 172.21.15.37:43750) 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.181+0000 7f12aad9d640 1 -- 172.21.15.37:0/2109648470 learned_addr learned my addr 172.21.15.37:0/2109648470 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.181+0000 7f12aad9d640 1 -- 172.21.15.37:0/2109648470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac1077f0 con 0x7f12ac106640 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.182+0000 7f12aad9d640 1 --2- 172.21.15.37:0/2109648470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac106a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f129c00bac0 tx=0x7f129c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1fee8e365103fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.182+0000 7f12a9d9b640 1 -- 172.21.15.37:0/2109648470 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129c0326d0 con 0x7f12ac106640 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.182+0000 7f12a9d9b640 1 -- 172.21.15.37:0/2109648470 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f129c038070 con 0x7f12ac106640 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.182+0000 7f12a9d9b640 1 -- 172.21.15.37:0/2109648470 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129c033d70 con 0x7f12ac106640 2024-09-19T06:32:18.847 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.182+0000 7f12b10e1640 1 -- 172.21.15.37:0/2109648470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 msgr2=0x7f12ac106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.182+0000 7f12b10e1640 1 --2- 172.21.15.37:0/2109648470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac106a40 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f129c00bac0 tx=0x7f129c02c920 comp rx=0 tx=0).stop 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.182+0000 7f12b10e1640 1 -- 172.21.15.37:0/2109648470 shutdown_connections 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.183+0000 7f12b10e1640 1 --2- 172.21.15.37:0/2109648470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac106a40 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.183+0000 7f12b10e1640 1 -- 172.21.15.37:0/2109648470 >> 172.21.15.37:0/2109648470 conn(0x7f12ac101db0 msgr2=0x7f12ac104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.183+0000 7f12b10e1640 1 -- 172.21.15.37:0/2109648470 shutdown_connections 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.183+0000 7f12b10e1640 1 -- 172.21.15.37:0/2109648470 wait complete. 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.183+0000 7f12b10e1640 1 Processor -- start 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.183+0000 7f12b10e1640 1 -- start start 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f12b10e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f12b10e1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f12ac07b120 con 0x7f12ac106640 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f12aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f12aad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac199530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43764/0 (socket says 172.21.15.37:43764) 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f12aad9d640 1 -- 172.21.15.37:0/2870231454 learned_addr learned my addr 172.21.15.37:0/2870231454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:18.848 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f12aad9d640 1 -- 172.21.15.37:0/2870231454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12ac199a70 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f12aad9d640 1 --2- 172.21.15.37:0/2870231454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac199530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f129c00b2c0 tx=0x7f129c032820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.184+0000 7f1293fff640 1 -- 172.21.15.37:0/2870231454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129c042070 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.185+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12ac199d00 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.185+0000 7f1293fff640 1 -- 172.21.15.37:0/2870231454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f129c032c40 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.185+0000 7f1293fff640 1 -- 172.21.15.37:0/2870231454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f129c03c580 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.185+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12ac19c9f0 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.185+0000 7f1293fff640 1 -- 172.21.15.37:0/2870231454 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f129c005990 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.185+0000 7f1293fff640 1 -- 172.21.15.37:0/2870231454 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f129c030050 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.186+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12ac106a40 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.188+0000 7f1293fff640 1 -- 172.21.15.37:0/2870231454 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f129c054080 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.250+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f12ac1929a0 con 0x7f12ac106640 2024-09-19T06:32:18.849 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.250+0000 7f1293fff640 1 -- 172.21.15.37:0/2870231454 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f129c046950 con 0x7f12ac106640 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.251+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 msgr2=0x7f12ac199530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.251+0000 7f12b10e1640 1 --2- 172.21.15.37:0/2870231454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac199530 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f129c00b2c0 tx=0x7f129c032820 comp rx=0 tx=0).stop 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.251+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 shutdown_connections 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.251+0000 7f12b10e1640 1 --2- 172.21.15.37:0/2870231454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f12ac106640 0x7f12ac199530 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.251+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 >> 172.21.15.37:0/2870231454 conn(0x7f12ac101db0 msgr2=0x7f12ac18fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.251+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 shutdown_connections 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:18.251+0000 7f12b10e1640 1 -- 172.21.15.37:0/2870231454 wait complete. 2024-09-19T06:32:18.850 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (1/15)... 2024-09-19T06:32:19.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:18.989+0000 7ff33ebeb040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:32:20.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:19.916+0000 7ff33ebeb040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:32:20.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:19.987+0000 7ff33ebeb040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:32:21.019 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:20.742+0000 7ff33ebeb040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:32:21.019 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:20.865+0000 7ff33ebeb040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:32:21.273 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.017+0000 7ff33ebeb040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:32:21.273 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.086+0000 7ff33ebeb040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:32:21.273 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.249+0000 7ff33ebeb040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:32:21.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/751264799' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:32:21.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.350+0000 7ff33ebeb040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:32:21.526 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.425+0000 7ff33ebeb040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:32:21.897 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.598+0000 7ff33ebeb040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:32:21.897 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.670+0000 7ff33ebeb040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:32:21.897 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.743+0000 7ff33ebeb040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:32:21.897 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.831+0000 7ff33ebeb040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:32:21.898 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.894+0000 7ff33ebeb040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:32:21.958 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:32:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:32:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:32:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:32:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:32:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:32:21.959 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-09-19T06:32:21.960 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:32:21.961 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:32:21.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:32:21.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:32:21.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:32:21.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:21.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:32:04:919506+0000", 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:32:21.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:32:04.920291+0000", 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.271+0000 7fd5ce5e9640 1 Processor -- start 2024-09-19T06:32:21.972 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.271+0000 7fd5ce5e9640 1 -- start start 2024-09-19T06:32:21.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.272+0000 7fd5ce5e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c8106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:21.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.272+0000 7fd5ce5e9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c8107010 con 0x7fd5c8106640 2024-09-19T06:32:21.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.272+0000 7fd5c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c8106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:21.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.272+0000 7fd5c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c8106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34726/0 (socket says 172.21.15.37:34726) 2024-09-19T06:32:21.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.272+0000 7fd5c7fff640 1 -- 172.21.15.37:0/1659554716 learned_addr learned my addr 172.21.15.37:0/1659554716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:21.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.272+0000 7fd5c7fff640 1 -- 172.21.15.37:0/1659554716 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c81077f0 con 0x7fd5c8106640 2024-09-19T06:32:21.973 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.273+0000 7fd5c7fff640 1 --2- 172.21.15.37:0/1659554716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c8106a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009820 tx=0x7fd5b802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8977eb659a215464 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.274+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/1659554716 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8037070 con 0x7fd5c8106640 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.274+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/1659554716 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5b8032900 con 0x7fd5c8106640 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.274+0000 7fd5c6ffd640 1 -- 172.21.15.37:0/1659554716 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8030850 con 0x7fd5c8106640 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.275+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/1659554716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 msgr2=0x7fd5c8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.275+0000 7fd5ce5e9640 1 --2- 172.21.15.37:0/1659554716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c8106a40 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009820 tx=0x7fd5b802a270 comp rx=0 tx=0).stop 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.275+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/1659554716 shutdown_connections 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.275+0000 7fd5ce5e9640 1 --2- 172.21.15.37:0/1659554716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c8106a40 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.275+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/1659554716 >> 172.21.15.37:0/1659554716 conn(0x7fd5c8101db0 msgr2=0x7fd5c8104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.275+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/1659554716 shutdown_connections 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.275+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/1659554716 wait complete. 2024-09-19T06:32:21.974 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.276+0000 7fd5ce5e9640 1 Processor -- start 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.276+0000 7fd5ce5e9640 1 -- start start 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.276+0000 7fd5ce5e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c819b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.276+0000 7fd5ce5e9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd5c810aba0 con 0x7fd5c8106640 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.276+0000 7fd5c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c819b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.276+0000 7fd5c7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c819b830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34736/0 (socket says 172.21.15.37:34736) 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.276+0000 7fd5c7fff640 1 -- 172.21.15.37:0/751264799 learned_addr learned my addr 172.21.15.37:0/751264799 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5c7fff640 1 -- 172.21.15.37:0/751264799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5c819bd70 con 0x7fd5c8106640 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5c7fff640 1 --2- 172.21.15.37:0/751264799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c819b830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009940 tx=0x7fd5b80326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5c57fa640 1 -- 172.21.15.37:0/751264799 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8032df0 con 0x7fd5c8106640 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5c57fa640 1 -- 172.21.15.37:0/751264799 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd5b803aa40 con 0x7fd5c8106640 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5c819c000 con 0x7fd5c8106640 2024-09-19T06:32:21.975 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5c57fa640 1 -- 172.21.15.37:0/751264799 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd5b8045bb0 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5c818fd90 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.277+0000 7fd5c57fa640 1 -- 172.21.15.37:0/751264799 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd5b8043710 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.278+0000 7fd5c57fa640 1 -- 172.21.15.37:0/751264799 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd5b804e730 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.278+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5c810ad20 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.281+0000 7fd5c57fa640 1 -- 172.21.15.37:0/751264799 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd5b8044e10 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.376+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fd5c802d8c0 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.376+0000 7fd5c57fa640 1 -- 172.21.15.37:0/751264799 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fd5b803abf0 con 0x7fd5c8106640 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.377+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 msgr2=0x7fd5c819b830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.377+0000 7fd5ce5e9640 1 --2- 172.21.15.37:0/751264799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c819b830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd5b8009940 tx=0x7fd5b80326e0 comp rx=0 tx=0).stop 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.377+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 shutdown_connections 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.377+0000 7fd5ce5e9640 1 --2- 172.21.15.37:0/751264799 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd5c8106640 0x7fd5c819b830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:21.976 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.377+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 >> 172.21.15.37:0/751264799 conn(0x7fd5c8101db0 msgr2=0x7fd5c8102db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:21.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.378+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 shutdown_connections 2024-09-19T06:32:21.977 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:21.378+0000 7fd5ce5e9640 1 -- 172.21.15.37:0/751264799 wait complete. 2024-09-19T06:32:21.977 INFO:teuthology.orchestra.run.smithi037.stdout:mgr not available, waiting (2/15)... 2024-09-19T06:32:22.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:21.958+0000 7ff33ebeb040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:32:23.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:22.795+0000 7ff33ebeb040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:32:23.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: Activating manager daemon x 2024-09-19T06:32:23.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: mgrmap e2: x(active, starting, since 0.00290474s) 2024-09-19T06:32:23.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: Manager daemon x is now available 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' 2024-09-19T06:32:23.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:22 smithi037 ceph-mon[31211]: from='mgr.14100 172.21.15.37:0/1694829209' entity='mgr.x' 2024-09-19T06:32:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:24 smithi037 ceph-mon[31211]: mgrmap e3: x(active, since 1.00739s) 2024-09-19T06:32:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:24 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1837166631' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "health": { 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-19T06:32:25.206 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 0 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "a" 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:25.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-19T06:32:25.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:25.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-19T06:32:25.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-19T06:32:25.216 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "btime": "2024-09-19T06:32:04:919506+0000", 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:32:25.217 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "restful" 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ], 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-19T06:32:25.218 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "modified": "2024-09-19T06:32:04.920291+0000", 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout }, 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.382+0000 7ff8f88ee640 1 Processor -- start 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.382+0000 7ff8f88ee640 1 -- start start 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.382+0000 7ff8f88ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f4107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.382+0000 7ff8f88ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4108160 con 0x7ff8f4107790 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.383+0000 7ff8f259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f4107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.383+0000 7ff8f259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f4107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34816/0 (socket says 172.21.15.37:34816) 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.383+0000 7ff8f259c640 1 -- 172.21.15.37:0/177994962 learned_addr learned my addr 172.21.15.37:0/177994962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.383+0000 7ff8f259c640 1 -- 172.21.15.37:0/177994962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f4108940 con 0x7ff8f4107790 2024-09-19T06:32:25.219 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.384+0000 7ff8f259c640 1 --2- 172.21.15.37:0/177994962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f4107b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009e80 tx=0x7ff8ec02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc85921dd864653d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.384+0000 7ff8f159a640 1 -- 172.21.15.37:0/177994962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec036070 con 0x7ff8f4107790 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.384+0000 7ff8f159a640 1 -- 172.21.15.37:0/177994962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8ec0318c0 con 0x7ff8f4107790 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.384+0000 7ff8f159a640 1 -- 172.21.15.37:0/177994962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec007ab0 con 0x7ff8f4107790 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.385+0000 7ff8f88ee640 1 -- 172.21.15.37:0/177994962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 msgr2=0x7ff8f4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.385+0000 7ff8f88ee640 1 --2- 172.21.15.37:0/177994962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f4107b90 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009e80 tx=0x7ff8ec02a7a0 comp rx=0 tx=0).stop 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.385+0000 7ff8f88ee640 1 -- 172.21.15.37:0/177994962 shutdown_connections 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.385+0000 7ff8f88ee640 1 --2- 172.21.15.37:0/177994962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f4107b90 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.385+0000 7ff8f88ee640 1 -- 172.21.15.37:0/177994962 >> 172.21.15.37:0/177994962 conn(0x7ff8f4102fa0 msgr2=0x7ff8f41053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.385+0000 7ff8f88ee640 1 -- 172.21.15.37:0/177994962 shutdown_connections 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.385+0000 7ff8f88ee640 1 -- 172.21.15.37:0/177994962 wait complete. 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f88ee640 1 Processor -- start 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f88ee640 1 -- start start 2024-09-19T06:32:25.220 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f88ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f88ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f410bcf0 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34832/0 (socket says 172.21.15.37:34832) 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f259c640 1 -- 172.21.15.37:0/1837166631 learned_addr learned my addr 172.21.15.37:0/1837166631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.386+0000 7ff8f259c640 1 -- 172.21.15.37:0/1837166631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f419deb0 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.387+0000 7ff8f259c640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f419d970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02acc0 tx=0x7ff8ec031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.387+0000 7ff8eb7fe640 1 -- 172.21.15.37:0/1837166631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec036050 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.387+0000 7ff8eb7fe640 1 -- 172.21.15.37:0/1837166631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ff8ec037040 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.387+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f419e140 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.387+0000 7ff8eb7fe640 1 -- 172.21.15.37:0/1837166631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff8ec03d8c0 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.387+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41a0e30 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.388+0000 7ff8eb7fe640 1 -- 172.21.15.37:0/1837166631 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7ff8ec044020 con 0x7ff8f4107790 2024-09-19T06:32:25.221 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.388+0000 7ff8eb7fe640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7ff8e00390a0 0x7ff8e003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.388+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f410be70 con 0x7ff8f4107790 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.388+0000 7ff8eb7fe640 1 -- 172.21.15.37:0/1837166631 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff8ec042020 con 0x7ff8f4107790 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.388+0000 7ff8f1d9b640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7ff8e00390a0 0x7ff8e003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.389+0000 7ff8f1d9b640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7ff8e00390a0 0x7ff8e003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0098c0 tx=0x7ff8dc006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.393+0000 7ff8eb7fe640 1 -- 172.21.15.37:0/1837166631 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff8ec04b020 con 0x7ff8f4107790 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.645+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ff8f41a1470 con 0x7ff8f4107790 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.645+0000 7ff8eb7fe640 1 -- 172.21.15.37:0/1837166631 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7ff8ec0072f0 con 0x7ff8f4107790 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7ff8e00390a0 msgr2=0x7ff8e003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7ff8e00390a0 0x7ff8e003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff8dc0098c0 tx=0x7ff8dc006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 msgr2=0x7ff8f419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f419d970 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec02acc0 tx=0x7ff8ec031ca0 comp rx=0 tx=0).stop 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 shutdown_connections 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7ff8e00390a0 0x7ff8e003b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:25.222 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 --2- 172.21.15.37:0/1837166631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8f4107790 0x7ff8f419d970 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:25.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 >> 172.21.15.37:0/1837166631 conn(0x7ff8f4102fa0 msgr2=0x7ff8f4103ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:25.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 shutdown_connections 2024-09-19T06:32:25.223 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:24.648+0000 7ff8f88ee640 1 -- 172.21.15.37:0/1837166631 wait complete. 2024-09-19T06:32:25.223 INFO:teuthology.orchestra.run.smithi037.stdout:mgr is available 2024-09-19T06:32:26.108 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:25 smithi037 ceph-mon[31211]: mgrmap e4: x(active, since 2s) 2024-09-19T06:32:26.108 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:25 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3577256918' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-19T06:32:26.401 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [global] 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout fsid = b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.37:3300,v1:172.21.15.37:6789] 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 2024-09-19T06:32:26.402 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout [osd] 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.616+0000 7fc679dfe640 1 Processor -- start 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.616+0000 7fc679dfe640 1 -- start start 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.617+0000 7fc679dfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67410a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.617+0000 7fc679dfe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc67407bd70 con 0x7fc674108270 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.617+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67410a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.617+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67410a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34846/0 (socket says 172.21.15.37:34846) 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.617+0000 7fc6737fe640 1 -- 172.21.15.37:0/3566290404 learned_addr learned my addr 172.21.15.37:0/3566290404 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:26.403 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.617+0000 7fc6737fe640 1 -- 172.21.15.37:0/3566290404 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67410aba0 con 0x7fc674108270 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.618+0000 7fc6737fe640 1 --2- 172.21.15.37:0/3566290404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67410a660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc664009820 tx=0x7fc66402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9791b1dd5e24298b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.618+0000 7fc6727fc640 1 -- 172.21.15.37:0/3566290404 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc664037070 con 0x7fc674108270 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.618+0000 7fc6727fc640 1 -- 172.21.15.37:0/3566290404 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc664032900 con 0x7fc674108270 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.618+0000 7fc6727fc640 1 -- 172.21.15.37:0/3566290404 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc664030850 con 0x7fc674108270 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.618+0000 7fc679dfe640 1 -- 172.21.15.37:0/3566290404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 msgr2=0x7fc67410a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.618+0000 7fc679dfe640 1 --2- 172.21.15.37:0/3566290404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67410a660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fc664009820 tx=0x7fc66402a270 comp rx=0 tx=0).stop 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.619+0000 7fc679dfe640 1 -- 172.21.15.37:0/3566290404 shutdown_connections 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.619+0000 7fc679dfe640 1 --2- 172.21.15.37:0/3566290404 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67410a660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.619+0000 7fc679dfe640 1 -- 172.21.15.37:0/3566290404 >> 172.21.15.37:0/3566290404 conn(0x7fc674101db0 msgr2=0x7fc674104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.619+0000 7fc679dfe640 1 -- 172.21.15.37:0/3566290404 shutdown_connections 2024-09-19T06:32:26.404 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.619+0000 7fc679dfe640 1 -- 172.21.15.37:0/3566290404 wait complete. 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.619+0000 7fc679dfe640 1 Processor -- start 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.619+0000 7fc679dfe640 1 -- start start 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc679dfe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc679dfe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc67410bb20 con 0x7fc674108270 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc6737fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67419b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34862/0 (socket says 172.21.15.37:34862) 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc6737fe640 1 -- 172.21.15.37:0/3577256918 learned_addr learned my addr 172.21.15.37:0/3577256918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc6737fe640 1 -- 172.21.15.37:0/3577256918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc67419bc70 con 0x7fc674108270 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc6737fe640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67419b730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc664031000 tx=0x7fc6640326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc670ff9640 1 -- 172.21.15.37:0/3577256918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc664042070 con 0x7fc674108270 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc670ff9640 1 -- 172.21.15.37:0/3577256918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc664032be0 con 0x7fc674108270 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc67419bf00 con 0x7fc674108270 2024-09-19T06:32:26.405 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc670ff9640 1 -- 172.21.15.37:0/3577256918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc664037040 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.620+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc67419ebf0 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.621+0000 7fc670ff9640 1 -- 172.21.15.37:0/3577256918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7fc664045070 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.622+0000 7fc670ff9640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7fc65c039160 0x7fc65c03b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.622+0000 7fc670ff9640 1 -- 172.21.15.37:0/3577256918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc66406c130 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.622+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6741086d0 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.622+0000 7fc672ffd640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7fc65c039160 0x7fc65c03b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.622+0000 7fc672ffd640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7fc65c039160 0x7fc65c03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6680098c0 tx=0x7fc668006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.627+0000 7fc670ff9640 1 -- 172.21.15.37:0/3577256918 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc6640328a0 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.801+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc6741a02e0 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.802+0000 7fc670ff9640 1 -- 172.21.15.37:0/3577256918 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7fc664032d90 con 0x7fc674108270 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7fc65c039160 msgr2=0x7fc65c03b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:26.406 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7fc65c039160 0x7fc65c03b620 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc6680098c0 tx=0x7fc668006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 msgr2=0x7fc67419b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67419b730 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc664031000 tx=0x7fc6640326e0 comp rx=0 tx=0).stop 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 shutdown_connections 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7fc65c039160 0x7fc65c03b620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 --2- 172.21.15.37:0/3577256918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc674108270 0x7fc67419b730 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 >> 172.21.15.37:0/3577256918 conn(0x7fc674101db0 msgr2=0x7fc674102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 shutdown_connections 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:25.804+0000 7fc679dfe640 1 -- 172.21.15.37:0/3577256918 wait complete. 2024-09-19T06:32:26.407 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling cephadm module... 2024-09-19T06:32:27.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1900954104' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-19T06:32:27.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setuser ceph since I am not root 2024-09-19T06:32:27.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setgroup ceph since I am not root 2024-09-19T06:32:27.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:27.213+0000 7f93ac49b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:32:27.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:27.272+0000 7f93ac49b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:32:27.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:27.426+0000 7f93ac49b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:32:27.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:27.495+0000 7f93ac49b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:32:27.710 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.785+0000 7f09d0f4f640 1 Processor -- start 2024-09-19T06:32:27.710 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.785+0000 7f09d0f4f640 1 -- start start 2024-09-19T06:32:27.710 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.786+0000 7f09d0f4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc10a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.786+0000 7f09d0f4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc07be80 con 0x7f09cc108170 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.786+0000 7f09ca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc10a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.786+0000 7f09ca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc10a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34876/0 (socket says 172.21.15.37:34876) 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.786+0000 7f09ca59c640 1 -- 172.21.15.37:0/836601979 learned_addr learned my addr 172.21.15.37:0/836601979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.786+0000 7f09ca59c640 1 -- 172.21.15.37:0/836601979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc10aaa0 con 0x7f09cc108170 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.787+0000 7f09ca59c640 1 --2- 172.21.15.37:0/836601979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc10a560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009e80 tx=0x7f09bc02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508e2dac0a7990f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.787+0000 7f09c959a640 1 -- 172.21.15.37:0/836601979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09bc036070 con 0x7f09cc108170 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.787+0000 7f09c959a640 1 -- 172.21.15.37:0/836601979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09bc0318c0 con 0x7f09cc108170 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.787+0000 7f09c959a640 1 -- 172.21.15.37:0/836601979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09bc007ab0 con 0x7f09cc108170 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 -- 172.21.15.37:0/836601979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 msgr2=0x7f09cc10a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 --2- 172.21.15.37:0/836601979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc10a560 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f09bc009e80 tx=0x7f09bc02a7a0 comp rx=0 tx=0).stop 2024-09-19T06:32:27.711 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 -- 172.21.15.37:0/836601979 shutdown_connections 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 --2- 172.21.15.37:0/836601979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc10a560 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 -- 172.21.15.37:0/836601979 >> 172.21.15.37:0/836601979 conn(0x7f09cc101db0 msgr2=0x7f09cc1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 -- 172.21.15.37:0/836601979 shutdown_connections 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 -- 172.21.15.37:0/836601979 wait complete. 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.788+0000 7f09d0f4f640 1 Processor -- start 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09d0f4f640 1 -- start start 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09d0f4f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc19b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09d0f4f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09cc10bb40 con 0x7f09cc108170 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09ca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc19b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09ca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc19b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34882/0 (socket says 172.21.15.37:34882) 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09ca59c640 1 -- 172.21.15.37:0/1900954104 learned_addr learned my addr 172.21.15.37:0/1900954104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:27.712 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09ca59c640 1 -- 172.21.15.37:0/1900954104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09cc19bca0 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09ca59c640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc19b760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f09bc031ca0 tx=0x7f09bc031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09bc036050 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09cc19bf30 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09cc19ec20 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f09bc042070 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.789+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09bc0478a0 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.790+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f09bc03d450 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.790+0000 7f09bb7fe640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f09b4039160 0x7f09b403b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.791+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f09bc007e00 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.791+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09ac0052c0 con 0x7f09cc108170 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.791+0000 7f09c9d9b640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f09b4039160 0x7f09b403b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:27.713 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.791+0000 7f09c9d9b640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f09b4039160 0x7f09b403b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09c00098c0 tx=0x7f09c0006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:26.796+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f09bc0467c0 con 0x7f09cc108170 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.019+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f09ac0055b0 con 0x7f09cc108170 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.071+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f09bc06d040 con 0x7f09cc108170 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.073+0000 7f09bb7fe640 1 -- 172.21.15.37:0/1900954104 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f09bc04dcd0 con 0x7f09cc108170 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f09b4039160 msgr2=0x7f09b403b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f09b4039160 0x7f09b403b620 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f09c00098c0 tx=0x7f09c0006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 msgr2=0x7f09cc19b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc19b760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f09bc031ca0 tx=0x7f09bc031cd0 comp rx=0 tx=0).stop 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 shutdown_connections 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f09b4039160 0x7f09b403b620 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 --2- 172.21.15.37:0/1900954104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09cc108170 0x7f09cc19b760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 >> 172.21.15.37:0/1900954104 conn(0x7f09cc101db0 msgr2=0x7f09cc102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 shutdown_connections 2024-09-19T06:32:27.714 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:27.074+0000 7f09d0f4f640 1 -- 172.21.15.37:0/1900954104 wait complete. 2024-09-19T06:32:27.950 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:27.649+0000 7f93ac49b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:32:27.950 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:27.799+0000 7f93ac49b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:32:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:28 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1900954104' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-19T06:32:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:28 smithi037 ceph-mon[31211]: mgrmap e5: x(active, since 4s) 2024-09-19T06:32:28.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:28.133+0000 7f93ac49b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:32:28.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:28.197+0000 7f93ac49b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:32:28.662 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:28.397+0000 7f93ac49b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:32:28.946 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:32:28.946 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-19T06:32:28.946 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:32:28.946 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.122+0000 7f33c02e8640 1 Processor -- start 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.122+0000 7f33c02e8640 1 -- start start 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.123+0000 7f33c02e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b810a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.123+0000 7f33c02e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33b807c690 con 0x7f33b8108250 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.123+0000 7f33be084640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b810a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.123+0000 7f33be084640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b810a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34910/0 (socket says 172.21.15.37:34910) 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.123+0000 7f33be084640 1 -- 172.21.15.37:0/2951159592 learned_addr learned my addr 172.21.15.37:0/2951159592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.123+0000 7f33be084640 1 -- 172.21.15.37:0/2951159592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b810ab80 con 0x7f33b8108250 2024-09-19T06:32:28.947 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.124+0000 7f33be084640 1 --2- 172.21.15.37:0/2951159592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b810a640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009820 tx=0x7f33b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34f33b83c5fd67c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:28.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.124+0000 7f33bd082640 1 -- 172.21.15.37:0/2951159592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b4037070 con 0x7f33b8108250 2024-09-19T06:32:28.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.124+0000 7f33bd082640 1 -- 172.21.15.37:0/2951159592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33b4032900 con 0x7f33b8108250 2024-09-19T06:32:28.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.124+0000 7f33bd082640 1 -- 172.21.15.37:0/2951159592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b4030850 con 0x7f33b8108250 2024-09-19T06:32:28.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 -- 172.21.15.37:0/2951159592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 msgr2=0x7f33b810a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:28.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 --2- 172.21.15.37:0/2951159592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b810a640 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009820 tx=0x7f33b402a270 comp rx=0 tx=0).stop 2024-09-19T06:32:28.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 -- 172.21.15.37:0/2951159592 shutdown_connections 2024-09-19T06:32:28.948 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 --2- 172.21.15.37:0/2951159592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b810a640 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 -- 172.21.15.37:0/2951159592 >> 172.21.15.37:0/2951159592 conn(0x7f33b8101e50 msgr2=0x7f33b8104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 -- 172.21.15.37:0/2951159592 shutdown_connections 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 -- 172.21.15.37:0/2951159592 wait complete. 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.125+0000 7f33c02e8640 1 Processor -- start 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33c02e8640 1 -- start start 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33c02e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33c02e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f33b810bb60 con 0x7f33b8108250 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33be084640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33be084640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34916/0 (socket says 172.21.15.37:34916) 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33be084640 1 -- 172.21.15.37:0/1682871907 learned_addr learned my addr 172.21.15.37:0/1682871907 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33be084640 1 -- 172.21.15.37:0/1682871907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b819bcb0 con 0x7f33b8108250 2024-09-19T06:32:28.949 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33be084640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b819b770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33b4032ce0 tx=0x7f33b4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.126+0000 7f33af7fe640 1 -- 172.21.15.37:0/1682871907 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b4042070 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.127+0000 7f33af7fe640 1 -- 172.21.15.37:0/1682871907 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f33b40308c0 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.127+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b819bf40 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.127+0000 7f33af7fe640 1 -- 172.21.15.37:0/1682871907 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f33b4037040 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.127+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b819ec30 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.128+0000 7f33af7fe640 1 -- 172.21.15.37:0/1682871907 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f33b4045070 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.128+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b81086b0 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.128+0000 7f33af7fe640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 0x7f33a403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.128+0000 7f33af7fe640 1 -- 172.21.15.37:0/1682871907 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f33b406c250 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.128+0000 7f33bd883640 1 -- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 msgr2=0x7f33a403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2963797247 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.128+0000 7f33bd883640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 0x7f33a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.133+0000 7f33af7fe640 1 -- 172.21.15.37:0/1682871907 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f33b404b170 con 0x7f33b8108250 2024-09-19T06:32:28.950 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.329+0000 7f33bd883640 1 -- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 msgr2=0x7f33a403b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2963797247 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.329+0000 7f33bd883640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 0x7f33a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.357+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f33b819f2b0 con 0x7f33b8108250 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.358+0000 7f33af7fe640 1 -- 172.21.15.37:0/1682871907 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f33b4045520 con 0x7f33b8108250 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 msgr2=0x7f33a403b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 0x7f33a403b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 msgr2=0x7f33b819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b819b770 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f33b4032ce0 tx=0x7f33b4032d10 comp rx=0 tx=0).stop 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 shutdown_connections 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f33a4039200 0x7f33a403b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 --2- 172.21.15.37:0/1682871907 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33b8108250 0x7f33b819b770 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 >> 172.21.15.37:0/1682871907 conn(0x7f33b8101e50 msgr2=0x7f33b8102a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 shutdown_connections 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:28.360+0000 7f33c02e8640 1 -- 172.21.15.37:0/1682871907 wait complete. 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-09-19T06:32:28.951 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 5... 2024-09-19T06:32:29.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:29 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1682871907' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T06:32:29.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:29.331+0000 7f93ac49b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:32:29.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:29.402+0000 7f93ac49b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:32:30.267 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.144+0000 7f93ac49b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:32:30.267 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.265+0000 7f93ac49b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:32:30.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.413+0000 7f93ac49b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:32:30.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.477+0000 7f93ac49b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:32:30.933 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.633+0000 7f93ac49b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:32:30.934 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.704+0000 7f93ac49b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:32:30.934 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.766+0000 7f93ac49b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:32:30.934 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.931+0000 7f93ac49b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:32:31.219 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:30.999+0000 7f93ac49b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:32:31.219 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:31.069+0000 7f93ac49b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:32:31.219 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:31.155+0000 7f93ac49b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:32:31.219 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:31.217+0000 7f93ac49b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:32:31.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:31.278+0000 7f93ac49b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:32:32.372 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:32.105+0000 7f93ac49b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: Active manager daemon x restarted 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: Activating manager daemon x 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: osdmap e2: 0 total, 0 up, 0 in 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: mgrmap e6: x(active, starting, since 0.00390554s) 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: Manager daemon x is now available 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:32.373 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:32.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:32.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:32:32.374 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:32 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:32:33.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:32:33.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-19T06:32:33.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T06:32:33.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:32:33.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.403+0000 7f1e880eb640 1 Processor -- start 2024-09-19T06:32:33.578 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.403+0000 7f1e880eb640 1 -- start start 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.403+0000 7f1e880eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8010a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.403+0000 7f1e880eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8007bdd0 con 0x7f1e80108210 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.404+0000 7f1e85e87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8010a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.404+0000 7f1e85e87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8010a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34920/0 (socket says 172.21.15.37:34920) 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.404+0000 7f1e85e87640 1 -- 172.21.15.37:0/3221755562 learned_addr learned my addr 172.21.15.37:0/3221755562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.404+0000 7f1e85e87640 1 -- 172.21.15.37:0/3221755562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8010ab40 con 0x7f1e80108210 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.404+0000 7f1e85e87640 1 --2- 172.21.15.37:0/3221755562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8010a600 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009e80 tx=0x7f1e7002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10ea7912dfbc0b7f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e84e85640 1 -- 172.21.15.37:0/3221755562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e70036070 con 0x7f1e80108210 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e84e85640 1 -- 172.21.15.37:0/3221755562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e700318c0 con 0x7f1e80108210 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e84e85640 1 -- 172.21.15.37:0/3221755562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e70007ab0 con 0x7f1e80108210 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e880eb640 1 -- 172.21.15.37:0/3221755562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 msgr2=0x7f1e8010a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e880eb640 1 --2- 172.21.15.37:0/3221755562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8010a600 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009e80 tx=0x7f1e7002a7a0 comp rx=0 tx=0).stop 2024-09-19T06:32:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e880eb640 1 -- 172.21.15.37:0/3221755562 shutdown_connections 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e880eb640 1 --2- 172.21.15.37:0/3221755562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8010a600 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.405+0000 7f1e880eb640 1 -- 172.21.15.37:0/3221755562 >> 172.21.15.37:0/3221755562 conn(0x7f1e80101db0 msgr2=0x7f1e80104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e880eb640 1 -- 172.21.15.37:0/3221755562 shutdown_connections 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e880eb640 1 -- 172.21.15.37:0/3221755562 wait complete. 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e880eb640 1 Processor -- start 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e880eb640 1 -- start start 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e880eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8019d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e880eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e8010bb20 con 0x7f1e80108210 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e85e87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8019d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e85e87640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8019d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34932/0 (socket says 172.21.15.37:34932) 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.406+0000 7f1e85e87640 1 -- 172.21.15.37:0/4182094922 learned_addr learned my addr 172.21.15.37:0/4182094922 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.407+0000 7f1e85e87640 1 -- 172.21.15.37:0/4182094922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e8019ded0 con 0x7f1e80108210 2024-09-19T06:32:33.580 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.407+0000 7f1e85e87640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8019d990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e70031ca0 tx=0x7f1e70031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.407+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e70036050 con 0x7f1e80108210 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.407+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1e70042070 con 0x7f1e80108210 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.407+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e8019e160 con 0x7f1e80108210 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.407+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1e7003d940 con 0x7f1e80108210 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.407+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e801a0e50 con 0x7f1e80108210 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.408+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f1e7003dc40 con 0x7f1e80108210 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.408+0000 7f1e7effd640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 0x7f1e6803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.408+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 --> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1e6803bd80 con 0x7f1e680391b0 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.408+0000 7f1e85686640 1 -- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 msgr2=0x7f1e6803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2963797247 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.408+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1e70007e00 con 0x7f1e80108210 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.408+0000 7f1e85686640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 0x7f1e6803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:32:33.581 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.609+0000 7f1e85686640 1 -- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 msgr2=0x7f1e6803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2963797247 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:29.609+0000 7f1e85686640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 0x7f1e6803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:30.010+0000 7f1e85686640 1 -- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 msgr2=0x7f1e6803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2963797247 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:30.010+0000 7f1e85686640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 0x7f1e6803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:30.811+0000 7f1e85686640 1 -- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 msgr2=0x7f1e6803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2963797247 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:30.811+0000 7f1e85686640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 0x7f1e6803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:32.109+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7f1e7004dcf0 con 0x7f1e80108210 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:32.109+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 msgr2=0x7f1e6803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:32.109+0000 7f1e7effd640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 0x7f1e6803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.110+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f1e70047460 con 0x7f1e80108210 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.110+0000 7f1e7effd640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f1e6803cc00 0x7f1e6803eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.110+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1e6803bd80 con 0x7f1e6803cc00 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.110+0000 7f1e85686640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f1e6803cc00 0x7f1e6803eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:33.582 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.111+0000 7f1e85686640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f1e6803cc00 0x7f1e6803eff0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003980 tx=0x7f1e74006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.112+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f1e6803bd80 con 0x7f1e6803cc00 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f1e80108670 con 0x7f1e6803cc00 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e7effd640 1 -- 172.21.15.37:0/4182094922 <== mgr.14118 v2:172.21.15.37:6800/2601148313 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f1e80108670 con 0x7f1e6803cc00 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f1e6803cc00 msgr2=0x7f1e6803eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f1e6803cc00 0x7f1e6803eff0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f1e74003980 tx=0x7f1e74006e10 comp rx=0 tx=0).stop 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 msgr2=0x7f1e8019d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8019d990 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1e70031ca0 tx=0x7f1e70031cd0 comp rx=0 tx=0).stop 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 shutdown_connections 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f1e6803cc00 0x7f1e6803eff0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:6800/2963797247,v1:172.21.15.37:6801/2963797247] conn(0x7f1e680391b0 0x7f1e6803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 --2- 172.21.15.37:0/4182094922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e80108210 0x7f1e8019d990 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 >> 172.21.15.37:0/4182094922 conn(0x7f1e80101db0 msgr2=0x7f1e801029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:33.583 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 shutdown_connections 2024-09-19T06:32:33.584 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.119+0000 7f1e880eb640 1 -- 172.21.15.37:0/4182094922 wait complete. 2024-09-19T06:32:33.584 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 5 is available 2024-09-19T06:32:33.584 INFO:teuthology.orchestra.run.smithi037.stdout:Setting orchestrator backend to cephadm... 2024-09-19T06:32:34.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:33 smithi037 ceph-mon[31211]: Found migration_current of "None". Setting to last migration. 2024-09-19T06:32:34.156 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:33 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:33 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:33 smithi037 ceph-mon[31211]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:33 smithi037 ceph-mon[31211]: mgrmap e7: x(active, since 1.00529s) 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:33 smithi037 ceph-mon[31211]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:34.006+0000 7f9319000640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:06:32:34] ENGINE Error in HTTPServer.serve 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: Traceback (most recent call last): 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._connections.run(self.expiration_interval) 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:32:34.157 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._run(expiration_interval) 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: s = self.context.wrap_socket( 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: return self.sslsocket_class._create( 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self.do_handshake() 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._sslobj.do_handshake() 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:32:34.158 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:34.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.952+0000 7fd05840d640 1 Processor -- start 2024-09-19T06:32:34.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.952+0000 7fd05840d640 1 -- start start 2024-09-19T06:32:34.719 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.952+0000 7fd05840d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050106a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.952+0000 7fd05840d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd050106fd0 con 0x7fd050106600 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.953+0000 7fd0561a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050106a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.953+0000 7fd0561a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050106a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57428/0 (socket says 172.21.15.37:57428) 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.953+0000 7fd0561a9640 1 -- 172.21.15.37:0/2245631788 learned_addr learned my addr 172.21.15.37:0/2245631788 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.953+0000 7fd0561a9640 1 -- 172.21.15.37:0/2245631788 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd050107800 con 0x7fd050106600 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd0561a9640 1 --2- 172.21.15.37:0/2245631788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050106a00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009820 tx=0x7fd04c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df65a29795b15f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd0551a7640 1 -- 172.21.15.37:0/2245631788 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd04c037070 con 0x7fd050106600 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd0551a7640 1 -- 172.21.15.37:0/2245631788 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd04c032900 con 0x7fd050106600 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd0551a7640 1 -- 172.21.15.37:0/2245631788 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd04c030850 con 0x7fd050106600 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd05840d640 1 -- 172.21.15.37:0/2245631788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 msgr2=0x7fd050106a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd05840d640 1 --2- 172.21.15.37:0/2245631788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050106a00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009820 tx=0x7fd04c02a270 comp rx=0 tx=0).stop 2024-09-19T06:32:34.720 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd05840d640 1 -- 172.21.15.37:0/2245631788 shutdown_connections 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd05840d640 1 --2- 172.21.15.37:0/2245631788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050106a00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.954+0000 7fd05840d640 1 -- 172.21.15.37:0/2245631788 >> 172.21.15.37:0/2245631788 conn(0x7fd050101db0 msgr2=0x7fd0501041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd05840d640 1 -- 172.21.15.37:0/2245631788 shutdown_connections 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd05840d640 1 -- 172.21.15.37:0/2245631788 wait complete. 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd05840d640 1 Processor -- start 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd05840d640 1 -- start start 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd05840d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050191830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd05840d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd05010abb0 con 0x7fd050106600 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd0561a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050191830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd0561a9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050191830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57440/0 (socket says 172.21.15.37:57440) 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.955+0000 7fd0561a9640 1 -- 172.21.15.37:0/888867740 learned_addr learned my addr 172.21.15.37:0/888867740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.956+0000 7fd0561a9640 1 -- 172.21.15.37:0/888867740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd050191d70 con 0x7fd050106600 2024-09-19T06:32:34.721 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.956+0000 7fd0561a9640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050191830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009940 tx=0x7fd04c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.956+0000 7fd0477fe640 1 -- 172.21.15.37:0/888867740 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd04c042070 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.956+0000 7fd0477fe640 1 -- 172.21.15.37:0/888867740 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd04c032be0 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.956+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd05018ff60 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.956+0000 7fd0477fe640 1 -- 172.21.15.37:0/888867740 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd04c037040 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.956+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0501904a0 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.957+0000 7fd0477fe640 1 -- 172.21.15.37:0/888867740 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fd04c045070 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.957+0000 7fd0477fe640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7fd03c0390f0 0x7fd03c03b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.957+0000 7fd0477fe640 1 -- 172.21.15.37:0/888867740 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd04c06c040 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.957+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd05010ad30 con 0x7fd050106600 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.957+0000 7fd0559a8640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7fd03c0390f0 0x7fd03c03b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.958+0000 7fd0559a8640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7fd03c0390f0 0x7fd03c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098c0 tx=0x7fd040006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:34.722 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:33.962+0000 7fd0477fe640 1 -- 172.21.15.37:0/888867740 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd04c045360 con 0x7fd050106600 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.154+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7fd050195610 con 0x7fd03c0390f0 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.159+0000 7fd0477fe640 1 -- 172.21.15.37:0/888867740 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd050195610 con 0x7fd03c0390f0 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7fd03c0390f0 msgr2=0x7fd03c03b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7fd03c0390f0 0x7fd03c03b5b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd0400098c0 tx=0x7fd040006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 msgr2=0x7fd050191830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050191830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd04c009940 tx=0x7fd04c0326e0 comp rx=0 tx=0).stop 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 shutdown_connections 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7fd03c0390f0 0x7fd03c03b5b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 --2- 172.21.15.37:0/888867740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd050106600 0x7fd050191830 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 >> 172.21.15.37:0/888867740 conn(0x7fd050101db0 msgr2=0x7fd050102d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 shutdown_connections 2024-09-19T06:32:34.723 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:34.161+0000 7fd05840d640 1 -- 172.21.15.37:0/888867740 wait complete. 2024-09-19T06:32:34.974 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:33] ENGINE Bus STARTING 2024-09-19T06:32:34.974 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:34] ENGINE Serving on https://172.21.15.37:7150 2024-09-19T06:32:34.974 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:34] ENGINE Error in HTTPServer.serve 2024-09-19T06:32:34.974 INFO:journalctl@ceph.mon.a.smithi037.stdout: Traceback (most recent call last): 2024-09-19T06:32:34.974 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._run(expiration_interval) 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: s = self.context.wrap_socket( 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: return self.sslsocket_class._create( 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: self.do_handshake() 2024-09-19T06:32:34.975 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:32:34.976 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._sslobj.do_handshake() 2024-09-19T06:32:34.976 INFO:journalctl@ceph.mon.a.smithi037.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:32:34.976 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:34] ENGINE Serving on http://172.21.15.37:8765 2024-09-19T06:32:34.976 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:34] ENGINE Bus STARTED 2024-09-19T06:32:34.976 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:34.976 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:34.976 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:34 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.145+0000 7f2747098640 1 Processor -- start 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.145+0000 7f2747098640 1 -- start start 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.146+0000 7f2747098640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f2740107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.146+0000 7f2747098640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2740108170 con 0x7f27401077a0 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.146+0000 7f2744e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f2740107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.146+0000 7f2744e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f2740107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57450/0 (socket says 172.21.15.37:57450) 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.146+0000 7f2744e34640 1 -- 172.21.15.37:0/309390621 learned_addr learned my addr 172.21.15.37:0/309390621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:35.818 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.146+0000 7f2744e34640 1 -- 172.21.15.37:0/309390621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2740108950 con 0x7f27401077a0 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.147+0000 7f2744e34640 1 --2- 172.21.15.37:0/309390621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f2740107ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2734009820 tx=0x7f273402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db084d989026589 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.147+0000 7f273f7fe640 1 -- 172.21.15.37:0/309390621 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2734037070 con 0x7f27401077a0 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.147+0000 7f273f7fe640 1 -- 172.21.15.37:0/309390621 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2734032900 con 0x7f27401077a0 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.147+0000 7f273f7fe640 1 -- 172.21.15.37:0/309390621 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2734030850 con 0x7f27401077a0 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 -- 172.21.15.37:0/309390621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 msgr2=0x7f2740107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 --2- 172.21.15.37:0/309390621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f2740107ba0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2734009820 tx=0x7f273402a270 comp rx=0 tx=0).stop 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 -- 172.21.15.37:0/309390621 shutdown_connections 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 --2- 172.21.15.37:0/309390621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f2740107ba0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 -- 172.21.15.37:0/309390621 >> 172.21.15.37:0/309390621 conn(0x7f2740102fd0 msgr2=0x7f27401053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 -- 172.21.15.37:0/309390621 shutdown_connections 2024-09-19T06:32:35.819 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 -- 172.21.15.37:0/309390621 wait complete. 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 Processor -- start 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.148+0000 7f2747098640 1 -- start start 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2747098640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f274007e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2747098640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f274010bd00 con 0x7f27401077a0 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2744e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f274007e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2744e34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f274007e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57452/0 (socket says 172.21.15.37:57452) 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2744e34640 1 -- 172.21.15.37:0/2858718379 learned_addr learned my addr 172.21.15.37:0/2858718379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2744e34640 1 -- 172.21.15.37:0/2858718379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f274007e910 con 0x7f27401077a0 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2744e34640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f274007e3d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27340097f0 tx=0x7f2734030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f273dffb640 1 -- 172.21.15.37:0/2858718379 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2734037040 con 0x7f27401077a0 2024-09-19T06:32:35.820 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f274007cb00 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f273dffb640 1 -- 172.21.15.37:0/2858718379 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2734040070 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.149+0000 7f273dffb640 1 -- 172.21.15.37:0/2858718379 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f273403a390 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.150+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f274007cfe0 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.150+0000 7f273dffb640 1 -- 172.21.15.37:0/2858718379 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f273403a630 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.151+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2740107c20 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.151+0000 7f273dffb640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f27300391b0 0x7f273003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.151+0000 7f273dffb640 1 -- 172.21.15.37:0/2858718379 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f273406c8f0 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.151+0000 7f273ffff640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f27300391b0 0x7f273003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.151+0000 7f273ffff640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f27300391b0 0x7f273003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f272c0098c0 tx=0x7f272c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.156+0000 7f273dffb640 1 -- 172.21.15.37:0/2858718379 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27340439b0 con 0x7f27401077a0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.334+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f2740104220 con 0x7f27300391b0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.335+0000 7f273dffb640 1 -- 172.21.15.37:0/2858718379 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f2740104220 con 0x7f27300391b0 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f27300391b0 msgr2=0x7f273003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:35.821 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f27300391b0 0x7f273003b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f272c0098c0 tx=0x7f272c006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 msgr2=0x7f274007e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f274007e3d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f27340097f0 tx=0x7f2734030d80 comp rx=0 tx=0).stop 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 shutdown_connections 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f27300391b0 0x7f273003b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 --2- 172.21.15.37:0/2858718379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27401077a0 0x7f274007e3d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 >> 172.21.15.37:0/2858718379 conn(0x7f2740102fd0 msgr2=0x7f2740103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 shutdown_connections 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:35.337+0000 7f2747098640 1 -- 172.21.15.37:0/2858718379 wait complete. 2024-09-19T06:32:35.822 INFO:teuthology.orchestra.run.smithi037.stdout:Generating ssh key... 2024-09-19T06:32:36.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:35 smithi037 ceph-mon[31211]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:36.113 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:35 smithi037 ceph-mon[31211]: mgrmap e8: x(active, since 2s) 2024-09-19T06:32:36.775 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: Generating public/private rsa key pair. 2024-09-19T06:32:36.775 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: Your identification has been saved in /tmp/tmp6v8uzgif/key 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: Your public key has been saved in /tmp/tmp6v8uzgif/key.pub 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: The key fingerprint is: 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: SHA256:donFhJ4Qf/4fV9ooOG8AWs//QK/hXcyzB97NAgFIwoA ceph-b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: The key's randomart image is: 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: +---[RSA 3072]----+ 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | ..ooo.o. | 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | E oooo. | 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | o..+. | 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | == .. | 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | oS++ .. .| 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | .. .++...*.| 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | oo=o==B| 2024-09-19T06:32:36.776 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | +oBo+B| 2024-09-19T06:32:36.777 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: | .+.+o.| 2024-09-19T06:32:36.777 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: +----[SHA256]-----+ 2024-09-19T06:32:37.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-mon[31211]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:37.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:37.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:36 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:37.207 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.197+0000 7f67239a8640 1 Processor -- start 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.197+0000 7f67239a8640 1 -- start start 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.197+0000 7f67239a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c10a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.197+0000 7f67239a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f671c10a970 con 0x7f671c109fa0 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.197+0000 7f6721744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c10a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.197+0000 7f6721744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c10a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57460/0 (socket says 172.21.15.37:57460) 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.197+0000 7f6721744640 1 -- 172.21.15.37:0/3131785283 learned_addr learned my addr 172.21.15.37:0/3131785283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.198+0000 7f6721744640 1 -- 172.21.15.37:0/3131785283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c10aaf0 con 0x7f671c109fa0 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.198+0000 7f6721744640 1 --2- 172.21.15.37:0/3131785283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c10a3a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6718009820 tx=0x7f671802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974fd89518eb40d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f6713fff640 1 -- 172.21.15.37:0/3131785283 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6718037070 con 0x7f671c109fa0 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f6713fff640 1 -- 172.21.15.37:0/3131785283 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6718032900 con 0x7f671c109fa0 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f6713fff640 1 -- 172.21.15.37:0/3131785283 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6718030850 con 0x7f671c109fa0 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f67239a8640 1 -- 172.21.15.37:0/3131785283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 msgr2=0x7f671c10a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:37.208 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f67239a8640 1 --2- 172.21.15.37:0/3131785283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c10a3a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6718009820 tx=0x7f671802a270 comp rx=0 tx=0).stop 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f67239a8640 1 -- 172.21.15.37:0/3131785283 shutdown_connections 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f67239a8640 1 --2- 172.21.15.37:0/3131785283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c10a3a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f67239a8640 1 -- 172.21.15.37:0/3131785283 >> 172.21.15.37:0/3131785283 conn(0x7f671c07cfe0 msgr2=0x7f671c07d3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f67239a8640 1 -- 172.21.15.37:0/3131785283 shutdown_connections 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.199+0000 7f67239a8640 1 -- 172.21.15.37:0/3131785283 wait complete. 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.200+0000 7f67239a8640 1 Processor -- start 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.200+0000 7f67239a8640 1 -- start start 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.200+0000 7f67239a8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.200+0000 7f67239a8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f671c10df80 con 0x7f671c109fa0 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.200+0000 7f6721744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.200+0000 7f6721744640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c19d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57466/0 (socket says 172.21.15.37:57466) 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f6721744640 1 -- 172.21.15.37:0/1540160595 learned_addr learned my addr 172.21.15.37:0/1540160595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f6721744640 1 -- 172.21.15.37:0/1540160595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f671c19de60 con 0x7f671c109fa0 2024-09-19T06:32:37.209 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f6721744640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c19d920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f67180097f0 tx=0x7f6718030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f67127fc640 1 -- 172.21.15.37:0/1540160595 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6718037040 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f67127fc640 1 -- 172.21.15.37:0/1540160595 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f6718040070 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f671c19e0f0 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f67127fc640 1 -- 172.21.15.37:0/1540160595 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f671803a390 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.201+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f671c1a0de0 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.202+0000 7f67127fc640 1 -- 172.21.15.37:0/1540160595 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f671803b430 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.202+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f671c10a420 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.202+0000 7f67127fc640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f67080391b0 0x7f670803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.202+0000 7f67127fc640 1 -- 172.21.15.37:0/1540160595 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f671806c7a0 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.203+0000 7f6720f43640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f67080391b0 0x7f670803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.203+0000 7f6720f43640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f67080391b0 0x7f670803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098c0 tx=0x7f670c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.207+0000 7f67127fc640 1 -- 172.21.15.37:0/1540160595 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6718032900 con 0x7f671c109fa0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.388+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f671c108500 con 0x7f67080391b0 2024-09-19T06:32:37.210 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.606+0000 7f67127fc640 1 -- 172.21.15.37:0/1540160595 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f671c108500 con 0x7f67080391b0 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f67080391b0 msgr2=0x7f670803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f67080391b0 0x7f670803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f670c0098c0 tx=0x7f670c006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 msgr2=0x7f671c19d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c19d920 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f67180097f0 tx=0x7f6718030d80 comp rx=0 tx=0).stop 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 shutdown_connections 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f67080391b0 0x7f670803b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 --2- 172.21.15.37:0/1540160595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f671c109fa0 0x7f671c19d920 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 >> 172.21.15.37:0/1540160595 conn(0x7f671c07cfe0 msgr2=0x7f671c107df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 shutdown_connections 2024-09-19T06:32:37.211 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:36.608+0000 7f67239a8640 1 -- 172.21.15.37:0/1540160595 wait complete. 2024-09-19T06:32:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:37 smithi037 ceph-mon[31211]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:37 smithi037 ceph-mon[31211]: Generating ssh key... 2024-09-19T06:32:38.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncLbIdA0HBBVCzqYL8XNd9GmNpzpxIVYHuuuV79R3Z/FSIm71/ca45Z425fjVKCX2vt+UaHbcP+kBvnuSsfsVrcXl8VO7sKObdQyfkcjmwZ9sLgIcGM3ekw0+bKjaSBef4FPvtP1BPYXuOySZMO55u4cheFjNDC/mV4zpuTVe8qGhpOOSKe9NPiB8QQOk9iW2+kmGz90ra5tuNmhrBpRzafuSgbaOm7U8I5C0JoW8twK8N46v0LCsG9fZBTynnOssHBEKnx8yiQvtGlUY39vI2uud8tfXHwSCC9eQN6bhTL0VN/FqbnviJi9CFeS7nl4figGKR5AHhPMR1K1XPDUJs5Qmp+SQ1rDKrx5Cpa8IWZJSW/8Ss1Wldu+bnKDxUbdcxINl9Nh58JjRxL8m0S+opYvCjf5/OekR92sG1QGQzcTkkQrchS0V02f7P5edC4iLtYz+rMpBjf/g7OBnavzcnr+KnnT7L1C6omELycMjXwj4J6IDoX1+fKTuRsXeyQM= ceph-b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:32:38.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.682+0000 7f97910bb640 1 Processor -- start 2024-09-19T06:32:38.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.682+0000 7f97910bb640 1 -- start start 2024-09-19T06:32:38.456 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.682+0000 7f97910bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.683+0000 7f97910bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f978c108160 con 0x7f978c107790 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.683+0000 7f978ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.683+0000 7f978ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57480/0 (socket says 172.21.15.37:57480) 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.683+0000 7f978ad9d640 1 -- 172.21.15.37:0/1989108346 learned_addr learned my addr 172.21.15.37:0/1989108346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.683+0000 7f978ad9d640 1 -- 172.21.15.37:0/1989108346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c108940 con 0x7f978c107790 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.684+0000 7f978ad9d640 1 --2- 172.21.15.37:0/1989108346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c107b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f977c009e80 tx=0x7f977c02a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a23e7a0ede8b401 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.684+0000 7f9789d9b640 1 -- 172.21.15.37:0/1989108346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f977c036070 con 0x7f978c107790 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.684+0000 7f9789d9b640 1 -- 172.21.15.37:0/1989108346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f977c0318c0 con 0x7f978c107790 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.684+0000 7f9789d9b640 1 -- 172.21.15.37:0/1989108346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f977c007ab0 con 0x7f978c107790 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 -- 172.21.15.37:0/1989108346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 msgr2=0x7f978c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 --2- 172.21.15.37:0/1989108346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c107b90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f977c009e80 tx=0x7f977c02a7a0 comp rx=0 tx=0).stop 2024-09-19T06:32:38.457 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 -- 172.21.15.37:0/1989108346 shutdown_connections 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 --2- 172.21.15.37:0/1989108346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c107b90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 -- 172.21.15.37:0/1989108346 >> 172.21.15.37:0/1989108346 conn(0x7f978c102f20 msgr2=0x7f978c105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 -- 172.21.15.37:0/1989108346 shutdown_connections 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 -- 172.21.15.37:0/1989108346 wait complete. 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.685+0000 7f97910bb640 1 Processor -- start 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.686+0000 7f97910bb640 1 -- start start 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.686+0000 7f97910bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c07db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.686+0000 7f97910bb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f978c10bcf0 con 0x7f978c107790 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.686+0000 7f978ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c07db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.686+0000 7f978ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c07db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57482/0 (socket says 172.21.15.37:57482) 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.686+0000 7f978ad9d640 1 -- 172.21.15.37:0/223566818 learned_addr learned my addr 172.21.15.37:0/223566818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:38.458 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.686+0000 7f978ad9d640 1 -- 172.21.15.37:0/223566818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978c07e080 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.687+0000 7f978ad9d640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c07db40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f977c02acc0 tx=0x7f977c031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.687+0000 7f9773fff640 1 -- 172.21.15.37:0/223566818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f977c036050 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.687+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978c07c270 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.687+0000 7f9773fff640 1 -- 172.21.15.37:0/223566818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f977c037040 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.687+0000 7f9773fff640 1 -- 172.21.15.37:0/223566818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f977c03d8c0 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.687+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f978c07c7b0 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.688+0000 7f9773fff640 1 -- 172.21.15.37:0/223566818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f977c044020 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.688+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f978c10be70 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.688+0000 7f9773fff640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f9774039200 0x7f977403b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.688+0000 7f9773fff640 1 -- 172.21.15.37:0/223566818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f977c042020 con 0x7f978c107790 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.689+0000 7f978a59c640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f9774039200 0x7f977403b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:38.459 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.689+0000 7f978a59c640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f9774039200 0x7f977403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f97800098c0 tx=0x7f9780006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.693+0000 7f9773fff640 1 -- 172.21.15.37:0/223566818 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f977c04b020 con 0x7f978c107790 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.873+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f978c07d040 con 0x7f9774039200 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.874+0000 7f9773fff640 1 -- 172.21.15.37:0/223566818 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f978c07d040 con 0x7f9774039200 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f9774039200 msgr2=0x7f977403b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f9774039200 0x7f977403b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f97800098c0 tx=0x7f9780006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 msgr2=0x7f978c07db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c07db40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f977c02acc0 tx=0x7f977c031ca0 comp rx=0 tx=0).stop 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 shutdown_connections 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f9774039200 0x7f977403b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 --2- 172.21.15.37:0/223566818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f978c107790 0x7f978c07db40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 >> 172.21.15.37:0/223566818 conn(0x7f978c102f20 msgr2=0x7f978c103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:38.460 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 shutdown_connections 2024-09-19T06:32:38.461 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:37.876+0000 7f97910bb640 1 -- 172.21.15.37:0/223566818 wait complete. 2024-09-19T06:32:38.461 INFO:teuthology.orchestra.run.smithi037.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:32:38.461 INFO:teuthology.orchestra.run.smithi037.stdout:Adding key to root@localhost authorized_keys... 2024-09-19T06:32:38.461 INFO:teuthology.orchestra.run.smithi037.stdout:Adding host smithi037... 2024-09-19T06:32:39.230 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:38 smithi037 ceph-mon[31211]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:40.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:39 smithi037 ceph-mon[31211]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:41.273 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:41 smithi037 ceph-mon[31211]: Deploying cephadm binary to smithi037 2024-09-19T06:32:43.472 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Added host 'smithi037' with addr '172.21.15.37' 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.854+0000 7f559fb5e640 1 Processor -- start 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.854+0000 7f559fb5e640 1 -- start start 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.854+0000 7f559fb5e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f5598107b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.854+0000 7f559fb5e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5598108160 con 0x7f5598107790 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.854+0000 7f559d8fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f5598107b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.854+0000 7f559d8fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f5598107b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57490/0 (socket says 172.21.15.37:57490) 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.854+0000 7f559d8fa640 1 -- 172.21.15.37:0/4137091421 learned_addr learned my addr 172.21.15.37:0/4137091421 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.855+0000 7f559d8fa640 1 -- 172.21.15.37:0/4137091421 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5598108940 con 0x7f5598107790 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.855+0000 7f559d8fa640 1 --2- 172.21.15.37:0/4137091421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f5598107b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5588009820 tx=0x7f558802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d8768c693c3e788 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559c8f8640 1 -- 172.21.15.37:0/4137091421 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5588037070 con 0x7f5598107790 2024-09-19T06:32:43.473 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559c8f8640 1 -- 172.21.15.37:0/4137091421 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5588032900 con 0x7f5598107790 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559c8f8640 1 -- 172.21.15.37:0/4137091421 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5588030850 con 0x7f5598107790 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559fb5e640 1 -- 172.21.15.37:0/4137091421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 msgr2=0x7f5598107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559fb5e640 1 --2- 172.21.15.37:0/4137091421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f5598107b90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5588009820 tx=0x7f558802a270 comp rx=0 tx=0).stop 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559fb5e640 1 -- 172.21.15.37:0/4137091421 shutdown_connections 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559fb5e640 1 --2- 172.21.15.37:0/4137091421 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f5598107b90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559fb5e640 1 -- 172.21.15.37:0/4137091421 >> 172.21.15.37:0/4137091421 conn(0x7f5598102fa0 msgr2=0x7f55981053c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559fb5e640 1 -- 172.21.15.37:0/4137091421 shutdown_connections 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.856+0000 7f559fb5e640 1 -- 172.21.15.37:0/4137091421 wait complete. 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.857+0000 7f559fb5e640 1 Processor -- start 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.857+0000 7f559fb5e640 1 -- start start 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.857+0000 7f559fb5e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f559819b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.857+0000 7f559fb5e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f559810bcf0 con 0x7f5598107790 2024-09-19T06:32:43.474 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.857+0000 7f559d8fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f559819b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.857+0000 7f559d8fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f559819b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57506/0 (socket says 172.21.15.37:57506) 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.857+0000 7f559d8fa640 1 -- 172.21.15.37:0/2346322520 learned_addr learned my addr 172.21.15.37:0/2346322520 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.858+0000 7f559d8fa640 1 -- 172.21.15.37:0/2346322520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f559819bc80 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.858+0000 7f559d8fa640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f559819b740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55880097f0 tx=0x7f5588030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.858+0000 7f5596ffd640 1 -- 172.21.15.37:0/2346322520 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5588037040 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.858+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f559819bf10 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.858+0000 7f5596ffd640 1 -- 172.21.15.37:0/2346322520 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5588040070 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.858+0000 7f5596ffd640 1 -- 172.21.15.37:0/2346322520 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f558803a390 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.858+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f559819ec00 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.859+0000 7f5596ffd640 1 -- 172.21.15.37:0/2346322520 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f558803b430 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.859+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5598107c10 con 0x7f5598107790 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.859+0000 7f5596ffd640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f55800391b0 0x7f558003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:43.475 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.859+0000 7f5596ffd640 1 -- 172.21.15.37:0/2346322520 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f558806c750 con 0x7f5598107790 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.859+0000 7f559d0f9640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f55800391b0 0x7f558003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.860+0000 7f559d0f9640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f55800391b0 0x7f558003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f558c0098c0 tx=0x7f558c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:38.864+0000 7f5596ffd640 1 -- 172.21.15.37:0/2346322520 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f558806ecd0 con 0x7f5598107790 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:39.047+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi037", "addr": "172.21.15.37", "target": ["mon-mgr", ""]}) -- 0x7f5598104220 con 0x7f55800391b0 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.916+0000 7f5596ffd640 1 -- 172.21.15.37:0/2346322520 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f5598104220 con 0x7f55800391b0 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f55800391b0 msgr2=0x7f558003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f55800391b0 0x7f558003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f558c0098c0 tx=0x7f558c006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 msgr2=0x7f559819b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f559819b740 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f55880097f0 tx=0x7f5588030d80 comp rx=0 tx=0).stop 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 shutdown_connections 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f55800391b0 0x7f558003b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:43.476 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 --2- 172.21.15.37:0/2346322520 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5598107790 0x7f559819b740 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:43.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 >> 172.21.15.37:0/2346322520 conn(0x7f5598102fa0 msgr2=0x7f5598103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:43.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 shutdown_connections 2024-09-19T06:32:43.477 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:42.919+0000 7f559fb5e640 1 -- 172.21.15.37:0/2346322520 wait complete. 2024-09-19T06:32:43.477 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mon service... 2024-09-19T06:32:44.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:43 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:44.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:43 smithi037 ceph-mon[31211]: Added host smithi037 2024-09-19T06:32:44.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:43 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c9098640 1 Processor -- start 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c9098640 1 -- start start 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c9098640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c4106aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c9098640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43c4107070 con 0x7f43c41066a0 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c4106aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c4106aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56294/0 (socket says 172.21.15.37:56294) 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c2d9d640 1 -- 172.21.15.37:0/2628206848 learned_addr learned my addr 172.21.15.37:0/2628206848 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.875+0000 7f43c2d9d640 1 -- 172.21.15.37:0/2628206848 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c4107850 con 0x7f43c41066a0 2024-09-19T06:32:44.782 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.876+0000 7f43c2d9d640 1 --2- 172.21.15.37:0/2628206848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c4106aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43b400bac0 tx=0x7f43b402c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bec22c3a80e2004 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.876+0000 7f43c1d9b640 1 -- 172.21.15.37:0/2628206848 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b40326d0 con 0x7f43c41066a0 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.876+0000 7f43c1d9b640 1 -- 172.21.15.37:0/2628206848 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f43b4038070 con 0x7f43c41066a0 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.876+0000 7f43c1d9b640 1 -- 172.21.15.37:0/2628206848 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4033d70 con 0x7f43c41066a0 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.877+0000 7f43c9098640 1 -- 172.21.15.37:0/2628206848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 msgr2=0x7f43c4106aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.877+0000 7f43c9098640 1 --2- 172.21.15.37:0/2628206848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c4106aa0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f43b400bac0 tx=0x7f43b402c920 comp rx=0 tx=0).stop 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.877+0000 7f43c9098640 1 -- 172.21.15.37:0/2628206848 shutdown_connections 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.877+0000 7f43c9098640 1 --2- 172.21.15.37:0/2628206848 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c4106aa0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.877+0000 7f43c9098640 1 -- 172.21.15.37:0/2628206848 >> 172.21.15.37:0/2628206848 conn(0x7f43c4101ed0 msgr2=0x7f43c41042f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.877+0000 7f43c9098640 1 -- 172.21.15.37:0/2628206848 shutdown_connections 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.877+0000 7f43c9098640 1 -- 172.21.15.37:0/2628206848 wait complete. 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c9098640 1 Processor -- start 2024-09-19T06:32:44.783 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c9098640 1 -- start start 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c9098640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c419d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c9098640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43c407b9b0 con 0x7f43c41066a0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c419d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c419d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56300/0 (socket says 172.21.15.37:56300) 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c2d9d640 1 -- 172.21.15.37:0/4104736234 learned_addr learned my addr 172.21.15.37:0/4104736234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.878+0000 7f43c2d9d640 1 -- 172.21.15.37:0/4104736234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c419de80 con 0x7f43c41066a0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.879+0000 7f43c2d9d640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c419d940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43b400b2c0 tx=0x7f43b4032910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.879+0000 7f43abfff640 1 -- 172.21.15.37:0/4104736234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b4042070 con 0x7f43c41066a0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.879+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c419e110 con 0x7f43c41066a0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.879+0000 7f43abfff640 1 -- 172.21.15.37:0/4104736234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f43b403e900 con 0x7f43c41066a0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.879+0000 7f43abfff640 1 -- 172.21.15.37:0/4104736234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43b403c950 con 0x7f43c41066a0 2024-09-19T06:32:44.784 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.879+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c41a0e00 con 0x7f43c41066a0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.880+0000 7f43abfff640 1 -- 172.21.15.37:0/4104736234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f43b4045070 con 0x7f43c41066a0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.880+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c4106aa0 con 0x7f43c41066a0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.880+0000 7f43abfff640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f43ac0391b0 0x7f43ac03b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.880+0000 7f43abfff640 1 -- 172.21.15.37:0/4104736234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f43b4030050 con 0x7f43c41066a0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.880+0000 7f43c259c640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f43ac0391b0 0x7f43ac03b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.880+0000 7f43c259c640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f43ac0391b0 0x7f43ac03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43b80098c0 tx=0x7f43b8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:43.885+0000 7f43abfff640 1 -- 172.21.15.37:0/4104736234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f43b404fba0 con 0x7f43c41066a0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.075+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f43c4062630 con 0x7f43ac0391b0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.079+0000 7f43abfff640 1 -- 172.21.15.37:0/4104736234 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f43c4062630 con 0x7f43ac0391b0 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f43ac0391b0 msgr2=0x7f43ac03b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f43ac0391b0 0x7f43ac03b670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f43b80098c0 tx=0x7f43b8006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 msgr2=0x7f43c419d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:44.785 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c419d940 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f43b400b2c0 tx=0x7f43b4032910 comp rx=0 tx=0).stop 2024-09-19T06:32:44.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 shutdown_connections 2024-09-19T06:32:44.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f43ac0391b0 0x7f43ac03b670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:44.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 --2- 172.21.15.37:0/4104736234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43c41066a0 0x7f43c419d940 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:44.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 >> 172.21.15.37:0/4104736234 conn(0x7f43c4101ed0 msgr2=0x7f43c4193f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:44.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 shutdown_connections 2024-09-19T06:32:44.786 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:44.082+0000 7f43c9098640 1 -- 172.21.15.37:0/4104736234 wait complete. 2024-09-19T06:32:44.786 INFO:teuthology.orchestra.run.smithi037.stdout:Deploying unmanaged mgr service... 2024-09-19T06:32:45.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:45 smithi037 ceph-mon[31211]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:45.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:45 smithi037 ceph-mon[31211]: Saving service mon spec with placement count:5 2024-09-19T06:32:45.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:45 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:47.013 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:46 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:47.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:46 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:47.014 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:46 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:47.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-19T06:32:47.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.147+0000 7f35c98db640 1 Processor -- start 2024-09-19T06:32:47.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.147+0000 7f35c98db640 1 -- start start 2024-09-19T06:32:47.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.148+0000 7f35c98db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c410a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:47.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.148+0000 7f35c98db640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35c407be00 con 0x7f35c41081f0 2024-09-19T06:32:47.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.148+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c410a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:47.288 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.148+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c410a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56302/0 (socket says 172.21.15.37:56302) 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.148+0000 7f35c2ffd640 1 -- 172.21.15.37:0/3440484532 learned_addr learned my addr 172.21.15.37:0/3440484532 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.148+0000 7f35c2ffd640 1 -- 172.21.15.37:0/3440484532 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c410ab20 con 0x7f35c41081f0 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.149+0000 7f35c2ffd640 1 --2- 172.21.15.37:0/3440484532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c410a5e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009820 tx=0x7f35b402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=661ca0aaaa7f61c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.149+0000 7f35c27fc640 1 -- 172.21.15.37:0/3440484532 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35b4037070 con 0x7f35c41081f0 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.149+0000 7f35c27fc640 1 -- 172.21.15.37:0/3440484532 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b4032920 con 0x7f35c41081f0 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.149+0000 7f35c27fc640 1 -- 172.21.15.37:0/3440484532 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35b4030810 con 0x7f35c41081f0 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.150+0000 7f35c98db640 1 -- 172.21.15.37:0/3440484532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 msgr2=0x7f35c410a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.150+0000 7f35c98db640 1 --2- 172.21.15.37:0/3440484532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c410a5e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009820 tx=0x7f35b402a270 comp rx=0 tx=0).stop 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.150+0000 7f35c98db640 1 -- 172.21.15.37:0/3440484532 shutdown_connections 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.150+0000 7f35c98db640 1 --2- 172.21.15.37:0/3440484532 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c410a5e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.150+0000 7f35c98db640 1 -- 172.21.15.37:0/3440484532 >> 172.21.15.37:0/3440484532 conn(0x7f35c4101db0 msgr2=0x7f35c41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.150+0000 7f35c98db640 1 -- 172.21.15.37:0/3440484532 shutdown_connections 2024-09-19T06:32:47.289 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.150+0000 7f35c98db640 1 -- 172.21.15.37:0/3440484532 wait complete. 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c98db640 1 Processor -- start 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c98db640 1 -- start start 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c98db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c98db640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f35c410bb40 con 0x7f35c41081f0 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c419b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56310/0 (socket says 172.21.15.37:56310) 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c2ffd640 1 -- 172.21.15.37:0/90349243 learned_addr learned my addr 172.21.15.37:0/90349243 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.151+0000 7f35c2ffd640 1 -- 172.21.15.37:0/90349243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35c419bce0 con 0x7f35c41081f0 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.152+0000 7f35c2ffd640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c419b7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f35b4030420 tx=0x7f35b4032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.152+0000 7f35c88d9640 1 -- 172.21.15.37:0/90349243 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35b4037040 con 0x7f35c41081f0 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.152+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35c419bf70 con 0x7f35c41081f0 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.152+0000 7f35c88d9640 1 -- 172.21.15.37:0/90349243 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f35b4040070 con 0x7f35c41081f0 2024-09-19T06:32:47.290 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.152+0000 7f35c88d9640 1 -- 172.21.15.37:0/90349243 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35b403a6e0 con 0x7f35c41081f0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.152+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35c419ec60 con 0x7f35c41081f0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.153+0000 7f35c88d9640 1 -- 172.21.15.37:0/90349243 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f35b403a9e0 con 0x7f35c41081f0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.153+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35c4108650 con 0x7f35c41081f0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.153+0000 7f35c88d9640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f35b00391b0 0x7f35b003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.153+0000 7f35c88d9640 1 -- 172.21.15.37:0/90349243 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f35b406c9c0 con 0x7f35c41081f0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.153+0000 7f35bbfff640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f35b00391b0 0x7f35b003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.154+0000 7f35bbfff640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f35b00391b0 0x7f35b003b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0098c0 tx=0x7f35ac006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.162+0000 7f35c88d9640 1 -- 172.21.15.37:0/90349243 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f35b406ee20 con 0x7f35c41081f0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.363+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f35c41030c0 con 0x7f35b00391b0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.368+0000 7f35c88d9640 1 -- 172.21.15.37:0/90349243 <== mgr.14118 v2:172.21.15.37:6800/2601148313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f35c41030c0 con 0x7f35b00391b0 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f35b00391b0 msgr2=0x7f35b003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:47.291 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f35b00391b0 0x7f35b003b670 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f35ac0098c0 tx=0x7f35ac006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 msgr2=0x7f35c419b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c419b7a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f35b4030420 tx=0x7f35b4032670 comp rx=0 tx=0).stop 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 shutdown_connections 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f35b00391b0 0x7f35b003b670 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 --2- 172.21.15.37:0/90349243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f35c41081f0 0x7f35c419b7a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 >> 172.21.15.37:0/90349243 conn(0x7f35c4101db0 msgr2=0x7f35c41029b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 shutdown_connections 2024-09-19T06:32:47.292 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:46.370+0000 7f35c98db640 1 -- 172.21.15.37:0/90349243 wait complete. 2024-09-19T06:32:47.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:47 smithi037 ceph-mon[31211]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:47.864 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:47 smithi037 ceph-mon[31211]: Saving service mgr spec with placement count:2 2024-09-19T06:32:48.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.676+0000 7f232f8c3640 1 Processor -- start 2024-09-19T06:32:48.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.676+0000 7f232f8c3640 1 -- start start 2024-09-19T06:32:48.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.676+0000 7f232f8c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f2328107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:48.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.676+0000 7f232f8c3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2328108170 con 0x7f23281077a0 2024-09-19T06:32:48.546 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.676+0000 7f232d65f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f2328107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.676+0000 7f232d65f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f2328107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56322/0 (socket says 172.21.15.37:56322) 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.676+0000 7f232d65f640 1 -- 172.21.15.37:0/1303870148 learned_addr learned my addr 172.21.15.37:0/1303870148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.677+0000 7f232d65f640 1 -- 172.21.15.37:0/1303870148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2328108950 con 0x7f23281077a0 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.677+0000 7f232d65f640 1 --2- 172.21.15.37:0/1303870148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f2328107ba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2324009820 tx=0x7f232402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9857aa4ce5a5c20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.677+0000 7f231ffff640 1 -- 172.21.15.37:0/1303870148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2324037070 con 0x7f23281077a0 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.677+0000 7f231ffff640 1 -- 172.21.15.37:0/1303870148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2324032920 con 0x7f23281077a0 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.677+0000 7f231ffff640 1 -- 172.21.15.37:0/1303870148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2324030810 con 0x7f23281077a0 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.678+0000 7f232f8c3640 1 -- 172.21.15.37:0/1303870148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 msgr2=0x7f2328107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.678+0000 7f232f8c3640 1 --2- 172.21.15.37:0/1303870148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f2328107ba0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2324009820 tx=0x7f232402a270 comp rx=0 tx=0).stop 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.678+0000 7f232f8c3640 1 -- 172.21.15.37:0/1303870148 shutdown_connections 2024-09-19T06:32:48.547 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.678+0000 7f232f8c3640 1 --2- 172.21.15.37:0/1303870148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f2328107ba0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.678+0000 7f232f8c3640 1 -- 172.21.15.37:0/1303870148 >> 172.21.15.37:0/1303870148 conn(0x7f2328102fd0 msgr2=0x7f23281053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.678+0000 7f232f8c3640 1 -- 172.21.15.37:0/1303870148 shutdown_connections 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.678+0000 7f232f8c3640 1 -- 172.21.15.37:0/1303870148 wait complete. 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232f8c3640 1 Processor -- start 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232f8c3640 1 -- start start 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232f8c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f232819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232f8c3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f232810bd00 con 0x7f23281077a0 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232d65f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f232819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232d65f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f232819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56326/0 (socket says 172.21.15.37:56326) 2024-09-19T06:32:48.548 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232d65f640 1 -- 172.21.15.37:0/505549648 learned_addr learned my addr 172.21.15.37:0/505549648 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232d65f640 1 -- 172.21.15.37:0/505549648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f232819bcb0 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.679+0000 7f232d65f640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f232819b770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2324030560 tx=0x7f2324032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.680+0000 7f231e7fc640 1 -- 172.21.15.37:0/505549648 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2324037040 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.680+0000 7f231e7fc640 1 -- 172.21.15.37:0/505549648 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2324040070 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.680+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f232819bf40 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.680+0000 7f231e7fc640 1 -- 172.21.15.37:0/505549648 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f232403a740 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.680+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f232819ec30 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.680+0000 7f231e7fc640 1 -- 172.21.15.37:0/505549648 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f232403b420 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.681+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f232810be80 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.681+0000 7f231e7fc640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f23140391b0 0x7f231403b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.681+0000 7f231e7fc640 1 -- 172.21.15.37:0/505549648 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f232406c850 con 0x7f23281077a0 2024-09-19T06:32:48.549 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.681+0000 7f232ce5e640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f23140391b0 0x7f231403b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.681+0000 7f232ce5e640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f23140391b0 0x7f231403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f23180098c0 tx=0x7f2318006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.685+0000 7f231e7fc640 1 -- 172.21.15.37:0/505549648 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f232403a9b0 con 0x7f23281077a0 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.861+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f2328107c20 con 0x7f23281077a0 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.865+0000 7f231e7fc640 1 -- 172.21.15.37:0/505549648 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f232402fd10 con 0x7f23281077a0 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f23140391b0 msgr2=0x7f231403b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f23140391b0 0x7f231403b670 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f23180098c0 tx=0x7f2318006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 msgr2=0x7f232819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f232819b770 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2324030560 tx=0x7f2324032670 comp rx=0 tx=0).stop 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 shutdown_connections 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f23140391b0 0x7f231403b670 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 --2- 172.21.15.37:0/505549648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f23281077a0 0x7f232819b770 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 >> 172.21.15.37:0/505549648 conn(0x7f2328102fd0 msgr2=0x7f2328103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:48.550 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 shutdown_connections 2024-09-19T06:32:48.551 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:47.867+0000 7f232f8c3640 1 -- 172.21.15.37:0/505549648 wait complete. 2024-09-19T06:32:48.975 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:48 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/505549648' entity='client.admin' 2024-09-19T06:32:48.976 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:48 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:49.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.975+0000 7f0df7ed3640 1 Processor -- start 2024-09-19T06:32:49.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.975+0000 7f0df7ed3640 1 -- start start 2024-09-19T06:32:49.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.975+0000 7f0df7ed3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df0107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:49.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.975+0000 7f0df7ed3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0df0108170 con 0x7f0df01077a0 2024-09-19T06:32:49.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.975+0000 7f0df5c6f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df0107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:49.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.975+0000 7f0df5c6f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df0107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56334/0 (socket says 172.21.15.37:56334) 2024-09-19T06:32:49.805 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.975+0000 7f0df5c6f640 1 -- 172.21.15.37:0/3612464668 learned_addr learned my addr 172.21.15.37:0/3612464668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.976+0000 7f0df5c6f640 1 -- 172.21.15.37:0/3612464668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df0108950 con 0x7f0df01077a0 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.976+0000 7f0df5c6f640 1 --2- 172.21.15.37:0/3612464668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df0107ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009820 tx=0x7f0dec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a57c2a325f7e7f63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df4c6d640 1 -- 172.21.15.37:0/3612464668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dec037070 con 0x7f0df01077a0 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df4c6d640 1 -- 172.21.15.37:0/3612464668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dec032920 con 0x7f0df01077a0 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df4c6d640 1 -- 172.21.15.37:0/3612464668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dec030810 con 0x7f0df01077a0 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df7ed3640 1 -- 172.21.15.37:0/3612464668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 msgr2=0x7f0df0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df7ed3640 1 --2- 172.21.15.37:0/3612464668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df0107ba0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0dec009820 tx=0x7f0dec02a270 comp rx=0 tx=0).stop 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df7ed3640 1 -- 172.21.15.37:0/3612464668 shutdown_connections 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df7ed3640 1 --2- 172.21.15.37:0/3612464668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df0107ba0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.977+0000 7f0df7ed3640 1 -- 172.21.15.37:0/3612464668 >> 172.21.15.37:0/3612464668 conn(0x7f0df0102fd0 msgr2=0x7f0df01053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.978+0000 7f0df7ed3640 1 -- 172.21.15.37:0/3612464668 shutdown_connections 2024-09-19T06:32:49.806 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.978+0000 7f0df7ed3640 1 -- 172.21.15.37:0/3612464668 wait complete. 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.978+0000 7f0df7ed3640 1 Processor -- start 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.978+0000 7f0df7ed3640 1 -- start start 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.978+0000 7f0df7ed3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df007e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.978+0000 7f0df7ed3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0df010bd00 con 0x7f0df01077a0 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0df5c6f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df007e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0df5c6f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df007e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56342/0 (socket says 172.21.15.37:56342) 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0df5c6f640 1 -- 172.21.15.37:0/103581524 learned_addr learned my addr 172.21.15.37:0/103581524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0df5c6f640 1 -- 172.21.15.37:0/103581524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0df007e910 con 0x7f0df01077a0 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0df5c6f640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df007e3d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0dec030560 tx=0x7f0dec032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0de6ffd640 1 -- 172.21.15.37:0/103581524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dec037040 con 0x7f0df01077a0 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0de6ffd640 1 -- 172.21.15.37:0/103581524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dec040070 con 0x7f0df01077a0 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0df007cb00 con 0x7f0df01077a0 2024-09-19T06:32:49.807 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0df007d040 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.979+0000 7f0de6ffd640 1 -- 172.21.15.37:0/103581524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0dec03a740 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.981+0000 7f0de6ffd640 1 -- 172.21.15.37:0/103581524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f0dec03a8e0 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.981+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0df0107c20 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.981+0000 7f0de6ffd640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f0ddc039200 0x7f0ddc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.981+0000 7f0de6ffd640 1 -- 172.21.15.37:0/103581524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0dec06c840 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.981+0000 7f0df546e640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f0ddc039200 0x7f0ddc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.981+0000 7f0df546e640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f0ddc039200 0x7f0ddc03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098c0 tx=0x7f0de0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:48.986+0000 7f0de6ffd640 1 -- 172.21.15.37:0/103581524 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0dec044c70 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.178+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f0df007d2c0 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.181+0000 7f0de6ffd640 1 -- 172.21.15.37:0/103581524 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f0dec041270 con 0x7f0df01077a0 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f0ddc039200 msgr2=0x7f0ddc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:49.808 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f0ddc039200 0x7f0ddc03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0de00098c0 tx=0x7f0de0006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 msgr2=0x7f0df007e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df007e3d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0dec030560 tx=0x7f0dec032670 comp rx=0 tx=0).stop 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 shutdown_connections 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f0ddc039200 0x7f0ddc03b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 --2- 172.21.15.37:0/103581524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0df01077a0 0x7f0df007e3d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 >> 172.21.15.37:0/103581524 conn(0x7f0df0102fd0 msgr2=0x7f0df0103b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 shutdown_connections 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:49.184+0000 7f0df7ed3640 1 -- 172.21.15.37:0/103581524 wait complete. 2024-09-19T06:32:49.809 INFO:teuthology.orchestra.run.smithi037.stdout:Enabling the dashboard module... 2024-09-19T06:32:50.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:50 smithi037 ceph-mon[31211]: from='mgr.14118 172.21.15.37:0/4144352201' entity='mgr.x' 2024-09-19T06:32:50.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:50 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/103581524' entity='client.admin' 2024-09-19T06:32:51.177 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1893492217' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-19T06:32:51.178 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setuser ceph since I am not root 2024-09-19T06:32:51.178 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setgroup ceph since I am not root 2024-09-19T06:32:51.178 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:51.175+0000 7f02cb889040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:32:51.461 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:51.235+0000 7f02cb889040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:32:51.461 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:51.390+0000 7f02cb889040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:32:51.461 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:51.459+0000 7f02cb889040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.172+0000 7f2ff7680640 1 Processor -- start 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.172+0000 7f2ff7680640 1 -- start start 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.172+0000 7f2ff7680640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff01063e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.172+0000 7f2ff7680640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff007be00 con 0x7f2ff0105fe0 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.173+0000 7f2ff541c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff01063e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.173+0000 7f2ff541c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff01063e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46436/0 (socket says 172.21.15.37:46436) 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.173+0000 7f2ff541c640 1 -- 172.21.15.37:0/3257241213 learned_addr learned my addr 172.21.15.37:0/3257241213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.173+0000 7f2ff541c640 1 -- 172.21.15.37:0/3257241213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff0106920 con 0x7f2ff0105fe0 2024-09-19T06:32:51.658 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.174+0000 7f2ff541c640 1 --2- 172.21.15.37:0/3257241213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff01063e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009820 tx=0x7f2fe802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3794021a4ddce778 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.174+0000 7f2feffff640 1 -- 172.21.15.37:0/3257241213 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8037070 con 0x7f2ff0105fe0 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.174+0000 7f2feffff640 1 -- 172.21.15.37:0/3257241213 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe8032920 con 0x7f2ff0105fe0 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.174+0000 7f2feffff640 1 -- 172.21.15.37:0/3257241213 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8030810 con 0x7f2ff0105fe0 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.175+0000 7f2ff7680640 1 -- 172.21.15.37:0/3257241213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 msgr2=0x7f2ff01063e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.175+0000 7f2ff7680640 1 --2- 172.21.15.37:0/3257241213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff01063e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8009820 tx=0x7f2fe802a270 comp rx=0 tx=0).stop 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.175+0000 7f2ff7680640 1 -- 172.21.15.37:0/3257241213 shutdown_connections 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.175+0000 7f2ff7680640 1 --2- 172.21.15.37:0/3257241213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff01063e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.175+0000 7f2ff7680640 1 -- 172.21.15.37:0/3257241213 >> 172.21.15.37:0/3257241213 conn(0x7f2ff0101db0 msgr2=0x7f2ff01041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.175+0000 7f2ff7680640 1 -- 172.21.15.37:0/3257241213 shutdown_connections 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.175+0000 7f2ff7680640 1 -- 172.21.15.37:0/3257241213 wait complete. 2024-09-19T06:32:51.659 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff7680640 1 Processor -- start 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff7680640 1 -- start start 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff7680640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff019d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff7680640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ff0107a60 con 0x7f2ff0105fe0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff541c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff019d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff541c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff019d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46442/0 (socket says 172.21.15.37:46442) 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff541c640 1 -- 172.21.15.37:0/1893492217 learned_addr learned my addr 172.21.15.37:0/1893492217 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff541c640 1 -- 172.21.15.37:0/1893492217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ff019de50 con 0x7f2ff0105fe0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff541c640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff019d910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80097f0 tx=0x7f2fe80327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8037040 con 0x7f2ff0105fe0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2fe8030c40 con 0x7f2ff0105fe0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ff019e0e0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.176+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe803a5d0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.660 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.177+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ff01a0dd0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.177+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f2fe803b3f0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.177+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ff01063e0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.178+0000 7f2fee7fc640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f2fd80391b0 0x7f2fd803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.178+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2fe806c790 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.178+0000 7f2ff4c1b640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f2fd80391b0 0x7f2fd803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.178+0000 7f2ff4c1b640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f2fd80391b0 0x7f2fd803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40098c0 tx=0x7f2fe4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.184+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2fe803b6e0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:50.417+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f2ff0196db0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.022+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f2fe803a7c0 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.024+0000 7f2fee7fc640 1 -- 172.21.15.37:0/1893492217 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f2fe8049c60 con 0x7f2ff0105fe0 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f2fd80391b0 msgr2=0x7f2fd803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:51.661 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f2fd80391b0 0x7f2fd803b670 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f2fe40098c0 tx=0x7f2fe4006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 msgr2=0x7f2ff019d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff019d910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2fe80097f0 tx=0x7f2fe80327b0 comp rx=0 tx=0).stop 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 shutdown_connections 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f2fd80391b0 0x7f2fd803b670 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 --2- 172.21.15.37:0/1893492217 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ff0105fe0 0x7f2ff019d910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 >> 172.21.15.37:0/1893492217 conn(0x7f2ff0101db0 msgr2=0x7f2ff0191c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 shutdown_connections 2024-09-19T06:32:51.662 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:51.025+0000 7f2ff7680640 1 -- 172.21.15.37:0/1893492217 wait complete. 2024-09-19T06:32:51.763 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:51.612+0000 7f02cb889040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:32:51.763 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:51.761+0000 7f02cb889040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:32:52.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:52 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1893492217' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-19T06:32:52.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:52 smithi037 ceph-mon[31211]: mgrmap e9: x(active, since 18s) 2024-09-19T06:32:52.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:52 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:52.093+0000 7f02cb889040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:32:52.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:52 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:52.165+0000 7f02cb889040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:32:52.716 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:52 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:52.364+0000 7f02cb889040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:32:52.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:32:52.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-19T06:32:52.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-19T06:32:52.966 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.140+0000 7f05b325a640 1 Processor -- start 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.141+0000 7f05b325a640 1 -- start start 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.141+0000 7f05b325a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac106a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.141+0000 7f05b325a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac107000 con 0x7f05ac106630 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.141+0000 7f05b0ff6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac106a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.141+0000 7f05b0ff6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac106a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46470/0 (socket says 172.21.15.37:46470) 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.141+0000 7f05b0ff6640 1 -- 172.21.15.37:0/3968473746 learned_addr learned my addr 172.21.15.37:0/3968473746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.142+0000 7f05b0ff6640 1 -- 172.21.15.37:0/3968473746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac107830 con 0x7f05ac106630 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.142+0000 7f05b0ff6640 1 --2- 172.21.15.37:0/3968473746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac106a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009820 tx=0x7f05a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd390ef22527dd02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.142+0000 7f05ab7fe640 1 -- 172.21.15.37:0/3968473746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a0037070 con 0x7f05ac106630 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05ab7fe640 1 -- 172.21.15.37:0/3968473746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a0032920 con 0x7f05ac106630 2024-09-19T06:32:52.967 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05ab7fe640 1 -- 172.21.15.37:0/3968473746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a0030810 con 0x7f05ac106630 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05b325a640 1 -- 172.21.15.37:0/3968473746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 msgr2=0x7f05ac106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05b325a640 1 --2- 172.21.15.37:0/3968473746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac106a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f05a0009820 tx=0x7f05a002a270 comp rx=0 tx=0).stop 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05b325a640 1 -- 172.21.15.37:0/3968473746 shutdown_connections 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05b325a640 1 --2- 172.21.15.37:0/3968473746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac106a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05b325a640 1 -- 172.21.15.37:0/3968473746 >> 172.21.15.37:0/3968473746 conn(0x7f05ac101de0 msgr2=0x7f05ac104200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05b325a640 1 -- 172.21.15.37:0/3968473746 shutdown_connections 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.143+0000 7f05b325a640 1 -- 172.21.15.37:0/3968473746 wait complete. 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.144+0000 7f05b325a640 1 Processor -- start 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.144+0000 7f05b325a640 1 -- start start 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.144+0000 7f05b325a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac191860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.144+0000 7f05b325a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05ac10abe0 con 0x7f05ac106630 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.144+0000 7f05b0ff6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac191860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:52.968 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.144+0000 7f05b0ff6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac191860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46478/0 (socket says 172.21.15.37:46478) 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.144+0000 7f05b0ff6640 1 -- 172.21.15.37:0/289439192 learned_addr learned my addr 172.21.15.37:0/289439192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.145+0000 7f05b0ff6640 1 -- 172.21.15.37:0/289439192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ac191da0 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.145+0000 7f05b0ff6640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac191860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097f0 tx=0x7f05a00327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.145+0000 7f05a9ffb640 1 -- 172.21.15.37:0/289439192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a0037040 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.145+0000 7f05a9ffb640 1 -- 172.21.15.37:0/289439192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05a0030cc0 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.145+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ac18ff90 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.145+0000 7f05a9ffb640 1 -- 172.21.15.37:0/289439192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f05a003a640 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.145+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ac1904d0 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.146+0000 7f05a9ffb640 1 -- 172.21.15.37:0/289439192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f05a003a940 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.146+0000 7f05a9ffb640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 0x7f059c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.146+0000 7f05a9ffb640 1 -- 172.21.15.37:0/289439192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f05a006caf0 con 0x7f05ac106630 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.147+0000 7f05abfff640 1 -- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 msgr2=0x7f059c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2601148313 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.147+0000 7f05abfff640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 0x7f059c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:32:52.969 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.147+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05ac10ad60 con 0x7f05ac106630 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.151+0000 7f05a9ffb640 1 -- 172.21.15.37:0/289439192 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05a003ba40 con 0x7f05ac106630 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.348+0000 7f05abfff640 1 -- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 msgr2=0x7f059c03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2601148313 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.348+0000 7f05abfff640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 0x7f059c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.393+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f05ac190a80 con 0x7f05ac106630 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.393+0000 7f05a9ffb640 1 -- 172.21.15.37:0/289439192 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f05a0044920 con 0x7f05ac106630 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.395+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 msgr2=0x7f059c03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.395+0000 7f05b325a640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 0x7f059c03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 msgr2=0x7f05ac191860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac191860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f05a00097f0 tx=0x7f05a00327b0 comp rx=0 tx=0).stop 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 shutdown_connections 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f059c039200 0x7f059c03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:52.970 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 --2- 172.21.15.37:0/289439192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05ac106630 0x7f05ac191860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:52.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 >> 172.21.15.37:0/289439192 conn(0x7f05ac101de0 msgr2=0x7f05ac102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:52.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 shutdown_connections 2024-09-19T06:32:52.971 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:52.396+0000 7f05b325a640 1 -- 172.21.15.37:0/289439192 wait complete. 2024-09-19T06:32:52.971 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for the mgr to restart... 2024-09-19T06:32:52.971 INFO:teuthology.orchestra.run.smithi037.stdout:Waiting for mgr epoch 9... 2024-09-19T06:32:53.312 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:53 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/289439192' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-19T06:32:53.610 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:53 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:53.310+0000 7f02cb889040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:32:53.610 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:53 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:53.383+0000 7f02cb889040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:32:54.268 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.144+0000 7f02cb889040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:32:54.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.265+0000 7f02cb889040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:32:54.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.416+0000 7f02cb889040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:32:54.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.480+0000 7f02cb889040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:32:54.938 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.637+0000 7f02cb889040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:32:54.938 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.708+0000 7f02cb889040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:32:54.938 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.770+0000 7f02cb889040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:32:54.938 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:54.936+0000 7f02cb889040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:32:55.223 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:55.004+0000 7f02cb889040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:32:55.223 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:55.073+0000 7f02cb889040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:32:55.223 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:55.159+0000 7f02cb889040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:32:55.223 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:55.221+0000 7f02cb889040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:32:55.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:55.283+0000 7f02cb889040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:32:56.361 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:56.106+0000 7f02cb889040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: Active manager daemon x restarted 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: Activating manager daemon x 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: osdmap e3: 0 total, 0 up, 0 in 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: mgrmap e10: x(active, starting, since 0.00346248s) 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: Manager daemon x is now available 2024-09-19T06:32:56.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:32:56.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:32:56.363 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:32:58.010 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout { 2024-09-19T06:32:58.010 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout } 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.402+0000 7f3e9c2a2640 1 Processor -- start 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.402+0000 7f3e9c2a2640 1 -- start start 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.402+0000 7f3e9c2a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e94106a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.402+0000 7f3e9c2a2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e94107010 con 0x7f3e94106640 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.402+0000 7f3e9a03e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e94106a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.402+0000 7f3e9a03e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e94106a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46492/0 (socket says 172.21.15.37:46492) 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.402+0000 7f3e9a03e640 1 -- 172.21.15.37:0/2938191800 learned_addr learned my addr 172.21.15.37:0/2938191800 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.403+0000 7f3e9a03e640 1 -- 172.21.15.37:0/2938191800 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e941077f0 con 0x7f3e94106640 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.403+0000 7f3e9a03e640 1 --2- 172.21.15.37:0/2938191800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e94106a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009820 tx=0x7f3e9002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eef456a61afbc53a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:58.011 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.403+0000 7f3e9903c640 1 -- 172.21.15.37:0/2938191800 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e90037070 con 0x7f3e94106640 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.403+0000 7f3e9903c640 1 -- 172.21.15.37:0/2938191800 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e90032920 con 0x7f3e94106640 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.403+0000 7f3e9903c640 1 -- 172.21.15.37:0/2938191800 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e90030810 con 0x7f3e94106640 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.404+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/2938191800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 msgr2=0x7f3e94106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.404+0000 7f3e9c2a2640 1 --2- 172.21.15.37:0/2938191800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e94106a40 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009820 tx=0x7f3e9002a270 comp rx=0 tx=0).stop 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.404+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/2938191800 shutdown_connections 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.404+0000 7f3e9c2a2640 1 --2- 172.21.15.37:0/2938191800 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e94106a40 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.404+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/2938191800 >> 172.21.15.37:0/2938191800 conn(0x7f3e94101db0 msgr2=0x7f3e94104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.404+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/2938191800 shutdown_connections 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.404+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/2938191800 wait complete. 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.405+0000 7f3e9c2a2640 1 Processor -- start 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.405+0000 7f3e9c2a2640 1 -- start start 2024-09-19T06:32:58.012 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.405+0000 7f3e9c2a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e9419b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.405+0000 7f3e9c2a2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9410aba0 con 0x7f3e94106640 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.405+0000 7f3e9a03e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e9419b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.405+0000 7f3e9a03e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e9419b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46494/0 (socket says 172.21.15.37:46494) 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.405+0000 7f3e9a03e640 1 -- 172.21.15.37:0/426477340 learned_addr learned my addr 172.21.15.37:0/426477340 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.406+0000 7f3e9a03e640 1 -- 172.21.15.37:0/426477340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9419bd90 con 0x7f3e94106640 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.406+0000 7f3e9a03e640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e9419b850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e900097f0 tx=0x7f3e900327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.406+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e90037040 con 0x7f3e94106640 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.406+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9419c020 con 0x7f3e94106640 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.406+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e90030c40 con 0x7f3e94106640 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.406+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9419ed10 con 0x7f3e94106640 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.406+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e9003a5b0 con 0x7f3e94106640 2024-09-19T06:32:58.013 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.407+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f3e9003a7f0 con 0x7f3e94106640 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.407+0000 7f3e8b7fe640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 0x7f3e8003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.407+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 --> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e94106ac0 con 0x7f3e80039200 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.407+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3e9006c770 con 0x7f3e94106640 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.407+0000 7f3e9983d640 1 -- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 msgr2=0x7f3e8003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2601148313 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.407+0000 7f3e9983d640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 0x7f3e8003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.608+0000 7f3e9983d640 1 -- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 msgr2=0x7f3e8003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2601148313 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:53.608+0000 7f3e9983d640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 0x7f3e8003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:54.009+0000 7f3e9983d640 1 -- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 msgr2=0x7f3e8003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2601148313 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:54.009+0000 7f3e9983d640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 0x7f3e8003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:54.810+0000 7f3e9983d640 1 -- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 msgr2=0x7f3e8003b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/2601148313 2024-09-19T06:32:58.014 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:54.810+0000 7f3e9983d640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 0x7f3e8003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:56.110+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f3e9006b220 con 0x7f3e94106640 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:56.110+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 msgr2=0x7f3e8003b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:56.110+0000 7f3e8b7fe640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 0x7f3e8003b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.112+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f3e90044400 con 0x7f3e94106640 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.112+0000 7f3e8b7fe640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e8003cad0 0x7f3e8003eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.112+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f3e9002fec0 con 0x7f3e8003cad0 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.113+0000 7f3e9983d640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e8003cad0 0x7f3e8003eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.113+0000 7f3e9983d640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e8003cad0 0x7f3e8003eec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3e84003980 tx=0x7f3e84006e10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.114+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f3e9002fec0 con 0x7f3e8003cad0 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.121+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f3e9410ad20 con 0x7f3e8003cad0 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.121+0000 7f3e8b7fe640 1 -- 172.21.15.37:0/426477340 <== mgr.14150 v2:172.21.15.37:6800/871019432 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f3e9410ad20 con 0x7f3e8003cad0 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e8003cad0 msgr2=0x7f3e8003eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:58.015 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e8003cad0 0x7f3e8003eec0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3e84003980 tx=0x7f3e84006e10 comp rx=0 tx=0).stop 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 msgr2=0x7f3e9419b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e9419b850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3e900097f0 tx=0x7f3e900327b0 comp rx=0 tx=0).stop 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 shutdown_connections 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e8003cad0 0x7f3e8003eec0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:6800/2601148313,v1:172.21.15.37:6801/2601148313] conn(0x7f3e80039200 0x7f3e8003b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 --2- 172.21.15.37:0/426477340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e94106640 0x7f3e9419b850 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 >> 172.21.15.37:0/426477340 conn(0x7f3e94101db0 msgr2=0x7f3e94102dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 shutdown_connections 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:57.122+0000 7f3e9c2a2640 1 -- 172.21.15.37:0/426477340 wait complete. 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:mgr epoch 9 is available 2024-09-19T06:32:58.016 INFO:teuthology.orchestra.run.smithi037.stdout:Generating a dashboard self-signed certificate... 2024-09-19T06:32:58.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-mon[31211]: mgrmap e11: x(active, since 1.00654s) 2024-09-19T06:32:58.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:32:58.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:57] ENGINE Bus STARTING 2024-09-19T06:32:58.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:58.215+0000 7f0231352640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:06:32:58] ENGINE Error in HTTPServer.serve 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: Traceback (most recent call last): 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._connections.run(self.expiration_interval) 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._run(expiration_interval) 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:32:58.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: s = self.context.wrap_socket( 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: return self.sslsocket_class._create( 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self.do_handshake() 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._sslobj.do_handshake() 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:32:58.277 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:32:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:32:59.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:58] ENGINE Serving on http://172.21.15.37:8765 2024-09-19T06:32:59.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:58] ENGINE Serving on https://172.21.15.37:7150 2024-09-19T06:32:59.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:58] ENGINE Bus STARTED 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: [19/Sep/2024:06:32:58] ENGINE Error in HTTPServer.serve 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: Traceback (most recent call last): 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._run(expiration_interval) 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:32:59.495 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: s = self.context.wrap_socket( 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: return self.sslsocket_class._create( 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: self.do_handshake() 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._sslobj.do_handshake() 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: mgrmap e12: x(active, since 2s) 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:32:59.496 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:32:59 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:32:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-19T06:32:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.456+0000 7fb9960e7640 1 Processor -- start 2024-09-19T06:32:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.456+0000 7fb9960e7640 1 -- start start 2024-09-19T06:32:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.457+0000 7fb9960e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99010a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.457+0000 7fb9960e7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb99007be30 con 0x7fb9901081b0 2024-09-19T06:32:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.457+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99010a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:59.502 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.457+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99010a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46566/0 (socket says 172.21.15.37:46566) 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.457+0000 7fb98f7fe640 1 -- 172.21.15.37:0/1806846911 learned_addr learned my addr 172.21.15.37:0/1806846911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.457+0000 7fb98f7fe640 1 -- 172.21.15.37:0/1806846911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99010aae0 con 0x7fb9901081b0 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.458+0000 7fb98f7fe640 1 --2- 172.21.15.37:0/1806846911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99010a5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb980009820 tx=0x7fb98002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac342785774f7aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.458+0000 7fb98e7fc640 1 -- 172.21.15.37:0/1806846911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb980037070 con 0x7fb9901081b0 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.458+0000 7fb98e7fc640 1 -- 172.21.15.37:0/1806846911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb980032920 con 0x7fb9901081b0 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.458+0000 7fb98e7fc640 1 -- 172.21.15.37:0/1806846911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb980030810 con 0x7fb9901081b0 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.459+0000 7fb9960e7640 1 -- 172.21.15.37:0/1806846911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 msgr2=0x7fb99010a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.459+0000 7fb9960e7640 1 --2- 172.21.15.37:0/1806846911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99010a5a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb980009820 tx=0x7fb98002a270 comp rx=0 tx=0).stop 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.459+0000 7fb9960e7640 1 -- 172.21.15.37:0/1806846911 shutdown_connections 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.459+0000 7fb9960e7640 1 --2- 172.21.15.37:0/1806846911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99010a5a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.459+0000 7fb9960e7640 1 -- 172.21.15.37:0/1806846911 >> 172.21.15.37:0/1806846911 conn(0x7fb990101db0 msgr2=0x7fb990104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.459+0000 7fb9960e7640 1 -- 172.21.15.37:0/1806846911 shutdown_connections 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.459+0000 7fb9960e7640 1 -- 172.21.15.37:0/1806846911 wait complete. 2024-09-19T06:32:59.503 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.460+0000 7fb9960e7640 1 Processor -- start 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.460+0000 7fb9960e7640 1 -- start start 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.460+0000 7fb9960e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.460+0000 7fb9960e7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb99010bb60 con 0x7fb9901081b0 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb98f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99019b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46582/0 (socket says 172.21.15.37:46582) 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb98f7fe640 1 -- 172.21.15.37:0/3789838374 learned_addr learned my addr 172.21.15.37:0/3789838374 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb98f7fe640 1 -- 172.21.15.37:0/3789838374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99019bcb0 con 0x7fb9901081b0 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb98f7fe640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99019b770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb980030420 tx=0x7fb980032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb98cff9640 1 -- 172.21.15.37:0/3789838374 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb980037040 con 0x7fb9901081b0 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb98cff9640 1 -- 172.21.15.37:0/3789838374 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb980040070 con 0x7fb9901081b0 2024-09-19T06:32:59.504 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99019bf40 con 0x7fb9901081b0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.461+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb99019ec30 con 0x7fb9901081b0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.462+0000 7fb98cff9640 1 -- 172.21.15.37:0/3789838374 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb98003a6e0 con 0x7fb9901081b0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.462+0000 7fb98cff9640 1 -- 172.21.15.37:0/3789838374 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fb98003a9e0 con 0x7fb9901081b0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.463+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9700052c0 con 0x7fb9901081b0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.463+0000 7fb98cff9640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb97c039200 0x7fb97c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.463+0000 7fb98cff9640 1 -- 172.21.15.37:0/3789838374 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fb98006ccd0 con 0x7fb9901081b0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.463+0000 7fb98effd640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb97c039200 0x7fb97c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.463+0000 7fb98effd640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb97c039200 0x7fb97c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9780098c0 tx=0x7fb978006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.468+0000 7fb98cff9640 1 -- 172.21.15.37:0/3789838374 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb9800328c0 con 0x7fb9901081b0 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.653+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7fb970000bf0 con 0x7fb97c039200 2024-09-19T06:32:59.505 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.718+0000 7fb98cff9640 1 -- 172.21.15.37:0/3789838374 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7fb970000bf0 con 0x7fb97c039200 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb97c039200 msgr2=0x7fb97c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb97c039200 0x7fb97c03b6c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb9780098c0 tx=0x7fb978006db0 comp rx=0 tx=0).stop 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 msgr2=0x7fb99019b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99019b770 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb980030420 tx=0x7fb980032670 comp rx=0 tx=0).stop 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 shutdown_connections 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb97c039200 0x7fb97c03b6c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 --2- 172.21.15.37:0/3789838374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9901081b0 0x7fb99019b770 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 >> 172.21.15.37:0/3789838374 conn(0x7fb990101db0 msgr2=0x7fb990102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 shutdown_connections 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:58.720+0000 7fb9960e7640 1 -- 172.21.15.37:0/3789838374 wait complete. 2024-09-19T06:32:59.506 INFO:teuthology.orchestra.run.smithi037.stdout:Creating initial admin user... 2024-09-19T06:33:01.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$9Wca00VUcvnc6YKcKDNmaeFLmvwcAKyf84vUrbBsGqNiTkrI41u76", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726727580, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-19T06:33:01.121 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.905+0000 7fa492316640 1 Processor -- start 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa492316640 1 -- start start 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa492316640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c10a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa492316640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c07bdd0 con 0x7fa48c108210 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c10a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c10a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51168/0 (socket says 172.21.15.37:51168) 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48bfff640 1 -- 172.21.15.37:0/2309696758 learned_addr learned my addr 172.21.15.37:0/2309696758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48bfff640 1 -- 172.21.15.37:0/2309696758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c10ab40 con 0x7fa48c108210 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48bfff640 1 --2- 172.21.15.37:0/2309696758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c10a600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009820 tx=0x7fa47c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e11e3f490d245f27 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48b7fe640 1 -- 172.21.15.37:0/2309696758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa47c037070 con 0x7fa48c108210 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48b7fe640 1 -- 172.21.15.37:0/2309696758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47c032920 con 0x7fa48c108210 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.907+0000 7fa48b7fe640 1 -- 172.21.15.37:0/2309696758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa47c030810 con 0x7fa48c108210 2024-09-19T06:33:01.122 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.908+0000 7fa492316640 1 -- 172.21.15.37:0/2309696758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 msgr2=0x7fa48c10a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.908+0000 7fa492316640 1 --2- 172.21.15.37:0/2309696758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c10a600 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa47c009820 tx=0x7fa47c02a270 comp rx=0 tx=0).stop 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.908+0000 7fa492316640 1 -- 172.21.15.37:0/2309696758 shutdown_connections 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.908+0000 7fa492316640 1 --2- 172.21.15.37:0/2309696758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c10a600 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.908+0000 7fa492316640 1 -- 172.21.15.37:0/2309696758 >> 172.21.15.37:0/2309696758 conn(0x7fa48c101db0 msgr2=0x7fa48c104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.908+0000 7fa492316640 1 -- 172.21.15.37:0/2309696758 shutdown_connections 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.908+0000 7fa492316640 1 -- 172.21.15.37:0/2309696758 wait complete. 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa492316640 1 Processor -- start 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa492316640 1 -- start start 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa492316640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa492316640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c10bb60 con 0x7fa48c108210 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:01.123 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c19b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51174/0 (socket says 172.21.15.37:51174) 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa48bfff640 1 -- 172.21.15.37:0/983254610 learned_addr learned my addr 172.21.15.37:0/983254610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.909+0000 7fa48bfff640 1 -- 172.21.15.37:0/983254610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c19bcb0 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.910+0000 7fa48bfff640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c19b770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa47c030420 tx=0x7fa47c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.910+0000 7fa489ffb640 1 -- 172.21.15.37:0/983254610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa47c037040 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.910+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c19bf40 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.910+0000 7fa489ffb640 1 -- 172.21.15.37:0/983254610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47c040070 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.910+0000 7fa489ffb640 1 -- 172.21.15.37:0/983254610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa47c03a6e0 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.910+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c19ec30 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.911+0000 7fa489ffb640 1 -- 172.21.15.37:0/983254610 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fa47c03bb20 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.911+0000 7fa489ffb640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa478039200 0x7fa47803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.911+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa48c108670 con 0x7fa48c108210 2024-09-19T06:33:01.124 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.911+0000 7fa489ffb640 1 -- 172.21.15.37:0/983254610 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fa47c06ca20 con 0x7fa48c108210 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.911+0000 7fa483fff640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa478039200 0x7fa47803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.912+0000 7fa483fff640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa478039200 0x7fa47803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098c0 tx=0x7fa474006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:32:59.916+0000 7fa489ffb640 1 -- 172.21.15.37:0/983254610 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa47c03b7a0 con 0x7fa48c108210 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.109+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fa48c19eec0 con 0x7fa478039200 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.359+0000 7fa489ffb640 1 -- 172.21.15.37:0/983254610 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fa48c19eec0 con 0x7fa478039200 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa478039200 msgr2=0x7fa47803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa478039200 0x7fa47803b6c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa4740098c0 tx=0x7fa474006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 msgr2=0x7fa48c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:01.125 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c19b770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa47c030420 tx=0x7fa47c032670 comp rx=0 tx=0).stop 2024-09-19T06:33:01.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 shutdown_connections 2024-09-19T06:33:01.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa478039200 0x7fa47803b6c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:01.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 --2- 172.21.15.37:0/983254610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c108210 0x7fa48c19b770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:01.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 >> 172.21.15.37:0/983254610 conn(0x7fa48c101db0 msgr2=0x7fa48c102a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:01.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 shutdown_connections 2024-09-19T06:33:01.126 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:00.361+0000 7fa492316640 1 -- 172.21.15.37:0/983254610 wait complete. 2024-09-19T06:33:01.126 INFO:teuthology.orchestra.run.smithi037.stdout:Fetching dashboard port number... 2024-09-19T06:33:01.457 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:01 smithi037 ceph-mon[31211]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:01.458 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stdout 8443 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.538+0000 7f471083e640 1 Processor -- start 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.538+0000 7f471083e640 1 -- start start 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.538+0000 7f471083e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c107ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.538+0000 7f471083e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f470c108170 con 0x7f470c1077a0 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.538+0000 7f470a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c107ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.538+0000 7f470a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c107ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51180/0 (socket says 172.21.15.37:51180) 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.538+0000 7f470a59c640 1 -- 172.21.15.37:0/2054662093 learned_addr learned my addr 172.21.15.37:0/2054662093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:02.311 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.539+0000 7f470a59c640 1 -- 172.21.15.37:0/2054662093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c108950 con 0x7f470c1077a0 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.539+0000 7f470a59c640 1 --2- 172.21.15.37:0/2054662093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c107ba0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009820 tx=0x7f46fc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27bfcdd9a3c87971 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.539+0000 7f4709d9b640 1 -- 172.21.15.37:0/2054662093 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46fc037070 con 0x7f470c1077a0 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f4709d9b640 1 -- 172.21.15.37:0/2054662093 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46fc032920 con 0x7f470c1077a0 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f4709d9b640 1 -- 172.21.15.37:0/2054662093 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46fc030810 con 0x7f470c1077a0 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f471083e640 1 -- 172.21.15.37:0/2054662093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 msgr2=0x7f470c107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f471083e640 1 --2- 172.21.15.37:0/2054662093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c107ba0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46fc009820 tx=0x7f46fc02a270 comp rx=0 tx=0).stop 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f471083e640 1 -- 172.21.15.37:0/2054662093 shutdown_connections 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f471083e640 1 --2- 172.21.15.37:0/2054662093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c107ba0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f471083e640 1 -- 172.21.15.37:0/2054662093 >> 172.21.15.37:0/2054662093 conn(0x7f470c102fd0 msgr2=0x7f470c1053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f471083e640 1 -- 172.21.15.37:0/2054662093 shutdown_connections 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.540+0000 7f471083e640 1 -- 172.21.15.37:0/2054662093 wait complete. 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.541+0000 7f471083e640 1 Processor -- start 2024-09-19T06:33:02.312 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.541+0000 7f471083e640 1 -- start start 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.541+0000 7f471083e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c07e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.541+0000 7f471083e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f470c10bd00 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.541+0000 7f470a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c07e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.541+0000 7f470a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c07e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51194/0 (socket says 172.21.15.37:51194) 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.541+0000 7f470a59c640 1 -- 172.21.15.37:0/2587098935 learned_addr learned my addr 172.21.15.37:0/2587098935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.542+0000 7f470a59c640 1 -- 172.21.15.37:0/2587098935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c07e910 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.542+0000 7f470a59c640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c07e3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f46fc030560 tx=0x7f46fc032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.542+0000 7f47037fe640 1 -- 172.21.15.37:0/2587098935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46fc037040 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.542+0000 7f47037fe640 1 -- 172.21.15.37:0/2587098935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46fc040070 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.542+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c07cb00 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.542+0000 7f47037fe640 1 -- 172.21.15.37:0/2587098935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46fc03a740 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.542+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c07d040 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.543+0000 7f47037fe640 1 -- 172.21.15.37:0/2587098935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f46fc03a8e0 con 0x7f470c1077a0 2024-09-19T06:33:02.313 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.543+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c107c20 con 0x7f470c1077a0 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.543+0000 7f47037fe640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f8039260 0x7f46f803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.543+0000 7f47037fe640 1 -- 172.21.15.37:0/2587098935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f46fc06c9c0 con 0x7f470c1077a0 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.543+0000 7f4703fff640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f8039260 0x7f46f803b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.544+0000 7f4703fff640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f8039260 0x7f46f803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46f40098c0 tx=0x7f46f4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.548+0000 7f47037fe640 1 -- 172.21.15.37:0/2587098935 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f46fc044c70 con 0x7f470c1077a0 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.726+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f470c07d2c0 con 0x7f470c1077a0 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.727+0000 7f47037fe640 1 -- 172.21.15.37:0/2587098935 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f46fc041270 con 0x7f470c1077a0 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f8039260 msgr2=0x7f46f803b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f8039260 0x7f46f803b720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f46f40098c0 tx=0x7f46f4006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 msgr2=0x7f470c07e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c07e3d0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f46fc030560 tx=0x7f46fc032670 comp rx=0 tx=0).stop 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 shutdown_connections 2024-09-19T06:33:02.314 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f8039260 0x7f46f803b720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:02.315 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 --2- 172.21.15.37:0/2587098935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c1077a0 0x7f470c07e3d0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:02.315 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 >> 172.21.15.37:0/2587098935 conn(0x7f470c102fd0 msgr2=0x7f470c103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:02.315 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 shutdown_connections 2024-09-19T06:33:02.315 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:01.730+0000 7f471083e640 1 -- 172.21.15.37:0/2587098935 wait complete. 2024-09-19T06:33:02.319 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-19T06:33:02.320 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout disabled 2024-09-19T06:33:02.327 INFO:teuthology.orchestra.run.smithi037.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-19T06:33:02.327 INFO:teuthology.orchestra.run.smithi037.stdout:systemctl: stdout inactive 2024-09-19T06:33:02.327 INFO:teuthology.orchestra.run.smithi037.stdout:firewalld.service is not enabled 2024-09-19T06:33:02.327 INFO:teuthology.orchestra.run.smithi037.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-19T06:33:02.329 INFO:teuthology.orchestra.run.smithi037.stdout:Ceph Dashboard is now available at: 2024-09-19T06:33:02.329 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:02.329 INFO:teuthology.orchestra.run.smithi037.stdout: URL: https://smithi037.front.sepia.ceph.com:8443/ 2024-09-19T06:33:02.330 INFO:teuthology.orchestra.run.smithi037.stdout: User: admin 2024-09-19T06:33:02.330 INFO:teuthology.orchestra.run.smithi037.stdout: Password: xi0x3n0b68 2024-09-19T06:33:02.330 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:02.330 INFO:teuthology.orchestra.run.smithi037.stdout:Saving cluster configuration to /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config directory 2024-09-19T06:33:02.452 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:02 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2587098935' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-19T06:33:03.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:03 smithi037 ceph-mon[31211]: mgrmap e13: x(active, since 6s) 2024-09-19T06:33:03.585 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:03 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/79013156' entity='client.admin' 2024-09-19T06:33:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.732+0000 7f3c4603f640 1 Processor -- start 2024-09-19T06:33:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.732+0000 7f3c4603f640 1 -- start start 2024-09-19T06:33:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.732+0000 7f3c4603f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c40105e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:03.592 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.732+0000 7f3c4603f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c40106410 con 0x7f3c40105a40 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.732+0000 7f3c3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c40105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.733+0000 7f3c3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c40105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51198/0 (socket says 172.21.15.37:51198) 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.733+0000 7f3c3f7fe640 1 -- 172.21.15.37:0/4178619235 learned_addr learned my addr 172.21.15.37:0/4178619235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.733+0000 7f3c3f7fe640 1 -- 172.21.15.37:0/4178619235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c40108a20 con 0x7f3c40105a40 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.733+0000 7f3c3f7fe640 1 --2- 172.21.15.37:0/4178619235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c40105e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009820 tx=0x7f3c3002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a5314811f8c4d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c3e7fc640 1 -- 172.21.15.37:0/4178619235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c30037070 con 0x7f3c40105a40 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c3e7fc640 1 -- 172.21.15.37:0/4178619235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c30032920 con 0x7f3c40105a40 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c3e7fc640 1 -- 172.21.15.37:0/4178619235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c30030810 con 0x7f3c40105a40 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c4603f640 1 -- 172.21.15.37:0/4178619235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 msgr2=0x7f3c40105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c4603f640 1 --2- 172.21.15.37:0/4178619235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c40105e40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3c30009820 tx=0x7f3c3002a270 comp rx=0 tx=0).stop 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c4603f640 1 -- 172.21.15.37:0/4178619235 shutdown_connections 2024-09-19T06:33:03.593 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c4603f640 1 --2- 172.21.15.37:0/4178619235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c40105e40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c4603f640 1 -- 172.21.15.37:0/4178619235 >> 172.21.15.37:0/4178619235 conn(0x7f3c401015f0 msgr2=0x7f3c40103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c4603f640 1 -- 172.21.15.37:0/4178619235 shutdown_connections 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.734+0000 7f3c4603f640 1 -- 172.21.15.37:0/4178619235 wait complete. 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.735+0000 7f3c4603f640 1 Processor -- start 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.735+0000 7f3c4603f640 1 -- start start 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.735+0000 7f3c4603f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c401082a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.735+0000 7f3c4603f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4010bc50 con 0x7f3c40105a40 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c401082a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c401082a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51214/0 (socket says 172.21.15.37:51214) 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3f7fe640 1 -- 172.21.15.37:0/79013156 learned_addr learned my addr 172.21.15.37:0/79013156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3f7fe640 1 -- 172.21.15.37:0/79013156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c401087e0 con 0x7f3c40105a40 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3f7fe640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c401082a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c300097f0 tx=0x7f3c300327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:03.594 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3cff9640 1 -- 172.21.15.37:0/79013156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c30037040 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3cff9640 1 -- 172.21.15.37:0/79013156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c30030cc0 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c401069d0 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c3cff9640 1 -- 172.21.15.37:0/79013156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3c3003a640 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.736+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c40106f10 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.737+0000 7f3c3cff9640 1 -- 172.21.15.37:0/79013156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3c3003a8f0 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.737+0000 7f3c3cff9640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3c28039300 0x7f3c2803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.737+0000 7f3c3cff9640 1 -- 172.21.15.37:0/79013156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3c3006c980 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.737+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c40105ec0 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.738+0000 7f3c3effd640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3c28039300 0x7f3c2803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.738+0000 7f3c3effd640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3c28039300 0x7f3c2803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3c340098c0 tx=0x7f3c34006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:02.742+0000 7f3c3cff9640 1 -- 172.21.15.37:0/79013156 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3c300371e0 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.013+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3c40107190 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.015+0000 7f3c3cff9640 1 -- 172.21.15.37:0/79013156 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f3c30031c30 con 0x7f3c40105a40 2024-09-19T06:33:03.595 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3c28039300 msgr2=0x7f3c2803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3c28039300 0x7f3c2803b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3c340098c0 tx=0x7f3c34006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 msgr2=0x7f3c401082a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c401082a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3c300097f0 tx=0x7f3c300327b0 comp rx=0 tx=0).stop 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 shutdown_connections 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3c28039300 0x7f3c2803b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 --2- 172.21.15.37:0/79013156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c40105a40 0x7f3c401082a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 >> 172.21.15.37:0/79013156 conn(0x7f3c401015f0 msgr2=0x7f3c40102180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 shutdown_connections 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:/usr/bin/ceph: stderr 2024-09-19T06:33:03.018+0000 7f3c4603f640 1 -- 172.21.15.37:0/79013156 wait complete. 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-19T06:33:03.596 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:Or, if you are only running a single cluster on this host: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: ceph telemetry on 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout:For more information see: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-19T06:33:03.597 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:03.598 INFO:teuthology.orchestra.run.smithi037.stdout:Bootstrap complete. 2024-09-19T06:33:03.634 INFO:tasks.cephadm:Fetching config... 2024-09-19T06:33:03.634 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:33:03.634 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-19T06:33:03.652 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-19T06:33:03.653 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:33:03.653 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-19T06:33:03.709 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-19T06:33:03.709 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:33:03.709 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-19T06:33:03.775 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-19T06:33:03.775 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:33:03.776 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-19T06:33:03.832 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-19T06:33:03.832 DEBUG:teuthology.orchestra.run.smithi037:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncLbIdA0HBBVCzqYL8XNd9GmNpzpxIVYHuuuV79R3Z/FSIm71/ca45Z425fjVKCX2vt+UaHbcP+kBvnuSsfsVrcXl8VO7sKObdQyfkcjmwZ9sLgIcGM3ekw0+bKjaSBef4FPvtP1BPYXuOySZMO55u4cheFjNDC/mV4zpuTVe8qGhpOOSKe9NPiB8QQOk9iW2+kmGz90ra5tuNmhrBpRzafuSgbaOm7U8I5C0JoW8twK8N46v0LCsG9fZBTynnOssHBEKnx8yiQvtGlUY39vI2uud8tfXHwSCC9eQN6bhTL0VN/FqbnviJi9CFeS7nl4figGKR5AHhPMR1K1XPDUJs5Qmp+SQ1rDKrx5Cpa8IWZJSW/8Ss1Wldu+bnKDxUbdcxINl9Nh58JjRxL8m0S+opYvCjf5/OekR92sG1QGQzcTkkQrchS0V02f7P5edC4iLtYz+rMpBjf/g7OBnavzcnr+KnnT7L1C6omELycMjXwj4J6IDoX1+fKTuRsXeyQM= ceph-b715dee2-7650-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:33:03.907 INFO:teuthology.orchestra.run.smithi037.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncLbIdA0HBBVCzqYL8XNd9GmNpzpxIVYHuuuV79R3Z/FSIm71/ca45Z425fjVKCX2vt+UaHbcP+kBvnuSsfsVrcXl8VO7sKObdQyfkcjmwZ9sLgIcGM3ekw0+bKjaSBef4FPvtP1BPYXuOySZMO55u4cheFjNDC/mV4zpuTVe8qGhpOOSKe9NPiB8QQOk9iW2+kmGz90ra5tuNmhrBpRzafuSgbaOm7U8I5C0JoW8twK8N46v0LCsG9fZBTynnOssHBEKnx8yiQvtGlUY39vI2uud8tfXHwSCC9eQN6bhTL0VN/FqbnviJi9CFeS7nl4figGKR5AHhPMR1K1XPDUJs5Qmp+SQ1rDKrx5Cpa8IWZJSW/8Ss1Wldu+bnKDxUbdcxINl9Nh58JjRxL8m0S+opYvCjf5/OekR92sG1QGQzcTkkQrchS0V02f7P5edC4iLtYz+rMpBjf/g7OBnavzcnr+KnnT7L1C6omELycMjXwj4J6IDoX1+fKTuRsXeyQM= ceph-b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:03.920 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncLbIdA0HBBVCzqYL8XNd9GmNpzpxIVYHuuuV79R3Z/FSIm71/ca45Z425fjVKCX2vt+UaHbcP+kBvnuSsfsVrcXl8VO7sKObdQyfkcjmwZ9sLgIcGM3ekw0+bKjaSBef4FPvtP1BPYXuOySZMO55u4cheFjNDC/mV4zpuTVe8qGhpOOSKe9NPiB8QQOk9iW2+kmGz90ra5tuNmhrBpRzafuSgbaOm7U8I5C0JoW8twK8N46v0LCsG9fZBTynnOssHBEKnx8yiQvtGlUY39vI2uud8tfXHwSCC9eQN6bhTL0VN/FqbnviJi9CFeS7nl4figGKR5AHhPMR1K1XPDUJs5Qmp+SQ1rDKrx5Cpa8IWZJSW/8Ss1Wldu+bnKDxUbdcxINl9Nh58JjRxL8m0S+opYvCjf5/OekR92sG1QGQzcTkkQrchS0V02f7P5edC4iLtYz+rMpBjf/g7OBnavzcnr+KnnT7L1C6omELycMjXwj4J6IDoX1+fKTuRsXeyQM= ceph-b715dee2-7650-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:33:03.961 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncLbIdA0HBBVCzqYL8XNd9GmNpzpxIVYHuuuV79R3Z/FSIm71/ca45Z425fjVKCX2vt+UaHbcP+kBvnuSsfsVrcXl8VO7sKObdQyfkcjmwZ9sLgIcGM3ekw0+bKjaSBef4FPvtP1BPYXuOySZMO55u4cheFjNDC/mV4zpuTVe8qGhpOOSKe9NPiB8QQOk9iW2+kmGz90ra5tuNmhrBpRzafuSgbaOm7U8I5C0JoW8twK8N46v0LCsG9fZBTynnOssHBEKnx8yiQvtGlUY39vI2uud8tfXHwSCC9eQN6bhTL0VN/FqbnviJi9CFeS7nl4figGKR5AHhPMR1K1XPDUJs5Qmp+SQ1rDKrx5Cpa8IWZJSW/8Ss1Wldu+bnKDxUbdcxINl9Nh58JjRxL8m0S+opYvCjf5/OekR92sG1QGQzcTkkQrchS0V02f7P5edC4iLtYz+rMpBjf/g7OBnavzcnr+KnnT7L1C6omELycMjXwj4J6IDoX1+fKTuRsXeyQM= ceph-b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:03.974 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncLbIdA0HBBVCzqYL8XNd9GmNpzpxIVYHuuuV79R3Z/FSIm71/ca45Z425fjVKCX2vt+UaHbcP+kBvnuSsfsVrcXl8VO7sKObdQyfkcjmwZ9sLgIcGM3ekw0+bKjaSBef4FPvtP1BPYXuOySZMO55u4cheFjNDC/mV4zpuTVe8qGhpOOSKe9NPiB8QQOk9iW2+kmGz90ra5tuNmhrBpRzafuSgbaOm7U8I5C0JoW8twK8N46v0LCsG9fZBTynnOssHBEKnx8yiQvtGlUY39vI2uud8tfXHwSCC9eQN6bhTL0VN/FqbnviJi9CFeS7nl4figGKR5AHhPMR1K1XPDUJs5Qmp+SQ1rDKrx5Cpa8IWZJSW/8Ss1Wldu+bnKDxUbdcxINl9Nh58JjRxL8m0S+opYvCjf5/OekR92sG1QGQzcTkkQrchS0V02f7P5edC4iLtYz+rMpBjf/g7OBnavzcnr+KnnT7L1C6omELycMjXwj4J6IDoX1+fKTuRsXeyQM= ceph-b715dee2-7650-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-19T06:33:04.016 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCncLbIdA0HBBVCzqYL8XNd9GmNpzpxIVYHuuuV79R3Z/FSIm71/ca45Z425fjVKCX2vt+UaHbcP+kBvnuSsfsVrcXl8VO7sKObdQyfkcjmwZ9sLgIcGM3ekw0+bKjaSBef4FPvtP1BPYXuOySZMO55u4cheFjNDC/mV4zpuTVe8qGhpOOSKe9NPiB8QQOk9iW2+kmGz90ra5tuNmhrBpRzafuSgbaOm7U8I5C0JoW8twK8N46v0LCsG9fZBTynnOssHBEKnx8yiQvtGlUY39vI2uud8tfXHwSCC9eQN6bhTL0VN/FqbnviJi9CFeS7nl4figGKR5AHhPMR1K1XPDUJs5Qmp+SQ1rDKrx5Cpa8IWZJSW/8Ss1Wldu+bnKDxUbdcxINl9Nh58JjRxL8m0S+opYvCjf5/OekR92sG1QGQzcTkkQrchS0V02f7P5edC4iLtYz+rMpBjf/g7OBnavzcnr+KnnT7L1C6omELycMjXwj4J6IDoX1+fKTuRsXeyQM= ceph-b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:04.029 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-19T06:33:04.282 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:04.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.772+0000 7f2b03c86640 1 -- 172.21.15.37:0/3712226955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 msgr2=0x7f2afc100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:04.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.772+0000 7f2b03c86640 1 --2- 172.21.15.37:0/3712226955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc100940 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009880 tx=0x7f2af802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.772+0000 7f2b03c86640 1 -- 172.21.15.37:0/3712226955 shutdown_connections 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.772+0000 7f2b03c86640 1 --2- 172.21.15.37:0/3712226955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc100940 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.772+0000 7f2b03c86640 1 -- 172.21.15.37:0/3712226955 >> 172.21.15.37:0/3712226955 conn(0x7f2afc0fbcd0 msgr2=0x7f2afc0fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.772+0000 7f2b03c86640 1 -- 172.21.15.37:0/3712226955 shutdown_connections 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.772+0000 7f2b03c86640 1 -- 172.21.15.37:0/3712226955 wait complete. 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b03c86640 1 Processor -- start 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b03c86640 1 -- start start 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b03c86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc06a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b03c86640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2afc1065f0 con 0x7f2afc100540 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b01a22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc06a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b01a22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc06a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51236/0 (socket says 172.21.15.37:51236) 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b01a22640 1 -- 172.21.15.37:0/224680184 learned_addr learned my addr 172.21.15.37:0/224680184 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b01a22640 1 -- 172.21.15.37:0/224680184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2afc199ab0 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2b01a22640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc06a130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2af8004f60 tx=0x7f2af80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.773+0000 7f2af2ffd640 1 -- 172.21.15.37:0/224680184 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af803f070 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.774+0000 7f2af2ffd640 1 -- 172.21.15.37:0/224680184 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2af8004cc0 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.774+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2afc199c80 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.774+0000 7f2af2ffd640 1 -- 172.21.15.37:0/224680184 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2af8035030 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.774+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2afc19a020 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.775+0000 7f2af2ffd640 1 -- 172.21.15.37:0/224680184 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f2af8042070 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.775+0000 7f2af2ffd640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2ae8039300 0x7f2ae803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.775+0000 7f2af2ffd640 1 -- 172.21.15.37:0/224680184 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2af806d2b0 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.775+0000 7f2b01221640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2ae8039300 0x7f2ae803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.775+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2afc106d70 con 0x7f2afc100540 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.775+0000 7f2b01221640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2ae8039300 0x7f2ae803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0098c0 tx=0x7f2aec006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.780+0000 7f2af2ffd640 1 -- 172.21.15.37:0/224680184 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2af8003dd0 con 0x7f2afc100540 2024-09-19T06:33:04.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.959+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f2afc02d7c0 con 0x7f2afc100540 2024-09-19T06:33:04.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.962+0000 7f2af2ffd640 1 -- 172.21.15.37:0/224680184 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f2af8003b40 con 0x7f2afc100540 2024-09-19T06:33:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2ae8039300 msgr2=0x7f2ae803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2ae8039300 0x7f2ae803b7c0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2aec0098c0 tx=0x7f2aec006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 msgr2=0x7f2afc06a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:04.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc06a130 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2af8004f60 tx=0x7f2af80048f0 comp rx=0 tx=0).stop 2024-09-19T06:33:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 shutdown_connections 2024-09-19T06:33:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2ae8039300 0x7f2ae803b7c0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 --2- 172.21.15.37:0/224680184 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2afc100540 0x7f2afc06a130 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 >> 172.21.15.37:0/224680184 conn(0x7f2afc0fbcd0 msgr2=0x7f2afc0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.964+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 shutdown_connections 2024-09-19T06:33:04.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:04.965+0000 7f2b03c86640 1 -- 172.21.15.37:0/224680184 wait complete. 2024-09-19T06:33:05.512 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-19T06:33:05.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-19T06:33:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:06.067 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:05 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/224680184' entity='client.admin' 2024-09-19T06:33:06.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:06.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:06.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi037", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:33:06.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:06.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:06.068 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.778+0000 7f92b3f0f640 1 -- 172.21.15.37:0/1094509729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 msgr2=0x7f92ac101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.778+0000 7f92b3f0f640 1 --2- 172.21.15.37:0/1094509729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac101a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009880 tx=0x7f92a802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.779+0000 7f92b3f0f640 1 -- 172.21.15.37:0/1094509729 shutdown_connections 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.779+0000 7f92b3f0f640 1 --2- 172.21.15.37:0/1094509729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac101a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.779+0000 7f92b3f0f640 1 -- 172.21.15.37:0/1094509729 >> 172.21.15.37:0/1094509729 conn(0x7f92ac0fcdb0 msgr2=0x7f92ac0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.779+0000 7f92b3f0f640 1 -- 172.21.15.37:0/1094509729 shutdown_connections 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.779+0000 7f92b3f0f640 1 -- 172.21.15.37:0/1094509729 wait complete. 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.779+0000 7f92b3f0f640 1 Processor -- start 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.779+0000 7f92b3f0f640 1 -- start start 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b3f0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:06.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b3f0f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92ac107730 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b1cab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b1cab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51248/0 (socket says 172.21.15.37:51248) 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b1cab640 1 -- 172.21.15.37:0/2536888308 learned_addr learned my addr 172.21.15.37:0/2536888308 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b1cab640 1 -- 172.21.15.37:0/2536888308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92ac0763f0 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b1cab640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92a8004f60 tx=0x7f92a80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92a2ffd640 1 -- 172.21.15.37:0/2536888308 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92a803f070 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92ac0745e0 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92a2ffd640 1 -- 172.21.15.37:0/2536888308 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92a8004cc0 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92a2ffd640 1 -- 172.21.15.37:0/2536888308 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92a8035030 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.780+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92ac074b20 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.781+0000 7f92a2ffd640 1 -- 172.21.15.37:0/2536888308 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f92a8042070 con 0x7f92ac101620 2024-09-19T06:33:06.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.782+0000 7f92a2ffd640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9298039300 0x7f929803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:06.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.782+0000 7f92a2ffd640 1 -- 172.21.15.37:0/2536888308 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f92a806d060 con 0x7f92ac101620 2024-09-19T06:33:06.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.782+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92ac107eb0 con 0x7f92ac101620 2024-09-19T06:33:06.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.782+0000 7f92b14aa640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9298039300 0x7f929803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:06.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.782+0000 7f92b14aa640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9298039300 0x7f929803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f929c0098c0 tx=0x7f929c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:06.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.787+0000 7f92a2ffd640 1 -- 172.21.15.37:0/2536888308 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f92a80433b0 con 0x7f92ac101620 2024-09-19T06:33:06.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.971+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f92ac190220 con 0x7f9298039300 2024-09-19T06:33:06.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.975+0000 7f92a2ffd640 1 -- 172.21.15.37:0/2536888308 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f92ac190220 con 0x7f9298039300 2024-09-19T06:33:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9298039300 msgr2=0x7f929803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9298039300 0x7f929803b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f929c0098c0 tx=0x7f929c006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:06.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 msgr2=0x7f92ac075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f92a8004f60 tx=0x7f92a80048f0 comp rx=0 tx=0).stop 2024-09-19T06:33:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 shutdown_connections 2024-09-19T06:33:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9298039300 0x7f929803b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 --2- 172.21.15.37:0/2536888308 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92ac101620 0x7f92ac075eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 >> 172.21.15.37:0/2536888308 conn(0x7f92ac0fcdb0 msgr2=0x7f92ac0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.977+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 shutdown_connections 2024-09-19T06:33:06.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:06.978+0000 7f92b3f0f640 1 -- 172.21.15.37:0/2536888308 wait complete. 2024-09-19T06:33:07.773 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi096 2024-09-19T06:33:07.774 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:33:07.774 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.conf 2024-09-19T06:33:07.794 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:33:07.794 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:07.853 INFO:tasks.cephadm:Adding host smithi096 to orchestrator... 2024-09-19T06:33:07.853 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch host add smithi096 2024-09-19T06:33:08.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:08.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:08.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:08.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:08.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:08.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: Updating smithi037:/etc/ceph/ceph.conf 2024-09-19T06:33:08.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:08.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:08.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:08.644 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 -- 172.21.15.37:0/728080064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 msgr2=0x7f3328102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 --2- 172.21.15.37:0/728080064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f3328102b90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3324009880 tx=0x7f332402a670 comp rx=0 tx=0).stop 2024-09-19T06:33:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 -- 172.21.15.37:0/728080064 shutdown_connections 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 --2- 172.21.15.37:0/728080064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f3328102b90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 -- 172.21.15.37:0/728080064 >> 172.21.15.37:0/728080064 conn(0x7f33280fc1b0 msgr2=0x7f33280fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 -- 172.21.15.37:0/728080064 shutdown_connections 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 -- 172.21.15.37:0/728080064 wait complete. 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 Processor -- start 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 -- start start 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f33281971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.080+0000 7f33301e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f332810bcb0 con 0x7f33281027b0 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f332df80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f33281971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f332df80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f33281971b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51272/0 (socket says 172.21.15.37:51272) 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f332df80640 1 -- 172.21.15.37:0/1942242158 learned_addr learned my addr 172.21.15.37:0/1942242158 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f332df80640 1 -- 172.21.15.37:0/1942242158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33281976f0 con 0x7f33281027b0 2024-09-19T06:33:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f332df80640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f33281971b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3324009230 tx=0x7f3324003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f331effd640 1 -- 172.21.15.37:0/1942242158 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f332403f070 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f331effd640 1 -- 172.21.15.37:0/1942242158 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f332403bc80 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3328197980 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f331effd640 1 -- 172.21.15.37:0/1942242158 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3324035030 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.081+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3328197de0 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.082+0000 7f331effd640 1 -- 172.21.15.37:0/1942242158 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3324042070 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.082+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f332806a750 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.082+0000 7f331effd640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f33140392a0 0x7f331403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.083+0000 7f331effd640 1 -- 172.21.15.37:0/1942242158 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f332406cff0 con 0x7f33281027b0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.083+0000 7f332d77f640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f33140392a0 0x7f331403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.083+0000 7f332d77f640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f33140392a0 0x7f331403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33180098c0 tx=0x7f3318006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:09.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.087+0000 7f331effd640 1 -- 172.21.15.37:0/1942242158 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3324031e20 con 0x7f33281027b0 2024-09-19T06:33:09.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:09.265+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}) -- 0x7f3328198570 con 0x7f33140392a0 2024-09-19T06:33:09.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:08 smithi037 ceph-mon[31211]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:09.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:08 smithi037 ceph-mon[31211]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:09.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:08 smithi037 ceph-mon[31211]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:33:10.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:10 smithi037 ceph-mon[31211]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:11.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:11 smithi037 ceph-mon[31211]: Deploying cephadm binary to smithi096 2024-09-19T06:33:13.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.066+0000 7f331effd640 1 -- 172.21.15.37:0/1942242158 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f3328198570 con 0x7f33140392a0 2024-09-19T06:33:13.068 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi096' with addr '172.21.15.96' 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f33140392a0 msgr2=0x7f331403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f33140392a0 0x7f331403b760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33180098c0 tx=0x7f3318006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 msgr2=0x7f33281971b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f33281971b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3324009230 tx=0x7f3324003d40 comp rx=0 tx=0).stop 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 shutdown_connections 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f33140392a0 0x7f331403b760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 --2- 172.21.15.37:0/1942242158 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f33281027b0 0x7f33281971b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 >> 172.21.15.37:0/1942242158 conn(0x7f33280fc1b0 msgr2=0x7f3328103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 shutdown_connections 2024-09-19T06:33:13.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:13.069+0000 7f33301e4640 1 -- 172.21.15.37:0/1942242158 wait complete. 2024-09-19T06:33:13.681 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T06:33:13.931 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:14.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:14.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:14 smithi037 ceph-mon[31211]: Added host smithi096 2024-09-19T06:33:14.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:14.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:14.345 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:14.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 -- 172.21.15.37:0/599846163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 msgr2=0x7fc540106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:14.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 --2- 172.21.15.37:0/599846163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540106940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc530009880 tx=0x7fc53002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:14.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 -- 172.21.15.37:0/599846163 shutdown_connections 2024-09-19T06:33:14.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 --2- 172.21.15.37:0/599846163 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540106940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:14.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 -- 172.21.15.37:0/599846163 >> 172.21.15.37:0/599846163 conn(0x7fc5400fc1b0 msgr2=0x7fc5400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:14.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 -- 172.21.15.37:0/599846163 shutdown_connections 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 -- 172.21.15.37:0/599846163 wait complete. 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.370+0000 7fc544cd2640 1 Processor -- start 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc544cd2640 1 -- start start 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc544cd2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540197170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc544cd2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc54010ab90 con 0x7fc540106560 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc53e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540197170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc53e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540197170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34744/0 (socket says 172.21.15.37:34744) 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc53e59c640 1 -- 172.21.15.37:0/3597160598 learned_addr learned my addr 172.21.15.37:0/3597160598 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc53e59c640 1 -- 172.21.15.37:0/3597160598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5401976b0 con 0x7fc540106560 2024-09-19T06:33:14.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc53e59c640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540197170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc530009230 tx=0x7fc530003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.371+0000 7fc52f7fe640 1 -- 172.21.15.37:0/3597160598 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc53003f070 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.372+0000 7fc52f7fe640 1 -- 172.21.15.37:0/3597160598 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc53003bc80 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.372+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc540197940 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.372+0000 7fc52f7fe640 1 -- 172.21.15.37:0/3597160598 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc530035030 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.372+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc540197da0 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.372+0000 7fc52f7fe640 1 -- 172.21.15.37:0/3597160598 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc530042070 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.373+0000 7fc52f7fe640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5280392a0 0x7fc52803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.373+0000 7fc52f7fe640 1 -- 172.21.15.37:0/3597160598 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc53003b7a0 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.373+0000 7fc53dd9b640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5280392a0 0x7fc52803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.373+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc540101bd0 con 0x7fc540106560 2024-09-19T06:33:14.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.373+0000 7fc53dd9b640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5280392a0 0x7fc52803b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5340098c0 tx=0x7fc534006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:14.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.378+0000 7fc52f7fe640 1 -- 172.21.15.37:0/3597160598 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc53004c350 con 0x7fc540106560 2024-09-19T06:33:14.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.558+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc540198530 con 0x7fc5280392a0 2024-09-19T06:33:14.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.559+0000 7fc52f7fe640 1 -- 172.21.15.37:0/3597160598 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fc540198530 con 0x7fc5280392a0 2024-09-19T06:33:14.560 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:14.560 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}] 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5280392a0 msgr2=0x7fc52803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5280392a0 0x7fc52803b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc5340098c0 tx=0x7fc534006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 msgr2=0x7fc540197170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540197170 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc530009230 tx=0x7fc530003d40 comp rx=0 tx=0).stop 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 shutdown_connections 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5280392a0 0x7fc52803b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 --2- 172.21.15.37:0/3597160598 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc540106560 0x7fc540197170 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 >> 172.21.15.37:0/3597160598 conn(0x7fc5400fc1b0 msgr2=0x7fc540069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:14.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.561+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 shutdown_connections 2024-09-19T06:33:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:14.562+0000 7fc544cd2640 1 -- 172.21.15.37:0/3597160598 wait complete. 2024-09-19T06:33:15.199 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-09-19T06:33:15.199 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:33:15.199 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-09-19T06:33:15.219 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:33:15.219 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:15.277 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-09-19T06:33:15.277 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch host add smithi151 2024-09-19T06:33:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:15 smithi037 ceph-mon[31211]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:33:15.533 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:15.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 -- 172.21.15.37:0/2893764289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 msgr2=0x7fcad8106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:15.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 --2- 172.21.15.37:0/2893764289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009880 tx=0x7fcad402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:15.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 -- 172.21.15.37:0/2893764289 shutdown_connections 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 --2- 172.21.15.37:0/2893764289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 -- 172.21.15.37:0/2893764289 >> 172.21.15.37:0/2893764289 conn(0x7fcad80fc1b0 msgr2=0x7fcad80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 -- 172.21.15.37:0/2893764289 shutdown_connections 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 -- 172.21.15.37:0/2893764289 wait complete. 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 Processor -- start 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.951+0000 7fcae0116640 1 -- start start 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcae0116640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcae0116640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcad810ab90 con 0x7fcad8106560 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaddeb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaddeb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34748/0 (socket says 172.21.15.37:34748) 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaddeb2640 1 -- 172.21.15.37:0/4131554438 learned_addr learned my addr 172.21.15.37:0/4131554438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:15.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaddeb2640 1 -- 172.21.15.37:0/4131554438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad8197640 con 0x7fcad8106560 2024-09-19T06:33:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaddeb2640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8197100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009230 tx=0x7fcad4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaceffd640 1 -- 172.21.15.37:0/4131554438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad403f070 con 0x7fcad8106560 2024-09-19T06:33:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaceffd640 1 -- 172.21.15.37:0/4131554438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcad403bc80 con 0x7fcad8106560 2024-09-19T06:33:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad81978d0 con 0x7fcad8106560 2024-09-19T06:33:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.952+0000 7fcaceffd640 1 -- 172.21.15.37:0/4131554438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcad4035030 con 0x7fcad8106560 2024-09-19T06:33:15.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.953+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad819b9d0 con 0x7fcad8106560 2024-09-19T06:33:15.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.953+0000 7fcaceffd640 1 -- 172.21.15.37:0/4131554438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fcad4042070 con 0x7fcad8106560 2024-09-19T06:33:15.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.954+0000 7fcaceffd640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcac40392a0 0x7fcac403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:15.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.953+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcad8101bd0 con 0x7fcad8106560 2024-09-19T06:33:15.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.954+0000 7fcaceffd640 1 -- 172.21.15.37:0/4131554438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcad406d0b0 con 0x7fcad8106560 2024-09-19T06:33:15.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.954+0000 7fcadd6b1640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcac40392a0 0x7fcac403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:15.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.954+0000 7fcadd6b1640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcac40392a0 0x7fcac403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcac80098c0 tx=0x7fcac8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:15.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:15.959+0000 7fcaceffd640 1 -- 172.21.15.37:0/4131554438 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcad40314c0 con 0x7fcad8106560 2024-09-19T06:33:16.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:16.141+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) -- 0x7fcad81069f0 con 0x7fcac40392a0 2024-09-19T06:33:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:33:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T06:33:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:19.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:19 smithi037 ceph-mon[31211]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:19.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:19 smithi037 ceph-mon[31211]: Deploying cephadm binary to smithi151 2024-09-19T06:33:19.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:19 smithi037 ceph-mon[31211]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:19.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:19 smithi037 ceph-mon[31211]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:33:20.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.059+0000 7fcaceffd640 1 -- 172.21.15.37:0/4131554438 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcad81069f0 con 0x7fcac40392a0 2024-09-19T06:33:20.060 INFO:teuthology.orchestra.run.smithi037.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-09-19T06:33:20.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcac40392a0 msgr2=0x7fcac403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:20.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcac40392a0 0x7fcac403b760 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcac80098c0 tx=0x7fcac8006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 msgr2=0x7fcad8197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8197100 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcad4009230 tx=0x7fcad4003d40 comp rx=0 tx=0).stop 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 shutdown_connections 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcac40392a0 0x7fcac403b760 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 --2- 172.21.15.37:0/4131554438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcad8106560 0x7fcad8197100 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 >> 172.21.15.37:0/4131554438 conn(0x7fcad80fc1b0 msgr2=0x7fcad8069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 shutdown_connections 2024-09-19T06:33:20.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:20.061+0000 7fcae0116640 1 -- 172.21.15.37:0/4131554438 wait complete. 2024-09-19T06:33:20.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:20 smithi037 ceph-mon[31211]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:20.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:20.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:20.629 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-19T06:33:20.880 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:21.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:21 smithi037 ceph-mon[31211]: Added host smithi151 2024-09-19T06:33:21.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:21.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:21.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.300+0000 7fd27d4bd640 1 -- 172.21.15.37:0/3069477070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 msgr2=0x7fd278106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.300+0000 7fd27d4bd640 1 --2- 172.21.15.37:0/3069477070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278106940 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd268009880 tx=0x7fd26802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.300+0000 7fd27d4bd640 1 -- 172.21.15.37:0/3069477070 shutdown_connections 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.300+0000 7fd27d4bd640 1 --2- 172.21.15.37:0/3069477070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278106940 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.300+0000 7fd27d4bd640 1 -- 172.21.15.37:0/3069477070 >> 172.21.15.37:0/3069477070 conn(0x7fd2780fc1b0 msgr2=0x7fd2780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.300+0000 7fd27d4bd640 1 -- 172.21.15.37:0/3069477070 shutdown_connections 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.300+0000 7fd27d4bd640 1 -- 172.21.15.37:0/3069477070 wait complete. 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd27d4bd640 1 Processor -- start 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd27d4bd640 1 -- start start 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd27d4bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd27d4bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd27810ab90 con 0x7fd278106560 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd276ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49686/0 (socket says 172.21.15.37:49686) 2024-09-19T06:33:21.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd276ffd640 1 -- 172.21.15.37:0/2487600625 learned_addr learned my addr 172.21.15.37:0/2487600625 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd276ffd640 1 -- 172.21.15.37:0/2487600625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2781976a0 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd276ffd640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278197160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd268009230 tx=0x7fd268003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd274ff9640 1 -- 172.21.15.37:0/2487600625 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd26803f070 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd278197930 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd274ff9640 1 -- 172.21.15.37:0/2487600625 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd26803bc80 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd274ff9640 1 -- 172.21.15.37:0/2487600625 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd268035030 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.301+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27819ba30 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.302+0000 7fd274ff9640 1 -- 172.21.15.37:0/2487600625 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd268042070 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.303+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd278101bd0 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.303+0000 7fd274ff9640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd2640392a0 0x7fd26403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.303+0000 7fd274ff9640 1 -- 172.21.15.37:0/2487600625 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd26806cd50 con 0x7fd278106560 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.303+0000 7fd26ffff640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd2640392a0 0x7fd26403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:21.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.303+0000 7fd26ffff640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd2640392a0 0x7fd26403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd2600098c0 tx=0x7fd260006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:21.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.308+0000 7fd274ff9640 1 -- 172.21.15.37:0/2487600625 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd268031570 con 0x7fd278106560 2024-09-19T06:33:21.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.489+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2781069c0 con 0x7fd2640392a0 2024-09-19T06:33:21.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.490+0000 7fd274ff9640 1 -- 172.21.15.37:0/2487600625 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7fd2781069c0 con 0x7fd2640392a0 2024-09-19T06:33:21.491 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:21.491 INFO:teuthology.orchestra.run.smithi037.stdout:[{"addr": "172.21.15.37", "hostname": "smithi037", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd2640392a0 msgr2=0x7fd26403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd2640392a0 0x7fd26403b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd2600098c0 tx=0x7fd260006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 msgr2=0x7fd278197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278197160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd268009230 tx=0x7fd268003d40 comp rx=0 tx=0).stop 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 shutdown_connections 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd2640392a0 0x7fd26403b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 --2- 172.21.15.37:0/2487600625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd278106560 0x7fd278197160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 >> 172.21.15.37:0/2487600625 conn(0x7fd2780fc1b0 msgr2=0x7fd278069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 shutdown_connections 2024-09-19T06:33:21.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:21.492+0000 7fd27d4bd640 1 -- 172.21.15.37:0/2487600625 wait complete. 2024-09-19T06:33:22.128 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:22 smithi037 ceph-mon[31211]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:22.128 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:22.129 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-19T06:33:22.130 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-19T06:33:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:22.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.833+0000 7f64dfa46640 1 -- 172.21.15.37:0/2202450265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 msgr2=0x7f64d8102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.833+0000 7f64dfa46640 1 --2- 172.21.15.37:0/2202450265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8102b70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64d4009810 tx=0x7f64d402a540 comp rx=0 tx=0).stop 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.833+0000 7f64dfa46640 1 -- 172.21.15.37:0/2202450265 shutdown_connections 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.833+0000 7f64dfa46640 1 --2- 172.21.15.37:0/2202450265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8102b70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.833+0000 7f64dfa46640 1 -- 172.21.15.37:0/2202450265 >> 172.21.15.37:0/2202450265 conn(0x7f64d8077310 msgr2=0x7f64d80fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.833+0000 7f64dfa46640 1 -- 172.21.15.37:0/2202450265 shutdown_connections 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.833+0000 7f64dfa46640 1 -- 172.21.15.37:0/2202450265 wait complete. 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.834+0000 7f64dfa46640 1 Processor -- start 2024-09-19T06:33:22.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.834+0000 7f64dfa46640 1 -- start start 2024-09-19T06:33:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.834+0000 7f64dfa46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.834+0000 7f64dfa46640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f64d81011e0 con 0x7f64d8102790 2024-09-19T06:33:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.834+0000 7f64dd7e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.834+0000 7f64dd7e2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8199400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49704/0 (socket says 172.21.15.37:49704) 2024-09-19T06:33:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.834+0000 7f64dd7e2640 1 -- 172.21.15.37:0/3204860257 learned_addr learned my addr 172.21.15.37:0/3204860257 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:22.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.835+0000 7f64dd7e2640 1 -- 172.21.15.37:0/3204860257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64d8199940 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.835+0000 7f64dd7e2640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8199400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64d40045e0 tx=0x7f64d4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.835+0000 7f64ce7fc640 1 -- 172.21.15.37:0/3204860257 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64d4030e70 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.835+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64d8199bd0 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.835+0000 7f64ce7fc640 1 -- 172.21.15.37:0/3204860257 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64d40318e0 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.835+0000 7f64ce7fc640 1 -- 172.21.15.37:0/3204860257 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64d40049c0 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.835+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64d819a030 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.836+0000 7f64ce7fc640 1 -- 172.21.15.37:0/3204860257 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f64d4039040 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.836+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64d8069f30 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.836+0000 7f64ce7fc640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f64c4039300 0x7f64c403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.836+0000 7f64ce7fc640 1 -- 172.21.15.37:0/3204860257 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f64d406d640 con 0x7f64d8102790 2024-09-19T06:33:22.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.836+0000 7f64dcfe1640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f64c4039300 0x7f64c403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:22.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.837+0000 7f64dcfe1640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f64c4039300 0x7f64c403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098c0 tx=0x7f64c8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:22.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:22.841+0000 7f64ce7fc640 1 -- 172.21.15.37:0/3204860257 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f64d4040030 con 0x7f64d8102790 2024-09-19T06:33:23.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.023+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f64d819a760 con 0x7f64d8102790 2024-09-19T06:33:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.076+0000 7f64ce7fc640 1 -- 172.21.15.37:0/3204860257 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f64d4003780 con 0x7f64d8102790 2024-09-19T06:33:23.077 INFO:teuthology.orchestra.run.smithi037.stderr:adjusted tunables profile to default 2024-09-19T06:33:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.078+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f64c4039300 msgr2=0x7f64c403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.078+0000 7f64dfa46640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f64c4039300 0x7f64c403b7c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f64c80098c0 tx=0x7f64c8006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.078+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 msgr2=0x7f64d8199400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:23.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.078+0000 7f64dfa46640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8199400 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f64d40045e0 tx=0x7f64d4004610 comp rx=0 tx=0).stop 2024-09-19T06:33:23.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.078+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 shutdown_connections 2024-09-19T06:33:23.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.079+0000 7f64dfa46640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f64c4039300 0x7f64c403b7c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:23.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.079+0000 7f64dfa46640 1 --2- 172.21.15.37:0/3204860257 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f64d8102790 0x7f64d8199400 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:23.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.079+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 >> 172.21.15.37:0/3204860257 conn(0x7f64d8077310 msgr2=0x7f64d80fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:23.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.079+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 shutdown_connections 2024-09-19T06:33:23.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:23.079+0000 7f64dfa46640 1 -- 172.21.15.37:0/3204860257 wait complete. 2024-09-19T06:33:23.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:23 smithi037 ceph-mon[31211]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:33:23.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:23 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3204860257' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T06:33:23.671 INFO:tasks.cephadm:Adding mon.a on smithi037 2024-09-19T06:33:23.672 INFO:tasks.cephadm:Adding mon.b on smithi096 2024-09-19T06:33:23.672 INFO:tasks.cephadm:Adding mon.c on smithi151 2024-09-19T06:33:23.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi037:172.21.15.37=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c' 2024-09-19T06:33:24.320 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:33:24.366 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-19T06:33:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:24 smithi037 ceph-mon[31211]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:24 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3204860257' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T06:33:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:24 smithi037 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:24.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.844+0000 7f83cbd40640 1 -- 172.21.15.151:0/3158814450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 msgr2=0x7f83c4100940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:24.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.844+0000 7f83cbd40640 1 --2- 172.21.15.151:0/3158814450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c4100940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f83c0009880 tx=0x7f83c002a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:24.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.844+0000 7f83cbd40640 1 -- 172.21.15.151:0/3158814450 shutdown_connections 2024-09-19T06:33:24.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.844+0000 7f83cbd40640 1 --2- 172.21.15.151:0/3158814450 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c4100940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:24.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.844+0000 7f83cbd40640 1 -- 172.21.15.151:0/3158814450 >> 172.21.15.151:0/3158814450 conn(0x7f83c40fbcd0 msgr2=0x7f83c40fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:24.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.844+0000 7f83cbd40640 1 -- 172.21.15.151:0/3158814450 shutdown_connections 2024-09-19T06:33:24.861 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.844+0000 7f83cbd40640 1 -- 172.21.15.151:0/3158814450 wait complete. 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83cbd40640 1 Processor -- start 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83cbd40640 1 -- start start 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83cbd40640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c406a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83cbd40640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83c41065f0 con 0x7f83c4100540 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83c9adc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c406a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83c9adc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c406a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.151:60792/0 (socket says 172.21.15.151:60792) 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83c9adc640 1 -- 172.21.15.151:0/2104020117 learned_addr learned my addr 172.21.15.151:0/2104020117 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.845+0000 7f83c9adc640 1 -- 172.21.15.151:0/2104020117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83c4199ab0 con 0x7f83c4100540 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.846+0000 7f83c9adc640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c406a130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83c0004f60 tx=0x7f83c00048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.846+0000 7f83baffd640 1 -- 172.21.15.151:0/2104020117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c003f070 con 0x7f83c4100540 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.846+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83c4199c80 con 0x7f83c4100540 2024-09-19T06:33:24.862 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.846+0000 7f83baffd640 1 -- 172.21.15.151:0/2104020117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83c0004d40 con 0x7f83c4100540 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.846+0000 7f83baffd640 1 -- 172.21.15.151:0/2104020117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0035030 con 0x7f83c4100540 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.846+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83c419a020 con 0x7f83c4100540 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.847+0000 7f83baffd640 1 -- 172.21.15.151:0/2104020117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f83c0042070 con 0x7f83c4100540 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.847+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83c4106d70 con 0x7f83c4100540 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.847+0000 7f83baffd640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83b00392a0 0x7f83b003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.847+0000 7f83baffd640 1 -- 172.21.15.151:0/2104020117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f83c006d0c0 con 0x7f83c4100540 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.847+0000 7f83c92db640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83b00392a0 0x7f83b003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.848+0000 7f83c92db640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83b00392a0 0x7f83b003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83b40098c0 tx=0x7f83b4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:24.863 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:24.852+0000 7f83baffd640 1 -- 172.21.15.151:0/2104020117 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f83c004a260 con 0x7f83c4100540 2024-09-19T06:33:25.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.042+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}) -- 0x7f83c402db40 con 0x7f83b00392a0 2024-09-19T06:33:25.049 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.048+0000 7f83baffd640 1 -- 172.21.15.151:0/2104020117 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f83c402db40 con 0x7f83b00392a0 2024-09-19T06:33:25.050 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mon update... 2024-09-19T06:33:25.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83b00392a0 msgr2=0x7f83b003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:25.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83b00392a0 0x7f83b003b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f83b40098c0 tx=0x7f83b4006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:25.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 msgr2=0x7f83c406a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:25.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c406a130 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f83c0004f60 tx=0x7f83c00048f0 comp rx=0 tx=0).stop 2024-09-19T06:33:25.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 shutdown_connections 2024-09-19T06:33:25.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83b00392a0 0x7f83b003b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:25.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 --2- 172.21.15.151:0/2104020117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83c4100540 0x7f83c406a130 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:25.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.050+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 >> 172.21.15.151:0/2104020117 conn(0x7f83c40fbcd0 msgr2=0x7f83c40fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:25.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.051+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 shutdown_connections 2024-09-19T06:33:25.053 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:25.051+0000 7f83cbd40640 1 -- 172.21.15.151:0/2104020117 wait complete. 2024-09-19T06:33:25.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:25.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:25.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:25.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:25 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:25.666 DEBUG:teuthology.orchestra.run.smithi096:mon.b> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.b.service 2024-09-19T06:33:25.669 DEBUG:teuthology.orchestra.run.smithi151:mon.c> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.c.service 2024-09-19T06:33:25.672 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:33:25.672 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:33:26.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:26 smithi037 ceph-mon[31211]: Deploying daemon mon.c on smithi151 2024-09-19T06:33:26.772 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:27.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:27 smithi037 ceph-mon[31211]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.782+0000 7fd4fa80d640 1 -- 172.21.15.151:0/2554347204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 msgr2=0x7fd4f4106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.782+0000 7fd4fa80d640 1 --2- 172.21.15.151:0/2554347204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4106940 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009880 tx=0x7fd4e402a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.782+0000 7fd4fa80d640 1 -- 172.21.15.151:0/2554347204 shutdown_connections 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.782+0000 7fd4fa80d640 1 --2- 172.21.15.151:0/2554347204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4106940 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.782+0000 7fd4fa80d640 1 -- 172.21.15.151:0/2554347204 >> 172.21.15.151:0/2554347204 conn(0x7fd4f40fc1b0 msgr2=0x7fd4f40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.782+0000 7fd4fa80d640 1 -- 172.21.15.151:0/2554347204 shutdown_connections 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.782+0000 7fd4fa80d640 1 -- 172.21.15.151:0/2554347204 wait complete. 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.783+0000 7fd4fa80d640 1 Processor -- start 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.783+0000 7fd4fa80d640 1 -- start start 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.783+0000 7fd4fa80d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:27.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.783+0000 7fd4fa80d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f410ab90 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.784+0000 7fd4f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.784+0000 7fd4f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.151:60810/0 (socket says 172.21.15.151:60810) 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.784+0000 7fd4f3fff640 1 -- 172.21.15.151:0/902543445 learned_addr learned my addr 172.21.15.151:0/902543445 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.784+0000 7fd4f3fff640 1 -- 172.21.15.151:0/902543445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f4197660 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.784+0000 7fd4f3fff640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009230 tx=0x7fd4e4003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.784+0000 7fd4f17fa640 1 -- 172.21.15.151:0/902543445 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4e403f070 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.784+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f41978f0 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.785+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f419b9f0 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.785+0000 7fd4f17fa640 1 -- 172.21.15.151:0/902543445 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd4e4005500 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.785+0000 7fd4f17fa640 1 -- 172.21.15.151:0/902543445 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4e4049060 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.785+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e0005290 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.785+0000 7fd4f17fa640 1 -- 172.21.15.151:0/902543445 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd4e4042070 con 0x7fd4f4106560 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.786+0000 7fd4f17fa640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd4dc0392a0 0x7fd4dc03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:27.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.786+0000 7fd4f17fa640 1 -- 172.21.15.151:0/902543445 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd4e406cb40 con 0x7fd4f4106560 2024-09-19T06:33:27.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.786+0000 7fd4f37fe640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd4dc0392a0 0x7fd4dc03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:27.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.786+0000 7fd4f37fe640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd4dc0392a0 0x7fd4dc03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80098c0 tx=0x7fd4e8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:27.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:27.791+0000 7fd4f17fa640 1 -- 172.21.15.151:0/902543445 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd4e403b850 con 0x7fd4f4106560 2024-09-19T06:33:28.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.026+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd4e0005580 con 0x7fd4f4106560 2024-09-19T06:33:28.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.027+0000 7fd4f17fa640 1 -- 172.21.15.151:0/902543445 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7fd4e40046c0 con 0x7fd4f4106560 2024-09-19T06:33:28.028 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:33:28.028 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","modified":"2024-09-19T06:31:58.000017Z","created":"2024-09-19T06:31:58.000017Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-19T06:33:28.029 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 1 2024-09-19T06:33:28.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd4dc0392a0 msgr2=0x7fd4dc03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:28.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd4dc0392a0 0x7fd4dc03b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd4e80098c0 tx=0x7fd4e8006db0 comp rx=0 tx=0).stop 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 msgr2=0x7fd4f4197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009230 tx=0x7fd4e4003d40 comp rx=0 tx=0).stop 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 shutdown_connections 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fd4dc0392a0 0x7fd4dc03b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 --2- 172.21.15.151:0/902543445 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4f4106560 0x7fd4f4197120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 >> 172.21.15.151:0/902543445 conn(0x7fd4f40fc1b0 msgr2=0x7fd4f4069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 shutdown_connections 2024-09-19T06:33:28.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:28.029+0000 7fd4fa80d640 1 -- 172.21.15.151:0/902543445 wait complete. 2024-09-19T06:33:28.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:28 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/902543445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:29.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:29 smithi037 ceph-mon[31211]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:30.007 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:33:30.008 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:33:30.261 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:33:35.104 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.102+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2527577090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aac003530 msgr2=0x7f4aac0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:35.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.102+0000 7f4ac8a6c640 1 --2- 172.21.15.151:0/2527577090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aac003530 0x7f4aac0059c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4ac4102af0 tx=0x7f4ab402c490 comp rx=0 tx=0).stop 2024-09-19T06:33:35.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.102+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2527577090 shutdown_connections 2024-09-19T06:33:35.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.102+0000 7f4ac8a6c640 1 --2- 172.21.15.151:0/2527577090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4aac003530 0x7f4aac0059c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:35.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.102+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2527577090 >> 172.21.15.151:0/2527577090 conn(0x7f4ac40fd320 msgr2=0x7f4ac40ff740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:35.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.103+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2527577090 shutdown_connections 2024-09-19T06:33:35.105 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.103+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2527577090 wait complete. 2024-09-19T06:33:35.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.104+0000 7f4ac8a6c640 1 Processor -- start 2024-09-19T06:33:35.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.104+0000 7f4ac8a6c640 1 -- start start 2024-09-19T06:33:35.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.104+0000 7f4ac8a6c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 0x7f4ac41994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:35.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.104+0000 7f4ac8a6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac41999f0 0x7f4ac419dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:35.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.104+0000 7f4ac8a6c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac4109780 con 0x7f4ac41999f0 2024-09-19T06:33:35.107 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.104+0000 7f4ac8a6c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ac4109600 con 0x7f4aac003530 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 0x7f4ac41994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 0x7f4ac41994b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47038/0 (socket says 172.21.15.151:47038) 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac259c640 1 -- 172.21.15.151:0/2995676619 learned_addr learned my addr 172.21.15.151:0/2995676619 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac1d9b640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac41999f0 0x7f4ac419dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac259c640 1 -- 172.21.15.151:0/2995676619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 msgr2=0x7f4ac41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac259c640 1 -- 172.21.15.151:0/2995676619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 msgr2=0x7f4ac41994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac259c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 0x7f4ac41994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac259c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 0x7f4ac41994b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:33:35.108 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac1d9b640 1 -- 172.21.15.151:0/2995676619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 msgr2=0x7f4ac41994b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:33:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac1d9b640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 0x7f4ac41994b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac1d9b640 1 -- 172.21.15.151:0/2995676619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ac419a010 con 0x7f4ac41999f0 2024-09-19T06:33:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.105+0000 7f4ac1d9b640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac41999f0 0x7f4ac419dd80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b3d0 tx=0x7f4ab800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.106+0000 7f4ab37fe640 1 -- 172.21.15.151:0/2995676619 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4ab8011630 con 0x7f4ac41999f0 2024-09-19T06:33:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.106+0000 7f4ab37fe640 1 -- 172.21.15.151:0/2995676619 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab8011cb0 con 0x7f4ac41999f0 2024-09-19T06:33:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.106+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ac419e2c0 con 0x7f4ac41999f0 2024-09-19T06:33:35.109 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.106+0000 7f4ab37fe640 1 -- 172.21.15.151:0/2995676619 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4ab8010df0 con 0x7f4ac41999f0 2024-09-19T06:33:35.110 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.106+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ac419e800 con 0x7f4ac41999f0 2024-09-19T06:33:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.107+0000 7f4ab37fe640 1 -- 172.21.15.151:0/2995676619 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f4ab8018020 con 0x7f4ac41999f0 2024-09-19T06:33:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.107+0000 7f4ab37fe640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f4aa4039090 0x7f4aa403b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.107+0000 7f4ab37fe640 1 -- 172.21.15.151:0/2995676619 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4ab804bf80 con 0x7f4ac41999f0 2024-09-19T06:33:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.107+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ac4102da0 con 0x7f4ac41999f0 2024-09-19T06:33:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.107+0000 7f4ac259c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f4aa4039090 0x7f4aa403b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:35.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.108+0000 7f4ac259c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f4aa4039090 0x7f4aa403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4012c10 tx=0x7f4ab40323a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:35.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.113+0000 7f4ab37fe640 1 -- 172.21.15.151:0/2995676619 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4ab801f020 con 0x7f4ac41999f0 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: mon.a calling monitor election 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: mon.c calling monitor election 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: monmap epoch 2 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: last_changed 2024-09-19T06:33:29.904710+0000 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: election_strategy: 1 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: fsmap 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: mgrmap e13: x(active, since 38s) 2024-09-19T06:33:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: overall HEALTH_OK 2024-09-19T06:33:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:35.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.349+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4ac4106880 con 0x7f4ac41999f0 2024-09-19T06:33:35.351 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.349+0000 7f4ab37fe640 1 -- 172.21.15.151:0/2995676619 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4ab8018310 con 0x7f4ac41999f0 2024-09-19T06:33:35.351 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:33:35.351 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","modified":"2024-09-19T06:33:29.904710Z","created":"2024-09-19T06:31:58.000017Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T06:33:35.351 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 2 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f4aa4039090 msgr2=0x7f4aa403b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f4aa4039090 0x7f4aa403b550 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4012c10 tx=0x7f4ab40323a0 comp rx=0 tx=0).stop 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac41999f0 msgr2=0x7f4ac419dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac41999f0 0x7f4ac419dd80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4ab800b3d0 tx=0x7f4ab800b890 comp rx=0 tx=0).stop 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 shutdown_connections 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f4aa4039090 0x7f4aa403b550 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ac41999f0 0x7f4ac419dd80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 --2- 172.21.15.151:0/2995676619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4aac003530 0x7f4ac41994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 >> 172.21.15.151:0/2995676619 conn(0x7f4ac40fd320 msgr2=0x7f4ac4074ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:35.355 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 shutdown_connections 2024-09-19T06:33:35.356 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:35.353+0000 7f4ac8a6c640 1 -- 172.21.15.151:0/2995676619 wait complete. 2024-09-19T06:33:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:35 smithi037 ceph-mon[31211]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:35 smithi037 ceph-mon[31211]: Deploying daemon mon.b on smithi096 2024-09-19T06:33:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:35 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/2995676619' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:36.916 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:33:36.916 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:33:37.172 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:33:37.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.586+0000 7fdfb339f640 1 -- 172.21.15.151:0/3588188675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac106630 msgr2=0x7fdfac106a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:37.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.586+0000 7fdfb339f640 1 --2- 172.21.15.151:0/3588188675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac106630 0x7fdfac106a10 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8009880 tx=0x7fdfa802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:37.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 -- 172.21.15.151:0/3588188675 shutdown_connections 2024-09-19T06:33:37.592 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 --2- 172.21.15.151:0/3588188675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac106630 0x7fdfac106a10 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 -- 172.21.15.151:0/3588188675 >> 172.21.15.151:0/3588188675 conn(0x7fdfac0fc220 msgr2=0x7fdfac0fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 -- 172.21.15.151:0/3588188675 shutdown_connections 2024-09-19T06:33:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 -- 172.21.15.151:0/3588188675 wait complete. 2024-09-19T06:33:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 Processor -- start 2024-09-19T06:33:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 -- start start 2024-09-19T06:33:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 0x7fdfac1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac196820 0x7fdfac19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.587+0000 7fdfb339f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10ac60 con 0x7fdfac196820 2024-09-19T06:33:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb339f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdfac10aae0 con 0x7fdfac106630 2024-09-19T06:33:37.594 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb113b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 0x7fdfac1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb113b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 0x7fdfac1962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:47044/0 (socket says 172.21.15.151:47044) 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb113b640 1 -- 172.21.15.151:0/1723743256 learned_addr learned my addr 172.21.15.151:0/1723743256 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb093a640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac196820 0x7fdfac19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb113b640 1 -- 172.21.15.151:0/1723743256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 msgr2=0x7fdfac1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb113b640 1 -- 172.21.15.151:0/1723743256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 msgr2=0x7fdfac1962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb113b640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 0x7fdfac1962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb113b640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 0x7fdfac1962e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb093a640 1 -- 172.21.15.151:0/1723743256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 msgr2=0x7fdfac1962e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb093a640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 0x7fdfac1962e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:37.595 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb093a640 1 -- 172.21.15.151:0/1723743256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfac196e40 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfb093a640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac196820 0x7fdfac19abb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00b3d0 tx=0x7fdf9c00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.588+0000 7fdfa27fc640 1 -- 172.21.15.151:0/1723743256 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fdf9c011630 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.589+0000 7fdfa27fc640 1 -- 172.21.15.151:0/1723743256 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf9c011cb0 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.589+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfac19b0f0 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.589+0000 7fdfa27fc640 1 -- 172.21.15.151:0/1723743256 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fdf9c010df0 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.589+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfac19b630 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.589+0000 7fdfa27fc640 1 -- 172.21.15.151:0/1723743256 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fdf9c018020 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.589+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfac101ca0 con 0x7fdfac196820 2024-09-19T06:33:37.596 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.590+0000 7fdfa27fc640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fdf98039040 0x7fdf9803b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.590+0000 7fdfa27fc640 1 -- 172.21.15.151:0/1723743256 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdf9c04bfe0 con 0x7fdfac196820 2024-09-19T06:33:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.590+0000 7fdfb113b640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fdf98039040 0x7fdf9803b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.590+0000 7fdfb113b640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fdf98039040 0x7fdf9803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80096c0 tx=0x7fdfa8005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:37.597 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.595+0000 7fdfa27fc640 1 -- 172.21.15.151:0/1723743256 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fdf9c01f020 con 0x7fdfac196820 2024-09-19T06:33:37.826 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.825+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fdfac19b940 con 0x7fdfac196820 2024-09-19T06:33:37.827 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.825+0000 7fdfa27fc640 1 -- 172.21.15.151:0/1723743256 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fdf9c0172c0 con 0x7fdfac196820 2024-09-19T06:33:37.827 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:33:37.827 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":2,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","modified":"2024-09-19T06:33:29.904710Z","created":"2024-09-19T06:31:58.000017Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T06:33:37.828 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 2 2024-09-19T06:33:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fdf98039040 msgr2=0x7fdf9803b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fdf98039040 0x7fdf9803b500 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdfa80096c0 tx=0x7fdfa8005a40 comp rx=0 tx=0).stop 2024-09-19T06:33:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac196820 msgr2=0x7fdfac19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac196820 0x7fdfac19abb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00b3d0 tx=0x7fdf9c00b890 comp rx=0 tx=0).stop 2024-09-19T06:33:37.829 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 shutdown_connections 2024-09-19T06:33:37.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fdf98039040 0x7fdf9803b500 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:37.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdfac196820 0x7fdfac19abb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:37.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 --2- 172.21.15.151:0/1723743256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdfac106630 0x7fdfac1962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:37.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 >> 172.21.15.151:0/1723743256 conn(0x7fdfac0fc220 msgr2=0x7fdfac0fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:37.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 shutdown_connections 2024-09-19T06:33:37.830 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:37.828+0000 7fdfb339f640 1 -- 172.21.15.151:0/1723743256 wait complete. 2024-09-19T06:33:38.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:37 smithi151 ceph-mon[35236]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:38.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:37 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/1723743256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:37 smithi037 ceph-mon[31211]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:37 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/1723743256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:38.971 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-19T06:33:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-19T06:33:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-19T06:33:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3204860257' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3204860257' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:39.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:33:39.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi037:172.21.15.37=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:39.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: Saving service mon spec with placement smithi037:172.21.15.37=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-09-19T06:33:39.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: Deploying daemon mon.c on smithi151 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/902543445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.a calling monitor election 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.c calling monitor election 2024-09-19T06:33:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: monmap epoch 2 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: last_changed 2024-09-19T06:33:29.904710+0000 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: min_mon_release 19 (squid) 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: election_strategy: 1 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: fsmap 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mgrmap e13: x(active, since 38s) 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: overall HEALTH_OK 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: Deploying daemon mon.b on smithi096 2024-09-19T06:33:39.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/2995676619' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:39.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:39.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:39.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/1723743256' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:39.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:38 smithi096 ceph-mon[35004]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-19T06:33:39.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:39 smithi096 bash[34985]: fbdc43f0f542a7d8849b61ed6162e7bde5ae5896cbe7c9a01760285533c2e6ab 2024-09-19T06:33:39.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:39 smithi096 systemd[1]: Started Ceph mon.b for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:33:39.533 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-19T06:33:39.533 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-19T06:33:39.786 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:33:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:33:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: mon.c calling monitor election 2024-09-19T06:33:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: mon.a calling monitor election 2024-09-19T06:33:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: monmap epoch 3 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: last_changed 2024-09-19T06:33:38.975661+0000 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:44.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: min_mon_release 19 (squid) 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: election_strategy: 1 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: fsmap 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: mgrmap e13: x(active, since 47s) 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: overall HEALTH_OK 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: mon.c calling monitor election 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: mon.a calling monitor election 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: monmap epoch 3 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: last_changed 2024-09-19T06:33:38.975661+0000 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: election_strategy: 1 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: fsmap 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:44.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: mgrmap e13: x(active, since 47s) 2024-09-19T06:33:44.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: overall HEALTH_OK 2024-09-19T06:33:44.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:44.600 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.596+0000 7f176dbf0640 1 -- 172.21.15.151:0/2612012352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 msgr2=0x7f17500059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.596+0000 7f176dbf0640 1 --2- 172.21.15.151:0/2612012352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 0x7f17500059c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1768069a30 tx=0x7f175802c490 comp rx=0 tx=0).stop 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.596+0000 7f176dbf0640 1 -- 172.21.15.151:0/2612012352 shutdown_connections 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.596+0000 7f176dbf0640 1 --2- 172.21.15.151:0/2612012352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 0x7f17500059c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.596+0000 7f176dbf0640 1 -- 172.21.15.151:0/2612012352 >> 172.21.15.151:0/2612012352 conn(0x7f17680fb9a0 msgr2=0x7f17680fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.596+0000 7f176dbf0640 1 -- 172.21.15.151:0/2612012352 shutdown_connections 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.596+0000 7f176dbf0640 1 -- 172.21.15.151:0/2612012352 wait complete. 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 Processor -- start 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 -- start start 2024-09-19T06:33:44.601 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 0x7f1768199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 0x7f176819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 0x7f176819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f176810b920 con 0x7f1750003530 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f176810b7a0 con 0x7f1768199ba0 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.597+0000 7f176dbf0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f176810baa0 con 0x7f176819e470 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1767fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 0x7f176819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1767fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 0x7f176819e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44584/0 (socket says 172.21.15.151:44584) 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1767fff640 1 -- 172.21.15.151:0/546133703 learned_addr learned my addr 172.21.15.151:0/546133703 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1767fff640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 msgr2=0x7f176819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1767fff640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 msgr2=0x7f176819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1767fff640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 0x7f176819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:33:44.602 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1767fff640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 0x7f176819e920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f17677fe640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 0x7f1768199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1766ffd640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 0x7f176819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1766ffd640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 msgr2=0x7f176819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1766ffd640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 msgr2=0x7f176819df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1766ffd640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 0x7f176819df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f1766ffd640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 0x7f176819df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f17677fe640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 msgr2=0x7f176819e920 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f17677fe640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 0x7f176819e920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f17677fe640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 msgr2=0x7f176819df30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f17677fe640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 0x7f176819df30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f17677fe640 1 -- 172.21.15.151:0/546133703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17681a61e0 con 0x7f1750003530 2024-09-19T06:33:44.603 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.598+0000 7f17677fe640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 0x7f1768199660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c00 tx=0x7f17580339a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:44.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.599+0000 7f1764ff9640 1 -- 172.21.15.151:0/546133703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758038040 con 0x7f1750003530 2024-09-19T06:33:44.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.599+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17681a63b0 con 0x7f1750003530 2024-09-19T06:33:44.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.599+0000 7f1764ff9640 1 -- 172.21.15.151:0/546133703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1758005550 con 0x7f1750003530 2024-09-19T06:33:44.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.599+0000 7f1764ff9640 1 -- 172.21.15.151:0/546133703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1758004cf0 con 0x7f1750003530 2024-09-19T06:33:44.604 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.599+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17681a6910 con 0x7f1750003530 2024-09-19T06:33:44.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.600+0000 7f1764ff9640 1 -- 172.21.15.151:0/546133703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f1758005070 con 0x7f1750003530 2024-09-19T06:33:44.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.600+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f176810cad0 con 0x7f1750003530 2024-09-19T06:33:44.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.600+0000 7f1764ff9640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1754039730 0x7f175403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:44.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.600+0000 7f1764ff9640 1 -- 172.21.15.151:0/546133703 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f175806e7c0 con 0x7f1750003530 2024-09-19T06:33:44.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.600+0000 7f1766ffd640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1754039730 0x7f175403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:44.605 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.601+0000 7f1766ffd640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1754039730 0x7f175403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f175c006fd0 tx=0x7f175c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:44.606 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.605+0000 7f1764ff9640 1 -- 172.21.15.151:0/546133703 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1758004e90 con 0x7f1750003530 2024-09-19T06:33:44.846 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.845+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1768193160 con 0x7f1750003530 2024-09-19T06:33:44.847 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.845+0000 7f1764ff9640 1 -- 172.21.15.151:0/546133703 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f1758005360 con 0x7f1750003530 2024-09-19T06:33:44.847 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:33:44.847 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","modified":"2024-09-19T06:33:38.975661Z","created":"2024-09-19T06:31:58.000017Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:3300","nonce":0},{"type":"v1","addr":"172.21.15.37:6789","nonce":0}]},"addr":"172.21.15.37:6789/0","public_addr":"172.21.15.37:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-19T06:33:44.847 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 3 2024-09-19T06:33:44.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1754039730 msgr2=0x7f175403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:44.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1754039730 0x7f175403bbf0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f175c006fd0 tx=0x7f175c008040 comp rx=0 tx=0).stop 2024-09-19T06:33:44.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 msgr2=0x7f1768199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:44.849 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 0x7f1768199660 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1758000c00 tx=0x7f17580339a0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 shutdown_connections 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1754039730 0x7f175403bbf0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f176819e470 0x7f176819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1768199ba0 0x7f176819df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 --2- 172.21.15.151:0/546133703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1750003530 0x7f1768199660 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 >> 172.21.15.151:0/546133703 conn(0x7f17680fb9a0 msgr2=0x7f17680fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 shutdown_connections 2024-09-19T06:33:44.850 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:44.848+0000 7f176dbf0640 1 -- 172.21.15.151:0/546133703 wait complete. 2024-09-19T06:33:45.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:45 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:45.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:45 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/546133703' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:45.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:45 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:45.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:45 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:45.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:45 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/546133703' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:45.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:45 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:45.793 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-19T06:33:45.793 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-19T06:33:46.057 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: mon.c calling monitor election 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: mon.a calling monitor election 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:46.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: monmap epoch 3 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: last_changed 2024-09-19T06:33:38.975661+0000 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: min_mon_release 19 (squid) 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: election_strategy: 1 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: fsmap 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: mgrmap e13: x(active, since 47s) 2024-09-19T06:33:46.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: overall HEALTH_OK 2024-09-19T06:33:46.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:46.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:46.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:46.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:46.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:46.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/546133703' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-19T06:33:46.277 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:47.075 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: mon.b calling monitor election 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Updating smithi037:/etc/ceph/ceph.conf 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: Reconfiguring daemon mon.a on smithi037 2024-09-19T06:33:47.076 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: mon.b calling monitor election 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: mon.c calling monitor election 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: mon.a calling monitor election 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: monmap epoch 3 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: last_changed 2024-09-19T06:33:38.975661+0000 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: min_mon_release 19 (squid) 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: election_strategy: 1 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T06:33:47.077 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: fsmap 2024-09-19T06:33:47.078 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:47.078 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: mgrmap e13: x(active, since 49s) 2024-09-19T06:33:47.078 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: overall HEALTH_OK 2024-09-19T06:33:47.078 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:46 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.073+0000 7f7da7a80640 1 -- 172.21.15.37:0/684600628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da0106560 msgr2=0x7f7da0106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.073+0000 7f7da7a80640 1 --2- 172.21.15.37:0/684600628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da0106560 0x7f7da0106940 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009880 tx=0x7f7d9c02a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.073+0000 7f7da7a80640 1 -- 172.21.15.37:0/684600628 shutdown_connections 2024-09-19T06:33:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.073+0000 7f7da7a80640 1 --2- 172.21.15.37:0/684600628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da0106560 0x7f7da0106940 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.073+0000 7f7da7a80640 1 -- 172.21.15.37:0/684600628 >> 172.21.15.37:0/684600628 conn(0x7f7da00fc1b0 msgr2=0x7f7da00fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.073+0000 7f7da7a80640 1 -- 172.21.15.37:0/684600628 shutdown_connections 2024-09-19T06:33:47.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.073+0000 7f7da7a80640 1 -- 172.21.15.37:0/684600628 wait complete. 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 Processor -- start 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 -- start start 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7da0106560 0x7f7da01981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da0198710 0x7f7da0198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da01a1240 0x7f7da01a3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010ab40 con 0x7f7da01a1240 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010a9c0 con 0x7f7da0198710 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da7a80640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7da010acc0 con 0x7f7da0106560 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da01a1240 0x7f7da01a3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da01a1240 0x7f7da01a3630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46368/0 (socket says 172.21.15.37:46368) 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 -- 172.21.15.37:0/3877549803 learned_addr learned my addr 172.21.15.37:0/3877549803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:47.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da501b640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da0198710 0x7f7da0198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 -- 172.21.15.37:0/3877549803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7da0106560 msgr2=0x7f7da01981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7da0106560 0x7f7da01981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 -- 172.21.15.37:0/3877549803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da0198710 msgr2=0x7f7da0198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da0198710 0x7f7da0198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da601d640 1 -- 172.21.15.37:0/3877549803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7da01a3dd0 con 0x7f7da01a1240 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.074+0000 7f7da501b640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da0198710 0x7f7da0198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.075+0000 7f7da601d640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da01a1240 0x7f7da01a3630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800ba50 tx=0x7f7d9800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.075+0000 7f7d96ffd640 1 -- 172.21.15.37:0/3877549803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d98019070 con 0x7f7da01a1240 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.075+0000 7f7d96ffd640 1 -- 172.21.15.37:0/3877549803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d9800cbe0 con 0x7f7da01a1240 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.075+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7da01a40c0 con 0x7f7da01a1240 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.075+0000 7f7d96ffd640 1 -- 172.21.15.37:0/3877549803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d980076a0 con 0x7f7da01a1240 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.075+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7da01a45b0 con 0x7f7da01a1240 2024-09-19T06:33:47.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.076+0000 7f7d96ffd640 1 -- 172.21.15.37:0/3877549803 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7d9801c070 con 0x7f7da01a1240 2024-09-19T06:33:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.076+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7da01069c0 con 0x7f7da01a1240 2024-09-19T06:33:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.076+0000 7f7d96ffd640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7d8c039730 0x7f7d8c03bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.076+0000 7f7d96ffd640 1 -- 172.21.15.37:0/3877549803 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7d9804be80 con 0x7f7da01a1240 2024-09-19T06:33:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.076+0000 7f7da581c640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7d8c039730 0x7f7d8c03bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.076+0000 7f7da581c640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7d8c039730 0x7f7d8c03bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009880 tx=0x7f7d9c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.081+0000 7f7d96ffd640 1 -- 172.21.15.37:0/3877549803 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d98012340 con 0x7f7da01a1240 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: mon.b calling monitor election 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Updating smithi037:/etc/ceph/ceph.conf 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T06:33:47.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: Reconfiguring daemon mon.a on smithi037 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: mon.b calling monitor election 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: mon.c calling monitor election 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: mon.a calling monitor election 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: monmap epoch 3 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: last_changed 2024-09-19T06:33:38.975661+0000 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: min_mon_release 19 (squid) 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: election_strategy: 1 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T06:33:47.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: fsmap 2024-09-19T06:33:47.115 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:47.115 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: mgrmap e13: x(active, since 49s) 2024-09-19T06:33:47.115 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: overall HEALTH_OK 2024-09-19T06:33:47.115 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:46 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:47.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.262+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7da01a4940 con 0x7f7da01a1240 2024-09-19T06:33:47.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.262+0000 7f7d96ffd640 1 -- 172.21.15.37:0/3877549803 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f7d980133a0 con 0x7f7da01a1240 2024-09-19T06:33:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:# minimal ceph.conf for b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:47.264 INFO:teuthology.orchestra.run.smithi037.stdout:[global] 2024-09-19T06:33:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: fsid = b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:47.264 INFO:teuthology.orchestra.run.smithi037.stdout: mon_host = [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] 2024-09-19T06:33:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7d8c039730 msgr2=0x7f7d8c03bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7d8c039730 0x7f7d8c03bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7d9c009880 tx=0x7f7d9c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da01a1240 msgr2=0x7f7da01a3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:47.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da01a1240 0x7f7da01a3630 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7d9800ba50 tx=0x7f7d9800bf10 comp rx=0 tx=0).stop 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 shutdown_connections 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7d8c039730 0x7f7d8c03bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7da01a1240 0x7f7da01a3630 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7da0198710 0x7f7da0198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 --2- 172.21.15.37:0/3877549803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7da0106560 0x7f7da01981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 >> 172.21.15.37:0/3877549803 conn(0x7f7da00fc1b0 msgr2=0x7f7da00fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 shutdown_connections 2024-09-19T06:33:47.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:47.264+0000 7f7da7a80640 1 -- 172.21.15.37:0/3877549803 wait complete. 2024-09-19T06:33:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: mon.b calling monitor election 2024-09-19T06:33:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Updating smithi037:/etc/ceph/ceph.conf 2024-09-19T06:33:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T06:33:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:33:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Reconfiguring mon.a (unknown last config time)... 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: Reconfiguring daemon mon.a on smithi037 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: mon.b calling monitor election 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: mon.c calling monitor election 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: mon.a calling monitor election 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: monmap epoch 3 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: last_changed 2024-09-19T06:33:38.975661+0000 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: created 2024-09-19T06:31:58.000017+0000 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: min_mon_release 19 (squid) 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: election_strategy: 1 2024-09-19T06:33:47.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: 0: [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] mon.a 2024-09-19T06:33:47.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: 1: [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] mon.c 2024-09-19T06:33:47.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-09-19T06:33:47.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: fsmap 2024-09-19T06:33:47.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: osdmap e4: 0 total, 0 up, 0 in 2024-09-19T06:33:47.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: mgrmap e13: x(active, since 49s) 2024-09-19T06:33:47.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: overall HEALTH_OK 2024-09-19T06:33:47.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:46 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:48.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:48.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3877549803' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:33:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3877549803' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:33:48.082 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:48.111 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-19T06:33:48.112 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:33:48.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:33:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:33:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3877549803' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:33:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:48.142 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:33:48.142 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:48.209 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:33:48.209 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:33:48.237 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:33:48.237 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:48.302 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:33:48.302 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-09-19T06:33:48.334 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:33:48.334 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:33:48.404 INFO:tasks.cephadm:Adding mgr.x on smithi037 2024-09-19T06:33:48.404 INFO:tasks.cephadm:Adding mgr.y on smithi096 2024-09-19T06:33:48.404 INFO:tasks.cephadm:Adding mgr.z on smithi151 2024-09-19T06:33:48.404 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi037=x;smithi096=y;smithi151=z' 2024-09-19T06:33:48.689 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:33:49.108 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:48 smithi151 ceph-mon[35236]: Reconfiguring mon.b (monmap changed)... 2024-09-19T06:33:49.109 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:48 smithi151 ceph-mon[35236]: Reconfiguring daemon mon.b on smithi096 2024-09-19T06:33:49.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.110+0000 7f8aecc5d640 1 -- 172.21.15.151:0/3859087342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 msgr2=0x7f8ae81009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:49.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.110+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/3859087342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae81009e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8009880 tx=0x7f8ad802a5d0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.110+0000 7f8aecc5d640 1 -- 172.21.15.151:0/3859087342 shutdown_connections 2024-09-19T06:33:49.111 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.110+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/3859087342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae81009e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.110+0000 7f8aecc5d640 1 -- 172.21.15.151:0/3859087342 >> 172.21.15.151:0/3859087342 conn(0x7f8ae80fbdb0 msgr2=0x7f8ae80fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.110+0000 7f8aecc5d640 1 -- 172.21.15.151:0/3859087342 shutdown_connections 2024-09-19T06:33:49.112 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.110+0000 7f8aecc5d640 1 -- 172.21.15.151:0/3859087342 wait complete. 2024-09-19T06:33:49.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 Processor -- start 2024-09-19T06:33:49.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 -- start start 2024-09-19T06:33:49.114 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae806a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8199d20 0x7f8ae819c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ae819c650 0x7f8ae819eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae81066d0 con 0x7f8ae81005e0 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8106550 con 0x7f8ae819c650 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8ae5d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8199d20 0x7f8ae819c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8ae659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae806a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8ae659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae806a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.151:40862/0 (socket says 172.21.15.151:40862) 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8ae5d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8199d20 0x7f8ae819c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44588/0 (socket says 172.21.15.151:44588) 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8ae659c640 1 -- 172.21.15.151:0/4122038470 learned_addr learned my addr 172.21.15.151:0/4122038470 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:33:49.115 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8ae8106850 con 0x7f8ae8199d20 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.111+0000 7f8ae6d9d640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ae819c650 0x7f8ae819eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ae659c640 1 -- 172.21.15.151:0/4122038470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8199d20 msgr2=0x7f8ae819c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ae659c640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8199d20 0x7f8ae819c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ae659c640 1 -- 172.21.15.151:0/4122038470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ae819c650 msgr2=0x7f8ae819eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ae659c640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ae819c650 0x7f8ae819eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ae659c640 1 -- 172.21.15.151:0/4122038470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ae819f420 con 0x7f8ae81005e0 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ae5d9b640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8199d20 0x7f8ae819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ae659c640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae806a770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8031610 tx=0x7f8ad80043c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ad77fe640 1 -- 172.21.15.151:0/4122038470 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad803f030 con 0x7f8ae81005e0 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ad77fe640 1 -- 172.21.15.151:0/4122038470 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8ad8003ce0 con 0x7f8ae81005e0 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ae819f650 con 0x7f8ae81005e0 2024-09-19T06:33:49.116 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8ad77fe640 1 -- 172.21.15.151:0/4122038470 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad8035070 con 0x7f8ae81005e0 2024-09-19T06:33:49.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.112+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ae819fc00 con 0x7f8ae81005e0 2024-09-19T06:33:49.117 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.113+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ae818f490 con 0x7f8ae81005e0 2024-09-19T06:33:49.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.114+0000 7f8ad77fe640 1 -- 172.21.15.151:0/4122038470 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f8ad8042030 con 0x7f8ae81005e0 2024-09-19T06:33:49.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.114+0000 7f8ad77fe640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8ad0039730 0x7f8ad003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:49.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.114+0000 7f8ad77fe640 1 -- 172.21.15.151:0/4122038470 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8ad806cfc0 con 0x7f8ae81005e0 2024-09-19T06:33:49.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.114+0000 7f8ae5d9b640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8ad0039730 0x7f8ad003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:49.118 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.115+0000 7f8ae5d9b640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8ad0039730 0x7f8ad003bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806acf0 tx=0x7f8adc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:49.120 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.119+0000 7f8ad77fe640 1 -- 172.21.15.151:0/4122038470 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8ad803bde0 con 0x7f8ae81005e0 2024-09-19T06:33:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:48 smithi096 ceph-mon[35004]: Reconfiguring mon.b (monmap changed)... 2024-09-19T06:33:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:48 smithi096 ceph-mon[35004]: Reconfiguring daemon mon.b on smithi096 2024-09-19T06:33:49.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:48 smithi037 ceph-mon[31211]: Reconfiguring mon.b (monmap changed)... 2024-09-19T06:33:49.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:48 smithi037 ceph-mon[31211]: Reconfiguring daemon mon.b on smithi096 2024-09-19T06:33:49.306 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.304+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}) -- 0x7f8ae80626c0 con 0x7f8ad0039730 2024-09-19T06:33:49.310 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.309+0000 7f8ad77fe640 1 -- 172.21.15.151:0/4122038470 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f8ae80626c0 con 0x7f8ad0039730 2024-09-19T06:33:49.311 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mgr update... 2024-09-19T06:33:49.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8ad0039730 msgr2=0x7f8ad003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:49.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8ad0039730 0x7f8ad003bbf0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8ae806acf0 tx=0x7f8adc009190 comp rx=0 tx=0).stop 2024-09-19T06:33:49.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 msgr2=0x7f8ae806a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:49.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae806a770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8ad8031610 tx=0x7f8ad80043c0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 shutdown_connections 2024-09-19T06:33:49.313 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8ad0039730 0x7f8ad003bbf0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8ae819c650 0x7f8ae819eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8ae8199d20 0x7f8ae819c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.311+0000 7f8aecc5d640 1 --2- 172.21.15.151:0/4122038470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8ae81005e0 0x7f8ae806a770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:49.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.312+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 >> 172.21.15.151:0/4122038470 conn(0x7f8ae80fbdb0 msgr2=0x7f8ae80fc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:49.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.312+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 shutdown_connections 2024-09-19T06:33:49.314 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:33:49.312+0000 7f8aecc5d640 1 -- 172.21.15.151:0/4122038470 wait complete. 2024-09-19T06:33:49.876 DEBUG:teuthology.orchestra.run.smithi096:mgr.y> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.y.service 2024-09-19T06:33:49.879 DEBUG:teuthology.orchestra.run.smithi151:mgr.z> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.z.service 2024-09-19T06:33:49.881 INFO:tasks.cephadm:Deploying OSDs... 2024-09-19T06:33:49.882 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:33:49.882 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:33:49.900 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:33:49.900 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_1 2024-09-19T06:33:49.958 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:33:49.958 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:49.958 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-19T06:33:49.958 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:49.958 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:49.958 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-19 06:33:06.894288389 +0000 2024-09-19T06:33:49.958 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-19 06:28:11.639696458 +0000 2024-09-19T06:33:49.959 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-19 06:28:11.639696458 +0000 2024-09-19T06:33:49.959 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-19 06:28:11.639696458 +0000 2024-09-19T06:33:49.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:33:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-19T06:33:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-19T06:33:50.026 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.00021985 s, 2.3 MB/s 2024-09-19T06:33:50.027 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:33:50.086 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_2 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: Reconfiguring mon.c (monmap changed)... 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: Reconfiguring daemon mon.c on smithi151 2024-09-19T06:33:50.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-19 06:33:06.920288022 +0000 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-19 06:28:11.975691623 +0000 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-19 06:28:11.975691623 +0000 2024-09-19T06:33:50.143 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-19 06:28:11.975691623 +0000 2024-09-19T06:33:50.144 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:33:50.206 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: Reconfiguring mon.c (monmap changed)... 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: Reconfiguring daemon mon.c on smithi151 2024-09-19T06:33:50.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.209 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-19T06:33:50.209 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-19T06:33:50.209 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000289863 s, 1.8 MB/s 2024-09-19T06:33:50.211 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:33:50.268 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_3 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: Reconfiguring mon.c (monmap changed)... 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: Reconfiguring daemon mon.c on smithi151 2024-09-19T06:33:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:50.325 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:33:50.325 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:50.325 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-19T06:33:50.325 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:50.325 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:50.325 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-19 06:33:06.947287641 +0000 2024-09-19T06:33:50.326 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-19 06:28:12.305686875 +0000 2024-09-19T06:33:50.326 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-19 06:28:12.305686875 +0000 2024-09-19T06:33:50.326 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-19 06:28:12.305686875 +0000 2024-09-19T06:33:50.326 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:33:50.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-19T06:33:50.391 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-19T06:33:50.391 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000217643 s, 2.4 MB/s 2024-09-19T06:33:50.392 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:33:50.450 DEBUG:teuthology.orchestra.run.smithi037:> stat /dev/vg_nvme/lv_4 2024-09-19T06:33:50.506 INFO:teuthology.orchestra.run.smithi037.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:33:50.506 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:50.506 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-19T06:33:50.506 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:50.506 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:50.506 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-19 06:33:06.974287260 +0000 2024-09-19T06:33:50.507 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-19 06:28:12.635682126 +0000 2024-09-19T06:33:50.507 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-19 06:28:12.635682126 +0000 2024-09-19T06:33:50.507 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-19 06:28:12.635682126 +0000 2024-09-19T06:33:50.507 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:33:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records in 2024-09-19T06:33:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:1+0 records out 2024-09-19T06:33:50.571 INFO:teuthology.orchestra.run.smithi037.stderr:512 bytes copied, 0.000216489 s, 2.4 MB/s 2024-09-19T06:33:50.572 DEBUG:teuthology.orchestra.run.smithi037:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:33:50.630 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-09-19T06:33:50.630 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:33:50.649 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:33:50.650 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-09-19T06:33:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:33:50.708 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:50.708 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-19T06:33:50.709 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:50.709 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:50.709 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 06:33:38.415566157 +0000 2024-09-19T06:33:50.709 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 06:28:20.753172480 +0000 2024-09-19T06:33:50.709 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 06:28:20.753172480 +0000 2024-09-19T06:33:50.709 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 06:28:20.753172480 +0000 2024-09-19T06:33:50.709 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:33:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T06:33:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T06:33:50.778 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000290314 s, 1.8 MB/s 2024-09-19T06:33:50.780 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:33:50.840 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-09-19T06:33:50.897 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 06:33:38.415566157 +0000 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 06:28:21.074165430 +0000 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 06:28:21.074165430 +0000 2024-09-19T06:33:50.898 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 06:28:21.074165430 +0000 2024-09-19T06:33:50.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:33:50.964 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T06:33:50.964 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T06:33:50.964 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000191635 s, 2.7 MB/s 2024-09-19T06:33:50.965 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:33:51.024 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 06:33:38.416566135 +0000 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 06:28:21.388158534 +0000 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 06:28:21.388158534 +0000 2024-09-19T06:33:51.081 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 06:28:21.388158534 +0000 2024-09-19T06:33:51.082 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: Saving service mgr spec with placement smithi037=x;smithi096=y;smithi151=z;count:3 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:51.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:33:51.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:51.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:51.143 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: Saving service mgr spec with placement smithi037=x;smithi096=y;smithi151=z;count:3 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:51.144 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:51.146 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T06:33:51.146 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T06:33:51.146 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000195896 s, 2.6 MB/s 2024-09-19T06:33:51.147 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:33:51.205 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-09-19T06:33:51.261 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-09-19 06:33:38.416566135 +0000 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-09-19 06:28:21.714151374 +0000 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-09-19 06:28:21.714151374 +0000 2024-09-19T06:33:51.262 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-09-19 06:28:21.714151374 +0000 2024-09-19T06:33:51.262 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:33:51.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi037=x;smithi096=y;smithi151=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:51.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: Saving service mgr spec with placement smithi037=x;smithi096=y;smithi151=z;count:3 2024-09-19T06:33:51.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:51.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:51.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:51.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:51.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:51.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:33:51.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:51.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:51.326 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-09-19T06:33:51.327 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-09-19T06:33:51.327 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000322099 s, 1.6 MB/s 2024-09-19T06:33:51.328 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:33:51.385 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-09-19T06:33:51.385 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-09-19T06:33:51.401 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-19T06:33:51.401 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-09-19T06:33:51.457 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-19T06:33:51.457 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:51.457 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-19T06:33:51.457 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:51.458 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:51.458 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 06:33:29.325667281 +0000 2024-09-19T06:33:51.458 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 06:28:26.414611855 +0000 2024-09-19T06:33:51.458 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 06:28:26.414611855 +0000 2024-09-19T06:33:51.458 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 06:28:26.414611855 +0000 2024-09-19T06:33:51.458 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-19T06:33:51.521 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T06:33:51.521 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T06:33:51.521 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000190767 s, 2.7 MB/s 2024-09-19T06:33:51.522 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-19T06:33:51.578 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-09-19T06:33:51.633 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 06:33:29.326667268 +0000 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 06:28:26.726607852 +0000 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 06:28:26.726607852 +0000 2024-09-19T06:33:51.634 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 06:28:26.726607852 +0000 2024-09-19T06:33:51.635 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-19T06:33:51.697 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T06:33:51.698 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T06:33:51.698 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000189376 s, 2.7 MB/s 2024-09-19T06:33:51.699 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-19T06:33:51.756 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-09-19T06:33:51.811 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-19T06:33:51.811 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:51.811 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-19T06:33:51.812 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:51.812 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:51.812 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 06:33:29.326667268 +0000 2024-09-19T06:33:51.812 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 06:28:27.053603660 +0000 2024-09-19T06:33:51.812 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 06:28:27.053603660 +0000 2024-09-19T06:33:51.812 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 06:28:27.053603660 +0000 2024-09-19T06:33:51.812 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-19T06:33:51.875 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T06:33:51.875 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T06:33:51.875 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.00020943 s, 2.4 MB/s 2024-09-19T06:33:51.877 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-19T06:33:51.934 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout:Context: system_u:object_r:device_t:s0 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-09-19 06:33:29.326667268 +0000 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-09-19 06:28:27.371599582 +0000 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-09-19 06:28:27.371599582 +0000 2024-09-19T06:33:51.990 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: 2024-09-19 06:28:27.371599582 +0000 2024-09-19T06:33:51.991 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-19T06:33:52.053 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-09-19T06:33:52.054 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-09-19T06:33:52.054 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000175967 s, 2.9 MB/s 2024-09-19T06:33:52.056 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-19T06:33:52.114 INFO:tasks.cephadm:Deploying osd.0 on smithi037 with /dev/vg_nvme/lv_4... 2024-09-19T06:33:52.114 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:33:52.118 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:51 smithi151 ceph-mon[35236]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:52.118 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:51 smithi151 ceph-mon[35236]: Deploying daemon mgr.z on smithi151 2024-09-19T06:33:52.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:51 smithi037 ceph-mon[31211]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:52.141 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:51 smithi037 ceph-mon[31211]: Deploying daemon mgr.z on smithi151 2024-09-19T06:33:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:51 smithi096 ceph-mon[35004]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:51 smithi096 ceph-mon[35004]: Deploying daemon mgr.z on smithi151 2024-09-19T06:33:52.376 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:52.759 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:52 smithi151 systemd[1]: Starting Ceph mgr.z for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:33:53.029 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:52 smithi151 podman[37030]: 2024-09-19 06:33:52.660326977 +0000 UTC m=+0.015640260 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:33:53.030 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:52 smithi151 podman[37030]: 2024-09-19 06:33:52.759845957 +0000 UTC m=+0.115159283 container create ef213e9b1b0853b529be0e7340f0c885ede1ecd87ca08c2daba71df91169f351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:33:53.030 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:52 smithi151 podman[37030]: 2024-09-19 06:33:52.954152359 +0000 UTC m=+0.309465657 container init ef213e9b1b0853b529be0e7340f0c885ede1ecd87ca08c2daba71df91169f351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4) 2024-09-19T06:33:53.030 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:52 smithi151 podman[37030]: 2024-09-19 06:33:52.957293716 +0000 UTC m=+0.312607002 container start ef213e9b1b0853b529be0e7340f0c885ede1ecd87ca08c2daba71df91169f351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T06:33:53.030 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:52 smithi151 ceph-mgr[37049]: -- 172.21.15.151:0/4115525168 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd80c4f0e0 con 0x55cd80c2c800 2024-09-19T06:33:53.030 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 bash[37030]: ef213e9b1b0853b529be0e7340f0c885ede1ecd87ca08c2daba71df91169f351 2024-09-19T06:33:53.287 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 systemd[1]: Started Ceph mgr.z for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:33:53.288 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:53.080+0000 7f47fff7b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:33:53.288 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:53.139+0000 7f47fff7b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:33:53.288 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:53.286+0000 7f47fff7b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:33:53.612 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:53.351+0000 7f47fff7b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:33:53.613 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:53.496+0000 7f47fff7b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:33:53.944 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:53.633+0000 7f47fff7b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:53.945 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:54.185 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:54.186 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:54.186 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:54.186 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:54.186 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:54.186 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:33:54.186 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:54.186 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:54.362 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:53.943+0000 7f47fff7b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:33:54.363 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:54 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:54.003+0000 7f47fff7b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:33:54.363 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:54 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:54.183+0000 7f47fff7b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:33:54.673 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:33:54.701 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_4 2024-09-19T06:33:54.996 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:33:55.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:54 smithi096 ceph-mon[35004]: Deploying daemon mgr.y on smithi096 2024-09-19T06:33:55.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:54 smithi096 systemd[1]: Starting Ceph mgr.y for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:33:55.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:54 smithi037 ceph-mon[31211]: Deploying daemon mgr.y on smithi096 2024-09-19T06:33:55.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:54 smithi151 ceph-mon[35236]: Deploying daemon mgr.y on smithi096 2024-09-19T06:33:55.363 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:55.051+0000 7f47fff7b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:33:55.363 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:55.117+0000 7f47fff7b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:33:55.399 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 podman[36349]: 2024-09-19 06:33:55.026490508 +0000 UTC m=+0.015629399 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:33:55.399 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 podman[36349]: 2024-09-19 06:33:55.154028247 +0000 UTC m=+0.143167128 container create 3b0d4123ccc87faca78e307bdc443f4b78f2ea0d34e3661af7e32485a9ffc8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-19T06:33:55.399 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 podman[36349]: 2024-09-19 06:33:55.339801863 +0000 UTC m=+0.328940744 container init 3b0d4123ccc87faca78e307bdc443f4b78f2ea0d34e3661af7e32485a9ffc8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T06:33:55.399 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 podman[36349]: 2024-09-19 06:33:55.342742739 +0000 UTC m=+0.331881620 container start 3b0d4123ccc87faca78e307bdc443f4b78f2ea0d34e3661af7e32485a9ffc8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T06:33:55.399 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mgr[36368]: -- 172.21.15.96:0/928665349 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563a63f890e0 con 0x563a63f66800 2024-09-19T06:33:55.399 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 bash[36349]: 3b0d4123ccc87faca78e307bdc443f4b78f2ea0d34e3661af7e32485a9ffc8c8 2024-09-19T06:33:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.448+0000 7ff032fe2640 1 -- 172.21.15.37:0/3193218408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c101d10 msgr2=0x7ff02c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.448+0000 7ff032fe2640 1 --2- 172.21.15.37:0/3193218408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c101d10 0x7ff02c102190 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff020009900 tx=0x7ff02002a410 comp rx=0 tx=0).stop 2024-09-19T06:33:55.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.448+0000 7ff032fe2640 1 -- 172.21.15.37:0/3193218408 shutdown_connections 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.448+0000 7ff032fe2640 1 --2- 172.21.15.37:0/3193218408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff02c1026d0 0x7ff02c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.448+0000 7ff032fe2640 1 --2- 172.21.15.37:0/3193218408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c101d10 0x7ff02c102190 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.448+0000 7ff032fe2640 1 --2- 172.21.15.37:0/3193218408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 0x7ff02c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.448+0000 7ff032fe2640 1 -- 172.21.15.37:0/3193218408 >> 172.21.15.37:0/3193218408 conn(0x7ff02c0fc2a0 msgr2=0x7ff02c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 -- 172.21.15.37:0/3193218408 shutdown_connections 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 -- 172.21.15.37:0/3193218408 wait complete. 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 Processor -- start 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 -- start start 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 0x7ff02c104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff02c101d10 0x7ff02c1030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c1026d0 0x7ff02c103630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c10c7c0 con 0x7ff02c1026d0 2024-09-19T06:33:55.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c10c640 con 0x7ff02c101d10 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff032fe2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff02c10c940 con 0x7ff02c100b10 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff03157f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c1026d0 0x7ff02c103630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff030d7e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 0x7ff02c104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff03157f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c1026d0 0x7ff02c103630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38638/0 (socket says 172.21.15.37:38638) 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff03157f640 1 -- 172.21.15.37:0/1608755389 learned_addr learned my addr 172.21.15.37:0/1608755389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff030d7e640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 0x7ff02c104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:34736/0 (socket says 172.21.15.37:34736) 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.449+0000 7ff02bfff640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff02c101d10 0x7ff02c1030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff03157f640 1 -- 172.21.15.37:0/1608755389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 msgr2=0x7ff02c104aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff03157f640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 0x7ff02c104aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff03157f640 1 -- 172.21.15.37:0/1608755389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff02c101d10 msgr2=0x7ff02c1030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff03157f640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff02c101d10 0x7ff02c1030f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff03157f640 1 -- 172.21.15.37:0/1608755389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff02c103ef0 con 0x7ff02c1026d0 2024-09-19T06:33:55.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff030d7e640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 0x7ff02c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff03157f640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c1026d0 0x7ff02c103630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eaa0 tx=0x7ff02400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff02400cdb0 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff024005020 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff02c1a20c0 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240116b0 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.450+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff02c1a25d0 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.451+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff024005560 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.451+0000 7ff029ffb640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff0140394d0 0x7ff01403b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.451+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff02404c9b0 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.451+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff02c1a2860 con 0x7ff02c1026d0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.452+0000 7ff030d7e640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff0140394d0 0x7ff01403b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.452+0000 7ff030d7e640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff0140394d0 0x7ff01403b990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff02c101b70 tx=0x7ff01c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:33:55.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.456+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff024016030 con 0x7ff02c1026d0 2024-09-19T06:33:55.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:55.644+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff02c0fe4f0 con 0x7ff0140394d0 2024-09-19T06:33:55.682 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 systemd[1]: Started Ceph mgr.y for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:33:55.682 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:55.466+0000 7febef81b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:33:55.682 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:55.524+0000 7febef81b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:33:55.682 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:55.680+0000 7febef81b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:33:56.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:55.749+0000 7febef81b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:33:56.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:55.913+0000 7febef81b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:33:56.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:56.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:56.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:56.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:33:56.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:33:56.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:56.113 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:55.845+0000 7f47fff7b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:33:56.113 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:55.979+0000 7f47fff7b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:33:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:56.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:33:56.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:33:56.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:56.326 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:56.061+0000 7febef81b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:33:56.382 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.121+0000 7f47fff7b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:33:56.383 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.186+0000 7f47fff7b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:33:56.383 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.344+0000 7f47fff7b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:33:56.645 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:56.389+0000 7febef81b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:33:56.646 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:56.451+0000 7febef81b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:33:56.647 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.414+0000 7f47fff7b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:33:56.647 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.474+0000 7f47fff7b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:33:56.648 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.646+0000 7f47fff7b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:33:56.922 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.713+0000 7f47fff7b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:33:56.922 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.780+0000 7f47fff7b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:33:56.922 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.860+0000 7f47fff7b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:33:56.922 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.920+0000 7f47fff7b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:56.923 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:56.924 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:57.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:56.644+0000 7febef81b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:33:57.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:57.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:57.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:57.175 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:57.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:57.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-19T06:33:57.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-19T06:33:57.176 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:57.364 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:56.980+0000 7f47fff7b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:33:57.881 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:57.533+0000 7febef81b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:33:57.882 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:57.600+0000 7febef81b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:33:57.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:33:57.888+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7ff024011850 con 0x7ff02c1026d0 2024-09-19T06:33:58.112 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:33:57.766+0000 7f47fff7b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: Reconfiguring daemon mgr.x on smithi037 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: Standby manager daemon z started 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:33:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:57 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: Reconfiguring daemon mgr.x on smithi037 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: Standby manager daemon z started 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:33:58.146 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:57 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: Reconfiguring mgr.x (unknown last config time)... 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: Reconfiguring daemon mgr.x on smithi037 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: Standby manager daemon z started 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:33:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:57 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1124348727' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:33:58.573 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:58.318+0000 7febef81b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:33:58.574 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:58.432+0000 7febef81b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:33:58.574 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:58.572+0000 7febef81b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:33:58.851 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:58.633+0000 7febef81b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:33:58.852 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:58.783+0000 7febef81b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:33:58.852 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:58.850+0000 7febef81b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:58.910+0000 7febef81b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:59.067+0000 7febef81b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:59.131+0000 7febef81b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:59.133 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:59.134 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:59.134 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T06:33:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:33:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: mgrmap e14: x(active, since 61s), standbys: z 2024-09-19T06:33:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:33:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:33:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:33:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:33:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:33:59.395 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:59.197+0000 7febef81b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:33:59.395 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:59.278+0000 7febef81b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:33:59.396 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:59.336+0000 7febef81b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:33:59.396 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:33:59.394+0000 7febef81b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:34:00.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-mon[35004]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:00.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/755249433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49e46443-fcf6-403e-95af-871ca077aee4"}]: dispatch 2024-09-19T06:34:00.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/755249433' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49e46443-fcf6-403e-95af-871ca077aee4"}]': finished 2024-09-19T06:34:00.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-mon[35004]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T06:34:00.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:00.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:33:59 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1268175009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:00.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:59 smithi037 ceph-mon[31211]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:00.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:59 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/755249433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49e46443-fcf6-403e-95af-871ca077aee4"}]: dispatch 2024-09-19T06:34:00.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:59 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/755249433' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49e46443-fcf6-403e-95af-871ca077aee4"}]': finished 2024-09-19T06:34:00.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:59 smithi037 ceph-mon[31211]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T06:34:00.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:59 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:00.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:33:59 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1268175009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:59 smithi151 ceph-mon[35236]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:59 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/755249433' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49e46443-fcf6-403e-95af-871ca077aee4"}]: dispatch 2024-09-19T06:34:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:59 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/755249433' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49e46443-fcf6-403e-95af-871ca077aee4"}]': finished 2024-09-19T06:34:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:59 smithi151 ceph-mon[35236]: osdmap e5: 1 total, 0 up, 1 in 2024-09-19T06:34:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:59 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:33:59 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1268175009' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:00.524 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:34:00 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:34:00.173+0000 7febef81b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:34:00.907 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:00.904+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7ff024004a40 con 0x7ff02c1026d0 2024-09-19T06:34:01.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:00 smithi096 ceph-mon[35004]: Standby manager daemon y started 2024-09-19T06:34:01.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:00 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:34:01.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:00 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:34:01.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:00 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:34:01.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:00 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:34:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:00 smithi037 ceph-mon[31211]: Standby manager daemon y started 2024-09-19T06:34:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:00 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:34:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:00 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:34:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:00 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:34:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:00 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:34:01.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:00 smithi151 ceph-mon[35236]: Standby manager daemon y started 2024-09-19T06:34:01.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:00 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:34:01.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:00 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:34:01.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:00 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:34:01.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:00 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3717932719' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:34:02.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:01 smithi037 ceph-mon[31211]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:02.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:01 smithi037 ceph-mon[31211]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T06:34:02.207 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:34:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:01 smithi096 ceph-mon[35004]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:01 smithi096 ceph-mon[35004]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T06:34:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:34:02.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:01 smithi151 ceph-mon[35236]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:02.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:01 smithi151 ceph-mon[35236]: mgrmap e15: x(active, since 64s), standbys: z, y 2024-09-19T06:34:02.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:34:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:03 smithi096 ceph-mon[35004]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:03 smithi037 ceph-mon[31211]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:03 smithi151 ceph-mon[35236]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:06.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:05 smithi151 ceph-mon[35236]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:06.180 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:05 smithi037 ceph-mon[31211]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:06.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:05 smithi096 ceph-mon[35004]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:08.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:07 smithi037 ceph-mon[31211]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:08.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:34:08.236 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:07 smithi096 ceph-mon[35004]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:07 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:34:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:07 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:08.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:07 smithi151 ceph-mon[35236]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:08.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:07 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-19T06:34:08.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:07 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:09.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:08 smithi037 ceph-mon[31211]: Deploying daemon osd.0 on smithi037 2024-09-19T06:34:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:08 smithi096 ceph-mon[35004]: Deploying daemon osd.0 on smithi037 2024-09-19T06:34:09.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:08 smithi151 ceph-mon[35236]: Deploying daemon osd.0 on smithi037 2024-09-19T06:34:10.224 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:09 smithi037 ceph-mon[31211]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:09 smithi096 ceph-mon[35004]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:09 smithi151 ceph-mon[35236]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:11 smithi096 ceph-mon[35004]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:12.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:11 smithi037 ceph-mon[31211]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:11 smithi151 ceph-mon[35236]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:13 smithi096 ceph-mon[35004]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:13 smithi037 ceph-mon[31211]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:13 smithi151 ceph-mon[35236]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:15.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:15.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:15.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:15.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:15.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:15.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:16.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:15 smithi037 ceph-mon[31211]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:15 smithi096 ceph-mon[35004]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:15 smithi151 ceph-mon[35236]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:17.773 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:17 smithi037 ceph-mon[31211]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:17.773 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.773 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.773 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:17.773 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:17.773 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:17 smithi096 ceph-mon[35004]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:17 smithi151 ceph-mon[35236]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:18.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.267+0000 7ff029ffb640 1 -- 172.21.15.37:0/1608755389 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff02c0fe4f0 con 0x7ff0140394d0 2024-09-19T06:34:18.269 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 0 on host 'smithi037' 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff0140394d0 msgr2=0x7ff01403b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff0140394d0 0x7ff01403b990 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff02c101b70 tx=0x7ff01c00b390 comp rx=0 tx=0).stop 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c1026d0 msgr2=0x7ff02c103630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c1026d0 0x7ff02c103630 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff02400eaa0 tx=0x7ff02400ef60 comp rx=0 tx=0).stop 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 shutdown_connections 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff0140394d0 0x7ff01403b990 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff02c1026d0 0x7ff02c103630 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff02c101d10 0x7ff02c1030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 --2- 172.21.15.37:0/1608755389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff02c100b10 0x7ff02c104aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:18.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 >> 172.21.15.37:0/1608755389 conn(0x7ff02c0fc2a0 msgr2=0x7ff02c0fdde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:34:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 shutdown_connections 2024-09-19T06:34:18.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:34:18.272+0000 7ff032fe2640 1 -- 172.21.15.37:0/1608755389 wait complete. 2024-09-19T06:34:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:18 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:18 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:18 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:18.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:18 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:18.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:18 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:18.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:18 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:18.919 DEBUG:teuthology.orchestra.run.smithi037:osd.0> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.0.service 2024-09-19T06:34:18.921 INFO:tasks.cephadm:Deploying osd.1 on smithi096 with /dev/vg_nvme/lv_4... 2024-09-19T06:34:18.921 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:34:19.162 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:34:19.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:19 smithi037 ceph-mon[31211]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:19.577 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:19 smithi096 ceph-mon[35004]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:19.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:19 smithi151 ceph-mon[35236]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:20.090 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:34:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0[45094]: 2024-09-19T06:34:19.816+0000 7fe60837c640 -1 osd.0 0 log_to_monitors true 2024-09-19T06:34:20.747 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:20 smithi096 ceph-mon[35004]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:34:20.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:20 smithi037 ceph-mon[31211]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:34:20.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:20 smithi151 ceph-mon[35236]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-19T06:34:21.465 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:34:21.489 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-09-19T06:34:21.706 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:34:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0[45094]: 2024-09-19T06:34:21.415+0000 7fe6042ff640 -1 osd.0 0 waiting for initial osdmap 2024-09-19T06:34:21.706 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:34:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0[45094]: 2024-09-19T06:34:21.418+0000 7fe600127640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:34:21.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:21 smithi037 ceph-mon[31211]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:21.706 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:21 smithi037 ceph-mon[31211]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:34:21.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:21 smithi037 ceph-mon[31211]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:34:21.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:21 smithi037 ceph-mon[31211]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:34:21.707 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:21.758 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:34:21.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:21 smithi096 ceph-mon[35004]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:21.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:21 smithi096 ceph-mon[35004]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:34:21.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:21 smithi096 ceph-mon[35004]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:34:21.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:21 smithi096 ceph-mon[35004]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:34:21.761 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:21.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:21 smithi151 ceph-mon[35236]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:21 smithi151 ceph-mon[35236]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-19T06:34:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:21 smithi151 ceph-mon[35236]: osdmap e6: 1 total, 0 up, 1 in 2024-09-19T06:34:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:21 smithi151 ceph-mon[35236]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:34:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2355823363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08107980 msgr2=0x7fbf08109d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2355823363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08107980 0x7fbf08109d70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009ca0 tx=0x7fbf0002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2355823363 shutdown_connections 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2355823363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08107980 0x7fbf08109d70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2355823363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf08105050 0x7fbf08107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2355823363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf08102720 0x7fbf08104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2355823363 >> 172.21.15.96:0/2355823363 conn(0x7fbf080fc2a0 msgr2=0x7fbf080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2355823363 shutdown_connections 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2355823363 wait complete. 2024-09-19T06:34:22.257 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 Processor -- start 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 -- start start 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf08102720 0x7fbf08197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08105050 0x7fbf08198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf08107980 0x7fbf0819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0810d970 con 0x7fbf08102720 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0810d7f0 con 0x7fbf08105050 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0fd1a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbf0810daf0 con 0x7fbf08107980 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08105050 0x7fbf08198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08105050 0x7fbf08198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:59400/0 (socket says 172.21.15.96:59400) 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 -- 172.21.15.96:0/2029473735 learned_addr learned my addr 172.21.15.96:0/2029473735 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T06:34:22.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0dab6640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf08102720 0x7fbf08197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 -- 172.21.15.96:0/2029473735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf08107980 msgr2=0x7fbf0819f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf08107980 0x7fbf0819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 -- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf08102720 msgr2=0x7fbf08197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf08102720 0x7fbf08197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.215+0000 7fbf0d2b5640 1 -- 172.21.15.96:0/2029473735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf0819fb60 con 0x7fbf08105050 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.216+0000 7fbf0d2b5640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08105050 0x7fbf08198470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d770 tx=0x7fbef800db70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.216+0000 7fbefeffd640 1 -- 172.21.15.96:0/2029473735 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8017070 con 0x7fbf08105050 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.216+0000 7fbefeffd640 1 -- 172.21.15.96:0/2029473735 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbef8010460 con 0x7fbf08105050 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.216+0000 7fbefeffd640 1 -- 172.21.15.96:0/2029473735 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef800f620 con 0x7fbf08105050 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.216+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf0819fe50 con 0x7fbf08105050 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.216+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf081a0330 con 0x7fbf08105050 2024-09-19T06:34:22.259 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.217+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf080fdd90 con 0x7fbf08105050 2024-09-19T06:34:22.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.218+0000 7fbefeffd640 1 -- 172.21.15.96:0/2029473735 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7fbef801a070 con 0x7fbf08105050 2024-09-19T06:34:22.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.219+0000 7fbefeffd640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbef40a3260 0x7fbef40a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:22.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.219+0000 7fbefeffd640 1 -- 172.21.15.96:0/2029473735 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7fbef80cd700 con 0x7fbf08105050 2024-09-19T06:34:22.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.219+0000 7fbf0dab6640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbef40a3260 0x7fbef40a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:34:22.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.219+0000 7fbf0dab6640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbef40a3260 0x7fbef40a5720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009990 tx=0x7fbf04009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:34:22.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.223+0000 7fbefeffd640 1 -- 172.21.15.96:0/2029473735 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbef809bfb0 con 0x7fbf08105050 2024-09-19T06:34:22.403 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:22.402+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fbf08062630 con 0x7fbef40a3260 2024-09-19T06:34:22.751 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi037 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi037 to 17092M 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.752 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:34:22.753 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:34:22.753 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:22.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:34:22.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:34:22.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:22.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi037 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi037 to 17092M 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:34:22.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: osdmap e7: 1 total, 0 up, 1 in 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi037 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi037 to 17092M 2024-09-19T06:34:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:22.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:22.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:22.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:34:22.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:34:22.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:23.714 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:23 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:34:23.715 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:23 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:34:23.715 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:23 smithi096 ceph-mon[35004]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:23.715 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:23 smithi096 ceph-mon[35004]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:34:23.715 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:23 smithi096 ceph-mon[35004]: osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] boot 2024-09-19T06:34:23.715 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:23 smithi096 ceph-mon[35004]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T06:34:23.715 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:23 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:23.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:23 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:34:23.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:23 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:34:23.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:23 smithi037 ceph-mon[31211]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:23.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:23 smithi037 ceph-mon[31211]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:34:23.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:23 smithi037 ceph-mon[31211]: osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] boot 2024-09-19T06:34:23.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:23 smithi037 ceph-mon[31211]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T06:34:23.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:23 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:23.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:23 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:34:23.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:23 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:34:23.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:23 smithi151 ceph-mon[35236]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-19T06:34:23.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:23 smithi151 ceph-mon[35236]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:34:23.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:23 smithi151 ceph-mon[35236]: osd.0 [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] boot 2024-09-19T06:34:23.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:23 smithi151 ceph-mon[35236]: osdmap e8: 1 total, 1 up, 1 in 2024-09-19T06:34:23.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:23 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:34:25.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:24 smithi096 ceph-mon[35004]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:34:25.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:24 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/3934899053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]: dispatch 2024-09-19T06:34:25.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:24 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]: dispatch 2024-09-19T06:34:25.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:24 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]': finished 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:24 smithi096 ceph-mon[35004]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:24 smithi037 ceph-mon[31211]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:24 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/3934899053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]: dispatch 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:24 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]: dispatch 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:24 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]': finished 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:24 smithi037 ceph-mon[31211]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T06:34:25.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:25.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:24 smithi151 ceph-mon[35236]: osdmap e9: 1 total, 1 up, 1 in 2024-09-19T06:34:25.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:24 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/3934899053' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]: dispatch 2024-09-19T06:34:25.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:24 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]: dispatch 2024-09-19T06:34:25.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:24 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "971dd8c5-2680-4d3a-812a-a7e391ff5560"}]': finished 2024-09-19T06:34:25.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:24 smithi151 ceph-mon[35236]: osdmap e10: 2 total, 1 up, 2 in 2024-09-19T06:34:25.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:26.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:25 smithi096 ceph-mon[35004]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:26.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:25 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/215511826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:26.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:25 smithi037 ceph-mon[31211]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:26.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:25 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/215511826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:26.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:25 smithi151 ceph-mon[35236]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:25 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/215511826' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:28.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:27 smithi096 ceph-mon[35004]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:28.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:27 smithi037 ceph-mon[31211]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:28.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:27 smithi151 ceph-mon[35236]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:30.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:29 smithi096 ceph-mon[35004]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:30.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:29 smithi037 ceph-mon[31211]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:30.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:29 smithi151 ceph-mon[35236]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:32.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:31 smithi096 ceph-mon[35004]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:32.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:31 smithi037 ceph-mon[31211]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:32.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:31 smithi151 ceph-mon[35236]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:34.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:33 smithi037 ceph-mon[31211]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:34.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:34:34.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:34.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:33 smithi151 ceph-mon[35236]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:34.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:34:34.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:34.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:33 smithi096 ceph-mon[35004]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:34.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-19T06:34:34.165 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:35.001 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:34 smithi096 ceph-mon[35004]: Deploying daemon osd.1 on smithi096 2024-09-19T06:34:35.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:34 smithi151 ceph-mon[35236]: Deploying daemon osd.1 on smithi096 2024-09-19T06:34:35.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:34 smithi037 ceph-mon[31211]: Deploying daemon osd.1 on smithi096 2024-09-19T06:34:35.899 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:35 smithi096 ceph-mon[35004]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:36.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:35 smithi151 ceph-mon[35236]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:35 smithi037 ceph-mon[31211]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:38.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:37 smithi151 ceph-mon[35236]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:38.222 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:37 smithi096 ceph-mon[35004]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:37 smithi037 ceph-mon[31211]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:39.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:39 smithi096 ceph-mon[35004]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:39.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:39.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:39.801 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:40.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:39 smithi151 ceph-mon[35236]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:40.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:40.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:40.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:40.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:39 smithi037 ceph-mon[31211]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:40.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:40.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:40.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:42.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:41 smithi096 ceph-mon[35004]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:42.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:41 smithi151 ceph-mon[35236]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:42.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:41 smithi037 ceph-mon[31211]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:43.174 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.174 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.174 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:43.174 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:43.174 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.213 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 1 on host 'smithi096' 2024-09-19T06:34:43.214 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.211+0000 7fbefeffd640 1 -- 172.21.15.96:0/2029473735 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf08062630 con 0x7fbef40a3260 2024-09-19T06:34:43.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbef40a3260 msgr2=0x7fbef40a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:43.215 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbef40a3260 0x7fbef40a5720 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009990 tx=0x7fbf04009110 comp rx=0 tx=0).stop 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08105050 msgr2=0x7fbf08198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08105050 0x7fbf08198470 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fbef800d770 tx=0x7fbef800db70 comp rx=0 tx=0).stop 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 shutdown_connections 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbef40a3260 0x7fbef40a5720 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbf08107980 0x7fbf0819f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbf08105050 0x7fbf08198470 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 --2- 172.21.15.96:0/2029473735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbf08102720 0x7fbf08197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 >> 172.21.15.96:0/2029473735 conn(0x7fbf080fc2a0 msgr2=0x7fbf08102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 shutdown_connections 2024-09-19T06:34:43.216 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:34:43.214+0000 7fbf0fd1a640 1 -- 172.21.15.96:0/2029473735 wait complete. 2024-09-19T06:34:43.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:43.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:43.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:43.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:43.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:43.847 DEBUG:teuthology.orchestra.run.smithi096:osd.1> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.1.service 2024-09-19T06:34:43.849 INFO:tasks.cephadm:Deploying osd.2 on smithi151 with /dev/vg_nvme/lv_4... 2024-09-19T06:34:43.849 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-19T06:34:44.106 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:34:44.146 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:44 smithi096 ceph-mon[35004]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:44.147 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:44.147 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:44.147 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:44.214 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:44 smithi151 ceph-mon[35236]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:44.214 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:44.214 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:44.214 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:44.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:44 smithi037 ceph-mon[31211]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:44.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:34:44.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:44.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:45.922 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:34:45 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1[39479]: 2024-09-19T06:34:45.629+0000 7fa06e1df640 -1 osd.1 0 log_to_monitors true 2024-09-19T06:34:46.235 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:46 smithi096 ceph-mon[35004]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:46.235 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:46 smithi096 ceph-mon[35004]: from='osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:34:46.235 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:46 smithi096 ceph-mon[35004]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:34:46.275 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:46 smithi151 ceph-mon[35236]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:46.275 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:46 smithi151 ceph-mon[35236]: from='osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:34:46.275 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:46 smithi151 ceph-mon[35236]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:34:46.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:46 smithi037 ceph-mon[31211]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:46.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:46 smithi037 ceph-mon[31211]: from='osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:34:46.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:46 smithi037 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-19T06:34:46.278 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:34:46.304 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-09-19T06:34:46.550 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:34:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 -- 172.21.15.151:0/2191874053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc1037c0 msgr2=0x7f55cc10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/2191874053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc1037c0 0x7f55cc10a050 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009ca0 tx=0x7f55c402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:34:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 -- 172.21.15.151:0/2191874053 shutdown_connections 2024-09-19T06:34:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/2191874053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc1037c0 0x7f55cc10a050 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:47.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/2191874053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc102e00 0x7f55cc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/2191874053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55cc101c00 0x7f55cc102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 -- 172.21.15.151:0/2191874053 >> 172.21.15.151:0/2191874053 conn(0x7f55cc0fd390 msgr2=0x7f55cc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 -- 172.21.15.151:0/2191874053 shutdown_connections 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.025+0000 7f55d0ff0640 1 -- 172.21.15.151:0/2191874053 wait complete. 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 Processor -- start 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 -- start start 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc101c00 0x7f55cc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc102e00 0x7f55cc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:47.030 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55cc1037c0 0x7f55cc19f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc10d8b0 con 0x7f55cc102e00 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc10d730 con 0x7f55cc1037c0 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.026+0000 7f55d0ff0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc10da30 con 0x7f55cc101c00 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc101c00 0x7f55cc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc101c00 0x7f55cc197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:42524/0 (socket says 172.21.15.151:42524) 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 -- 172.21.15.151:0/1013494430 learned_addr learned my addr 172.21.15.151:0/1013494430 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55c9d9b640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc102e00 0x7f55cc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:34:47.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55cad9d640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55cc1037c0 0x7f55cc19f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 -- 172.21.15.151:0/1013494430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55cc1037c0 msgr2=0x7f55cc19f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55cc1037c0 0x7f55cc19f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 -- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc102e00 msgr2=0x7f55cc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc102e00 0x7f55cc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 -- 172.21.15.151:0/1013494430 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc19fa40 con 0x7f55cc101c00 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55ca59c640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc101c00 0x7f55cc197e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c970 tx=0x7f55bc00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55bb7fe640 1 -- 172.21.15.151:0/1013494430 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc007c10 con 0x7f55cc101c00 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55bb7fe640 1 -- 172.21.15.151:0/1013494430 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f55bc007db0 con 0x7f55cc101c00 2024-09-19T06:34:47.032 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc19fd30 con 0x7f55cc101c00 2024-09-19T06:34:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55bb7fe640 1 -- 172.21.15.151:0/1013494430 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55bc00f940 con 0x7f55cc101c00 2024-09-19T06:34:47.033 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.027+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc1a01c0 con 0x7f55cc101c00 2024-09-19T06:34:47.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.028+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55cc108f30 con 0x7f55cc101c00 2024-09-19T06:34:47.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.030+0000 7f55bb7fe640 1 -- 172.21.15.151:0/1013494430 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f55bc00fae0 con 0x7f55cc101c00 2024-09-19T06:34:47.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.030+0000 7f55bb7fe640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f55b40a31e0 0x7f55b40a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:34:47.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.031+0000 7f55c9d9b640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f55b40a31e0 0x7f55b40a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:34:47.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.031+0000 7f55c9d9b640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f55b40a31e0 0x7f55b40a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55cc199330 tx=0x7f55c000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:34:47.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.033+0000 7f55bb7fe640 1 -- 172.21.15.151:0/1013494430 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 2029+0+0 (secure 0 0 0) 0x7f55bc0cda70 con 0x7f55cc101c00 2024-09-19T06:34:47.042 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.041+0000 7f55bb7fe640 1 -- 172.21.15.151:0/1013494430 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f55bc09ccf0 con 0x7f55cc101c00 2024-09-19T06:34:47.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:34:47.221+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f55cc062630 con 0x7f55b40a31e0 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.223 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:47.224 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:47.224 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.224 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:47 smithi151 ceph-mon[35236]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:34:47.524 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1[39479]: 2024-09-19T06:34:47.024+0000 7fa06a162640 -1 osd.1 0 waiting for initial osdmap 2024-09-19T06:34:47.524 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1[39479]: 2024-09-19T06:34:47.027+0000 7fa065f8a640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:34:47.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:47 smithi096 ceph-mon[35004]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: osdmap e11: 2 total, 1 up, 2 in 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:34:47.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:47.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:34:47.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:34:47.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:47 smithi037 ceph-mon[31211]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:34:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi096 2024-09-19T06:34:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-19T06:34:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:34:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:34:48.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:34:48.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:48.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi096 2024-09-19T06:34:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-19T06:34:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:48.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi096 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi096 to 17088M 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: osdmap e12: 2 total, 1 up, 2 in 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:48.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:49.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:49 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:34:49.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:49 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:34:49.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:49 smithi151 ceph-mon[35236]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:34:49.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:49 smithi151 ceph-mon[35236]: osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] boot 2024-09-19T06:34:49.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:49 smithi151 ceph-mon[35236]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T06:34:49.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:49 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:34:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:49 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:34:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:49 smithi096 ceph-mon[35004]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:34:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:49 smithi096 ceph-mon[35004]: osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] boot 2024-09-19T06:34:49.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:49 smithi096 ceph-mon[35004]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T06:34:49.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:49.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:49 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:34:49.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:49 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:34:49.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:49 smithi037 ceph-mon[31211]: from='client.24154 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:34:49.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:49 smithi037 ceph-mon[31211]: osd.1 [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] boot 2024-09-19T06:34:49.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:49 smithi037 ceph-mon[31211]: osdmap e13: 2 total, 2 up, 2 in 2024-09-19T06:34:49.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:34:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:50 smithi151 ceph-mon[35236]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:50 smithi151 ceph-mon[35236]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T06:34:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:50 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/444983340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]: dispatch 2024-09-19T06:34:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:50 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]: dispatch 2024-09-19T06:34:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:50 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]': finished 2024-09-19T06:34:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:50 smithi151 ceph-mon[35236]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T06:34:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:50 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:34:50.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:50 smithi096 ceph-mon[35004]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:50.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:50 smithi096 ceph-mon[35004]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T06:34:50.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:50 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/444983340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]: dispatch 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:50 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]: dispatch 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:50 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]': finished 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:50 smithi096 ceph-mon[35004]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:50 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:50 smithi037 ceph-mon[31211]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:50 smithi037 ceph-mon[31211]: osdmap e14: 2 total, 2 up, 2 in 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:50 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/444983340' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]: dispatch 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:50 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]: dispatch 2024-09-19T06:34:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:50 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd59ee43-9b0c-4440-a57c-7f944560b77c"}]': finished 2024-09-19T06:34:50.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:50 smithi037 ceph-mon[31211]: osdmap e15: 3 total, 2 up, 3 in 2024-09-19T06:34:50.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:50 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:34:51.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:51 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/827898034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:51.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:51 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/827898034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:51.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:51 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/827898034' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:34:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:52 smithi151 ceph-mon[35236]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:52.426 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:52 smithi037 ceph-mon[31211]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:52.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:52 smithi096 ceph-mon[35004]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:54 smithi151 ceph-mon[35236]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:54 smithi096 ceph-mon[35004]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:54.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:54 smithi037 ceph-mon[31211]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:56.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:56 smithi151 ceph-mon[35236]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:56 smithi096 ceph-mon[35004]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:56.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:56 smithi037 ceph-mon[31211]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:58 smithi151 ceph-mon[35236]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:58.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:34:58.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:58 smithi096 ceph-mon[35004]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:34:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:58 smithi037 ceph-mon[31211]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:34:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-19T06:34:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:34:59.115 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:34:59 smithi151 ceph-mon[35236]: Deploying daemon osd.2 on smithi151 2024-09-19T06:34:59.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:34:59 smithi096 ceph-mon[35004]: Deploying daemon osd.2 on smithi151 2024-09-19T06:34:59.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:34:59 smithi037 ceph-mon[31211]: Deploying daemon osd.2 on smithi151 2024-09-19T06:35:00.353 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:00 smithi151 ceph-mon[35236]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:00 smithi096 ceph-mon[35004]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:00 smithi037 ceph-mon[31211]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:02.217 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:02 smithi151 ceph-mon[35236]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:02.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:02 smithi096 ceph-mon[35004]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:02.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:02 smithi037 ceph-mon[31211]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:04.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:04 smithi151 ceph-mon[35236]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:04.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:04 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:04.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:04 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:04.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:04 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:04 smithi096 ceph-mon[35004]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:04 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:04 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:04 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:04 smithi037 ceph-mon[31211]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:04 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:04 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:04 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.326 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:06 smithi096 ceph-mon[35004]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:06.326 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:06 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.327 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:06 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.327 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:06 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:06.327 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:06 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:06.327 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:06 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:06 smithi151 ceph-mon[35236]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:06 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:06 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:06 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:06 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:06 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:06 smithi037 ceph-mon[31211]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:06.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:06 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:06 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:06.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:06 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:06.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:06 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:06 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:07.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.361+0000 7f55bb7fe640 1 -- 172.21.15.151:0/1013494430 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f55cc062630 con 0x7f55b40a31e0 2024-09-19T06:35:07.364 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 2 on host 'smithi151' 2024-09-19T06:35:07.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.363+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f55b40a31e0 msgr2=0x7f55b40a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:07.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.363+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f55b40a31e0 0x7f55b40a56a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f55cc199330 tx=0x7f55c000b390 comp rx=0 tx=0).stop 2024-09-19T06:35:07.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.363+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc101c00 msgr2=0x7f55cc197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:07.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.363+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc101c00 0x7f55cc197e10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c970 tx=0x7f55bc00ce30 comp rx=0 tx=0).stop 2024-09-19T06:35:07.365 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 shutdown_connections 2024-09-19T06:35:07.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f55b40a31e0 0x7f55b40a56a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:07.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f55cc1037c0 0x7f55cc19f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:07.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f55cc102e00 0x7f55cc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:07.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 --2- 172.21.15.151:0/1013494430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f55cc101c00 0x7f55cc197e10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:07.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 >> 172.21.15.151:0/1013494430 conn(0x7f55cc0fd390 msgr2=0x7f55cc106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:35:07.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 shutdown_connections 2024-09-19T06:35:07.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:35:07.364+0000 7f55d0ff0640 1 -- 172.21.15.151:0/1013494430 wait complete. 2024-09-19T06:35:07.892 DEBUG:teuthology.orchestra.run.smithi151:osd.2> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.2.service 2024-09-19T06:35:07.895 INFO:tasks.cephadm:Deploying osd.3 on smithi037 with /dev/vg_nvme/lv_3... 2024-09-19T06:35:07.895 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:35:08.161 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:35:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:08 smithi151 ceph-mon[35236]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:08.188 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:08.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:08.189 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:08 smithi037 ceph-mon[31211]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:08.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:08 smithi096 ceph-mon[35004]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:08.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:08.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:08.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:09.613 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:09 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2[40199]: 2024-09-19T06:35:09.423+0000 7f72a8a5f640 -1 osd.2 0 log_to_monitors true 2024-09-19T06:35:10.252 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:10 smithi151 ceph-mon[35236]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:10.252 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:10 smithi151 ceph-mon[35236]: from='osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:35:10.252 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:10 smithi151 ceph-mon[35236]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:35:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:10 smithi037 ceph-mon[31211]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:10 smithi037 ceph-mon[31211]: from='osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:35:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:10 smithi037 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:35:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:10 smithi096 ceph-mon[35004]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:10 smithi096 ceph-mon[35004]: from='osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:35:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:10 smithi096 ceph-mon[35004]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-19T06:35:10.806 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:35:10.833 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_3 2024-09-19T06:35:11.087 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:35:11.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:11.367 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:11.368 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: osdmap e16: 3 total, 2 up, 3 in 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:11.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:11.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:11.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 -- 172.21.15.37:0/3555430704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc102e10 msgr2=0x7f49dc103290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:11.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 --2- 172.21.15.37:0/3555430704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc102e10 0x7f49dc103290 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009900 tx=0x7f49cc02a410 comp rx=0 tx=0).stop 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 -- 172.21.15.37:0/3555430704 shutdown_connections 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 --2- 172.21.15.37:0/3555430704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc1037d0 0x7f49dc10a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 --2- 172.21.15.37:0/3555430704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc102e10 0x7f49dc103290 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 --2- 172.21.15.37:0/3555430704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49dc101c10 0x7f49dc102010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 -- 172.21.15.37:0/3555430704 >> 172.21.15.37:0/3555430704 conn(0x7f49dc0fd3c0 msgr2=0x7f49dc0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 -- 172.21.15.37:0/3555430704 shutdown_connections 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 -- 172.21.15.37:0/3555430704 wait complete. 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.599+0000 7f49e351e640 1 Processor -- start 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e351e640 1 -- start start 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e351e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc101c10 0x7f49dc197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:11.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e351e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49dc102e10 0x7f49dc198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e351e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc1037d0 0x7f49dc19f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e351e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10d890 con 0x7f49dc101c10 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e351e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10d710 con 0x7f49dc1037d0 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e351e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49dc10da10 con 0x7f49dc102e10 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc101c10 0x7f49dc197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc101c10 0x7f49dc197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59468/0 (socket says 172.21.15.37:59468) 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 -- 172.21.15.37:0/3469501482 learned_addr learned my addr 172.21.15.37:0/3469501482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e0ab9640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49dc102e10 0x7f49dc198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e1abb640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc1037d0 0x7f49dc19f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 -- 172.21.15.37:0/3469501482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49dc102e10 msgr2=0x7f49dc198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49dc102e10 0x7f49dc198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 -- 172.21.15.37:0/3469501482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc1037d0 msgr2=0x7f49dc19f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:11.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc1037d0 0x7f49dc19f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 -- 172.21.15.37:0/3469501482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49dc19fb80 con 0x7f49dc101c10 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e1abb640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc1037d0 0x7f49dc19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.600+0000 7f49e12ba640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc101c10 0x7f49dc197f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49d800eaa0 tx=0x7f49d800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.601+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8016070 con 0x7f49dc101c10 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.601+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f49d8003e50 con 0x7f49dc101c10 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.601+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49dc19fe70 con 0x7f49dc101c10 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.601+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d80116d0 con 0x7f49dc101c10 2024-09-19T06:35:11.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.601+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49dc1a0270 con 0x7f49dc101c10 2024-09-19T06:35:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.602+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49dc109030 con 0x7f49dc101c10 2024-09-19T06:35:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.603+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f49d8019070 con 0x7f49dc101c10 2024-09-19T06:35:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.604+0000 7f49d27fc640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f49c80a31e0 0x7f49c80a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.604+0000 7f49e0ab9640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f49c80a31e0 0x7f49c80a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.604+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f49d80ce200 con 0x7f49dc101c10 2024-09-19T06:35:11.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.604+0000 7f49e0ab9640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f49c80a31e0 0x7f49c80a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0314e0 tx=0x7f49cc000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:35:11.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.608+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49d809c670 con 0x7f49dc101c10 2024-09-19T06:35:11.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:11.795+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f49dc062630 con 0x7f49c80a31e0 2024-09-19T06:35:11.862 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2[40199]: 2024-09-19T06:35:11.493+0000 7f72a51f6640 -1 osd.2 0 waiting for initial osdmap 2024-09-19T06:35:11.862 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2[40199]: 2024-09-19T06:35:11.498+0000 7f729c802640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:35:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi151 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: from='osd.2 ' entity='osd.2' 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:35:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:12.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:12.442 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi151 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: from='osd.2 ' entity='osd.2' 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:35:12.443 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi151 2024-09-19T06:35:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi151 to 17088M 2024-09-19T06:35:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:35:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: osdmap e17: 3 total, 2 up, 3 in 2024-09-19T06:35:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: from='osd.2 ' entity='osd.2' 2024-09-19T06:35:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:35:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:35:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:13.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:13 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:35:13.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:13 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:35:13.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:13 smithi037 ceph-mon[31211]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:35:13.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:13 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:13.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:13 smithi037 ceph-mon[31211]: osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] boot 2024-09-19T06:35:13.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:13 smithi037 ceph-mon[31211]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T06:35:13.214 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:13 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:13 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:35:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:13 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:35:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:13 smithi151 ceph-mon[35236]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:35:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:13 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:13 smithi151 ceph-mon[35236]: osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] boot 2024-09-19T06:35:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:13 smithi151 ceph-mon[35236]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T06:35:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:13 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:13 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:35:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:13 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:35:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:13 smithi096 ceph-mon[35004]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:35:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:13 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:13 smithi096 ceph-mon[35004]: osd.2 [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] boot 2024-09-19T06:35:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:13 smithi096 ceph-mon[35004]: osdmap e18: 3 total, 3 up, 3 in 2024-09-19T06:35:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:13 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:35:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:14 smithi151 ceph-mon[35236]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:14 smithi096 ceph-mon[35004]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:14 smithi037 ceph-mon[31211]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: pgmap v78: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4080219423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77fbc8c9-cfb7-49d4-a395-c747b5061988"}]: dispatch 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4080219423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77fbc8c9-cfb7-49d4-a395-c747b5061988"}]': finished 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:35:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T06:35:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: pgmap v78: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:35:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4080219423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77fbc8c9-cfb7-49d4-a395-c747b5061988"}]: dispatch 2024-09-19T06:35:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:35:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4080219423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77fbc8c9-cfb7-49d4-a395-c747b5061988"}]': finished 2024-09-19T06:35:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T06:35:15.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:15.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:35:15.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: osdmap e19: 3 total, 3 up, 3 in 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: pgmap v78: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4080219423' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "77fbc8c9-cfb7-49d4-a395-c747b5061988"}]: dispatch 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4080219423' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "77fbc8c9-cfb7-49d4-a395-c747b5061988"}]': finished 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: osdmap e20: 4 total, 3 up, 4 in 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:35:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:16 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3703776969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:35:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:16 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:35:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:16 smithi096 ceph-mon[35004]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T06:35:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:16 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:16.525 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49743]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:35:16.525 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49743]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:16.525 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49743]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:16.525 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49743]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:16.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:16 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3703776969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:35:16.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:16 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:35:16.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:16 smithi037 ceph-mon[31211]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T06:35:16.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:16 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:16.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:16 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3703776969' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:35:16.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:16 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:35:16.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:16 smithi151 ceph-mon[35236]: osdmap e21: 4 total, 3 up, 4 in 2024-09-19T06:35:16.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:16 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:17.025 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49930]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:35:17.025 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49930]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:17.025 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49930]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:17.025 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:35:16 smithi037 sudo[49930]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:17.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:17.133+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f49d80985e0 con 0x7f49dc101c10 2024-09-19T06:35:17.160 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:16 smithi096 sudo[43177]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:35:17.160 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:16 smithi096 sudo[43177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:17.160 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:16 smithi096 sudo[43177]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:17.160 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:16 smithi096 sudo[43177]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:17.524 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:17 smithi096 sudo[43182]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:35:17.524 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:17 smithi096 sudo[43182]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:17.524 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:17 smithi096 sudo[43182]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:17.524 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:35:17 smithi096 sudo[43182]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:17.686 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:17 smithi151 ceph-mon[35236]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:17.686 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:17 smithi151 ceph-mon[35236]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T06:35:17.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:17.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:17.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:17 smithi151 ceph-mon[35236]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T06:35:17.687 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:17.687 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44116]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:35:17.687 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44116]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:17.687 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44116]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:17.687 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44116]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:18.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:17 smithi096 ceph-mon[35004]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:18.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:17 smithi096 ceph-mon[35004]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T06:35:18.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:18.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:18.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:17 smithi096 ceph-mon[35004]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T06:35:18.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:18.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:17 smithi037 ceph-mon[31211]: pgmap v81: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:18.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:17 smithi037 ceph-mon[31211]: osdmap e22: 4 total, 3 up, 4 in 2024-09-19T06:35:18.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:18.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:18.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:17 smithi037 ceph-mon[31211]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-19T06:35:18.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:18.101 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44120]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:35:18.101 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44120]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:18.101 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44120]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:18.101 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:35:17 smithi151 sudo[44120]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50306]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:35:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50306]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50306]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:19.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50492]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:35:19.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50492]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:19.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50492]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:19.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:18 smithi037 sudo[50492]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:19.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:18 smithi151 sudo[44123]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:35:19.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:18 smithi151 sudo[44123]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:19.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:18 smithi151 sudo[44123]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:19.103 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:18 smithi151 sudo[44123]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:19.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 sudo[44126]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:35:19.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 sudo[44126]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:19.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 sudo[44126]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:19.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 sudo[44126]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:19.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43185]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-19T06:35:19.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43185]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:19.604 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43185]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:19.605 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43185]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:19.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:19.889 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:19.890 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:20.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43188]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-19T06:35:20.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43188]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-19T06:35:20.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43188]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-19T06:35:20.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 sudo[43188]: pam_unix(sudo:session): session closed for user root 2024-09-19T06:35:20.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:20.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:20.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:20.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:20.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:20.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:20.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:20.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:20.869 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:20.870 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:20 smithi037 ceph-mon[31211]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:21.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:21.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:21.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:21.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:21.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:21.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:21.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:21.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:20 smithi096 ceph-mon[35004]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:21.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: pgmap v83: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:21.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:21.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:21.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-19T06:35:21.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:35:21.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:35:21.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:35:21.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:20 smithi151 ceph-mon[35236]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-19T06:35:21.916 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:21 smithi037 ceph-mon[31211]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:22.166 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:21 smithi151 ceph-mon[35236]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:22.166 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:21 smithi096 ceph-mon[35004]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:24.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:23 smithi096 ceph-mon[35004]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:24.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:23 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:35:24.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:23 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:24.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:23 smithi096 ceph-mon[35004]: Deploying daemon osd.3 on smithi037 2024-09-19T06:35:24.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:23 smithi037 ceph-mon[31211]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:24.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:23 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:35:24.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:23 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:24.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:23 smithi037 ceph-mon[31211]: Deploying daemon osd.3 on smithi037 2024-09-19T06:35:24.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:23 smithi151 ceph-mon[35236]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:24.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:23 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-19T06:35:24.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:23 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:24.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:23 smithi151 ceph-mon[35236]: Deploying daemon osd.3 on smithi037 2024-09-19T06:35:25.953 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:25 smithi037 ceph-mon[31211]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T06:35:26.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:25 smithi096 ceph-mon[35004]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T06:35:26.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:25 smithi151 ceph-mon[35236]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 65 KiB/s wr, 6 op/s 2024-09-19T06:35:28.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:27 smithi037 ceph-mon[31211]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T06:35:28.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:27 smithi096 ceph-mon[35004]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T06:35:28.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:27 smithi151 ceph-mon[35236]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.7 KiB/s rd, 44 KiB/s wr, 4 op/s 2024-09-19T06:35:29.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:29 smithi037 ceph-mon[31211]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T06:35:29.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:29.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:29.919 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:30.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:29 smithi096 ceph-mon[35004]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T06:35:30.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:30.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:30.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:30.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:29 smithi151 ceph-mon[35236]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 33 KiB/s wr, 3 op/s 2024-09-19T06:35:30.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:30.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:30.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:31.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:31 smithi037 ceph-mon[31211]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T06:35:32.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:31 smithi096 ceph-mon[35004]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T06:35:32.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:31 smithi151 ceph-mon[35236]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 1023 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-19T06:35:33.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:33.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:33.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.384 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:33.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:33.385 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.422+0000 7f49d27fc640 1 -- 172.21.15.37:0/3469501482 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49dc062630 con 0x7f49c80a31e0 2024-09-19T06:35:33.423 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 3 on host 'smithi037' 2024-09-19T06:35:33.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f49c80a31e0 msgr2=0x7f49c80a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f49c80a31e0 0x7f49c80a56a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0314e0 tx=0x7f49cc000bc0 comp rx=0 tx=0).stop 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc101c10 msgr2=0x7f49dc197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc101c10 0x7f49dc197f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f49d800eaa0 tx=0x7f49d800ef60 comp rx=0 tx=0).stop 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 shutdown_connections 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f49c80a31e0 0x7f49c80a56a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49dc1037d0 0x7f49dc19f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f49dc102e10 0x7f49dc198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 --2- 172.21.15.37:0/3469501482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f49dc101c10 0x7f49dc197f50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.424+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 >> 172.21.15.37:0/3469501482 conn(0x7f49dc0fd3c0 msgr2=0x7f49dc106200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.425+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 shutdown_connections 2024-09-19T06:35:33.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:35:33.425+0000 7f49e351e640 1 -- 172.21.15.37:0/3469501482 wait complete. 2024-09-19T06:35:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:34.136 DEBUG:teuthology.orchestra.run.smithi037:osd.3> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.3.service 2024-09-19T06:35:34.138 INFO:tasks.cephadm:Deploying osd.4 on smithi096 with /dev/vg_nvme/lv_3... 2024-09-19T06:35:34.138 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:35:34.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:34 smithi151 ceph-mon[35236]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T06:35:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:34 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:34 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:34 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:34.393 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:35:34.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:34 smithi037 ceph-mon[31211]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T06:35:34.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:34.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:34.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:34.513 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:34 smithi096 ceph-mon[35004]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T06:35:34.513 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:34 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:34.513 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:34 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:34.513 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:34 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:35.271 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:35:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:35:35.183+0000 7f0384b95640 -1 osd.3 0 log_to_monitors true 2024-09-19T06:35:36.435 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:36 smithi151 ceph-mon[35236]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T06:35:36.435 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:36 smithi151 ceph-mon[35236]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:35:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:36 smithi096 ceph-mon[35004]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T06:35:36.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:36 smithi096 ceph-mon[35004]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:35:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:36 smithi037 ceph-mon[31211]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-19T06:35:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:36 smithi037 ceph-mon[31211]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-19T06:35:36.916 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:35:36.945 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-09-19T06:35:37.194 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:35:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:35:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T06:35:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:35:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi037 2024-09-19T06:35:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi037 to 8546M 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:37 smithi096 ceph-mon[35004]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi037 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi037 to 8546M 2024-09-19T06:35:37.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:37.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:37.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-mon[31211]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:35:37.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-19T06:35:37.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: osdmap e23: 4 total, 3 up, 4 in 2024-09-19T06:35:37.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:35:37.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:37.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-19T06:35:37.394 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi037 2024-09-19T06:35:37.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:37.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:35:37.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi037 to 8546M 2024-09-19T06:35:37.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.395 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:37.396 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:37.396 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:37.396 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:37 smithi151 ceph-mon[35236]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:35:37.692 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 -- 172.21.15.96:0/1584808304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988074b40 msgr2=0x7fa988074fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 --2- 172.21.15.96:0/1584808304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988074b40 0x7fa988074fa0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009900 tx=0x7fa97c02a410 comp rx=0 tx=0).stop 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 -- 172.21.15.96:0/1584808304 shutdown_connections 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 --2- 172.21.15.96:0/1584808304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa988109a70 0x7fa98810bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 --2- 172.21.15.96:0/1584808304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988074b40 0x7fa988074fa0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 --2- 172.21.15.96:0/1584808304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9880761a0 0x7fa988074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 -- 172.21.15.96:0/1584808304 >> 172.21.15.96:0/1584808304 conn(0x7fa9880fd390 msgr2=0x7fa9880ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.673+0000 7fa98e532640 1 -- 172.21.15.96:0/1584808304 shutdown_connections 2024-09-19T06:35:37.693 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.674+0000 7fa98e532640 1 -- 172.21.15.96:0/1584808304 wait complete. 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.674+0000 7fa98e532640 1 Processor -- start 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.674+0000 7fa98e532640 1 -- start start 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.674+0000 7fa98e532640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa988074b40 0x7fa98819a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.674+0000 7fa98e532640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9880761a0 0x7fa98819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98e532640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988109a70 0x7fa9881a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98e532640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa98810fa70 con 0x7fa9880761a0 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98e532640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa98810f8f0 con 0x7fa988109a70 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98e532640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa98810fbf0 con 0x7fa988074b40 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988109a70 0x7fa9881a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988109a70 0x7fa9881a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:46890/0 (socket says 172.21.15.96:46890) 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa987fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa988074b40 0x7fa98819a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:37.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 -- 172.21.15.96:0/3495356564 learned_addr learned my addr 172.21.15.96:0/3495356564 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa9877fe640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9880761a0 0x7fa98819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 -- 172.21.15.96:0/3495356564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa988074b40 msgr2=0x7fa98819a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa988074b40 0x7fa98819a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 -- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9880761a0 msgr2=0x7fa98819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9880761a0 0x7fa98819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 -- 172.21.15.96:0/3495356564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9881a1d50 con 0x7fa988109a70 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98cacf640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988109a70 0x7fa9881a16e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa98000cad0 tx=0x7fa98000cf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa9857fa640 1 -- 172.21.15.96:0/3495356564 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98000ae20 con 0x7fa988109a70 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa9857fa640 1 -- 172.21.15.96:0/3495356564 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa980005020 con 0x7fa988109a70 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9881a1fe0 con 0x7fa988109a70 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa9857fa640 1 -- 172.21.15.96:0/3495356564 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98000f6f0 con 0x7fa988109a70 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.675+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9881a24a0 con 0x7fa988109a70 2024-09-19T06:35:37.695 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.676+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa988074fa0 con 0x7fa988109a70 2024-09-19T06:35:37.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.677+0000 7fa9857fa640 1 -- 172.21.15.96:0/3495356564 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa9800051d0 con 0x7fa988109a70 2024-09-19T06:35:37.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.678+0000 7fa9857fa640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa9700a3270 0x7fa9700a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:35:37.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.678+0000 7fa9857fa640 1 -- 172.21.15.96:0/3495356564 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7fa9800ce560 con 0x7fa988109a70 2024-09-19T06:35:37.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.678+0000 7fa987fff640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa9700a3270 0x7fa9700a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:35:37.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.679+0000 7fa987fff640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa9700a3270 0x7fa9700a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa978005fd0 tx=0x7fa97800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:35:37.696 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.682+0000 7fa9857fa640 1 -- 172.21.15.96:0/3495356564 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa980014030 con 0x7fa988109a70 2024-09-19T06:35:37.775 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:35:37.505+0000 7f0380b18640 -1 osd.3 0 waiting for initial osdmap 2024-09-19T06:35:37.775 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:35:37 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:35:37.511+0000 7f0378938640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:35:37.859 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:37.857+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa988062630 con 0x7fa9700a3270 2024-09-19T06:35:38.419 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] boot 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T06:35:38.420 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T06:35:38.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] boot 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T06:35:38.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:38 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: osdmap e24: 4 total, 3 up, 4 in 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019]' entity='osd.3' 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: osd.3 [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] boot 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: osdmap e25: 4 total, 4 up, 4 in 2024-09-19T06:35:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:38 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:35:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:39 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:35:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:39 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:35:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:39 smithi096 ceph-mon[35004]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:39 smithi096 ceph-mon[35004]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T06:35:39.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:39 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:35:39.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:39 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:35:39.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:39 smithi037 ceph-mon[31211]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:39.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:39 smithi037 ceph-mon[31211]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T06:35:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:39 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:35:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:39 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:35:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:39 smithi151 ceph-mon[35236]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:39 smithi151 ceph-mon[35236]: osdmap e26: 4 total, 4 up, 4 in 2024-09-19T06:35:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T06:35:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/2303976442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]: dispatch 2024-09-19T06:35:41.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]: dispatch 2024-09-19T06:35:41.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]': finished 2024-09-19T06:35:41.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:35:41.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:35:41.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:41 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/3347995361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:35:41.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T06:35:41.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:41.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/2303976442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]: dispatch 2024-09-19T06:35:41.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]: dispatch 2024-09-19T06:35:41.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]': finished 2024-09-19T06:35:41.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:35:41.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:35:41.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:41 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/3347995361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:35:41.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: osdmap e27: 4 total, 4 up, 4 in 2024-09-19T06:35:41.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: pgmap v99: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:41.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/2303976442' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]: dispatch 2024-09-19T06:35:41.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]: dispatch 2024-09-19T06:35:41.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c58779a9-0c41-444b-b1c1-93e50dbca0eb"}]': finished 2024-09-19T06:35:41.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: osdmap e28: 5 total, 4 up, 5 in 2024-09-19T06:35:41.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:35:41.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:41 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/3347995361' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:35:42.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:42 smithi151 ceph-mon[35236]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:35:42.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:42 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:35:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:42 smithi096 ceph-mon[35004]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:35:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:42 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:35:42.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:42 smithi037 ceph-mon[31211]: osdmap e29: 5 total, 4 up, 5 in 2024-09-19T06:35:42.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:42 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:35:43.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:43 smithi151 ceph-mon[35236]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:43 smithi096 ceph-mon[35004]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:43.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:43 smithi037 ceph-mon[31211]: pgmap v102: 1 pgs: 1 remapped+peering; 577 KiB data, 125 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:45.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:45 smithi096 ceph-mon[35004]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-19T06:35:45.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:45 smithi151 ceph-mon[35236]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-19T06:35:45.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:45 smithi037 ceph-mon[31211]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-19T06:35:47.601 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:47 smithi096 ceph-mon[35004]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:35:47.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:47 smithi151 ceph-mon[35236]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:35:47.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:47 smithi037 ceph-mon[31211]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-19T06:35:49.371 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:49 smithi096 ceph-mon[35004]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:35:49.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:35:49.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:49.372 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:49 smithi096 ceph-mon[35004]: Deploying daemon osd.4 on smithi096 2024-09-19T06:35:49.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:49 smithi151 ceph-mon[35236]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:35:49.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:35:49.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:49.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:49 smithi151 ceph-mon[35236]: Deploying daemon osd.4 on smithi096 2024-09-19T06:35:49.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:49 smithi037 ceph-mon[31211]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:35:49.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-19T06:35:49.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:49.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:49 smithi037 ceph-mon[31211]: Deploying daemon osd.4 on smithi096 2024-09-19T06:35:51.389 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:51 smithi096 ceph-mon[35004]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T06:35:51.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:51 smithi151 ceph-mon[35236]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T06:35:51.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:51 smithi037 ceph-mon[31211]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T06:35:53.577 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:53 smithi037 ceph-mon[31211]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T06:35:53.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:53 smithi151 ceph-mon[35236]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T06:35:53.623 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:53 smithi096 ceph-mon[35004]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-19T06:35:55.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:55 smithi096 ceph-mon[35004]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:35:55.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:55.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:55.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:55.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:55 smithi151 ceph-mon[35236]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:35:55.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:55.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:55.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:55.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:55 smithi037 ceph-mon[31211]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:35:55.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:55.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:55.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:57 smithi151 ceph-mon[35236]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:57.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:57 smithi037 ceph-mon[31211]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:57.777 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:57 smithi096 ceph-mon[35004]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:35:58.258 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 4 on host 'smithi096' 2024-09-19T06:35:58.258 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.256+0000 7fa9857fa640 1 -- 172.21.15.96:0/3495356564 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa988062630 con 0x7fa9700a3270 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa9700a3270 msgr2=0x7fa9700a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa9700a3270 0x7fa9700a5730 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fa978005fd0 tx=0x7fa97800b390 comp rx=0 tx=0).stop 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988109a70 msgr2=0x7fa9881a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988109a70 0x7fa9881a16e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa98000cad0 tx=0x7fa98000cf90 comp rx=0 tx=0).stop 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 shutdown_connections 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa9700a3270 0x7fa9700a5730 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa988109a70 0x7fa9881a16e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9880761a0 0x7fa98819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:58.260 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 --2- 172.21.15.96:0/3495356564 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa988074b40 0x7fa98819a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:35:58.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 >> 172.21.15.96:0/3495356564 conn(0x7fa9880fd390 msgr2=0x7fa988105a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:35:58.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 shutdown_connections 2024-09-19T06:35:58.261 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:35:58.258+0000 7fa98e532640 1 -- 172.21.15.96:0/3495356564 wait complete. 2024-09-19T06:35:58.849 DEBUG:teuthology.orchestra.run.smithi096:osd.4> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.4.service 2024-09-19T06:35:58.851 INFO:tasks.cephadm:Deploying osd.5 on smithi151 with /dev/vg_nvme/lv_3... 2024-09-19T06:35:58.851 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.090 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.108 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:35:59.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:59.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:59.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:59.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:35:59.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:35:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:35:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:35:59.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:00.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:35:59 smithi096 ceph-mon[35004]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:00.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:35:59 smithi151 ceph-mon[35236]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:00.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:35:59 smithi037 ceph-mon[31211]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:00.378 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:36:00 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:36:00.153+0000 7f7f14fa6640 -1 osd.4 0 log_to_monitors true 2024-09-19T06:36:01.091 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:00 smithi096 ceph-mon[35004]: from='osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:36:01.091 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:00 smithi096 ceph-mon[35004]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:36:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:00 smithi151 ceph-mon[35236]: from='osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:36:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:00 smithi151 ceph-mon[35236]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:36:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:00 smithi037 ceph-mon[31211]: from='osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:36:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:00 smithi037 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-19T06:36:01.609 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:36:01.635 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:01.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:01.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:01.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:01.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:01.893 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:36:02.024 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:36:01.841+0000 7f7f10f29640 -1 osd.4 0 waiting for initial osdmap 2024-09-19T06:36:02.024 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:36:01.848+0000 7f7f0cd51640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:36:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:02.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-19T06:36:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: osdmap e30: 5 total, 4 up, 5 in 2024-09-19T06:36:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:02.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:02.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 -- 172.21.15.151:0/3783872837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 msgr2=0x7f4484105c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:02.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 --2- 172.21.15.151:0/3783872837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f4484105c40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f447c009ca0 tx=0x7f447c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:36:02.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 -- 172.21.15.151:0/3783872837 shutdown_connections 2024-09-19T06:36:02.359 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 --2- 172.21.15.151:0/3783872837 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f4484105c40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 --2- 172.21.15.151:0/3783872837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4484102de0 0x7f4484105250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 --2- 172.21.15.151:0/3783872837 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4484100490 0x7f44841028a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 -- 172.21.15.151:0/3783872837 >> 172.21.15.151:0/3783872837 conn(0x7f44840fc2a0 msgr2=0x7f44840fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 -- 172.21.15.151:0/3783872837 shutdown_connections 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 -- 172.21.15.151:0/3783872837 wait complete. 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 Processor -- start 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.350+0000 7f448b444640 1 -- start start 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f448b444640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4484100490 0x7f448419e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f448b444640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4484102de0 0x7f448419e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f448b444640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f44841a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f448b444640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4484113ea0 con 0x7f4484100490 2024-09-19T06:36:02.360 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f448b444640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4484113d20 con 0x7f4484102de0 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f448b444640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4484114020 con 0x7f4484105790 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f44841a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f44841a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:34194/0 (socket says 172.21.15.151:34194) 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 -- 172.21.15.151:0/1267801819 learned_addr learned my addr 172.21.15.151:0/1267801819 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44889df640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4484102de0 0x7f448419e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44891e0640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4484100490 0x7f448419e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 -- 172.21.15.151:0/1267801819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4484102de0 msgr2=0x7f448419e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4484102de0 0x7f448419e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 -- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4484100490 msgr2=0x7f448419e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4484100490 0x7f448419e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 -- 172.21.15.151:0/1267801819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44841a60e0 con 0x7f4484105790 2024-09-19T06:36:02.361 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44891e0640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4484100490 0x7f448419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f44899e1640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f44841a5a70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f447c005e30 tx=0x7f447c031ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f447a7fc640 1 -- 172.21.15.151:0/1267801819 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c003c30 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44841a6370 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f447a7fc640 1 -- 172.21.15.151:0/1267801819 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f447c003dd0 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.351+0000 7f447a7fc640 1 -- 172.21.15.151:0/1267801819 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447c02fde0 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.352+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44841a68b0 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.353+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44840fdd90 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.354+0000 7f447a7fc640 1 -- 172.21.15.151:0/1267801819 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f447c004050 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.355+0000 7f447a7fc640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f44700a3270 0x7f44700a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.355+0000 7f447a7fc640 1 -- 172.21.15.151:0/1267801819 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f447c0ef9b0 con 0x7f4484105790 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.355+0000 7f44891e0640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f44700a3270 0x7f44700a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:02.362 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.355+0000 7f44891e0640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f44700a3270 0x7f44700a5730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44740058c0 tx=0x7f4474005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:36:02.363 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.359+0000 7f447a7fc640 1 -- 172.21.15.151:0/1267801819 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f447c0bd860 con 0x7f4484105790 2024-09-19T06:36:02.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:02.531+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f4484062630 con 0x7f44700a3270 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi096 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:03.104 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi096 2024-09-19T06:36:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-19T06:36:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:36:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T06:36:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi096 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi096 to 8544M 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: osdmap e31: 5 total, 4 up, 5 in 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:03.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:03.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:03.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:03.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:03.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:36:03.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:36:03.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:03.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:03.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:03.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] boot 2024-09-19T06:36:03.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T06:36:03.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:03 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:36:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:36:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] boot 2024-09-19T06:36:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:03 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: from='client.24229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: osd.4 [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] boot 2024-09-19T06:36:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: osdmap e32: 5 total, 5 up, 5 in 2024-09-19T06:36:04.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:03 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:36:05.118 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:04 smithi151 ceph-mon[35236]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T06:36:05.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:04 smithi096 ceph-mon[35004]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T06:36:05.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:04 smithi037 ceph-mon[31211]: osdmap e33: 5 total, 5 up, 5 in 2024-09-19T06:36:06.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:05 smithi096 ceph-mon[35004]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:06.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:05 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/1129405940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]: dispatch 2024-09-19T06:36:06.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:05 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]: dispatch 2024-09-19T06:36:06.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:05 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]': finished 2024-09-19T06:36:06.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:05 smithi096 ceph-mon[35004]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T06:36:06.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:05 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:06.176 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:05 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/1033982518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:05 smithi037 ceph-mon[31211]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:05 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/1129405940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]: dispatch 2024-09-19T06:36:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:05 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]: dispatch 2024-09-19T06:36:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:05 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]': finished 2024-09-19T06:36:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:05 smithi037 ceph-mon[31211]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T06:36:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:06.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:05 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/1033982518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:05 smithi151 ceph-mon[35236]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:05 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/1129405940' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]: dispatch 2024-09-19T06:36:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:05 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]: dispatch 2024-09-19T06:36:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:05 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9318d75e-c472-4c7e-b758-53c7f9cb12e9"}]': finished 2024-09-19T06:36:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:05 smithi151 ceph-mon[35236]: osdmap e34: 6 total, 5 up, 6 in 2024-09-19T06:36:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:05 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:05 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/1033982518' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:07 smithi096 ceph-mon[35004]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:07 smithi037 ceph-mon[31211]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:07 smithi151 ceph-mon[35236]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:09.994 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:09 smithi151 ceph-mon[35236]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:09 smithi096 ceph-mon[35004]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:09 smithi037 ceph-mon[31211]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:12.055 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:11 smithi151 ceph-mon[35236]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:11 smithi096 ceph-mon[35004]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:12.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:11 smithi037 ceph-mon[31211]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:14.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:13 smithi151 ceph-mon[35236]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:14.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:13 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:36:14.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:13 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:13 smithi096 ceph-mon[35004]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:13 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:36:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:13 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:13 smithi037 ceph-mon[31211]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:13 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-19T06:36:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:13 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:15.098 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:14 smithi151 ceph-mon[35236]: Deploying daemon osd.5 on smithi151 2024-09-19T06:36:15.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:14 smithi096 ceph-mon[35004]: Deploying daemon osd.5 on smithi151 2024-09-19T06:36:15.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:14 smithi037 ceph-mon[31211]: Deploying daemon osd.5 on smithi151 2024-09-19T06:36:16.142 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:15 smithi151 ceph-mon[35236]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:15 smithi096 ceph-mon[35004]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:15 smithi037 ceph-mon[31211]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:17 smithi096 ceph-mon[35004]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:18.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:17 smithi037 ceph-mon[31211]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:18.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:17 smithi151 ceph-mon[35236]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:20.148 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:19 smithi151 ceph-mon[35236]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:20.149 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:20.149 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:20.149 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:20.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:19 smithi096 ceph-mon[35004]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:20.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:20.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:20.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:20.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:19 smithi037 ceph-mon[31211]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:20.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:20.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:20.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:21 smithi151 ceph-mon[35236]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:21 smithi096 ceph-mon[35004]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:21 smithi037 ceph-mon[31211]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:23.341 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 5 on host 'smithi151' 2024-09-19T06:36:23.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.339+0000 7f447a7fc640 1 -- 172.21.15.151:0/1267801819 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4484062630 con 0x7f44700a3270 2024-09-19T06:36:23.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.341+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f44700a3270 msgr2=0x7f44700a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:23.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.341+0000 7f448b444640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f44700a3270 0x7f44700a5730 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f44740058c0 tx=0x7f4474005d30 comp rx=0 tx=0).stop 2024-09-19T06:36:23.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 msgr2=0x7f44841a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:23.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f44841a5a70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f447c005e30 tx=0x7f447c031ee0 comp rx=0 tx=0).stop 2024-09-19T06:36:23.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 shutdown_connections 2024-09-19T06:36:23.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f44700a3270 0x7f44700a5730 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:23.343 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4484105790 0x7f44841a5a70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:23.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4484102de0 0x7f448419e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:23.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 --2- 172.21.15.151:0/1267801819 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4484100490 0x7f448419e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:23.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 >> 172.21.15.151:0/1267801819 conn(0x7f44840fc2a0 msgr2=0x7f4484100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:36:23.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 shutdown_connections 2024-09-19T06:36:23.344 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:36:23.342+0000 7f448b444640 1 -- 172.21.15.151:0/1267801819 wait complete. 2024-09-19T06:36:23.919 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:23 smithi151 ceph-mon[35236]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:23.919 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:23 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:23.919 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:23 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:23.919 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:23 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:23.921 DEBUG:teuthology.orchestra.run.smithi151:osd.5> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.5.service 2024-09-19T06:36:23.924 INFO:tasks.cephadm:Deploying osd.6 on smithi037 with /dev/vg_nvme/lv_2... 2024-09-19T06:36:23.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:36:24.190 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:36:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:23 smithi096 ceph-mon[35004]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:23 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:23 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:23 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:24.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:23 smithi037 ceph-mon[31211]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:24.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:23 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:24.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:23 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:24.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:23 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:25.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:36:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:36:25.403+0000 7f85e96d7640 -1 osd.5 0 log_to_monitors true 2024-09-19T06:36:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:25 smithi096 ceph-mon[35004]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:25 smithi096 ceph-mon[35004]: from='osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:36:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:25 smithi096 ceph-mon[35004]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:36:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:25 smithi037 ceph-mon[31211]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:25 smithi037 ceph-mon[31211]: from='osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:36:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:25 smithi037 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:36:26.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:25 smithi151 ceph-mon[35236]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:26.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:25 smithi151 ceph-mon[35236]: from='osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:36:26.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:25 smithi151 ceph-mon[35236]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-19T06:36:27.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:36:27.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T06:36:27.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:27.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:36:27.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:36:27.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:27.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:27.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:27.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:27.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.042 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:36:27.068 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_2 2024-09-19T06:36:27.113 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:36:26.913+0000 7f85e565a640 -1 osd.5 0 waiting for initial osdmap 2024-09-19T06:36:27.113 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:36:26.919+0000 7f85e1482640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:36:27.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:27.115 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:27.115 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-19T06:36:27.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: osdmap e35: 6 total, 5 up, 6 in 2024-09-19T06:36:27.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:27.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:36:27.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:36:27.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:27.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:27.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:27.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:27.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:27.323 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:36:27.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 -- 172.21.15.37:0/3841893319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48105030 msgr2=0x7f6f48107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:27.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/3841893319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48105030 0x7f6f48107420 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6f3c009900 tx=0x7f6f3c02a410 comp rx=0 tx=0).stop 2024-09-19T06:36:27.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 -- 172.21.15.37:0/3841893319 shutdown_connections 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/3841893319 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f48107960 0x7f6f48109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/3841893319 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48105030 0x7f6f48107420 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/3841893319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f48102700 0x7f6f48104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 -- 172.21.15.37:0/3841893319 >> 172.21.15.37:0/3841893319 conn(0x7f6f480fc280 msgr2=0x7f6f480fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 -- 172.21.15.37:0/3841893319 shutdown_connections 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 -- 172.21.15.37:0/3841893319 wait complete. 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 Processor -- start 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.816+0000 7f6f4dffc640 1 -- start start 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f4dffc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f48102700 0x7f6f4819a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f4dffc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f48105030 0x7f6f4819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f4dffc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48107960 0x7f6f481a1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f4dffc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d830 con 0x7f6f48107960 2024-09-19T06:36:27.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f4dffc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d6b0 con 0x7f6f48102700 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f4dffc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6f4810d9b0 con 0x7f6f48105030 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48107960 0x7f6f481a1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48107960 0x7f6f481a1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49132/0 (socket says 172.21.15.37:49132) 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 -- 172.21.15.37:0/256665712 learned_addr learned my addr 172.21.15.37:0/256665712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f46ffd640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f48105030 0x7f6f4819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f477fe640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f48102700 0x7f6f4819a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 -- 172.21.15.37:0/256665712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f48105030 msgr2=0x7f6f4819a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f48105030 0x7f6f4819a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 -- 172.21.15.37:0/256665712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f48102700 msgr2=0x7f6f4819a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f48102700 0x7f6f4819a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 -- 172.21.15.37:0/256665712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f481a1dc0 con 0x7f6f48107960 2024-09-19T06:36:27.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f46ffd640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f48105030 0x7f6f4819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f477fe640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f48102700 0x7f6f4819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f47fff640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48107960 0x7f6f481a1750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000cad0 tx=0x7f6f4000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.817+0000 7f6f44ff9640 1 -- 172.21.15.37:0/256665712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000ae20 con 0x7f6f48107960 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.818+0000 7f6f44ff9640 1 -- 172.21.15.37:0/256665712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6f40005020 con 0x7f6f48107960 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.818+0000 7f6f44ff9640 1 -- 172.21.15.37:0/256665712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f4000f6f0 con 0x7f6f48107960 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.818+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f481a20b0 con 0x7f6f48107960 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.818+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f481a2570 con 0x7f6f48107960 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.819+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f48109450 con 0x7f6f48107960 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.820+0000 7f6f44ff9640 1 -- 172.21.15.37:0/256665712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6f400051d0 con 0x7f6f48107960 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.821+0000 7f6f44ff9640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6f300a32f0 0x7f6f300a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.821+0000 7f6f477fe640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6f300a32f0 0x7f6f300a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:27.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.821+0000 7f6f44ff9640 1 -- 172.21.15.37:0/256665712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7f6f400cedf0 con 0x7f6f48107960 2024-09-19T06:36:27.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.821+0000 7f6f477fe640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6f300a32f0 0x7f6f300a57b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009b40 tx=0x7f6f38005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:36:27.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:27.825+0000 7f6f44ff9640 1 -- 172.21.15.37:0/256665712 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6f40014030 con 0x7f6f48107960 2024-09-19T06:36:28.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:28.014+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6f48062630 con 0x7f6f300a32f0 2024-09-19T06:36:28.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:27 smithi037 ceph-mon[31211]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:28.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:27 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi151 2024-09-19T06:36:28.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:27 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-19T06:36:28.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:27 smithi037 ceph-mon[31211]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:36:28.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:27 smithi037 ceph-mon[31211]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T06:36:28.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:27 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:28.017 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:27 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:27 smithi096 ceph-mon[35004]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:27 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi151 2024-09-19T06:36:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:27 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-19T06:36:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:27 smithi096 ceph-mon[35004]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:36:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:27 smithi096 ceph-mon[35004]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T06:36:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:27 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:27 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:27 smithi151 ceph-mon[35236]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-09-19T06:36:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:27 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi151 2024-09-19T06:36:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:27 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi151 to 8544M 2024-09-19T06:36:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:27 smithi151 ceph-mon[35236]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:36:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:27 smithi151 ceph-mon[35236]: osdmap e36: 6 total, 5 up, 6 in 2024-09-19T06:36:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:27 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:27 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:29.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:36:29.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:36:29.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] boot 2024-09-19T06:36:29.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T06:36:29.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:29.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:29.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:29.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:29.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:28 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:36:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:36:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] boot 2024-09-19T06:36:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T06:36:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:28 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: osd.5 [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] boot 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: osdmap e37: 6 total, 6 up, 6 in 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:29.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:28 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:30.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:29 smithi037 ceph-mon[31211]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:30.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:29 smithi037 ceph-mon[31211]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T06:36:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:29 smithi096 ceph-mon[35004]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:29 smithi096 ceph-mon[35004]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T06:36:30.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:29 smithi151 ceph-mon[35236]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 168 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:30.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:29 smithi151 ceph-mon[35236]: osdmap e38: 6 total, 6 up, 6 in 2024-09-19T06:36:31.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:30 smithi037 ceph-mon[31211]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T06:36:31.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:30 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2739198471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b76fc6b5-0f24-447c-9541-f19b2dcf0066"}]: dispatch 2024-09-19T06:36:31.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:30 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2739198471' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b76fc6b5-0f24-447c-9541-f19b2dcf0066"}]': finished 2024-09-19T06:36:31.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:30 smithi037 ceph-mon[31211]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T06:36:31.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:30 smithi096 ceph-mon[35004]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T06:36:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:30 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2739198471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b76fc6b5-0f24-447c-9541-f19b2dcf0066"}]: dispatch 2024-09-19T06:36:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:30 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2739198471' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b76fc6b5-0f24-447c-9541-f19b2dcf0066"}]': finished 2024-09-19T06:36:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:30 smithi096 ceph-mon[35004]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T06:36:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:31.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:30 smithi151 ceph-mon[35236]: osdmap e39: 6 total, 6 up, 6 in 2024-09-19T06:36:31.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:30 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2739198471' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b76fc6b5-0f24-447c-9541-f19b2dcf0066"}]: dispatch 2024-09-19T06:36:31.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:30 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2739198471' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b76fc6b5-0f24-447c-9541-f19b2dcf0066"}]': finished 2024-09-19T06:36:31.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:30 smithi151 ceph-mon[35236]: osdmap e40: 7 total, 6 up, 7 in 2024-09-19T06:36:31.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:31 smithi096 ceph-mon[35004]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:31 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3450920650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:31 smithi096 ceph-mon[35004]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T06:36:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:32.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:31 smithi037 ceph-mon[31211]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:32.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:31 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3450920650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:32.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:31 smithi037 ceph-mon[31211]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T06:36:32.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:32.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:31 smithi151 ceph-mon[35236]: pgmap v136: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:32.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:31 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3450920650' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:32.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:31 smithi151 ceph-mon[35236]: osdmap e41: 7 total, 6 up, 7 in 2024-09-19T06:36:32.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:33 smithi096 ceph-mon[35004]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:34.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:33 smithi037 ceph-mon[31211]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:34.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:33 smithi151 ceph-mon[35236]: pgmap v139: 1 pgs: 1 remapped+peering; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:35 smithi096 ceph-mon[35004]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-19T06:36:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:35 smithi037 ceph-mon[31211]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-19T06:36:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:35 smithi151 ceph-mon[35236]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 110 KiB/s, 0 objects/s recovering 2024-09-19T06:36:38.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:37 smithi037 ceph-mon[31211]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T06:36:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:37 smithi096 ceph-mon[35004]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T06:36:38.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:37 smithi151 ceph-mon[35236]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 93 KiB/s, 0 objects/s recovering 2024-09-19T06:36:40.272 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:39 smithi037 ceph-mon[31211]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:36:40.273 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:36:40.273 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:40.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:39 smithi096 ceph-mon[35004]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:36:40.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:36:40.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:39 smithi151 ceph-mon[35236]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:36:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-19T06:36:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:41.148 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:40 smithi037 ceph-mon[31211]: Deploying daemon osd.6 on smithi037 2024-09-19T06:36:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:40 smithi096 ceph-mon[35004]: Deploying daemon osd.6 on smithi037 2024-09-19T06:36:41.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:40 smithi151 ceph-mon[35236]: Deploying daemon osd.6 on smithi037 2024-09-19T06:36:42.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:41 smithi037 ceph-mon[31211]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T06:36:42.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:41 smithi096 ceph-mon[35004]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T06:36:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:41 smithi151 ceph-mon[35236]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-19T06:36:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:43 smithi096 ceph-mon[35004]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T06:36:44.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:43 smithi037 ceph-mon[31211]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T06:36:44.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:43 smithi151 ceph-mon[35236]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 54 KiB/s, 0 objects/s recovering 2024-09-19T06:36:46.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:45 smithi037 ceph-mon[31211]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:36:46.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:45 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:46.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:45 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:46.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:45 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:45 smithi096 ceph-mon[35004]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:36:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:45 smithi151 ceph-mon[35236]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:36:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:45 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:45 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:45 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:47 smithi096 ceph-mon[35004]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:47 smithi037 ceph-mon[31211]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:48.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:48.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:48.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:47 smithi151 ceph-mon[35236]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:48.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:48.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:48.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:48.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.860+0000 7f6f44ff9640 1 -- 172.21.15.37:0/256665712 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6f48062630 con 0x7f6f300a32f0 2024-09-19T06:36:48.861 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 6 on host 'smithi037' 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6f300a32f0 msgr2=0x7f6f300a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6f300a32f0 0x7f6f300a57b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f38009b40 tx=0x7f6f38005e50 comp rx=0 tx=0).stop 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48107960 msgr2=0x7f6f481a1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48107960 0x7f6f481a1750 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6f4000cad0 tx=0x7f6f4000cf90 comp rx=0 tx=0).stop 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 shutdown_connections 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6f300a32f0 0x7f6f300a57b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6f48107960 0x7f6f481a1750 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6f48105030 0x7f6f4819a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 --2- 172.21.15.37:0/256665712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f48102700 0x7f6f4819a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 >> 172.21.15.37:0/256665712 conn(0x7f6f480fc280 msgr2=0x7f6f48101e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 shutdown_connections 2024-09-19T06:36:48.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:36:48.863+0000 7f6f4dffc640 1 -- 172.21.15.37:0/256665712 wait complete. 2024-09-19T06:36:49.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:49.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:49.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:48 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:48 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:49.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:36:49.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:48 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:49.580 DEBUG:teuthology.orchestra.run.smithi037:osd.6> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.6.service 2024-09-19T06:36:49.582 INFO:tasks.cephadm:Deploying osd.7 on smithi096 with /dev/vg_nvme/lv_2... 2024-09-19T06:36:49.583 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:36:49.834 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:36:50.266 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:49 smithi096 ceph-mon[35004]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:50.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:49 smithi037 ceph-mon[31211]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:49 smithi151 ceph-mon[35236]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:51.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:50 smithi037 ceph-mon[31211]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:36:51.025 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:36:50 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:36:50.772+0000 7fa7ff652640 -1 osd.6 0 log_to_monitors true 2024-09-19T06:36:51.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:50 smithi096 ceph-mon[35004]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:36:51.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:50 smithi151 ceph-mon[35236]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-19T06:36:52.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:51 smithi037 ceph-mon[31211]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:52.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:51 smithi037 ceph-mon[31211]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:36:52.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:51 smithi037 ceph-mon[31211]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T06:36:52.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:51 smithi037 ceph-mon[31211]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:36:52.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:51 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:51 smithi096 ceph-mon[35004]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:51 smithi096 ceph-mon[35004]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:36:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:51 smithi096 ceph-mon[35004]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T06:36:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:51 smithi096 ceph-mon[35004]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:36:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:51 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:51 smithi151 ceph-mon[35236]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:52.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:51 smithi151 ceph-mon[35236]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-19T06:36:52.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:51 smithi151 ceph-mon[35236]: osdmap e42: 7 total, 6 up, 7 in 2024-09-19T06:36:52.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:51 smithi151 ceph-mon[35236]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:36:52.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:51 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:52.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:36:52.414+0000 7fa7fb5d5640 -1 osd.6 0 waiting for initial osdmap 2024-09-19T06:36:52.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:36:52.422+0000 7fa7f33f5640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:36:52.632 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:36:52.659 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-09-19T06:36:52.912 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:36:53.271 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:53.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:52 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:52 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: osdmap e43: 7 total, 6 up, 7 in 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148]' entity='osd.6' 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:36:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:36:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:52 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:36:53.382 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 -- 172.21.15.96:0/3096273777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc102eb0 msgr2=0x7f3ddc103330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 --2- 172.21.15.96:0/3096273777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc102eb0 0x7f3ddc103330 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc009900 tx=0x7f3dcc02a410 comp rx=0 tx=0).stop 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 -- 172.21.15.96:0/3096273777 shutdown_connections 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 --2- 172.21.15.96:0/3096273777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ddc103870 0x7f3ddc10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 --2- 172.21.15.96:0/3096273777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc102eb0 0x7f3ddc103330 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 --2- 172.21.15.96:0/3096273777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ddc0ffed0 0x7f3ddc1002d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 -- 172.21.15.96:0/3096273777 >> 172.21.15.96:0/3096273777 conn(0x7f3ddc0fba40 msgr2=0x7f3ddc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 -- 172.21.15.96:0/3096273777 shutdown_connections 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 -- 172.21.15.96:0/3096273777 wait complete. 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 Processor -- start 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.372+0000 7f3de4528640 1 -- start start 2024-09-19T06:36:53.383 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ddc0ffed0 0x7f3ddc197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ddc102eb0 0x7f3ddc198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc103870 0x7f3ddc19f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc10d960 con 0x7f3ddc0ffed0 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc10d7e0 con 0x7f3ddc103870 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3ddc10dae0 con 0x7f3ddc102eb0 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc103870 0x7f3ddc19f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc103870 0x7f3ddc19f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:49808/0 (socket says 172.21.15.96:49808) 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 -- 172.21.15.96:0/3403504327 learned_addr learned my addr 172.21.15.96:0/3403504327 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de22c4640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ddc0ffed0 0x7f3ddc197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de1ac3640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ddc102eb0 0x7f3ddc198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:53.384 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 -- 172.21.15.96:0/3403504327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ddc102eb0 msgr2=0x7f3ddc198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ddc102eb0 0x7f3ddc198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 -- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ddc0ffed0 msgr2=0x7f3ddc197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ddc0ffed0 0x7f3ddc197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 -- 172.21.15.96:0/3403504327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ddc19fb60 con 0x7f3ddc103870 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de1ac3640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ddc102eb0 0x7f3ddc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de2ac5640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc103870 0x7f3ddc19f4f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400b970 tx=0x7f3dd400be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3dd37fe640 1 -- 172.21.15.96:0/3403504327 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4019070 con 0x7f3ddc103870 2024-09-19T06:36:53.385 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3dd37fe640 1 -- 172.21.15.96:0/3403504327 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3dd400cb40 con 0x7f3ddc103870 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ddc19fe50 con 0x7f3ddc103870 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3dd37fe640 1 -- 172.21.15.96:0/3403504327 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dd4007650 con 0x7f3ddc103870 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.373+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ddc1a0330 con 0x7f3ddc103870 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.374+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ddc108fe0 con 0x7f3ddc103870 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.375+0000 7f3dd37fe640 1 -- 172.21.15.96:0/3403504327 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3dd401c070 con 0x7f3ddc103870 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.376+0000 7f3dd37fe640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3dc80a3270 0x7f3dc80a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.376+0000 7f3dd37fe640 1 -- 172.21.15.96:0/3403504327 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7f3dd40ce070 con 0x7f3ddc103870 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.376+0000 7f3de22c4640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3dc80a3270 0x7f3dc80a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.377+0000 7f3de22c4640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3dc80a3270 0x7f3dc80a5730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc06a590 tx=0x7f3dd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:36:53.386 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.380+0000 7f3dd37fe640 1 -- 172.21.15.96:0/3403504327 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3dd409b570 con 0x7f3ddc103870 2024-09-19T06:36:53.558 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:36:53.557+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3ddc062630 con 0x7f3dc80a3270 2024-09-19T06:36:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:36:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:36:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi037 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi037 to 5697M 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] boot 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:54.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi037 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi037 to 5697M 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] boot 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:54.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:54.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 169 MiB used, 536 GiB / 536 GiB avail 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi037 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi037 to 5697M 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: osd.6 [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] boot 2024-09-19T06:36:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: osdmap e44: 7 total, 7 up, 7 in 2024-09-19T06:36:54.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:36:54.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:36:54.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:36:54.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:36:55.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:55 smithi096 ceph-mon[35004]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:55.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:55 smithi096 ceph-mon[35004]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T06:36:55.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:55 smithi037 ceph-mon[31211]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:55.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:55 smithi037 ceph-mon[31211]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T06:36:55.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:55 smithi151 ceph-mon[35236]: from='client.24275 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:36:55.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:55 smithi151 ceph-mon[35236]: osdmap e45: 7 total, 7 up, 7 in 2024-09-19T06:36:56.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:56 smithi096 ceph-mon[35004]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:36:56.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:56 smithi037 ceph-mon[31211]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:36:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:56 smithi151 ceph-mon[35236]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:36:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:57 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/1943948693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]: dispatch 2024-09-19T06:36:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:57 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]: dispatch 2024-09-19T06:36:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:57 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]': finished 2024-09-19T06:36:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:57 smithi096 ceph-mon[35004]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T06:36:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:36:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:57 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/3232584036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:57 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/1943948693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]: dispatch 2024-09-19T06:36:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:57 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]: dispatch 2024-09-19T06:36:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:57 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]': finished 2024-09-19T06:36:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:57 smithi037 ceph-mon[31211]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T06:36:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:36:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:57 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/3232584036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:57 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/1943948693' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]: dispatch 2024-09-19T06:36:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:57 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]: dispatch 2024-09-19T06:36:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:57 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57af0c79-adbc-4e9c-a848-4586e582c9b6"}]': finished 2024-09-19T06:36:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:57 smithi151 ceph-mon[35236]: osdmap e46: 8 total, 7 up, 8 in 2024-09-19T06:36:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:36:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:57 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/3232584036' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:36:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:36:58 smithi151 ceph-mon[35236]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:36:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:36:58 smithi096 ceph-mon[35004]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:36:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:36:58 smithi037 ceph-mon[31211]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:00.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:00 smithi151 ceph-mon[35236]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:00 smithi096 ceph-mon[35004]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:00 smithi037 ceph-mon[31211]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:02 smithi096 ceph-mon[35004]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:02 smithi151 ceph-mon[35236]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:02.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:02 smithi037 ceph-mon[31211]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:04 smithi096 ceph-mon[35004]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:04 smithi151 ceph-mon[35236]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:04 smithi037 ceph-mon[31211]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:05.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:37:05.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:05.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:05 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:37:05.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:05 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:05.382 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:05 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-19T06:37:05.382 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:05 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:06.170 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:06 smithi096 ceph-mon[35004]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:06.170 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:06 smithi096 ceph-mon[35004]: Deploying daemon osd.7 on smithi096 2024-09-19T06:37:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:06 smithi151 ceph-mon[35236]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:06 smithi151 ceph-mon[35236]: Deploying daemon osd.7 on smithi096 2024-09-19T06:37:06.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:06 smithi037 ceph-mon[31211]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:06.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:06 smithi037 ceph-mon[31211]: Deploying daemon osd.7 on smithi096 2024-09-19T06:37:08.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:08 smithi096 ceph-mon[35004]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:08 smithi151 ceph-mon[35236]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:08.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:08 smithi037 ceph-mon[31211]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:10.254 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:10 smithi096 ceph-mon[35004]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:10 smithi151 ceph-mon[35236]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:10.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:10 smithi037 ceph-mon[31211]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:11.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:11 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:11.408 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:11.409 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:11.409 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:11 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:11.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:11.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:11.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:11 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:12.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:12 smithi151 ceph-mon[35236]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:12.428 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:12 smithi096 ceph-mon[35004]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:12.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:12 smithi037 ceph-mon[31211]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:14 smithi096 ceph-mon[35004]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:14 smithi151 ceph-mon[35236]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:14 smithi037 ceph-mon[31211]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:14.708 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.706+0000 7f3dd37fe640 1 -- 172.21.15.96:0/3403504327 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3ddc062630 con 0x7f3dc80a3270 2024-09-19T06:37:14.709 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 7 on host 'smithi096' 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3dc80a3270 msgr2=0x7f3dc80a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3dc80a3270 0x7f3dc80a5730 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc06a590 tx=0x7f3dd8005f90 comp rx=0 tx=0).stop 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc103870 msgr2=0x7f3ddc19f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc103870 0x7f3ddc19f4f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3dd400b970 tx=0x7f3dd400be30 comp rx=0 tx=0).stop 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 shutdown_connections 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3dc80a3270 0x7f3dc80a5730 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ddc103870 0x7f3ddc19f4f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3ddc102eb0 0x7f3ddc198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 --2- 172.21.15.96:0/3403504327 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3ddc0ffed0 0x7f3ddc197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:14.710 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.708+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 >> 172.21.15.96:0/3403504327 conn(0x7f3ddc0fba40 msgr2=0x7f3ddc1061b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:37:14.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.709+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 shutdown_connections 2024-09-19T06:37:14.711 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:37:14.709+0000 7f3de4528640 1 -- 172.21.15.96:0/3403504327 wait complete. 2024-09-19T06:37:15.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:15.389 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:15.389 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:15.389 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:15.415 DEBUG:teuthology.orchestra.run.smithi096:osd.7> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.7.service 2024-09-19T06:37:15.417 INFO:tasks.cephadm:Deploying osd.8 on smithi151 with /dev/vg_nvme/lv_2... 2024-09-19T06:37:15.417 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-19T06:37:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:15.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:15.673 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:37:16.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:16 smithi151 ceph-mon[35236]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:16 smithi096 ceph-mon[35004]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:16.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:16 smithi037 ceph-mon[31211]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:16.829 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:37:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:37:16.618+0000 7fa6a31b0640 -1 osd.7 0 log_to_monitors true 2024-09-19T06:37:17.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:17 smithi151 ceph-mon[35236]: from='osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:37:17.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:17 smithi151 ceph-mon[35236]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:37:17.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:17 smithi096 ceph-mon[35004]: from='osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:37:17.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:17 smithi096 ceph-mon[35004]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:37:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:17 smithi037 ceph-mon[31211]: from='osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:37:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:17 smithi037 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-19T06:37:18.217 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:37:18.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:18 smithi151 ceph-mon[35236]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:18.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:18 smithi151 ceph-mon[35236]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:37:18.241 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:18 smithi151 ceph-mon[35236]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T06:37:18.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:18 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:18.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:18 smithi151 ceph-mon[35236]: from='osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:37:18.242 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:18 smithi151 ceph-mon[35236]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:37:18.243 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-09-19T06:37:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-mon[35004]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-mon[35004]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:37:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-mon[35004]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T06:37:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-mon[35004]: from='osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:37:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-mon[35004]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:37:18.275 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:37:18.119+0000 7fa69f133640 -1 osd.7 0 waiting for initial osdmap 2024-09-19T06:37:18.275 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:37:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:37:18.131+0000 7fa69af5b640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:37:18.499 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:37:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:18 smithi037 ceph-mon[31211]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:18 smithi037 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-19T06:37:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:18 smithi037 ceph-mon[31211]: osdmap e47: 8 total, 7 up, 8 in 2024-09-19T06:37:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:18 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:18 smithi037 ceph-mon[31211]: from='osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:37:18.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:18 smithi037 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 -- 172.21.15.151:0/1532838553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30100ad0 msgr2=0x7f5b30100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/1532838553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30100ad0 0x7f5b30100ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009900 tx=0x7f5b1802a410 comp rx=0 tx=0).stop 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 -- 172.21.15.151:0/1532838553 shutdown_connections 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/1532838553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b30102690 0x7f5b30108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/1532838553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b30101cd0 0x7f5b30102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/1532838553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30100ad0 0x7f5b30100ed0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 -- 172.21.15.151:0/1532838553 >> 172.21.15.151:0/1532838553 conn(0x7f5b300fc280 msgr2=0x7f5b300fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:37:18.985 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 -- 172.21.15.151:0/1532838553 shutdown_connections 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.961+0000 7f5b34cfa640 1 -- 172.21.15.151:0/1532838553 wait complete. 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 Processor -- start 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 -- start start 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b30100ad0 0x7f5b30196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b30101cd0 0x7f5b30197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30102690 0x7f5b3019e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010c760 con 0x7f5b30100ad0 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010c5e0 con 0x7f5b30101cd0 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b34cfa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5b3010c8e0 con 0x7f5b30102690 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b2ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30102690 0x7f5b3019e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b2ed9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30102690 0x7f5b3019e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52222/0 (socket says 172.21.15.151:52222) 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b2ed9d640 1 -- 172.21.15.151:0/336038881 learned_addr learned my addr 172.21.15.151:0/336038881 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:37:18.986 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.962+0000 7f5b2e59c640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b30100ad0 0x7f5b30196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b2dd9b640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b30101cd0 0x7f5b30197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b2ed9d640 1 -- 172.21.15.151:0/336038881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b30101cd0 msgr2=0x7f5b30197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b2ed9d640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b30101cd0 0x7f5b30197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b2ed9d640 1 -- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b30100ad0 msgr2=0x7f5b30196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b2ed9d640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b30100ad0 0x7f5b30196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b2ed9d640 1 -- 172.21.15.151:0/336038881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b3019ea20 con 0x7f5b30102690 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b2ed9d640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30102690 0x7f5b3019e3b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800eb10 tx=0x7f5b2800c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b277fe640 1 -- 172.21.15.151:0/336038881 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28004b20 con 0x7f5b30102690 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b3019ed10 con 0x7f5b30102690 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b277fe640 1 -- 172.21.15.151:0/336038881 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5b28004cc0 con 0x7f5b30102690 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b277fe640 1 -- 172.21.15.151:0/336038881 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28010760 con 0x7f5b30102690 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.963+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b3019f1f0 con 0x7f5b30102690 2024-09-19T06:37:18.987 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.964+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b30107990 con 0x7f5b30102690 2024-09-19T06:37:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.966+0000 7f5b277fe640 1 -- 172.21.15.151:0/336038881 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5b28005020 con 0x7f5b30102690 2024-09-19T06:37:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.967+0000 7f5b277fe640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5b1c0a3270 0x7f5b1c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.967+0000 7f5b277fe640 1 -- 172.21.15.151:0/336038881 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 5117+0+0 (secure 0 0 0) 0x7f5b280cf200 con 0x7f5b30102690 2024-09-19T06:37:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.967+0000 7f5b2e59c640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5b1c0a3270 0x7f5b1c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.968+0000 7f5b2e59c640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5b1c0a3270 0x7f5b1c0a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005ab0 tx=0x7f5b18000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:37:18.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:18.971+0000 7f5b277fe640 1 -- 172.21.15.151:0/336038881 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5b2809cba0 con 0x7f5b30102690 2024-09-19T06:37:19.158 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:19.156+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5b3006a900 con 0x7f5b1c0a3270 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.158 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:19.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:19.159 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:19 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:37:19.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T06:37:19.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:19.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:19.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.528 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:19 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:37:19.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: osdmap e48: 8 total, 7 up, 8 in 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:19.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:19 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi096 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] boot 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:20.400 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:37:20.401 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:37:20.401 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:20.401 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:20 smithi151 ceph-mon[35236]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T06:37:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:37:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:37:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi096 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] boot 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:20 smithi096 ceph-mon[35004]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T06:37:20.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi096 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi096 to 5696M 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: osd.7 [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] boot 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: osdmap e49: 8 total, 8 up, 8 in 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:20.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:20 smithi037 ceph-mon[31211]: osdmap e50: 8 total, 8 up, 8 in 2024-09-19T06:37:21.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:21 smithi151 ceph-mon[35236]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:37:21.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:21 smithi151 ceph-mon[35236]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T06:37:21.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:21 smithi096 ceph-mon[35004]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:37:21.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:21 smithi096 ceph-mon[35004]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T06:37:21.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:21 smithi037 ceph-mon[31211]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:37:21.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:21 smithi037 ceph-mon[31211]: osdmap e51: 8 total, 8 up, 8 in 2024-09-19T06:37:22.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:22 smithi151 ceph-mon[35236]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:22.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:22 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/351024599' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]: dispatch 2024-09-19T06:37:22.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:22 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]: dispatch 2024-09-19T06:37:22.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:22 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]': finished 2024-09-19T06:37:22.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:22 smithi151 ceph-mon[35236]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T06:37:22.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:22 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:22.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:22 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/447924186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:37:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:22 smithi096 ceph-mon[35004]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:22 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/351024599' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]: dispatch 2024-09-19T06:37:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:22 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]: dispatch 2024-09-19T06:37:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:22 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]': finished 2024-09-19T06:37:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:22 smithi096 ceph-mon[35004]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T06:37:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:22 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:22 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/447924186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:37:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:22 smithi037 ceph-mon[31211]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:22 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/351024599' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]: dispatch 2024-09-19T06:37:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:22 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]: dispatch 2024-09-19T06:37:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:22 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f560456d-ef7e-4b12-8bcf-be9bab3c3def"}]': finished 2024-09-19T06:37:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:22 smithi037 ceph-mon[31211]: osdmap e52: 9 total, 8 up, 9 in 2024-09-19T06:37:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:22 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:22 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/447924186' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:37:23.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:23 smithi096 ceph-mon[35004]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:23.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:23 smithi096 ceph-mon[35004]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T06:37:23.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:23 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:23.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:23 smithi037 ceph-mon[31211]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:23.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:23 smithi037 ceph-mon[31211]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T06:37:23.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:23 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:23.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:23 smithi151 ceph-mon[35236]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 227 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:23.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:23 smithi151 ceph-mon[35236]: osdmap e53: 9 total, 8 up, 9 in 2024-09-19T06:37:23.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:23 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:25.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:25 smithi096 ceph-mon[35004]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:25.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:25 smithi037 ceph-mon[31211]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:25.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:25 smithi151 ceph-mon[35236]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:27.712 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:27 smithi151 ceph-mon[35236]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:27.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:27 smithi096 ceph-mon[35004]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:27.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:27 smithi037 ceph-mon[31211]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:29.724 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:29 smithi151 ceph-mon[35236]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:29.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:29 smithi096 ceph-mon[35004]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:29.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:29 smithi037 ceph-mon[31211]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:30.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T06:37:30.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:30.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:30 smithi096 ceph-mon[35004]: Deploying daemon osd.8 on smithi151 2024-09-19T06:37:30.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T06:37:30.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:30.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:30 smithi037 ceph-mon[31211]: Deploying daemon osd.8 on smithi151 2024-09-19T06:37:30.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-19T06:37:30.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:30.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:30 smithi151 ceph-mon[35236]: Deploying daemon osd.8 on smithi151 2024-09-19T06:37:31.709 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:31 smithi151 ceph-mon[35236]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:31.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:31 smithi096 ceph-mon[35004]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:31.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:31 smithi037 ceph-mon[31211]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:33.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:33 smithi096 ceph-mon[35004]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:33.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:33 smithi037 ceph-mon[31211]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:33.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:33 smithi151 ceph-mon[35236]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:35.692 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:35 smithi151 ceph-mon[35236]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:35.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:35 smithi096 ceph-mon[35004]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:35.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:35 smithi037 ceph-mon[31211]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:36.687 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:36.688 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:36.688 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:36.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:36.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:36.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:36.778 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:36.778 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:36.778 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:37.780 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:37 smithi151 ceph-mon[35236]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:38.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:37 smithi096 ceph-mon[35004]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:38.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:37 smithi037 ceph-mon[31211]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:39.397 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.395+0000 7f5b277fe640 1 -- 172.21.15.151:0/336038881 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5b3006a900 con 0x7f5b1c0a3270 2024-09-19T06:37:39.397 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 8 on host 'smithi151' 2024-09-19T06:37:39.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5b1c0a3270 msgr2=0x7f5b1c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:39.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5b1c0a3270 0x7f5b1c0a5730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5b18005ab0 tx=0x7f5b18000bc0 comp rx=0 tx=0).stop 2024-09-19T06:37:39.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30102690 msgr2=0x7f5b3019e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:39.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30102690 0x7f5b3019e3b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800eb10 tx=0x7f5b2800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 shutdown_connections 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5b1c0a3270 0x7f5b1c0a5730 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b30102690 0x7f5b3019e3b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b30101cd0 0x7f5b30197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 --2- 172.21.15.151:0/336038881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5b30100ad0 0x7f5b30196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 >> 172.21.15.151:0/336038881 conn(0x7f5b300fc280 msgr2=0x7f5b301050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 shutdown_connections 2024-09-19T06:37:39.400 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:37:39.398+0000 7f5b34cfa640 1 -- 172.21.15.151:0/336038881 wait complete. 2024-09-19T06:37:39.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:39 smithi151 ceph-mon[35236]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:39.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:39 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:39 smithi096 ceph-mon[35004]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:39.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:39.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:39.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:39 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:39 smithi037 ceph-mon[31211]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:39.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:39.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:39.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:39.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:39 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:40.027 DEBUG:teuthology.orchestra.run.smithi151:osd.8> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.8.service 2024-09-19T06:37:40.029 INFO:tasks.cephadm:Deploying osd.9 on smithi037 with /dev/vg_nvme/lv_1... 2024-09-19T06:37:40.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:37:40.296 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:37:40.595 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:40.595 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:40.595 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:40.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:40.637 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:40.641 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:40.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:37:40.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:40.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:41.537 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:41 smithi151 ceph-mon[35236]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:41.713 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:41 smithi037 ceph-mon[31211]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:41.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:41 smithi096 ceph-mon[35004]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:42.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:37:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:37:42.104+0000 7f9b9561b640 -1 osd.8 0 log_to_monitors true 2024-09-19T06:37:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:42 smithi096 ceph-mon[35004]: from='osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:37:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:42 smithi096 ceph-mon[35004]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:37:42.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:42 smithi037 ceph-mon[31211]: from='osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:37:42.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:42 smithi037 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:37:42.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:42 smithi151 ceph-mon[35236]: from='osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:37:42.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:42 smithi151 ceph-mon[35236]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-19T06:37:43.213 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:37:43.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi037:vg_nvme/lv_1 2024-09-19T06:37:43.494 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:37:43.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:43.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T06:37:43.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T06:37:43.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi151 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:43.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T06:37:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T06:37:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi151 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:43.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:43.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-19T06:37:43.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: osdmap e54: 9 total, 8 up, 9 in 2024-09-19T06:37:43.800 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi151 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi151 to 5696M 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:37:43.801 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:37:43.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 -- 172.21.15.37:0/4216538060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 msgr2=0x7fcf28104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:43.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 --2- 172.21.15.37:0/4216538060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28104a90 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009900 tx=0x7fcf1802a410 comp rx=0 tx=0).stop 2024-09-19T06:37:43.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 -- 172.21.15.37:0/4216538060 shutdown_connections 2024-09-19T06:37:43.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 --2- 172.21.15.37:0/4216538060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf28107900 0x7fcf28109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:43.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 --2- 172.21.15.37:0/4216538060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf28104fd0 0x7fcf281073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:43.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 --2- 172.21.15.37:0/4216538060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28104a90 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:43.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 -- 172.21.15.37:0/4216538060 >> 172.21.15.37:0/4216538060 conn(0x7fcf280fc2a0 msgr2=0x7fcf280fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:37:43.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 -- 172.21.15.37:0/4216538060 shutdown_connections 2024-09-19T06:37:43.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.982+0000 7fcf2da22640 1 -- 172.21.15.37:0/4216538060 wait complete. 2024-09-19T06:37:43.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 Processor -- start 2024-09-19T06:37:43.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 -- start start 2024-09-19T06:37:44.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:44.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf28104fd0 0x7fcf281984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:44.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf28107900 0x7fcf2819f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:44.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2810d8d0 con 0x7fcf281026a0 2024-09-19T06:37:44.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2810d750 con 0x7fcf28107900 2024-09-19T06:37:44.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf2da22640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2810da50 con 0x7fcf28104fd0 2024-09-19T06:37:44.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf26ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:44.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.983+0000 7fcf26ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58188/0 (socket says 172.21.15.37:58188) 2024-09-19T06:37:44.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf26ffd640 1 -- 172.21.15.37:0/181855560 learned_addr learned my addr 172.21.15.37:0/181855560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:37:44.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf267fc640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf28104fd0 0x7fcf281984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:44.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf277fe640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf28107900 0x7fcf2819f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:44.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf26ffd640 1 -- 172.21.15.37:0/181855560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf28104fd0 msgr2=0x7fcf281984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:44.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf26ffd640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf28104fd0 0x7fcf281984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:44.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf26ffd640 1 -- 172.21.15.37:0/181855560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf28107900 msgr2=0x7fcf2819f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:37:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf26ffd640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf28107900 0x7fcf2819f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:37:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf26ffd640 1 -- 172.21.15.37:0/181855560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2819fbc0 con 0x7fcf281026a0 2024-09-19T06:37:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf26ffd640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28197f90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009230 tx=0x7fcf18004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:37:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf2ca20640 1 -- 172.21.15.37:0/181855560 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1803f030 con 0x7fcf281026a0 2024-09-19T06:37:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf2ca20640 1 -- 172.21.15.37:0/181855560 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf18003ce0 con 0x7fcf281026a0 2024-09-19T06:37:44.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2819fe50 con 0x7fcf281026a0 2024-09-19T06:37:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf2ca20640 1 -- 172.21.15.37:0/181855560 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf18035070 con 0x7fcf281026a0 2024-09-19T06:37:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.984+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf281a0250 con 0x7fcf281026a0 2024-09-19T06:37:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.985+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf280fdd90 con 0x7fcf281026a0 2024-09-19T06:37:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.987+0000 7fcf2ca20640 1 -- 172.21.15.37:0/181855560 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fcf18042030 con 0x7fcf281026a0 2024-09-19T06:37:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.987+0000 7fcf2ca20640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf140a3270 0x7fcf140a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:37:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.987+0000 7fcf2ca20640 1 -- 172.21.15.37:0/181855560 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(55..55 src has 1..55) ==== 5549+0+0 (secure 0 0 0) 0x7fcf180ef450 con 0x7fcf281026a0 2024-09-19T06:37:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.987+0000 7fcf267fc640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf140a3270 0x7fcf140a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:37:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.988+0000 7fcf267fc640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf140a3270 0x7fcf140a5730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf281994b0 tx=0x7fcf1000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:37:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:43.992+0000 7fcf2ca20640 1 -- 172.21.15.37:0/181855560 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf180bcc40 con 0x7fcf281026a0 2024-09-19T06:37:44.112 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:37:43.798+0000 7f9b9159e640 -1 osd.8 0 waiting for initial osdmap 2024-09-19T06:37:44.112 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:37:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:37:43.807+0000 7f9b893be640 -1 osd.8 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:37:44.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:37:44.177+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fcf28062630 con 0x7fcf140a3270 2024-09-19T06:37:44.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:37:44.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T06:37:44.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:44.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: from='osd.8 ' entity='osd.8' 2024-09-19T06:37:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:37:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:37:44.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:37:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T06:37:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:44.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: from='osd.8 ' entity='osd.8' 2024-09-19T06:37:44.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:37:44.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:37:44.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:44.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:37:44.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: osdmap e55: 9 total, 8 up, 9 in 2024-09-19T06:37:44.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:44.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:44.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: from='osd.8 ' entity='osd.8' 2024-09-19T06:37:44.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:37:44.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:37:44.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:45.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:37:45.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:37:45.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:45.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:37:45.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:45.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] boot 2024-09-19T06:37:45.756 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T06:37:45.757 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:45.757 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:45 smithi037 ceph-mon[31211]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] boot 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:45 smithi096 ceph-mon[35004]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: pgmap v189: 1 pgs: 1 peering; 577 KiB data, 228 MiB used, 715 GiB / 715 GiB avail 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: from='client.14418 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi037:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: osd.8 [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] boot 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: osdmap e56: 9 total, 9 up, 9 in 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:37:45.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:45 smithi151 ceph-mon[35236]: osdmap e57: 9 total, 9 up, 9 in 2024-09-19T06:37:47.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:46 smithi096 ceph-mon[35004]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T06:37:47.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:46 smithi037 ceph-mon[31211]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T06:37:47.050 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:46 smithi151 ceph-mon[35236]: osdmap e58: 9 total, 9 up, 9 in 2024-09-19T06:37:47.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:47 smithi096 ceph-mon[35004]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:37:47.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:47 smithi096 ceph-mon[35004]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T06:37:47.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:47 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2325524330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab6dfa58-1497-4cee-9145-64b049b10e93"}]: dispatch 2024-09-19T06:37:47.950 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:47 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2325524330' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab6dfa58-1497-4cee-9145-64b049b10e93"}]': finished 2024-09-19T06:37:47.951 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:47 smithi096 ceph-mon[35004]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T06:37:47.951 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:37:47.951 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:47 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1316126721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:37:48.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:47 smithi037 ceph-mon[31211]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:37:48.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:47 smithi037 ceph-mon[31211]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T06:37:48.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:47 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2325524330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab6dfa58-1497-4cee-9145-64b049b10e93"}]: dispatch 2024-09-19T06:37:48.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:47 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2325524330' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab6dfa58-1497-4cee-9145-64b049b10e93"}]': finished 2024-09-19T06:37:48.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:47 smithi037 ceph-mon[31211]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T06:37:48.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:37:48.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:47 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1316126721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:37:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:47 smithi151 ceph-mon[35236]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:37:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:47 smithi151 ceph-mon[35236]: osdmap e59: 9 total, 9 up, 9 in 2024-09-19T06:37:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:47 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2325524330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ab6dfa58-1497-4cee-9145-64b049b10e93"}]: dispatch 2024-09-19T06:37:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:47 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2325524330' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab6dfa58-1497-4cee-9145-64b049b10e93"}]': finished 2024-09-19T06:37:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:47 smithi151 ceph-mon[35236]: osdmap e60: 10 total, 9 up, 10 in 2024-09-19T06:37:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:37:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:47 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1316126721' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:37:50.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:49 smithi096 ceph-mon[35004]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:37:50.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:49 smithi037 ceph-mon[31211]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:37:50.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:49 smithi151 ceph-mon[35236]: pgmap v196: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:37:51.969 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:51 smithi037 ceph-mon[31211]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T06:37:52.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:51 smithi096 ceph-mon[35004]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T06:37:52.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:51 smithi151 ceph-mon[35236]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 105 KiB/s, 0 objects/s recovering 2024-09-19T06:37:53.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:53 smithi037 ceph-mon[31211]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T06:37:54.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:53 smithi096 ceph-mon[35004]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T06:37:54.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:53 smithi151 ceph-mon[35236]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-09-19T06:37:56.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:55 smithi096 ceph-mon[35004]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:37:56.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T06:37:56.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:55 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:56.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:55 smithi037 ceph-mon[31211]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:37:56.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T06:37:56.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:55 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:56.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:55 smithi151 ceph-mon[35236]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:37:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-19T06:37:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:55 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:37:57.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:56 smithi096 ceph-mon[35004]: Deploying daemon osd.9 on smithi037 2024-09-19T06:37:57.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:56 smithi037 ceph-mon[31211]: Deploying daemon osd.9 on smithi037 2024-09-19T06:37:57.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:56 smithi151 ceph-mon[35236]: Deploying daemon osd.9 on smithi037 2024-09-19T06:37:58.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:57 smithi096 ceph-mon[35004]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T06:37:58.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:57 smithi037 ceph-mon[31211]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T06:37:58.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:57 smithi151 ceph-mon[35236]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-19T06:38:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:37:59 smithi096 ceph-mon[35004]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T06:38:00.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:37:59 smithi037 ceph-mon[31211]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T06:38:00.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:37:59 smithi151 ceph-mon[35236]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-19T06:38:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:01 smithi096 ceph-mon[35004]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:38:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:01 smithi037 ceph-mon[31211]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:38:02.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:01 smithi151 ceph-mon[35236]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-19T06:38:02.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:02.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:02.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:02 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:03.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:03.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:03.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:02 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:03.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:03.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:03.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:02 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:04.095 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:03 smithi037 ceph-mon[31211]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:04.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:03 smithi151 ceph-mon[35236]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:03 smithi096 ceph-mon[35004]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:05.501 INFO:teuthology.orchestra.run.smithi037.stdout:Created osd(s) 9 on host 'smithi037' 2024-09-19T06:38:05.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.500+0000 7fcf2ca20640 1 -- 172.21.15.37:0/181855560 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcf28062630 con 0x7fcf140a3270 2024-09-19T06:38:05.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.502+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf140a3270 msgr2=0x7fcf140a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:05.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.502+0000 7fcf2da22640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf140a3270 0x7fcf140a5730 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcf281994b0 tx=0x7fcf1000b390 comp rx=0 tx=0).stop 2024-09-19T06:38:05.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.502+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 msgr2=0x7fcf28197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.502+0000 7fcf2da22640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28197f90 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fcf18009230 tx=0x7fcf18004360 comp rx=0 tx=0).stop 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 shutdown_connections 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf140a3270 0x7fcf140a5730 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf28107900 0x7fcf2819f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf28104fd0 0x7fcf281984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 --2- 172.21.15.37:0/181855560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf281026a0 0x7fcf28197f90 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 >> 172.21.15.37:0/181855560 conn(0x7fcf280fc2a0 msgr2=0x7fcf28102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:38:05.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 shutdown_connections 2024-09-19T06:38:05.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:05.503+0000 7fcf2da22640 1 -- 172.21.15.37:0/181855560 wait complete. 2024-09-19T06:38:06.076 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:05 smithi096 ceph-mon[35004]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:06.076 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:05 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:06.077 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:05 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:06.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:05 smithi151 ceph-mon[35236]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:06.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:05 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:06.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:05 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:06.150 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:05 smithi037 ceph-mon[31211]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:06.150 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:06.151 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:05 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:06.192 DEBUG:teuthology.orchestra.run.smithi037:osd.9> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.9.service 2024-09-19T06:38:06.194 INFO:tasks.cephadm:Deploying osd.10 on smithi096 with /dev/vg_nvme/lv_1... 2024-09-19T06:38:06.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:38:06.452 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:38:07.384 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:38:07 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:38:07.143+0000 7ff1066f0640 -1 osd.9 0 log_to_monitors true 2024-09-19T06:38:07.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:07 smithi096 ceph-mon[35004]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:07.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:07 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:07.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:07 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:07.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:07 smithi096 ceph-mon[35004]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T06:38:07.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:07 smithi037 ceph-mon[31211]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:07.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:07.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:07 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:07.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:07 smithi037 ceph-mon[31211]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T06:38:07.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:07 smithi151 ceph-mon[35236]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:07.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:07 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:07.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:07 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:07.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:07 smithi151 ceph-mon[35236]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-19T06:38:09.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:08 smithi096 ceph-mon[35004]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T06:38:09.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:08 smithi096 ceph-mon[35004]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T06:38:09.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:08 smithi096 ceph-mon[35004]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:38:09.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:09.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:08 smithi037 ceph-mon[31211]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T06:38:09.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:08 smithi037 ceph-mon[31211]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T06:38:09.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:08 smithi037 ceph-mon[31211]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:38:09.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:09.058 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-09-19T06:38:09.083 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-09-19T06:38:09.091 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:08 smithi151 ceph-mon[35236]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-19T06:38:09.092 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:08 smithi151 ceph-mon[35236]: osdmap e61: 10 total, 9 up, 10 in 2024-09-19T06:38:09.092 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:08 smithi151 ceph-mon[35236]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]: dispatch 2024-09-19T06:38:09.092 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:09.333 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:38:09.467 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:38:09.282+0000 7ff102673640 -1 osd.9 0 waiting for initial osdmap 2024-09-19T06:38:09.467 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:38:09.295+0000 7ff0fa493640 -1 osd.9 62 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:38:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 -- 172.21.15.96:0/1162372571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c101d10 msgr2=0x7fe15c102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 --2- 172.21.15.96:0/1162372571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c101d10 0x7fe15c102190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe148009900 tx=0x7fe14802a410 comp rx=0 tx=0).stop 2024-09-19T06:38:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 -- 172.21.15.96:0/1162372571 shutdown_connections 2024-09-19T06:38:09.984 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 --2- 172.21.15.96:0/1162372571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15c1026d0 0x7fe15c108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 --2- 172.21.15.96:0/1162372571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c101d10 0x7fe15c102190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 --2- 172.21.15.96:0/1162372571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15c100b10 0x7fe15c100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 -- 172.21.15.96:0/1162372571 >> 172.21.15.96:0/1162372571 conn(0x7fe15c0fc2a0 msgr2=0x7fe15c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 -- 172.21.15.96:0/1162372571 shutdown_connections 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.955+0000 7fe162d56640 1 -- 172.21.15.96:0/1162372571 wait complete. 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 Processor -- start 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 -- start start 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c100b10 0x7fe15c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15c101d10 0x7fe15c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15c1026d0 0x7fe15c19f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10d960 con 0x7fe15c101d10 2024-09-19T06:38:09.985 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10d7e0 con 0x7fe15c100b10 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe162d56640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe15c10dae0 con 0x7fe15c1026d0 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe160af2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c100b10 0x7fe15c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe160af2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c100b10 0x7fe15c197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:40098/0 (socket says 172.21.15.96:40098) 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe160af2640 1 -- 172.21.15.96:0/1619864473 learned_addr learned my addr 172.21.15.96:0/1619864473 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe15bfff640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15c101d10 0x7fe15c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.956+0000 7fe1612f3640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15c1026d0 0x7fe15c19f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe160af2640 1 -- 172.21.15.96:0/1619864473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15c1026d0 msgr2=0x7fe15c19f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe160af2640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15c1026d0 0x7fe15c19f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe160af2640 1 -- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15c101d10 msgr2=0x7fe15c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe160af2640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15c101d10 0x7fe15c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:09.986 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe160af2640 1 -- 172.21.15.96:0/1619864473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15c19fbd0 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe160af2640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c100b10 0x7fe15c197fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b450 tx=0x7fe15000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe159ffb640 1 -- 172.21.15.96:0/1619864473 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150015020 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe159ffb640 1 -- 172.21.15.96:0/1619864473 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe150011d70 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15c19fec0 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe159ffb640 1 -- 172.21.15.96:0/1619864473 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15000f340 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.957+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe15c1a0320 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.958+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15c107eb0 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.959+0000 7fe159ffb640 1 -- 172.21.15.96:0/1619864473 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe150018020 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.960+0000 7fe159ffb640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe14c0a32f0 0x7fe14c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.961+0000 7fe159ffb640 1 -- 172.21.15.96:0/1619864473 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5981+0+0 (secure 0 0 0) 0x7fe1500ceed0 con 0x7fe15c100b10 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.961+0000 7fe15bfff640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe14c0a32f0 0x7fe14c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.961+0000 7fe15bfff640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe14c0a32f0 0x7fe14c0a57b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe148031440 tx=0x7fe1480315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:38:09.987 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:09.964+0000 7fe159ffb640 1 -- 172.21.15.96:0/1619864473 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe15001f020 con 0x7fe15c100b10 2024-09-19T06:38:10.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:10.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:09 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:38:10.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:10.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:09 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi037", "root=default"]}]': finished 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: osdmap e62: 10 total, 9 up, 10 in 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849]' entity='osd.9' 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:10.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:10.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:10.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:09 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:10.146 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:10.144+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fe15c062630 con 0x7fe14c0a32f0 2024-09-19T06:38:11.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi037 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi037 to 4273M 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:38:11.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:11.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] boot 2024-09-19T06:38:11.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T06:38:11.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi037 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi037 to 4273M 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] boot 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:38:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi037 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi037 to 4273M 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: osd.9 [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] boot 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: osdmap e63: 10 total, 10 up, 10 in 2024-09-19T06:38:11.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:38:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:11 smithi096 ceph-mon[35004]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:38:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:11 smithi096 ceph-mon[35004]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:12.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:11 smithi037 ceph-mon[31211]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:38:12.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:11 smithi037 ceph-mon[31211]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:11 smithi151 ceph-mon[35236]: from='client.24347 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:38:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:11 smithi151 ceph-mon[35236]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-09-19T06:38:13.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:12 smithi096 ceph-mon[35004]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T06:38:13.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:12 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/894393065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]: dispatch 2024-09-19T06:38:13.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:12 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]: dispatch 2024-09-19T06:38:13.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:12 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]': finished 2024-09-19T06:38:13.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:12 smithi096 ceph-mon[35004]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:38:13.202 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:13.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:12 smithi037 ceph-mon[31211]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T06:38:13.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:12 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/894393065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]: dispatch 2024-09-19T06:38:13.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:12 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]: dispatch 2024-09-19T06:38:13.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:12 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]': finished 2024-09-19T06:38:13.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:12 smithi037 ceph-mon[31211]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:38:13.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:12 smithi151 ceph-mon[35236]: osdmap e64: 10 total, 10 up, 10 in 2024-09-19T06:38:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:12 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/894393065' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]: dispatch 2024-09-19T06:38:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:12 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]: dispatch 2024-09-19T06:38:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:12 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8817f0ca-2ce3-483e-a70c-68d96cacdbd6"}]': finished 2024-09-19T06:38:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:12 smithi151 ceph-mon[35236]: osdmap e65: 11 total, 10 up, 11 in 2024-09-19T06:38:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:13 smithi096 ceph-mon[35004]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:13 smithi096 ceph-mon[35004]: from='client.? 172.21.15.96:0/922819894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:38:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:13 smithi037 ceph-mon[31211]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:13 smithi037 ceph-mon[31211]: from='client.? 172.21.15.96:0/922819894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:38:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:13 smithi151 ceph-mon[35236]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:13 smithi151 ceph-mon[35236]: from='client.? 172.21.15.96:0/922819894' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:38:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:15 smithi096 ceph-mon[35004]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:15 smithi037 ceph-mon[31211]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:15 smithi151 ceph-mon[35236]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:18.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:17 smithi096 ceph-mon[35004]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:18.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:17 smithi037 ceph-mon[31211]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:18.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:17 smithi151 ceph-mon[35236]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:19.984 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:19 smithi096 ceph-mon[35004]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:20.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:19 smithi037 ceph-mon[31211]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:20.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:19 smithi151 ceph-mon[35236]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:21 smithi096 ceph-mon[35004]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:38:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:21 smithi037 ceph-mon[31211]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:38:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:22.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:21 smithi151 ceph-mon[35236]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:22.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-19T06:38:22.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:23.227 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:22 smithi096 ceph-mon[35004]: Deploying daemon osd.10 on smithi096 2024-09-19T06:38:23.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:22 smithi037 ceph-mon[31211]: Deploying daemon osd.10 on smithi096 2024-09-19T06:38:23.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:22 smithi151 ceph-mon[35236]: Deploying daemon osd.10 on smithi096 2024-09-19T06:38:24.231 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:23 smithi096 ceph-mon[35004]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:24.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:23 smithi037 ceph-mon[31211]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:24.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:23 smithi151 ceph-mon[35236]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:26.132 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:25 smithi096 ceph-mon[35004]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:25 smithi037 ceph-mon[31211]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:26.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:25 smithi151 ceph-mon[35236]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:28.243 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:27 smithi096 ceph-mon[35004]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:28.244 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:27 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:28.244 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:27 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:28.244 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:27 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:27 smithi037 ceph-mon[31211]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:27 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:27 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:27 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:28.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:27 smithi151 ceph-mon[35236]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:28.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:27 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:27 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:28.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:27 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:29 smithi096 ceph-mon[35004]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:29 smithi037 ceph-mon[31211]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:30.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:29 smithi151 ceph-mon[35236]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:30.674 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.672+0000 7fe159ffb640 1 -- 172.21.15.96:0/1619864473 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fe15c062630 con 0x7fe14c0a32f0 2024-09-19T06:38:30.674 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 10 on host 'smithi096' 2024-09-19T06:38:30.676 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe14c0a32f0 msgr2=0x7fe14c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe14c0a32f0 0x7fe14c0a57b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe148031440 tx=0x7fe1480315b0 comp rx=0 tx=0).stop 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c100b10 msgr2=0x7fe15c197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c100b10 0x7fe15c197fa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe15000b450 tx=0x7fe15000b910 comp rx=0 tx=0).stop 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 shutdown_connections 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe14c0a32f0 0x7fe14c0a57b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe15c1026d0 0x7fe15c19f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe15c101d10 0x7fe15c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 --2- 172.21.15.96:0/1619864473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe15c100b10 0x7fe15c197fa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 >> 172.21.15.96:0/1619864473 conn(0x7fe15c0fc2a0 msgr2=0x7fe15c105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 shutdown_connections 2024-09-19T06:38:30.677 INFO:teuthology.orchestra.run.smithi096.stderr:2024-09-19T06:38:30.675+0000 7fe162d56640 1 -- 172.21.15.96:0/1619864473 wait complete. 2024-09-19T06:38:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:30 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:31.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:30 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.329 DEBUG:teuthology.orchestra.run.smithi096:osd.10> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.10.service 2024-09-19T06:38:31.332 INFO:tasks.cephadm:Deploying osd.11 on smithi151 with /dev/vg_nvme/lv_1... 2024-09-19T06:38:31.332 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-19T06:38:31.360 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.360 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:31.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:31.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:31.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.361 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:30 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:31.589 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:38:32.251 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:31 smithi096 ceph-mon[35004]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:32.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:31 smithi037 ceph-mon[31211]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:32.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:31 smithi151 ceph-mon[35236]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:33.501 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:38:33 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:38:33.142+0000 7feaaa5c0640 -1 osd.10 0 log_to_monitors true 2024-09-19T06:38:34.152 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:33 smithi096 ceph-mon[35004]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:34.153 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:33 smithi096 ceph-mon[35004]: from='osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:38:34.153 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:33 smithi096 ceph-mon[35004]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:38:34.246 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:33 smithi151 ceph-mon[35236]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:34.247 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:33 smithi151 ceph-mon[35236]: from='osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:38:34.247 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:33 smithi151 ceph-mon[35236]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:38:34.249 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-09-19T06:38:34.274 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-09-19T06:38:34.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:33 smithi037 ceph-mon[31211]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:34.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:33 smithi037 ceph-mon[31211]: from='osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:38:34.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:33 smithi037 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-19T06:38:34.560 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:38:35.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 -- 172.21.15.151:0/2583856128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 msgr2=0x7fbce8102190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:35.025 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 --2- 172.21.15.151:0/2583856128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce8102190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009900 tx=0x7fbcd002a410 comp rx=0 tx=0).stop 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 -- 172.21.15.151:0/2583856128 shutdown_connections 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 --2- 172.21.15.151:0/2583856128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce81026d0 0x7fbce8108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 --2- 172.21.15.151:0/2583856128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce8102190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 --2- 172.21.15.151:0/2583856128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbce8100b10 0x7fbce8100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 -- 172.21.15.151:0/2583856128 >> 172.21.15.151:0/2583856128 conn(0x7fbce80fc2a0 msgr2=0x7fbce80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 -- 172.21.15.151:0/2583856128 shutdown_connections 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.011+0000 7fbcee764640 1 -- 172.21.15.151:0/2583856128 wait complete. 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 Processor -- start 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 -- start start 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8100b10 0x7fbce806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:35.026 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbce81026d0 0x7fbce819f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbce810c790 con 0x7fbce81026d0 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbce810c610 con 0x7fbce8100b10 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcee764640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbce810c910 con 0x7fbce8101d10 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce819cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50864/0 (socket says 172.21.15.151:50864) 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 -- 172.21.15.151:0/1841235148 learned_addr learned my addr 172.21.15.151:0/1841235148 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbce7fff640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8100b10 0x7fbce806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcecd01640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbce81026d0 0x7fbce819f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:35.027 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 -- 172.21.15.151:0/1841235148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8100b10 msgr2=0x7fbce806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8100b10 0x7fbce806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 -- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbce81026d0 msgr2=0x7fbce819f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbce81026d0 0x7fbce819f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 -- 172.21.15.151:0/1841235148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbce819fba0 con 0x7fbce8101d10 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.012+0000 7fbcdffff640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce819cc70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009230 tx=0x7fbcd0031b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.013+0000 7fbce5ffb640 1 -- 172.21.15.151:0/1841235148 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd003f030 con 0x7fbce8101d10 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.013+0000 7fbce5ffb640 1 -- 172.21.15.151:0/1841235148 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbcd0003ad0 con 0x7fbce8101d10 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.013+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbce819fdd0 con 0x7fbce8101d10 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.013+0000 7fbce5ffb640 1 -- 172.21.15.151:0/1841235148 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0035070 con 0x7fbce8101d10 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.013+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbce81a0270 con 0x7fbce8101d10 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.014+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbce8107f10 con 0x7fbce8101d10 2024-09-19T06:38:35.028 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.015+0000 7fbce5ffb640 1 -- 172.21.15.151:0/1841235148 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fbcd0042030 con 0x7fbce8101d10 2024-09-19T06:38:35.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.015+0000 7fbce5ffb640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbcd40a32c0 0x7fbcd40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:35.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.016+0000 7fbce5ffb640 1 -- 172.21.15.151:0/1841235148 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6414+0+0 (secure 0 0 0) 0x7fbcd00ef9c0 con 0x7fbce8101d10 2024-09-19T06:38:35.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.016+0000 7fbce7fff640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbcd40a32c0 0x7fbcd40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:35.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.016+0000 7fbce7fff640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbcd40a32c0 0x7fbcd40a5780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbce8101b70 tx=0x7fbcd8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:38:35.029 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.019+0000 7fbce5ffb640 1 -- 172.21.15.151:0/1841235148 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbcd00bc800 con 0x7fbce8101d10 2024-09-19T06:38:35.197 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:35.196+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbce8062630 con 0x7fbcd40a32c0 2024-09-19T06:38:35.274 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:38:35.001+0000 7feaa6543640 -1 osd.10 0 waiting for initial osdmap 2024-09-19T06:38:35.274 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:38:35.014+0000 7feaa236b640 -1 osd.10 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:35 smithi096 ceph-mon[35004]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:38:35.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.278 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:35 smithi037 ceph-mon[31211]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: osdmap e66: 11 total, 10 up, 11 in 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:38:35.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:35.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:38:35.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:35.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:35 smithi151 ceph-mon[35236]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-09-19T06:38:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi096 2024-09-19T06:38:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-19T06:38:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T06:38:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] boot 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:36 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi096 2024-09-19T06:38:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] boot 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:38:36.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:36 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.298 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi096 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi096 to 4272M 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: osdmap e67: 11 total, 10 up, 11 in 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: osd.10 [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] boot 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: osdmap e68: 11 total, 11 up, 11 in 2024-09-19T06:38:36.299 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:36 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:38:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:37 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:38:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:37 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:38:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:37 smithi096 ceph-mon[35004]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:38:37.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:37 smithi096 ceph-mon[35004]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T06:38:37.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:37 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:38:37.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:37 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:38:37.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:37 smithi037 ceph-mon[31211]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:38:37.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:37 smithi037 ceph-mon[31211]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T06:38:37.282 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:37 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:38:37.282 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:37 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:38:37.282 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:37 smithi151 ceph-mon[35236]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:38:37.282 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:37 smithi151 ceph-mon[35236]: osdmap e69: 11 total, 11 up, 11 in 2024-09-19T06:38:38.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:38 smithi151 ceph-mon[35236]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:38.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:38 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/817348688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]: dispatch 2024-09-19T06:38:38.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:38 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]: dispatch 2024-09-19T06:38:38.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:38 smithi151 ceph-mon[35236]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]': finished 2024-09-19T06:38:38.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:38 smithi151 ceph-mon[35236]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:38:38.273 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:38 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:38 smithi096 ceph-mon[35004]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:38 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/817348688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]: dispatch 2024-09-19T06:38:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:38 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]: dispatch 2024-09-19T06:38:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:38 smithi096 ceph-mon[35004]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]': finished 2024-09-19T06:38:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:38 smithi096 ceph-mon[35004]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:38:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:38 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:38 smithi037 ceph-mon[31211]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:38 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/817348688' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]: dispatch 2024-09-19T06:38:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:38 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]: dispatch 2024-09-19T06:38:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:38 smithi037 ceph-mon[31211]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d075a370-835b-443b-a582-a19ac7a79167"}]': finished 2024-09-19T06:38:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:38 smithi037 ceph-mon[31211]: osdmap e70: 12 total, 11 up, 12 in 2024-09-19T06:38:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:38 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:39.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:39 smithi096 ceph-mon[35004]: from='client.? 172.21.15.151:0/2674172219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:38:39.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:39 smithi037 ceph-mon[31211]: from='client.? 172.21.15.151:0/2674172219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:38:39.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:39 smithi151 ceph-mon[35236]: from='client.? 172.21.15.151:0/2674172219' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-19T06:38:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:40 smithi151 ceph-mon[35236]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:40 smithi151 ceph-mon[35236]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:38:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:40.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:40 smithi096 ceph-mon[35004]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:40.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:40 smithi096 ceph-mon[35004]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:38:40.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:40.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:40 smithi037 ceph-mon[31211]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:40.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:40 smithi037 ceph-mon[31211]: osdmap e71: 12 total, 11 up, 12 in 2024-09-19T06:38:40.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:41.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:41 smithi151 ceph-mon[35236]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:38:41.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:41 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:41.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:41 smithi096 ceph-mon[35004]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:38:41.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:41 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:41.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:41 smithi037 ceph-mon[31211]: osdmap e72: 12 total, 11 up, 12 in 2024-09-19T06:38:41.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:41 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:42 smithi151 ceph-mon[35236]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:38:42.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:42 smithi096 ceph-mon[35004]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:38:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:42 smithi037 ceph-mon[31211]: pgmap v234: 1 pgs: 1 peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-19T06:38:44.347 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:44 smithi151 ceph-mon[35236]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T06:38:44.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:44 smithi096 ceph-mon[35004]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T06:38:44.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:44 smithi037 ceph-mon[31211]: pgmap v235: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2024-09-19T06:38:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:46 smithi151 ceph-mon[35236]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-19T06:38:46.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:46 smithi096 ceph-mon[35004]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-19T06:38:46.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:46 smithi037 ceph-mon[31211]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 90 KiB/s, 0 objects/s recovering 2024-09-19T06:38:47.163 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:38:47.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:38:47.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:47.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-19T06:38:47.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:38:48.286 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:48 smithi151 ceph-mon[35236]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:38:48.286 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:48 smithi151 ceph-mon[35236]: Deploying daemon osd.11 on smithi151 2024-09-19T06:38:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:48 smithi096 ceph-mon[35004]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:38:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:48 smithi096 ceph-mon[35004]: Deploying daemon osd.11 on smithi151 2024-09-19T06:38:48.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:48 smithi037 ceph-mon[31211]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-19T06:38:48.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:48 smithi037 ceph-mon[31211]: Deploying daemon osd.11 on smithi151 2024-09-19T06:38:50.274 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:50 smithi151 ceph-mon[35236]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T06:38:50.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:50 smithi096 ceph-mon[35004]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T06:38:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:50 smithi037 ceph-mon[31211]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-19T06:38:52.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:52 smithi151 ceph-mon[35236]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:52.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:52 smithi096 ceph-mon[35004]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:52.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:52 smithi037 ceph-mon[31211]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:53.269 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:53.270 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:53.270 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:53 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:53.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:53.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:53.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:53 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:53.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:38:53.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:53.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:53 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:54.339 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:54 smithi151 ceph-mon[35236]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:54 smithi096 ceph-mon[35004]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:54.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:54 smithi037 ceph-mon[31211]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:55.834 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 11 on host 'smithi151' 2024-09-19T06:38:55.834 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.832+0000 7fbce5ffb640 1 -- 172.21.15.151:0/1841235148 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fbce8062630 con 0x7fbcd40a32c0 2024-09-19T06:38:55.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbcd40a32c0 msgr2=0x7fbcd40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:55.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbcd40a32c0 0x7fbcd40a5780 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbce8101b70 tx=0x7fbcd8005f90 comp rx=0 tx=0).stop 2024-09-19T06:38:55.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 msgr2=0x7fbce819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:55.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce819cc70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fbcd0009230 tx=0x7fbcd0031b60 comp rx=0 tx=0).stop 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 shutdown_connections 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbcd40a32c0 0x7fbcd40a5780 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbce81026d0 0x7fbce819f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbce8101d10 0x7fbce819cc70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 --2- 172.21.15.151:0/1841235148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbce8100b10 0x7fbce806b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 >> 172.21.15.151:0/1841235148 conn(0x7fbce80fc2a0 msgr2=0x7fbce81050e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 shutdown_connections 2024-09-19T06:38:55.837 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:38:55.835+0000 7fbcee764640 1 -- 172.21.15.151:0/1841235148 wait complete. 2024-09-19T06:38:56.327 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:56 smithi037 ceph-mon[31211]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:56.327 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:56.328 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:56 smithi151 ceph-mon[35236]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:56.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:56.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:56 smithi096 ceph-mon[35004]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:56.535 DEBUG:teuthology.orchestra.run.smithi151:osd.11> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.11.service 2024-09-19T06:38:56.538 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-19T06:38:56.539 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:38:56.808 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:38:57.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 -- 172.21.15.37:0/52193471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101cd0 msgr2=0x7f0778102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:57.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 --2- 172.21.15.37:0/52193471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101cd0 0x7f0778102150 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0760009900 tx=0x7f076002a410 comp rx=0 tx=0).stop 2024-09-19T06:38:57.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 -- 172.21.15.37:0/52193471 shutdown_connections 2024-09-19T06:38:57.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 --2- 172.21.15.37:0/52193471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778102690 0x7f0778108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 --2- 172.21.15.37:0/52193471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101cd0 0x7f0778102150 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 --2- 172.21.15.37:0/52193471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0778100ad0 0x7f0778100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 -- 172.21.15.37:0/52193471 >> 172.21.15.37:0/52193471 conn(0x7f07780fc280 msgr2=0x7f07780fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 -- 172.21.15.37:0/52193471 shutdown_connections 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.296+0000 7f077cd29640 1 -- 172.21.15.37:0/52193471 wait complete. 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 Processor -- start 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 -- start start 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778100ad0 0x7f077806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0778101cd0 0x7f077819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778102690 0x7f077819f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f077810c7a0 con 0x7f0778102690 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f077810c620 con 0x7f0778101cd0 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.297+0000 7f077cd29640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f077810c920 con 0x7f0778100ad0 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778102690 0x7f077819f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:57.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778102690 0x7f077819f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44290/0 (socket says 172.21.15.37:44290) 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 -- 172.21.15.37:0/3588411298 learned_addr learned my addr 172.21.15.37:0/3588411298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f077659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778100ad0 0x7f077806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f076ffff640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0778101cd0 0x7f077819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 -- 172.21.15.37:0/3588411298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778100ad0 msgr2=0x7f077806aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778100ad0 0x7f077806aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 -- 172.21.15.37:0/3588411298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0778101cd0 msgr2=0x7f077819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0778101cd0 0x7f077819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 -- 172.21.15.37:0/3588411298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077819fb60 con 0x7f0778102690 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f077659c640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778100ad0 0x7f077806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f0776d9d640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778102690 0x7f077819f250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f077000eb10 tx=0x7f077000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f076f7fe640 1 -- 172.21.15.37:0/3588411298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770004b20 con 0x7f0778102690 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077819fdf0 con 0x7f0778102690 2024-09-19T06:38:57.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f076f7fe640 1 -- 172.21.15.37:0/3588411298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0770004cc0 con 0x7f0778102690 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f076f7fe640 1 -- 172.21.15.37:0/3588411298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770010760 con 0x7f0778102690 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.298+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781a02f0 con 0x7f0778102690 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.300+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0778107e80 con 0x7f0778102690 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.302+0000 7f076f7fe640 1 -- 172.21.15.37:0/3588411298 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0770005000 con 0x7f0778102690 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.302+0000 7f076f7fe640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07640a3240 0x7f07640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.303+0000 7f077659c640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07640a3240 0x7f07640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.303+0000 7f076f7fe640 1 -- 172.21.15.37:0/3588411298 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 6809+0+0 (secure 0 0 0) 0x7f07700cfd80 con 0x7f0778102690 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.303+0000 7f077659c640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07640a3240 0x7f07640a5700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0778101b30 tx=0x7f076800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:38:57.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.305+0000 7f076f7fe640 1 -- 172.21.15.37:0/3588411298 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f077009d080 con 0x7f0778102690 2024-09-19T06:38:57.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.482+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f077802d990 con 0x7f0778102690 2024-09-19T06:38:57.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.482+0000 7f076f7fe640 1 -- 172.21.15.37:0/3588411298 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f077009c980 con 0x7f0778102690 2024-09-19T06:38:57.483 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:38:57.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.484+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07640a3240 msgr2=0x7f07640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:57.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.484+0000 7f077cd29640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07640a3240 0x7f07640a5700 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0778101b30 tx=0x7f076800b390 comp rx=0 tx=0).stop 2024-09-19T06:38:57.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.484+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778102690 msgr2=0x7f077819f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.484+0000 7f077cd29640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778102690 0x7f077819f250 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f077000eb10 tx=0x7f077000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.484+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 shutdown_connections 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.484+0000 7f077cd29640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07640a3240 0x7f07640a5700 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.485+0000 7f077cd29640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778102690 0x7f077819f250 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.485+0000 7f077cd29640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0778101cd0 0x7f077819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.485+0000 7f077cd29640 1 --2- 172.21.15.37:0/3588411298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778100ad0 0x7f077806aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.485+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 >> 172.21.15.37:0/3588411298 conn(0x7f07780fc280 msgr2=0x7f0778105050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.485+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 shutdown_connections 2024-09-19T06:38:57.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:38:57.485+0000 7f077cd29640 1 -- 172.21.15.37:0/3588411298 wait complete. 2024-09-19T06:38:57.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:57 smithi037 ceph-mon[31211]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:57.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:57.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:57.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:57 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3588411298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:38:58.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:57 smithi096 ceph-mon[35004]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:58.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:58.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:58.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:57 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3588411298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:38:58.067 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:57 smithi151 ceph-mon[35236]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:58.067 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:58.068 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:38:58.068 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:57 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3588411298' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:38:58.431 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1726727915,"num_in_osds":12,"osd_in_since":1726727917,"num_remapped_pgs":0} 2024-09-19T06:38:58.613 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:38:58 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:38:58.520+0000 7f8f093f2640 -1 osd.11 0 log_to_monitors true 2024-09-19T06:38:59.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:58 smithi096 ceph-mon[35004]: from='osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:38:59.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:58 smithi096 ceph-mon[35004]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:38:59.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:58 smithi037 ceph-mon[31211]: from='osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:38:59.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:58 smithi037 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:38:59.087 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:58 smithi151 ceph-mon[35236]: from='osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:38:59.087 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:58 smithi151 ceph-mon[35236]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-19T06:38:59.431 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:38:59.700 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:38:59.851 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:59 smithi151 ceph-mon[35236]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:59.851 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:59 smithi151 ceph-mon[35236]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:38:59.851 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:59 smithi151 ceph-mon[35236]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:38:59.851 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:59 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:59.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:59 smithi151 ceph-mon[35236]: from='osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:38:59.852 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:38:59 smithi151 ceph-mon[35236]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:38:59.976 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:59 smithi037 ceph-mon[31211]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:38:59.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:59 smithi037 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:38:59.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:59 smithi037 ceph-mon[31211]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:38:59.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:59 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:38:59.977 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:59 smithi037 ceph-mon[31211]: from='osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:38:59.978 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:38:59 smithi037 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:39:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:59 smithi096 ceph-mon[35004]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:39:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:59 smithi096 ceph-mon[35004]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-19T06:39:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:59 smithi096 ceph-mon[35004]: osdmap e73: 12 total, 11 up, 12 in 2024-09-19T06:39:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:59 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:59 smithi096 ceph-mon[35004]: from='osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:39:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:38:59 smithi096 ceph-mon[35004]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 -- 172.21.15.37:0/1051892326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c107900 msgr2=0x7efc6c109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 --2- 172.21.15.37:0/1051892326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c107900 0x7efc6c109d50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efc64009ca0 tx=0x7efc6402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 -- 172.21.15.37:0/1051892326 shutdown_connections 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 --2- 172.21.15.37:0/1051892326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c107900 0x7efc6c109d50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 --2- 172.21.15.37:0/1051892326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc6c104fd0 0x7efc6c1073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 --2- 172.21.15.37:0/1051892326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6c1026a0 0x7efc6c104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 -- 172.21.15.37:0/1051892326 >> 172.21.15.37:0/1051892326 conn(0x7efc6c0fc2a0 msgr2=0x7efc6c0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 -- 172.21.15.37:0/1051892326 shutdown_connections 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.199+0000 7efc71490640 1 -- 172.21.15.37:0/1051892326 wait complete. 2024-09-19T06:39:00.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 Processor -- start 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 -- start start 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1026a0 0x7efc6c197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6c104fd0 0x7efc6c198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc6c107900 0x7efc6c19f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c10d8a0 con 0x7efc6c1026a0 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c10d720 con 0x7efc6c107900 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc71490640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc6c10da20 con 0x7efc6c104fd0 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1026a0 0x7efc6c197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1026a0 0x7efc6c197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55686/0 (socket says 172.21.15.37:55686) 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc6affd640 1 -- 172.21.15.37:0/3510914501 learned_addr learned my addr 172.21.15.37:0/3510914501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc6a7fc640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6c104fd0 0x7efc6c198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:00.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.200+0000 7efc6b7fe640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc6c107900 0x7efc6c19f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc6affd640 1 -- 172.21.15.37:0/3510914501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6c104fd0 msgr2=0x7efc6c198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc6affd640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6c104fd0 0x7efc6c198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc6affd640 1 -- 172.21.15.37:0/3510914501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc6c107900 msgr2=0x7efc6c19f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc6affd640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc6c107900 0x7efc6c19f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc6affd640 1 -- 172.21.15.37:0/3510914501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc6c19fb00 con 0x7efc6c1026a0 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc6affd640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1026a0 0x7efc6c197ed0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00d950 tx=0x7efc5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc53fff640 1 -- 172.21.15.37:0/3510914501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c017070 con 0x7efc6c1026a0 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc6c19fdf0 con 0x7efc6c1026a0 2024-09-19T06:39:00.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc53fff640 1 -- 172.21.15.37:0/3510914501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc5c009c30 con 0x7efc6c1026a0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc53fff640 1 -- 172.21.15.37:0/3510914501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc5c00f6b0 con 0x7efc6c1026a0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.201+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc6c1a02a0 con 0x7efc6c1026a0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.202+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc6c0fdd90 con 0x7efc6c1026a0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.203+0000 7efc53fff640 1 -- 172.21.15.37:0/3510914501 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efc5c0109d0 con 0x7efc6c1026a0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.204+0000 7efc53fff640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efc540a3270 0x7efc540a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.204+0000 7efc6a7fc640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efc540a3270 0x7efc540a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.204+0000 7efc53fff640 1 -- 172.21.15.37:0/3510914501 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6847+0+0 (secure 0 0 0) 0x7efc5c0cef10 con 0x7efc6c1026a0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.204+0000 7efc6a7fc640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efc540a3270 0x7efc540a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efc6c1993f0 tx=0x7efc6000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:00.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.207+0000 7efc53fff640 1 -- 172.21.15.37:0/3510914501 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc5c009de0 con 0x7efc6c1026a0 2024-09-19T06:39:00.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.382+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7efc6c02d9e0 con 0x7efc6c1026a0 2024-09-19T06:39:00.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.383+0000 7efc53fff640 1 -- 172.21.15.37:0/3510914501 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v74) ==== 74+0+133 (secure 0 0 0) 0x7efc5c09c1f0 con 0x7efc6c1026a0 2024-09-19T06:39:00.384 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efc540a3270 msgr2=0x7efc540a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efc540a3270 0x7efc540a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7efc6c1993f0 tx=0x7efc6000b410 comp rx=0 tx=0).stop 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1026a0 msgr2=0x7efc6c197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1026a0 0x7efc6c197ed0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7efc5c00d950 tx=0x7efc5c00de10 comp rx=0 tx=0).stop 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 shutdown_connections 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efc540a3270 0x7efc540a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc6c107900 0x7efc6c19f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc6c104fd0 0x7efc6c198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 --2- 172.21.15.37:0/3510914501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc6c1026a0 0x7efc6c197ed0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:00.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 >> 172.21.15.37:0/3510914501 conn(0x7efc6c0fc2a0 msgr2=0x7efc6c102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:00.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 shutdown_connections 2024-09-19T06:39:00.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:00.385+0000 7efc71490640 1 -- 172.21.15.37:0/3510914501 wait complete. 2024-09-19T06:39:00.612 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:39:00.117+0000 7f8f05375640 -1 osd.11 0 waiting for initial osdmap 2024-09-19T06:39:00.612 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:39:00.131+0000 7f8efd195640 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-19T06:39:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:39:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T06:39:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: Detected new or changed devices on smithi151 2024-09-19T06:39:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:01.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='osd.11 ' entity='osd.11' 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:00 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3510914501' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: Detected new or changed devices on smithi151 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.027 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='osd.11 ' entity='osd.11' 2024-09-19T06:39:01.028 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:00 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3510914501' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: osdmap e74: 12 total, 11 up, 12 in 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: Detected new or changed devices on smithi151 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi151 to 4272M 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='osd.11 ' entity='osd.11' 2024-09-19T06:39:01.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:00 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3510914501' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:39:01.145 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":74,"num_osds":12,"num_up_osds":11,"osd_up_since":1726727915,"num_in_osds":12,"osd_in_since":1726727917,"num_remapped_pgs":0} 2024-09-19T06:39:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:01 smithi096 ceph-mon[35004]: purged_snaps scrub starts 2024-09-19T06:39:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:01 smithi096 ceph-mon[35004]: purged_snaps scrub ok 2024-09-19T06:39:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:01 smithi096 ceph-mon[35004]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:39:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:01 smithi096 ceph-mon[35004]: osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] boot 2024-09-19T06:39:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:01 smithi096 ceph-mon[35004]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:39:02.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:01 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:01 smithi037 ceph-mon[31211]: purged_snaps scrub starts 2024-09-19T06:39:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:01 smithi037 ceph-mon[31211]: purged_snaps scrub ok 2024-09-19T06:39:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:01 smithi037 ceph-mon[31211]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:39:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:01 smithi037 ceph-mon[31211]: osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] boot 2024-09-19T06:39:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:01 smithi037 ceph-mon[31211]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:39:02.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:01 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:02.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:01 smithi151 ceph-mon[35236]: purged_snaps scrub starts 2024-09-19T06:39:02.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:01 smithi151 ceph-mon[35236]: purged_snaps scrub ok 2024-09-19T06:39:02.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:01 smithi151 ceph-mon[35236]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-09-19T06:39:02.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:02.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:01 smithi151 ceph-mon[35236]: osd.11 [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] boot 2024-09-19T06:39:02.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:01 smithi151 ceph-mon[35236]: osdmap e75: 12 total, 12 up, 12 in 2024-09-19T06:39:02.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:01 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:39:02.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-19T06:39:02.413 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:02.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.921+0000 7f05c070b640 1 -- 172.21.15.37:0/3745131284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b8104fd0 msgr2=0x7f05b81073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:02.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.921+0000 7f05c070b640 1 --2- 172.21.15.37:0/3745131284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b8104fd0 0x7f05b81073c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f05a8009900 tx=0x7f05a802a410 comp rx=0 tx=0).stop 2024-09-19T06:39:02.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.921+0000 7f05c070b640 1 -- 172.21.15.37:0/3745131284 shutdown_connections 2024-09-19T06:39:02.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.921+0000 7f05c070b640 1 --2- 172.21.15.37:0/3745131284 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05b8107900 0x7f05b8109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:02.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.921+0000 7f05c070b640 1 --2- 172.21.15.37:0/3745131284 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b8104fd0 0x7f05b81073c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:02.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.921+0000 7f05c070b640 1 --2- 172.21.15.37:0/3745131284 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05b81026a0 0x7f05b8104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.921+0000 7f05c070b640 1 -- 172.21.15.37:0/3745131284 >> 172.21.15.37:0/3745131284 conn(0x7f05b80fc2a0 msgr2=0x7f05b80fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 -- 172.21.15.37:0/3745131284 shutdown_connections 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 -- 172.21.15.37:0/3745131284 wait complete. 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 Processor -- start 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 -- start start 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b81026a0 0x7f05b8197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05b8104fd0 0x7f05b8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05b8107900 0x7f05b819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05b810d8a0 con 0x7f05b81026a0 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05b810d720 con 0x7f05b8104fd0 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05c070b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05b810da20 con 0x7f05b8107900 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05be4a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b81026a0 0x7f05b8197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:02.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05be4a7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b81026a0 0x7f05b8197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55704/0 (socket says 172.21.15.37:55704) 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.922+0000 7f05be4a7640 1 -- 172.21.15.37:0/2143849777 learned_addr learned my addr 172.21.15.37:0/2143849777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05be4a7640 1 -- 172.21.15.37:0/2143849777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05b8107900 msgr2=0x7f05b819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05bdca6640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05b8104fd0 0x7f05b8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05be4a7640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05b8107900 0x7f05b819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05be4a7640 1 -- 172.21.15.37:0/2143849777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05b8104fd0 msgr2=0x7f05b8198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05be4a7640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05b8104fd0 0x7f05b8198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05be4a7640 1 -- 172.21.15.37:0/2143849777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05b819fb30 con 0x7f05b81026a0 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05bdca6640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05b8104fd0 0x7f05b8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05be4a7640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b81026a0 0x7f05b8197f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c920 tx=0x7f05b400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05af7fe640 1 -- 172.21.15.37:0/2143849777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4007c70 con 0x7f05b81026a0 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05af7fe640 1 -- 172.21.15.37:0/2143849777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05b4007e10 con 0x7f05b81026a0 2024-09-19T06:39:02.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05af7fe640 1 -- 172.21.15.37:0/2143849777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b4018890 con 0x7f05b81026a0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05b819fe20 con 0x7f05b81026a0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.923+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05b81a0280 con 0x7f05b81026a0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.924+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05b80fdd90 con 0x7f05b81026a0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.925+0000 7f05af7fe640 1 -- 172.21.15.37:0/2143849777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f05b4018a30 con 0x7f05b81026a0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.926+0000 7f05af7fe640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f05a40a32f0 0x7f05a40a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.926+0000 7f05af7fe640 1 -- 172.21.15.37:0/2143849777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f05b40cf6d0 con 0x7f05b81026a0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.926+0000 7f05bdca6640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f05a40a32f0 0x7f05a40a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.927+0000 7f05bdca6640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f05a40a32f0 0x7f05a40a57b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a8031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:02.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:02.930+0000 7f05af7fe640 1 -- 172.21.15.37:0/2143849777 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f05b4014030 con 0x7f05b81026a0 2024-09-19T06:39:03.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.108+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f05b8062ef0 con 0x7f05b81026a0 2024-09-19T06:39:03.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.109+0000 7f05af7fe640 1 -- 172.21.15.37:0/2143849777 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f05b401f300 con 0x7f05b81026a0 2024-09-19T06:39:03.110 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f05a40a32f0 msgr2=0x7f05a40a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f05a40a32f0 0x7f05a40a57b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f05a8000c00 tx=0x7f05a8031400 comp rx=0 tx=0).stop 2024-09-19T06:39:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b81026a0 msgr2=0x7f05b8197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b81026a0 0x7f05b8197f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f05b400c920 tx=0x7f05b400cde0 comp rx=0 tx=0).stop 2024-09-19T06:39:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 shutdown_connections 2024-09-19T06:39:03.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f05a40a32f0 0x7f05a40a57b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05b8107900 0x7f05b819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05b8104fd0 0x7f05b8198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 --2- 172.21.15.37:0/2143849777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05b81026a0 0x7f05b8197f00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 >> 172.21.15.37:0/2143849777 conn(0x7f05b80fc2a0 msgr2=0x7f05b8102f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 shutdown_connections 2024-09-19T06:39:03.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:03.111+0000 7f05c070b640 1 -- 172.21.15.37:0/2143849777 wait complete. 2024-09-19T06:39:03.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:03 smithi096 ceph-mon[35004]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:39:03.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:03 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2143849777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:39:03.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:03 smithi037 ceph-mon[31211]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:39:03.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:03 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2143849777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:39:03.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:03 smithi151 ceph-mon[35236]: osdmap e76: 12 total, 12 up, 12 in 2024-09-19T06:39:03.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:03 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2143849777' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-19T06:39:03.760 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726727941,"num_in_osds":12,"osd_in_since":1726727917,"num_remapped_pgs":0} 2024-09-19T06:39:03.761 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:39:04.011 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:04.401 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:04 smithi037 ceph-mon[31211]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 -- 172.21.15.37:0/1527614463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c106a80 msgr2=0x7f3e7c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1527614463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c106a80 0x7f3e7c106e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009930 tx=0x7f3e6c02a420 comp rx=0 tx=0).stop 2024-09-19T06:39:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 -- 172.21.15.37:0/1527614463 shutdown_connections 2024-09-19T06:39:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1527614463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7c101420 0x7f3e7c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1527614463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7c100a80 0x7f3e7c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1527614463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c106a80 0x7f3e7c106e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 -- 172.21.15.37:0/1527614463 >> 172.21.15.37:0/1527614463 conn(0x7f3e7c0fc7a0 msgr2=0x7f3e7c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 -- 172.21.15.37:0/1527614463 shutdown_connections 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.484+0000 7f3e8258e640 1 -- 172.21.15.37:0/1527614463 wait complete. 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 Processor -- start 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 -- start start 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7c100a80 0x7f3e7c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c101420 0x7f3e7c19ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7c106a80 0x7f3e7c19efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c110e80 con 0x7f3e7c101420 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c110d00 con 0x7f3e7c100a80 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.485+0000 7f3e8258e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e7c111000 con 0x7f3e7c106a80 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c101420 0x7f3e7c19ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:04.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c101420 0x7f3e7c19ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55730/0 (socket says 172.21.15.37:55730) 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 -- 172.21.15.37:0/1832373974 learned_addr learned my addr 172.21.15.37:0/1832373974 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7bfff640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7c100a80 0x7f3e7c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e80b2b640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7c106a80 0x7f3e7c19efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 -- 172.21.15.37:0/1832373974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7c106a80 msgr2=0x7f3e7c19efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7c106a80 0x7f3e7c19efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 -- 172.21.15.37:0/1832373974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7c100a80 msgr2=0x7f3e7c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7c100a80 0x7f3e7c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 -- 172.21.15.37:0/1832373974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7c19f680 con 0x7f3e7c101420 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7bfff640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7c100a80 0x7f3e7c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e7b7fe640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c101420 0x7f3e7c19ac10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c970 tx=0x7f3e7000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e797fa640 1 -- 172.21.15.37:0/1832373974 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e70007bb0 con 0x7f3e7c101420 2024-09-19T06:39:04.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7c19f970 con 0x7f3e7c101420 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e797fa640 1 -- 172.21.15.37:0/1832373974 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e70007d50 con 0x7f3e7c101420 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e797fa640 1 -- 172.21.15.37:0/1832373974 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e7000f450 con 0x7f3e7c101420 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.486+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e7c1a7250 con 0x7f3e7c101420 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.487+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e7c100ee0 con 0x7f3e7c101420 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.488+0000 7f3e797fa640 1 -- 172.21.15.37:0/1832373974 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3e70004fe0 con 0x7f3e7c101420 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.489+0000 7f3e797fa640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e640a3240 0x7f3e640a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.489+0000 7f3e7bfff640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e640a3240 0x7f3e640a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.489+0000 7f3e797fa640 1 -- 172.21.15.37:0/1832373974 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3e700cf790 con 0x7f3e7c101420 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.490+0000 7f3e7bfff640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e640a3240 0x7f3e640a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:04.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.492+0000 7f3e797fa640 1 -- 172.21.15.37:0/1832373974 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3e70014030 con 0x7f3e7c101420 2024-09-19T06:39:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:04 smithi096 ceph-mon[35004]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:04 smithi151 ceph-mon[35236]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:04.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.668+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e7c19ba30 con 0x7f3e7c101420 2024-09-19T06:39:04.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.670+0000 7f3e797fa640 1 -- 172.21.15.37:0/1832373974 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) ==== 74+0+18457 (secure 0 0 0) 0x7f3e7c19ba30 con 0x7f3e7c101420 2024-09-19T06:39:04.672 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:04.672 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":76,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:39:02.131282+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:39:04.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.672+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e640a3240 msgr2=0x7f3e640a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:04.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.672+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e640a3240 0x7f3e640a5700 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c000c00 tx=0x7f3e6c005a20 comp rx=0 tx=0).stop 2024-09-19T06:39:04.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.672+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c101420 msgr2=0x7f3e7c19ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:04.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.672+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c101420 0x7f3e7c19ac10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3e7000c970 tx=0x7f3e7000ce30 comp rx=0 tx=0).stop 2024-09-19T06:39:04.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 shutdown_connections 2024-09-19T06:39:04.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e640a3240 0x7f3e640a5700 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e7c106a80 0x7f3e7c19efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e7c101420 0x7f3e7c19ac10 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 --2- 172.21.15.37:0/1832373974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e7c100a80 0x7f3e7c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:04.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 >> 172.21.15.37:0/1832373974 conn(0x7f3e7c0fc7a0 msgr2=0x7f3e7c06a970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:04.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 shutdown_connections 2024-09-19T06:39:04.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:04.673+0000 7f3e8258e640 1 -- 172.21.15.37:0/1832373974 wait complete. 2024-09-19T06:39:05.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:05 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1832373974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:05.403 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T06:35:14.622171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:39:05.403 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-19T06:39:05.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:05 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1832373974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:05.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:05 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1832373974' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:05.663 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 -- 172.21.15.37:0/274693193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 msgr2=0x7fe65c10bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 --2- 172.21.15.37:0/274693193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c10bbb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fe64c009900 tx=0x7fe64c02a480 comp rx=0 tx=0).stop 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 -- 172.21.15.37:0/274693193 shutdown_connections 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 --2- 172.21.15.37:0/274693193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe65c10c180 0x7fe65c10e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 --2- 172.21.15.37:0/274693193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c10bbb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 --2- 172.21.15.37:0/274693193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65c102da0 0x7fe65c103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 -- 172.21.15.37:0/274693193 >> 172.21.15.37:0/274693193 conn(0x7fe65c0fc7a0 msgr2=0x7fe65c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 -- 172.21.15.37:0/274693193 shutdown_connections 2024-09-19T06:39:06.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.128+0000 7fe6647a3640 1 -- 172.21.15.37:0/274693193 wait complete. 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 Processor -- start 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 -- start start 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65c102da0 0x7fe65c1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe65c10c180 0x7fe65c19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c111fc0 con 0x7fe65c1036c0 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c111e40 con 0x7fe65c10c180 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe6647a3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe65c112140 con 0x7fe65c102da0 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe661d3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:06.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe661d3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55742/0 (socket says 172.21.15.37:55742) 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.129+0000 7fe661d3e640 1 -- 172.21.15.37:0/1705901209 learned_addr learned my addr 172.21.15.37:0/1705901209 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe66253f640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65c102da0 0x7fe65c1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe662d40640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe65c10c180 0x7fe65c19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe661d3e640 1 -- 172.21.15.37:0/1705901209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65c102da0 msgr2=0x7fe65c1984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe661d3e640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65c102da0 0x7fe65c1984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe661d3e640 1 -- 172.21.15.37:0/1705901209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe65c10c180 msgr2=0x7fe65c19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe661d3e640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe65c10c180 0x7fe65c19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe661d3e640 1 -- 172.21.15.37:0/1705901209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe65c19d480 con 0x7fe65c1036c0 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe662d40640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe65c10c180 0x7fe65c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe661d3e640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c198a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe64c005ab0 tx=0x7fe64c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe6537fe640 1 -- 172.21.15.37:0/1705901209 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c03f030 con 0x7fe65c1036c0 2024-09-19T06:39:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe6537fe640 1 -- 172.21.15.37:0/1705901209 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe64c003ce0 con 0x7fe65c1036c0 2024-09-19T06:39:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe65c19d710 con 0x7fe65c1036c0 2024-09-19T06:39:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe6537fe640 1 -- 172.21.15.37:0/1705901209 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe64c035070 con 0x7fe65c1036c0 2024-09-19T06:39:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.130+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe65c1a5050 con 0x7fe65c1036c0 2024-09-19T06:39:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.131+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe65c103180 con 0x7fe65c1036c0 2024-09-19T06:39:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.133+0000 7fe6537fe640 1 -- 172.21.15.37:0/1705901209 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe64c042030 con 0x7fe65c1036c0 2024-09-19T06:39:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.133+0000 7fe6537fe640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe6480a32c0 0x7fe6480a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.133+0000 7fe6537fe640 1 -- 172.21.15.37:0/1705901209 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fe64c0eff90 con 0x7fe65c1036c0 2024-09-19T06:39:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.133+0000 7fe66253f640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe6480a32c0 0x7fe6480a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.134+0000 7fe66253f640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe6480a32c0 0x7fe6480a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe6580058c0 tx=0x7fe65800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.136+0000 7fe6537fe640 1 -- 172.21.15.37:0/1705901209 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe64c0bd150 con 0x7fe65c1036c0 2024-09-19T06:39:06.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.312+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe65c1071f0 con 0x7fe65c1036c0 2024-09-19T06:39:06.314 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:06 smithi037 ceph-mon[31211]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:06.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.313+0000 7fe6537fe640 1 -- 172.21.15.37:0/1705901209 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) ==== 93+0+10 (secure 0 0 0) 0x7fe64c0bca50 con 0x7fe65c1036c0 2024-09-19T06:39:06.315 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-19T06:39:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe6480a32c0 msgr2=0x7fe6480a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe6480a32c0 0x7fe6480a5780 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe6580058c0 tx=0x7fe65800a250 comp rx=0 tx=0).stop 2024-09-19T06:39:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 msgr2=0x7fe65c198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c198a10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fe64c005ab0 tx=0x7fe64c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 shutdown_connections 2024-09-19T06:39:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe6480a32c0 0x7fe6480a5780 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe65c10c180 0x7fe65c19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe65c1036c0 0x7fe65c198a10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 --2- 172.21.15.37:0/1705901209 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe65c102da0 0x7fe65c1984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 >> 172.21.15.37:0/1705901209 conn(0x7fe65c0fc7a0 msgr2=0x7fe65c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:06.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 shutdown_connections 2024-09-19T06:39:06.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:06.315+0000 7fe6647a3640 1 -- 172.21.15.37:0/1705901209 wait complete. 2024-09-19T06:39:06.425 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:06 smithi096 ceph-mon[35004]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:06 smithi151 ceph-mon[35236]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:07.017 INFO:tasks.cephadm:Adding mds.a on smithi037 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.d on smithi037 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.g on smithi037 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.j on smithi037 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.b on smithi096 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.e on smithi096 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.h on smithi096 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.k on smithi096 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.c on smithi151 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.f on smithi151 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.i on smithi151 2024-09-19T06:39:07.018 INFO:tasks.cephadm:Adding mds.l on smithi151 2024-09-19T06:39:07.019 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l' 2024-09-19T06:39:07.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:07 smithi037 ceph-mon[31211]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:07.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:07 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1705901209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:39:07.281 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:39:07.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:07 smithi151 ceph-mon[35236]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:07.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:07 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1705901209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:39:07.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:07 smithi096 ceph-mon[35004]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 350 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:07.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:07 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1705901209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:39:07.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.773+0000 7f3b070d8640 1 -- 172.21.15.151:0/1291233954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00105050 msgr2=0x7f3b00107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:07.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.773+0000 7f3b070d8640 1 --2- 172.21.15.151:0/1291233954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00105050 0x7f3b00107440 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3aec009900 tx=0x7f3aec02a410 comp rx=0 tx=0).stop 2024-09-19T06:39:07.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- 172.21.15.151:0/1291233954 shutdown_connections 2024-09-19T06:39:07.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 --2- 172.21.15.151:0/1291233954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00107980 0x7f3b00109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.788 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 --2- 172.21.15.151:0/1291233954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00105050 0x7f3b00107440 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 --2- 172.21.15.151:0/1291233954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b00102720 0x7f3b00104b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- 172.21.15.151:0/1291233954 >> 172.21.15.151:0/1291233954 conn(0x7f3b000fc280 msgr2=0x7f3b000fe6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- 172.21.15.151:0/1291233954 shutdown_connections 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- 172.21.15.151:0/1291233954 wait complete. 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 Processor -- start 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- start start 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102720 0x7f3b00102080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:07.789 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b00105050 0x7f3b001006d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00107980 0x7f3b00100c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0010d8d0 con 0x7f3b00105050 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0010d750 con 0x7f3b00107980 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.774+0000 7f3b070d8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3b0010da50 con 0x7f3b00102720 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102720 0x7f3b00102080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102720 0x7f3b00102080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:54966/0 (socket says 172.21.15.151:54966) 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 -- 172.21.15.151:0/3236809505 learned_addr learned my addr 172.21.15.151:0/3236809505 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-09-19T06:39:07.790 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3afffff640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b00105050 0x7f3b001006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:07.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 -- 172.21.15.151:0/3236809505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00107980 msgr2=0x7f3b00100c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:07.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00107980 0x7f3b00100c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 -- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b00105050 msgr2=0x7f3b001006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:07.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b00105050 0x7f3b001006d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 -- 172.21.15.151:0/3236809505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b001014d0 con 0x7f3b00102720 2024-09-19T06:39:07.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b04e74640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102720 0x7f3b00102080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c890 tx=0x7f3af400cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:07.791 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3afdffb640 1 -- 172.21.15.151:0/3236809505 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af4004220 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3afdffb640 1 -- 172.21.15.151:0/3236809505 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3af40043c0 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3afdffb640 1 -- 172.21.15.151:0/3236809505 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3af40049e0 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b001a2140 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.775+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b001a2680 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.776+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b000fdd70 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.777+0000 7f3afdffb640 1 -- 172.21.15.151:0/3236809505 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3af40079b0 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.778+0000 7f3afdffb640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3af00a3240 0x7f3af00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.779+0000 7f3afdffb640 1 -- 172.21.15.151:0/3236809505 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f3af40cf580 con 0x7f3b00102720 2024-09-19T06:39:07.792 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.779+0000 7f3afffff640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3af00a3240 0x7f3af00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.779+0000 7f3afffff640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3af00a3240 0x7f3af00a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3aec006010 tx=0x7f3aec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:07.793 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.783+0000 7f3afdffb640 1 -- 172.21.15.151:0/3236809505 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3af4014030 con 0x7f3b00102720 2024-09-19T06:39:07.967 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.966+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}) -- 0x7f3b00062730 con 0x7f3af00a3240 2024-09-19T06:39:07.975 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.974+0000 7f3afdffb640 1 -- 172.21.15.151:0/3236809505 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f3b00062730 con 0x7f3af00a3240 2024-09-19T06:39:07.975 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mds.all update... 2024-09-19T06:39:07.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3af00a3240 msgr2=0x7f3af00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:07.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3af00a3240 0x7f3af00a5700 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3aec006010 tx=0x7f3aec000bc0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.977 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102720 msgr2=0x7f3b00102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102720 0x7f3b00102080 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3af400c890 tx=0x7f3af400cd50 comp rx=0 tx=0).stop 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 shutdown_connections 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3af00a3240 0x7f3af00a5700 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3b00107980 0x7f3b00100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3b00105050 0x7f3b001006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 --2- 172.21.15.151:0/3236809505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3b00102720 0x7f3b00102080 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 >> 172.21.15.151:0/3236809505 conn(0x7f3b000fc280 msgr2=0x7f3b001030e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 shutdown_connections 2024-09-19T06:39:07.978 INFO:teuthology.orchestra.run.smithi151.stderr:2024-09-19T06:39:07.976+0000 7f3b070d8640 1 -- 172.21.15.151:0/3236809505 wait complete. 2024-09-19T06:39:08.668 DEBUG:teuthology.orchestra.run.smithi037:mds.a> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.a.service 2024-09-19T06:39:08.672 DEBUG:teuthology.orchestra.run.smithi037:mds.d> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.d.service 2024-09-19T06:39:08.717 DEBUG:teuthology.orchestra.run.smithi037:mds.g> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.g.service 2024-09-19T06:39:08.760 DEBUG:teuthology.orchestra.run.smithi037:mds.j> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.j.service 2024-09-19T06:39:08.802 DEBUG:teuthology.orchestra.run.smithi096:mds.b> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.b.service 2024-09-19T06:39:08.806 DEBUG:teuthology.orchestra.run.smithi096:mds.e> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.e.service 2024-09-19T06:39:08.850 DEBUG:teuthology.orchestra.run.smithi096:mds.h> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.h.service 2024-09-19T06:39:08.893 DEBUG:teuthology.orchestra.run.smithi096:mds.k> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.k.service 2024-09-19T06:39:08.936 DEBUG:teuthology.orchestra.run.smithi151:mds.c> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.c.service 2024-09-19T06:39:08.939 DEBUG:teuthology.orchestra.run.smithi151:mds.f> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.f.service 2024-09-19T06:39:08.982 DEBUG:teuthology.orchestra.run.smithi151:mds.i> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.i.service 2024-09-19T06:39:09.026 DEBUG:teuthology.orchestra.run.smithi151:mds.l> sudo journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.l.service 2024-09-19T06:39:09.069 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-19T06:39:09.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:09.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:39:09.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T06:39:09.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:09.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:09.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:09.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:09.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:09.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:09.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:09.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:08 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:39:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T06:39:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:08 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='client.24388 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:39:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: Saving service mds.all spec with placement smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T06:39:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:08 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:09.325 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:09.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 -- 172.21.15.37:0/601048870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101240 msgr2=0x7febd4101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 --2- 172.21.15.37:0/601048870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101240 0x7febd4101680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7febc8009900 tx=0x7febc802a410 comp rx=0 tx=0).stop 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 -- 172.21.15.37:0/601048870 shutdown_connections 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 --2- 172.21.15.37:0/601048870 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd4101bc0 0x7febd410d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 --2- 172.21.15.37:0/601048870 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101240 0x7febd4101680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 --2- 172.21.15.37:0/601048870 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7febd4105a90 0x7febd4105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 -- 172.21.15.37:0/601048870 >> 172.21.15.37:0/601048870 conn(0x7febd4077210 msgr2=0x7febd40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 -- 172.21.15.37:0/601048870 shutdown_connections 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 -- 172.21.15.37:0/601048870 wait complete. 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 Processor -- start 2024-09-19T06:39:09.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 -- start start 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.775+0000 7febdc42f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd4101240 0x7febd419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febdc42f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101bc0 0x7febd419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febdc42f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7febd4105a90 0x7febd419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febdc42f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7febd4110f70 con 0x7febd4101bc0 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febdc42f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7febd4110df0 con 0x7febd4101240 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febdc42f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7febd41110f0 con 0x7febd4105a90 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101bc0 0x7febd419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101bc0 0x7febd419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39140/0 (socket says 172.21.15.37:39140) 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 -- 172.21.15.37:0/3350289241 learned_addr learned my addr 172.21.15.37:0/3350289241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febda1cb640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd4101240 0x7febd419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febda9cc640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7febd4105a90 0x7febd419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:09.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 -- 172.21.15.37:0/3350289241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7febd4105a90 msgr2=0x7febd419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7febd4105a90 0x7febd419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 -- 172.21.15.37:0/3350289241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd4101240 msgr2=0x7febd419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd4101240 0x7febd419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 -- 172.21.15.37:0/3350289241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd419f6a0 con 0x7febd4101bc0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febda1cb640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd4101240 0x7febd419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febd99ca640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101bc0 0x7febd419ac30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febc77fe640 1 -- 172.21.15.37:0/3350289241 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc803f070 con 0x7febd4101bc0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd419f930 con 0x7febd4101bc0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febc77fe640 1 -- 172.21.15.37:0/3350289241 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7febc8031c30 con 0x7febd4101bc0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.776+0000 7febc77fe640 1 -- 172.21.15.37:0/3350289241 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febc8035030 con 0x7febd4101bc0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.777+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd41a7270 con 0x7febd4101bc0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.778+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd4101680 con 0x7febd4101bc0 2024-09-19T06:39:09.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.780+0000 7febc77fe640 1 -- 172.21.15.37:0/3350289241 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7febc8042070 con 0x7febd4101bc0 2024-09-19T06:39:09.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.780+0000 7febc77fe640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7febc00a3240 0x7febc00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:09.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.780+0000 7febda1cb640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7febc00a3240 0x7febc00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:09.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.780+0000 7febc77fe640 1 -- 172.21.15.37:0/3350289241 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7febc80f01e0 con 0x7febd4101bc0 2024-09-19T06:39:09.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.781+0000 7febda1cb640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7febc00a3240 0x7febc00a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7febd00058c0 tx=0x7febd0005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:09.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.784+0000 7febc77fe640 1 -- 172.21.15.37:0/3350289241 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7febc8031de0 con 0x7febd4101bc0 2024-09-19T06:39:09.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.996+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febd41a7440 con 0x7febd4101bc0 2024-09-19T06:39:09.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.996+0000 7febc77fe640 1 -- 172.21.15.37:0/3350289241 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7febc80bd460 con 0x7febd4101bc0 2024-09-19T06:39:09.997 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:09.998 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":1,"btime":"2024-09-19T06:32:04:919506+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-19T06:39:09.998 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 1 2024-09-19T06:39:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.998+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7febc00a3240 msgr2=0x7febc00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.998+0000 7febdc42f640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7febc00a3240 0x7febc00a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7febd00058c0 tx=0x7febd0005850 comp rx=0 tx=0).stop 2024-09-19T06:39:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.998+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101bc0 msgr2=0x7febd419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:09.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.998+0000 7febdc42f640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101bc0 0x7febd419ac30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc8003c10 comp rx=0 tx=0).stop 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 shutdown_connections 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7febc00a3240 0x7febc00a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7febd4105a90 0x7febd419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7febd4101bc0 0x7febd419ac30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 --2- 172.21.15.37:0/3350289241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7febd4101240 0x7febd419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 >> 172.21.15.37:0/3350289241 conn(0x7febd4077210 msgr2=0x7febd4104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 shutdown_connections 2024-09-19T06:39:10.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:09.999+0000 7febdc42f640 1 -- 172.21.15.37:0/3350289241 wait complete. 2024-09-19T06:39:10.261 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:09 smithi151 ceph-mon[35236]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:10.261 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:09 smithi151 ceph-mon[35236]: Deploying daemon mds.l on smithi151 2024-09-19T06:39:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:09 smithi096 ceph-mon[35004]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:09 smithi096 ceph-mon[35004]: Deploying daemon mds.l on smithi151 2024-09-19T06:39:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:09 smithi037 ceph-mon[31211]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:09 smithi037 ceph-mon[31211]: Deploying daemon mds.l on smithi151 2024-09-19T06:39:10.613 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:10 smithi151 systemd[1]: Starting Ceph mds.l for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:10.923 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:10 smithi151 podman[65347]: 2024-09-19 06:39:10.652447272 +0000 UTC m=+0.111556428 container create 1e9f1562a3dba6521593fcf04d4f98ed507e58f9c081c0d53e851e633922f851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:39:10.923 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:10 smithi151 podman[65347]: 2024-09-19 06:39:10.557100463 +0000 UTC m=+0.016209595 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:10.923 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:10 smithi151 podman[65347]: 2024-09-19 06:39:10.89637512 +0000 UTC m=+0.355484238 container init 1e9f1562a3dba6521593fcf04d4f98ed507e58f9c081c0d53e851e633922f851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:39:10.923 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:10 smithi151 podman[65347]: 2024-09-19 06:39:10.89923471 +0000 UTC m=+0.358343822 container start 1e9f1562a3dba6521593fcf04d4f98ed507e58f9c081c0d53e851e633922f851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T06:39:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:10 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3350289241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:10 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6832/3007825106,v1:172.21.15.151:6833/3007825106] up:boot 2024-09-19T06:39:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:10 smithi096 ceph-mon[35004]: fsmap 1 up:standby 2024-09-19T06:39:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:10 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:39:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:10 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3350289241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:10 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6832/3007825106,v1:172.21.15.151:6833/3007825106] up:boot 2024-09-19T06:39:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:10 smithi037 ceph-mon[31211]: fsmap 1 up:standby 2024-09-19T06:39:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:10 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:39:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:10 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3350289241' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:10 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6832/3007825106,v1:172.21.15.151:6833/3007825106] up:boot 2024-09-19T06:39:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:10 smithi151 ceph-mon[35236]: fsmap 1 up:standby 2024-09-19T06:39:11.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:10 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:39:11.363 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:10 smithi151 ceph-mds[65367]: -- 172.21.15.151:0/3216580028 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c550b03860 con 0x55c550ae0800 2024-09-19T06:39:11.363 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:10 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l[65363]: starting mds.l at 2024-09-19T06:39:11.363 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:11 smithi151 bash[65347]: 1e9f1562a3dba6521593fcf04d4f98ed507e58f9c081c0d53e851e633922f851 2024-09-19T06:39:11.364 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:39:11 smithi151 systemd[1]: Started Ceph mds.l for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:12 smithi151 ceph-mon[35236]: Deploying daemon mds.k on smithi096 2024-09-19T06:39:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:12.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:12 smithi096 ceph-mon[35004]: Deploying daemon mds.k on smithi096 2024-09-19T06:39:12.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:12.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:12.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:12.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:12.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:12.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:12 smithi037 ceph-mon[31211]: Deploying daemon mds.k on smithi096 2024-09-19T06:39:12.760 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:13.027 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:13.274 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 systemd[1]: Starting Ceph mds.k for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 -- 172.21.15.37:0/1451060245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90106a80 msgr2=0x7f5c90106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/1451060245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90106a80 0x7f5c90106e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5c80009930 tx=0x7f5c8002a420 comp rx=0 tx=0).stop 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 -- 172.21.15.37:0/1451060245 shutdown_connections 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/1451060245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c90101420 0x7f5c9010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/1451060245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c90100a80 0x7f5c90100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/1451060245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90106a80 0x7f5c90106e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 -- 172.21.15.37:0/1451060245 >> 172.21.15.37:0/1451060245 conn(0x7f5c900fc7a0 msgr2=0x7f5c900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 -- 172.21.15.37:0/1451060245 shutdown_connections 2024-09-19T06:39:13.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.495+0000 7f5c96fd3640 1 -- 172.21.15.37:0/1451060245 wait complete. 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 Processor -- start 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 -- start start 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c90100a80 0x7f5c90198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90101420 0x7f5c901989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c90106a80 0x7f5c9019cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90110ea0 con 0x7f5c90101420 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90110d20 con 0x7f5c90106a80 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c96fd3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5c90111020 con 0x7f5c90100a80 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90101420 0x7f5c901989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90101420 0x7f5c901989c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39152/0 (socket says 172.21.15.37:39152) 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 -- 172.21.15.37:0/889051628 learned_addr learned my addr 172.21.15.37:0/889051628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c95570640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c90106a80 0x7f5c9019cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:13.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c94d6f640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c90100a80 0x7f5c90198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 -- 172.21.15.37:0/889051628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c90100a80 msgr2=0x7f5c90198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c90100a80 0x7f5c90198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 -- 172.21.15.37:0/889051628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c90106a80 msgr2=0x7f5c9019cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c90106a80 0x7f5c9019cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c8ffff640 1 -- 172.21.15.37:0/889051628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c9019d430 con 0x7f5c90101420 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.496+0000 7f5c95570640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c90106a80 0x7f5c9019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.497+0000 7f5c8ffff640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90101420 0x7f5c901989c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400db80 tx=0x7f5c8400b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.497+0000 7f5c8dffb640 1 -- 172.21.15.37:0/889051628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c84017070 con 0x7f5c90101420 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.497+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c9019d720 con 0x7f5c90101420 2024-09-19T06:39:13.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.497+0000 7f5c8dffb640 1 -- 172.21.15.37:0/889051628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5c84003b70 con 0x7f5c90101420 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.497+0000 7f5c8dffb640 1 -- 172.21.15.37:0/889051628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c840106c0 con 0x7f5c90101420 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.497+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c901a5000 con 0x7f5c90101420 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.498+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c90100ee0 con 0x7f5c90101420 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.499+0000 7f5c8dffb640 1 -- 172.21.15.37:0/889051628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5c84004020 con 0x7f5c90101420 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.500+0000 7f5c8dffb640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5c780a3240 0x7f5c780a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.500+0000 7f5c94d6f640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5c780a3240 0x7f5c780a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.500+0000 7f5c8dffb640 1 -- 172.21.15.37:0/889051628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f5c840cf370 con 0x7f5c90101420 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.500+0000 7f5c94d6f640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5c780a3240 0x7f5c780a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000c00 tx=0x7f5c80005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:13.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.504+0000 7f5c8dffb640 1 -- 172.21.15.37:0/889051628 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5c84020070 con 0x7f5c90101420 2024-09-19T06:39:13.578 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 podman[64073]: 2024-09-19 06:39:13.22014723 +0000 UTC m=+0.015635962 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:13.578 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 podman[64073]: 2024-09-19 06:39:13.341270887 +0000 UTC m=+0.136759618 container create 89ff57701125d1a9035d4c7878d9c2d4efaedc85cfdf4244230e87d085898dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T06:39:13.578 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 podman[64073]: 2024-09-19 06:39:13.577328439 +0000 UTC m=+0.372817171 container init 89ff57701125d1a9035d4c7878d9c2d4efaedc85cfdf4244230e87d085898dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T06:39:13.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.716+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5c9002d7c0 con 0x7f5c90101420 2024-09-19T06:39:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.720+0000 7f5c8dffb640 1 -- 172.21.15.37:0/889051628 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f5c8409c490 con 0x7f5c90101420 2024-09-19T06:39:13.721 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:13.721 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":2,"btime":"2024-09-19T06:39:10:927727+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-19T06:39:13.721 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 2 2024-09-19T06:39:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5c780a3240 msgr2=0x7f5c780a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5c780a3240 0x7f5c780a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5c80000c00 tx=0x7f5c80005a20 comp rx=0 tx=0).stop 2024-09-19T06:39:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90101420 msgr2=0x7f5c901989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90101420 0x7f5c901989c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400db80 tx=0x7f5c8400b630 comp rx=0 tx=0).stop 2024-09-19T06:39:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 shutdown_connections 2024-09-19T06:39:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5c780a3240 0x7f5c780a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c90106a80 0x7f5c9019cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5c90101420 0x7f5c901989c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 --2- 172.21.15.37:0/889051628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5c90100a80 0x7f5c90198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 >> 172.21.15.37:0/889051628 conn(0x7f5c900fc7a0 msgr2=0x7f5c9006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 shutdown_connections 2024-09-19T06:39:13.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:13.722+0000 7f5c96fd3640 1 -- 172.21.15.37:0/889051628 wait complete. 2024-09-19T06:39:14.025 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 podman[64073]: 2024-09-19 06:39:13.580456743 +0000 UTC m=+0.375945481 container start 89ff57701125d1a9035d4c7878d9c2d4efaedc85cfdf4244230e87d085898dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T06:39:14.025 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 ceph-mds[64092]: -- 172.21.15.96:0/2267075635 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cec095d860 con 0x55cec093b400 2024-09-19T06:39:14.025 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k[64088]: starting mds.k at 2024-09-19T06:39:14.025 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 bash[64073]: 89ff57701125d1a9035d4c7878d9c2d4efaedc85cfdf4244230e87d085898dbc 2024-09-19T06:39:14.025 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:39:13 smithi096 systemd[1]: Started Ceph mds.k for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/889051628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:14 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:14.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:14.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/889051628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:14.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:14.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:14.494 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:14 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/889051628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:14 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:15.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:15 smithi037 ceph-mon[31211]: Deploying daemon mds.j on smithi037 2024-09-19T06:39:15.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:15 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6832/672218237,v1:172.21.15.96:6833/672218237] up:boot 2024-09-19T06:39:15.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:15 smithi037 ceph-mon[31211]: fsmap 2 up:standby 2024-09-19T06:39:15.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:15 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:39:15.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:15 smithi151 ceph-mon[35236]: Deploying daemon mds.j on smithi037 2024-09-19T06:39:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:15 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6832/672218237,v1:172.21.15.96:6833/672218237] up:boot 2024-09-19T06:39:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:15 smithi151 ceph-mon[35236]: fsmap 2 up:standby 2024-09-19T06:39:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:15 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:39:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:15 smithi096 ceph-mon[35004]: Deploying daemon mds.j on smithi037 2024-09-19T06:39:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:15 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6832/672218237,v1:172.21.15.96:6833/672218237] up:boot 2024-09-19T06:39:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:15 smithi096 ceph-mon[35004]: fsmap 2 up:standby 2024-09-19T06:39:15.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:15 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:39:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:16 smithi096 ceph-mon[35004]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:16 smithi037 ceph-mon[31211]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:16.275 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 systemd[1]: Starting Ceph mds.j for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:16 smithi151 ceph-mon[35236]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:16.524 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:16.623 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 podman[69791]: 2024-09-19 06:39:16.267534009 +0000 UTC m=+0.016299996 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:16.624 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 podman[69791]: 2024-09-19 06:39:16.370188579 +0000 UTC m=+0.118954588 container create 6512a1479098082483b74ac49b4cecd81bfd3ef6729a14e32a60f85226da85e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T06:39:16.624 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 podman[69791]: 2024-09-19 06:39:16.623043416 +0000 UTC m=+0.371809399 container init 6512a1479098082483b74ac49b4cecd81bfd3ef6729a14e32a60f85226da85e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-19T06:39:16.813 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:16.936 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 podman[69791]: 2024-09-19 06:39:16.626453924 +0000 UTC m=+0.375219897 container start 6512a1479098082483b74ac49b4cecd81bfd3ef6729a14e32a60f85226da85e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:39:16.936 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 ceph-mds[69829]: -- 172.21.15.37:0/2687673310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557e23c39860 con 0x557e23c16800 2024-09-19T06:39:16.936 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j[69807]: starting mds.j at 2024-09-19T06:39:16.936 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 bash[69791]: 6512a1479098082483b74ac49b4cecd81bfd3ef6729a14e32a60f85226da85e9 2024-09-19T06:39:16.937 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:39:16 smithi037 systemd[1]: Started Ceph mds.j for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 -- 172.21.15.37:0/3802489461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 msgr2=0x7fb2981067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 --2- 172.21.15.37:0/3802489461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981067e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fb28c009900 tx=0x7fb28c02a410 comp rx=0 tx=0).stop 2024-09-19T06:39:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 -- 172.21.15.37:0/3802489461 shutdown_connections 2024-09-19T06:39:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 --2- 172.21.15.37:0/3802489461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb298106f10 0x7fb29810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 --2- 172.21.15.37:0/3802489461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981067e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 --2- 172.21.15.37:0/3802489461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb298101de0 0x7fb2981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 -- 172.21.15.37:0/3802489461 >> 172.21.15.37:0/3802489461 conn(0x7fb2980fd8c0 msgr2=0x7fb2980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 -- 172.21.15.37:0/3802489461 shutdown_connections 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.302+0000 7fb29cf47640 1 -- 172.21.15.37:0/3802489461 wait complete. 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 Processor -- start 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 -- start start 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb298101de0 0x7fb298198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb298106f10 0x7fb29819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb298111f90 con 0x7fb298102790 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb298111e10 con 0x7fb298106f10 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb29cf47640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb298112110 con 0x7fb298101de0 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb295d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb295d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39194/0 (socket says 172.21.15.37:39194) 2024-09-19T06:39:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb296d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb298106f10 0x7fb29819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.303+0000 7fb295d9b640 1 -- 172.21.15.37:0/2694789977 learned_addr learned my addr 172.21.15.37:0/2694789977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb29659c640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb298101de0 0x7fb298198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb295d9b640 1 -- 172.21.15.37:0/2694789977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb298101de0 msgr2=0x7fb298198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb295d9b640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb298101de0 0x7fb298198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb295d9b640 1 -- 172.21.15.37:0/2694789977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb298106f10 msgr2=0x7fb29819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb295d9b640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb298106f10 0x7fb29819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb295d9b640 1 -- 172.21.15.37:0/2694789977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb29819d440 con 0x7fb298102790 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb296d9d640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb298106f10 0x7fb29819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb295d9b640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981989d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb28c006010 tx=0x7fb28c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb2877fe640 1 -- 172.21.15.37:0/2694789977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c03f030 con 0x7fb298102790 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb29819d6d0 con 0x7fb298102790 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb2877fe640 1 -- 172.21.15.37:0/2694789977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb28c003ce0 con 0x7fb298102790 2024-09-19T06:39:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb2877fe640 1 -- 172.21.15.37:0/2694789977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb28c035070 con 0x7fb298102790 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.304+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2981a5010 con 0x7fb298102790 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.305+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2981032c0 con 0x7fb298102790 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.307+0000 7fb2877fe640 1 -- 172.21.15.37:0/2694789977 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb28c042030 con 0x7fb298102790 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.308+0000 7fb2877fe640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2800a3240 0x7fb2800a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.308+0000 7fb2877fe640 1 -- 172.21.15.37:0/2694789977 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fb28c0efac0 con 0x7fb298102790 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.308+0000 7fb29659c640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2800a3240 0x7fb2800a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.308+0000 7fb29659c640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2800a3240 0x7fb2800a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb288009990 tx=0x7fb288009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.311+0000 7fb2877fe640 1 -- 172.21.15.37:0/2694789977 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb28c0bcc80 con 0x7fb298102790 2024-09-19T06:39:17.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.530+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb29810a0f0 con 0x7fb298102790 2024-09-19T06:39:17.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.530+0000 7fb2877fe640 1 -- 172.21.15.37:0/2694789977 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fb28c0bc580 con 0x7fb298102790 2024-09-19T06:39:17.532 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:17.533 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":4,"btime":"2024-09-19T06:39:17:072949+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T06:39:17.533 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 4 2024-09-19T06:39:17.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2800a3240 msgr2=0x7fb2800a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:17.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2800a3240 0x7fb2800a5700 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb288009990 tx=0x7fb288009110 comp rx=0 tx=0).stop 2024-09-19T06:39:17.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 msgr2=0x7fb2981989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981989d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb28c006010 tx=0x7fb28c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 shutdown_connections 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2800a3240 0x7fb2800a5700 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb298106f10 0x7fb29819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb298102790 0x7fb2981989d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 --2- 172.21.15.37:0/2694789977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb298101de0 0x7fb298198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 >> 172.21.15.37:0/2694789977 conn(0x7fb2980fd8c0 msgr2=0x7fb2980fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 shutdown_connections 2024-09-19T06:39:17.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:17.533+0000 7fb29cf47640 1 -- 172.21.15.37:0/2694789977 wait complete. 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: Deploying daemon mds.d on smithi037 2024-09-19T06:39:18.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.37:6834/146682084,v1:172.21.15.37:6835/146682084] up:boot 2024-09-19T06:39:18.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: fsmap 3 up:standby 2024-09-19T06:39:18.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:39:18.114 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:17 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2694789977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: Deploying daemon mds.d on smithi037 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.37:6834/146682084,v1:172.21.15.37:6835/146682084] up:boot 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: fsmap 3 up:standby 2024-09-19T06:39:18.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:17 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2694789977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:18.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:18.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: Deploying daemon mds.d on smithi037 2024-09-19T06:39:18.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.37:6834/146682084,v1:172.21.15.37:6835/146682084] up:boot 2024-09-19T06:39:18.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: fsmap 3 up:standby 2024-09-19T06:39:18.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:39:18.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:17 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2694789977' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:20.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:19 smithi037 ceph-mon[31211]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:20.026 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:19 smithi037 systemd[1]: Starting Ceph mds.d for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:20.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:19 smithi151 ceph-mon[35236]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:20.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:19 smithi096 ceph-mon[35004]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:20.323 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 podman[70207]: 2024-09-19 06:39:19.954918381 +0000 UTC m=+0.016679963 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:20.323 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 podman[70207]: 2024-09-19 06:39:20.061368849 +0000 UTC m=+0.123130416 container create 9059d2ccd3581c2b84c98b59ff5ee711e90b97dd7962dc07c72b59fe6193f509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-19T06:39:20.323 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 podman[70207]: 2024-09-19 06:39:20.322624736 +0000 UTC m=+0.384386311 container init 9059d2ccd3581c2b84c98b59ff5ee711e90b97dd7962dc07c72b59fe6193f509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-19T06:39:20.349 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:20.613 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:20.734 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 podman[70207]: 2024-09-19 06:39:20.326177883 +0000 UTC m=+0.387939451 container start 9059d2ccd3581c2b84c98b59ff5ee711e90b97dd7962dc07c72b59fe6193f509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:39:20.734 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 ceph-mds[70226]: -- 172.21.15.37:0/3121323991 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56465790d860 con 0x5646578eb000 2024-09-19T06:39:20.734 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d[70222]: starting mds.d at 2024-09-19T06:39:20.734 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 bash[70207]: 9059d2ccd3581c2b84c98b59ff5ee711e90b97dd7962dc07c72b59fe6193f509 2024-09-19T06:39:20.734 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:39:20 smithi037 systemd[1]: Started Ceph mds.d for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:21.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 -- 172.21.15.37:0/1094600129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724100e20 msgr2=0x7f0724101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:21.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 --2- 172.21.15.37:0/1094600129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724100e20 0x7f0724101200 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0720009900 tx=0x7f072002a480 comp rx=0 tx=0).stop 2024-09-19T06:39:21.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 -- 172.21.15.37:0/1094600129 shutdown_connections 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 --2- 172.21.15.37:0/1094600129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0724102ef0 0x7f072410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 --2- 172.21.15.37:0/1094600129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0724102570 0x7f07241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 --2- 172.21.15.37:0/1094600129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724100e20 0x7f0724101200 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 -- 172.21.15.37:0/1094600129 >> 172.21.15.37:0/1094600129 conn(0x7f07240fc7d0 msgr2=0x7f07240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 -- 172.21.15.37:0/1094600129 shutdown_connections 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.101+0000 7f072b619640 1 -- 172.21.15.37:0/1094600129 wait complete. 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 Processor -- start 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 -- start start 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0724100e20 0x7f0724198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724102570 0x7f0724198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0724102ef0 0x7f072419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:21.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0724110fc0 con 0x7f0724102570 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0724110e40 con 0x7f0724100e20 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f072b619640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0724111140 con 0x7f0724102ef0 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724102570 0x7f0724198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724102570 0x7f0724198ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52518/0 (socket says 172.21.15.37:52518) 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 -- 172.21.15.37:0/3514484315 learned_addr learned my addr 172.21.15.37:0/3514484315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0729bb6640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0724102ef0 0x7f072419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f07293b5640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0724100e20 0x7f0724198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 -- 172.21.15.37:0/3514484315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0724102ef0 msgr2=0x7f072419ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:21.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0724102ef0 0x7f072419ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 -- 172.21.15.37:0/3514484315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0724100e20 msgr2=0x7f0724198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0724100e20 0x7f0724198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f0728bb4640 1 -- 172.21.15.37:0/3514484315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f072419d530 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.102+0000 7f07293b5640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0724100e20 0x7f0724198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.103+0000 7f0728bb4640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724102570 0x7f0724198ac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f071400b4a0 tx=0x7f071400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.103+0000 7f071a7fc640 1 -- 172.21.15.37:0/3514484315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714014070 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.103+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f072419d820 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.103+0000 7f071a7fc640 1 -- 172.21.15.37:0/3514484315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0714010af0 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.103+0000 7f071a7fc640 1 -- 172.21.15.37:0/3514484315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0714017d10 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.103+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07241a5100 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.104+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0724101200 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.105+0000 7f071a7fc640 1 -- 172.21.15.37:0/3514484315 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0714017070 con 0x7f0724102570 2024-09-19T06:39:21.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.106+0000 7f071a7fc640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07100a3240 0x7f07100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:21.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.106+0000 7f071a7fc640 1 -- 172.21.15.37:0/3514484315 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f07140cec70 con 0x7f0724102570 2024-09-19T06:39:21.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.106+0000 7f07293b5640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07100a3240 0x7f07100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:21.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.106+0000 7f07293b5640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07100a3240 0x7f07100a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0720005ab0 tx=0x7f0720005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:21.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.109+0000 7f071a7fc640 1 -- 172.21.15.37:0/3514484315 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f071409bd90 con 0x7f0724102570 2024-09-19T06:39:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.322+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07241998e0 con 0x7f0724102570 2024-09-19T06:39:21.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.323+0000 7f071a7fc640 1 -- 172.21.15.37:0/3514484315 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7f07241998e0 con 0x7f0724102570 2024-09-19T06:39:21.325 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:21.325 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":5,"btime":"2024-09-19T06:39:20:797828+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-19T06:39:21.325 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 5 2024-09-19T06:39:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07100a3240 msgr2=0x7f07100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07100a3240 0x7f07100a5700 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0720005ab0 tx=0x7f0720005a20 comp rx=0 tx=0).stop 2024-09-19T06:39:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724102570 msgr2=0x7f0724198ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724102570 0x7f0724198ac0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f071400b4a0 tx=0x7f071400b960 comp rx=0 tx=0).stop 2024-09-19T06:39:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 shutdown_connections 2024-09-19T06:39:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f07100a3240 0x7f07100a5700 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0724102ef0 0x7f072419ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0724102570 0x7f0724198ac0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 --2- 172.21.15.37:0/3514484315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0724100e20 0x7f0724198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:21.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 >> 172.21.15.37:0/3514484315 conn(0x7f07240fc7d0 msgr2=0x7f0724069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:21.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 shutdown_connections 2024-09-19T06:39:21.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:21.325+0000 7f072b619640 1 -- 172.21.15.37:0/3514484315 wait complete. 2024-09-19T06:39:21.681 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: Deploying daemon mds.e on smithi096 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.37:6836/773501481,v1:172.21.15.37:6837/773501481] up:boot 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: fsmap 4 up:standby 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:39:21.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:21 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3514484315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: Deploying daemon mds.e on smithi096 2024-09-19T06:39:21.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.37:6836/773501481,v1:172.21.15.37:6837/773501481] up:boot 2024-09-19T06:39:21.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: fsmap 4 up:standby 2024-09-19T06:39:21.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:39:21.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:21 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3514484315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:21.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: Deploying daemon mds.e on smithi096 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.37:6836/773501481,v1:172.21.15.37:6837/773501481] up:boot 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: fsmap 4 up:standby 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:39:21.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:21 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3514484315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:22.751 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:22 smithi096 systemd[1]: Starting Ceph mds.e for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:23.002 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:22 smithi096 podman[64389]: 2024-09-19 06:39:22.652446111 +0000 UTC m=+0.015709012 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:23.002 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:22 smithi096 podman[64389]: 2024-09-19 06:39:22.770934442 +0000 UTC m=+0.134197340 container create 1c294244b3eef1c105e4cfdb1184095cdaf7504a65b7e64dc1d3f7857ea8e295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T06:39:23.002 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:22 smithi096 podman[64389]: 2024-09-19 06:39:22.998685592 +0000 UTC m=+0.361948487 container init 1c294244b3eef1c105e4cfdb1184095cdaf7504a65b7e64dc1d3f7857ea8e295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-19T06:39:23.002 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:23 smithi096 podman[64389]: 2024-09-19 06:39:23.001525853 +0000 UTC m=+0.364788749 container start 1c294244b3eef1c105e4cfdb1184095cdaf7504a65b7e64dc1d3f7857ea8e295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:39:23.274 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:23 smithi096 ceph-mds[64408]: -- 172.21.15.96:0/2050433744 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c8d65b860 con 0x563c8d639400 2024-09-19T06:39:23.274 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:23 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e[64404]: starting mds.e at 2024-09-19T06:39:23.274 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:23 smithi096 bash[64389]: 1c294244b3eef1c105e4cfdb1184095cdaf7504a65b7e64dc1d3f7857ea8e295 2024-09-19T06:39:23.275 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:39:23 smithi096 systemd[1]: Started Ceph mds.e for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:24.146 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:24.499 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:24.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:24.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] up:boot 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: fsmap 5 up:standby 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:24 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] up:boot 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: fsmap 5 up:standby 2024-09-19T06:39:24.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:24 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:39:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:24.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:24.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:24.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:24.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:24.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] up:boot 2024-09-19T06:39:24.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: fsmap 5 up:standby 2024-09-19T06:39:24.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:24 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:39:25.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 -- 172.21.15.37:0/4081615663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 msgr2=0x7fca70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:25.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 --2- 172.21.15.37:0/4081615663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70101200 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fca60009900 tx=0x7fca6002a480 comp rx=0 tx=0).stop 2024-09-19T06:39:25.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 -- 172.21.15.37:0/4081615663 shutdown_connections 2024-09-19T06:39:25.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 --2- 172.21.15.37:0/4081615663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70102ef0 0x7fca7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 --2- 172.21.15.37:0/4081615663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca70102570 0x7fca701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 --2- 172.21.15.37:0/4081615663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70101200 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 -- 172.21.15.37:0/4081615663 >> 172.21.15.37:0/4081615663 conn(0x7fca700fc7d0 msgr2=0x7fca700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 -- 172.21.15.37:0/4081615663 shutdown_connections 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 -- 172.21.15.37:0/4081615663 wait complete. 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 Processor -- start 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.992+0000 7fca762f9640 1 -- start start 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca762f9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca762f9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca70102570 0x7fca70199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca762f9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70102ef0 0x7fca701a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:25.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca762f9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca70110f80 con 0x7fca70100e20 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca762f9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca70110e00 con 0x7fca70102570 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca762f9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca70111100 con 0x7fca70102ef0 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52540/0 (socket says 172.21.15.37:52540) 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 -- 172.21.15.37:0/1682186924 learned_addr learned my addr 172.21.15.37:0/1682186924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6f7fe640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca70102570 0x7fca70199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 -- 172.21.15.37:0/1682186924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70102ef0 msgr2=0x7fca701a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70102ef0 0x7fca701a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 -- 172.21.15.37:0/1682186924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca70102570 msgr2=0x7fca70199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:25.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca70102570 0x7fca70199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 -- 172.21.15.37:0/1682186924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca701a48f0 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.993+0000 7fca6ffff640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70199330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca60006010 tx=0x7fca60003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.994+0000 7fca6d7fa640 1 -- 172.21.15.37:0/1682186924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6003f070 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.994+0000 7fca6d7fa640 1 -- 172.21.15.37:0/1682186924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca60031c30 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.994+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca701a4ac0 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.994+0000 7fca6d7fa640 1 -- 172.21.15.37:0/1682186924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca60035030 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.994+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca701a4f70 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.995+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca70101200 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.997+0000 7fca6d7fa640 1 -- 172.21.15.37:0/1682186924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fca60042070 con 0x7fca70100e20 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.997+0000 7fca6d7fa640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fca580a32c0 0x7fca580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.998+0000 7fca6f7fe640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fca580a32c0 0x7fca580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:25.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.998+0000 7fca6d7fa640 1 -- 172.21.15.37:0/1682186924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fca600f0090 con 0x7fca70100e20 2024-09-19T06:39:25.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:24.998+0000 7fca6f7fe640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fca580a32c0 0x7fca580a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca7006aac0 tx=0x7fca6400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:25.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.001+0000 7fca6d7fa640 1 -- 172.21.15.37:0/1682186924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fca600bd250 con 0x7fca70100e20 2024-09-19T06:39:25.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:25 smithi037 ceph-mon[31211]: Deploying daemon mds.f on smithi151 2024-09-19T06:39:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.212+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca70109090 con 0x7fca70100e20 2024-09-19T06:39:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.212+0000 7fca6d7fa640 1 -- 172.21.15.37:0/1682186924 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4726 (secure 0 0 0) 0x7fca600bcb50 con 0x7fca70100e20 2024-09-19T06:39:25.214 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:25.215 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":6,"btime":"2024-09-19T06:39:23:476420+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T06:39:25.215 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 6 2024-09-19T06:39:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.214+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fca580a32c0 msgr2=0x7fca580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.214+0000 7fca762f9640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fca580a32c0 0x7fca580a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fca7006aac0 tx=0x7fca6400a4c0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.214+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 msgr2=0x7fca70199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.214+0000 7fca762f9640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70199330 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fca60006010 tx=0x7fca60003c10 comp rx=0 tx=0).stop 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 shutdown_connections 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fca580a32c0 0x7fca580a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca70102ef0 0x7fca701a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca70102570 0x7fca70199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 --2- 172.21.15.37:0/1682186924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca70100e20 0x7fca70199330 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 >> 172.21.15.37:0/1682186924 conn(0x7fca700fc7d0 msgr2=0x7fca700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 shutdown_connections 2024-09-19T06:39:25.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:25.215+0000 7fca762f9640 1 -- 172.21.15.37:0/1682186924 wait complete. 2024-09-19T06:39:25.346 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:25 smithi151 ceph-mon[35236]: Deploying daemon mds.f on smithi151 2024-09-19T06:39:25.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:25 smithi096 ceph-mon[35004]: Deploying daemon mds.f on smithi151 2024-09-19T06:39:25.613 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:39:25 smithi151 podman[65649]: 2024-09-19 06:39:25.593574536 +0000 UTC m=+0.361941864 container start f12ce8dc46298eb7c905ac3dbc4379a396fcab58779621a0d4a6f08fd516ca88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T06:39:26.113 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:39:25 smithi151 ceph-mds[65669]: -- 172.21.15.151:0/808084710 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562d8d581860 con 0x562d8d55f000 2024-09-19T06:39:26.113 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:39:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-f[65665]: starting mds.f at 2024-09-19T06:39:26.113 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:39:25 smithi151 bash[65649]: f12ce8dc46298eb7c905ac3dbc4379a396fcab58779621a0d4a6f08fd516ca88 2024-09-19T06:39:26.113 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:39:25 smithi151 systemd[1]: Started Ceph mds.f for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:26.455 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:26.455 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1682186924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:26.455 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.455 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:26.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:26.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:26.456 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:26 smithi096 ceph-mon[35004]: Deploying daemon mds.c on smithi151 2024-09-19T06:39:26.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:26.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1682186924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:26.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.461 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:26.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:26.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:26.462 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:26 smithi151 ceph-mon[35236]: Deploying daemon mds.c on smithi151 2024-09-19T06:39:26.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1682186924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:26.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:26 smithi037 ceph-mon[31211]: Deploying daemon mds.c on smithi151 2024-09-19T06:39:27.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:27 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6834/2848315285,v1:172.21.15.151:6835/2848315285] up:boot 2024-09-19T06:39:27.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:27 smithi096 ceph-mon[35004]: fsmap 6 up:standby 2024-09-19T06:39:27.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:27 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:39:27.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:27 smithi096 ceph-mon[35004]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:27.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:27 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6834/2848315285,v1:172.21.15.151:6835/2848315285] up:boot 2024-09-19T06:39:27.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:27 smithi037 ceph-mon[31211]: fsmap 6 up:standby 2024-09-19T06:39:27.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:27 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:39:27.322 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:27 smithi037 ceph-mon[31211]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:27.383 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:27 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6834/2848315285,v1:172.21.15.151:6835/2848315285] up:boot 2024-09-19T06:39:27.383 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:27 smithi151 ceph-mon[35236]: fsmap 6 up:standby 2024-09-19T06:39:27.383 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:27 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:39:27.383 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:27 smithi151 ceph-mon[35236]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:27.780 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:27 smithi151 systemd[1]: Starting Ceph mds.c for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:27.865 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:28.070 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:27 smithi151 podman[65952]: 2024-09-19 06:39:27.681928709 +0000 UTC m=+0.016072991 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:28.071 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:27 smithi151 podman[65952]: 2024-09-19 06:39:27.817341628 +0000 UTC m=+0.151485890 container create 90aecbb0c8243b678f9d94666ef9758f500b30a1b1816bd6f7ddab01eff1a0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:39:28.071 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:28 smithi151 podman[65952]: 2024-09-19 06:39:28.069980666 +0000 UTC m=+0.404124928 container init 90aecbb0c8243b678f9d94666ef9758f500b30a1b1816bd6f7ddab01eff1a0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:39:28.128 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:28.363 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:28 smithi151 podman[65952]: 2024-09-19 06:39:28.073036662 +0000 UTC m=+0.407180933 container start 90aecbb0c8243b678f9d94666ef9758f500b30a1b1816bd6f7ddab01eff1a0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4) 2024-09-19T06:39:28.363 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:28 smithi151 ceph-mds[65971]: -- 172.21.15.151:0/1896006140 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559e88483860 con 0x559e88461400 2024-09-19T06:39:28.363 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c[65967]: starting mds.c at 2024-09-19T06:39:28.363 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:28 smithi151 bash[65952]: 90aecbb0c8243b678f9d94666ef9758f500b30a1b1816bd6f7ddab01eff1a0d4 2024-09-19T06:39:28.363 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:39:28 smithi151 systemd[1]: Started Ceph mds.c for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:28.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 -- 172.21.15.37:0/1544569212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac100a80 msgr2=0x7f61ac100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:28.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/1544569212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac100a80 0x7f61ac100ee0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6194009930 tx=0x7f619402a420 comp rx=0 tx=0).stop 2024-09-19T06:39:28.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 -- 172.21.15.37:0/1544569212 shutdown_connections 2024-09-19T06:39:28.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/1544569212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61ac101420 0x7f61ac10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/1544569212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac100a80 0x7f61ac100ee0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6194009930 tx=0x7f619402a420 comp rx=0 tx=0).stop 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/1544569212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac106a80 0x7f61ac106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 -- 172.21.15.37:0/1544569212 >> 172.21.15.37:0/1544569212 conn(0x7f61ac0fc7a0 msgr2=0x7f61ac0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 -- 172.21.15.37:0/1544569212 shutdown_connections 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.644+0000 7f61b0c7b640 1 -- 172.21.15.37:0/1544569212 wait complete. 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 Processor -- start 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 -- start start 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac101420 0x7f61ac199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61ac106a80 0x7f61ac199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac199f80 0x7f61ac1a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac110e80 con 0x7f61ac199f80 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac110d00 con 0x7f61ac101420 2024-09-19T06:39:28.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61b0c7b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61ac111000 con 0x7f61ac106a80 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.645+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac101420 0x7f61ac199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac101420 0x7f61ac199500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43888/0 (socket says 172.21.15.37:43888) 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 -- 172.21.15.37:0/3299316351 learned_addr learned my addr 172.21.15.37:0/3299316351 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aad9d640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac199f80 0x7f61ac1a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61a1d9b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61ac106a80 0x7f61ac199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 -- 172.21.15.37:0/3299316351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61ac106a80 msgr2=0x7f61ac199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61ac106a80 0x7f61ac199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 -- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac199f80 msgr2=0x7f61ac1a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac199f80 0x7f61ac1a4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 -- 172.21.15.37:0/3299316351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61ac1a4d50 con 0x7f61ac101420 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61aa59c640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac101420 0x7f61ac199500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9f0 tx=0x7f619c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61a3fff640 1 -- 172.21.15.37:0/3299316351 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cd80 con 0x7f61ac101420 2024-09-19T06:39:28.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61a3fff640 1 -- 172.21.15.37:0/3299316351 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f619c010910 con 0x7f61ac101420 2024-09-19T06:39:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61ac1a4fe0 con 0x7f61ac101420 2024-09-19T06:39:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61a3fff640 1 -- 172.21.15.37:0/3299316351 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c018990 con 0x7f61ac101420 2024-09-19T06:39:28.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.646+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61ac1a54a0 con 0x7f61ac101420 2024-09-19T06:39:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.647+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61ac100ee0 con 0x7f61ac101420 2024-09-19T06:39:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.649+0000 7f61a3fff640 1 -- 172.21.15.37:0/3299316351 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f619c010ac0 con 0x7f61ac101420 2024-09-19T06:39:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.649+0000 7f61a3fff640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f61980a3240 0x7f61980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.649+0000 7f61a1d9b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f61980a3240 0x7f61980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.649+0000 7f61a3fff640 1 -- 172.21.15.37:0/3299316351 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f619c0cfbb0 con 0x7f61ac101420 2024-09-19T06:39:28.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.650+0000 7f61a1d9b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f61980a3240 0x7f61980a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6194009230 tx=0x7f61940314b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:28.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.652+0000 7f61a3fff640 1 -- 172.21.15.37:0/3299316351 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f619c09cd70 con 0x7f61ac101420 2024-09-19T06:39:28.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.863+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61ac06ad80 con 0x7f61ac101420 2024-09-19T06:39:28.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.864+0000 7f61a3fff640 1 -- 172.21.15.37:0/3299316351 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6382 (secure 0 0 0) 0x7f619c09c670 con 0x7f61ac101420 2024-09-19T06:39:28.866 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:28.866 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":8,"btime":"2024-09-19T06:39:28:161752+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-19T06:39:28.867 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 8 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f61980a3240 msgr2=0x7f61980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f61980a3240 0x7f61980a5700 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f6194009230 tx=0x7f61940314b0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac101420 msgr2=0x7f61ac199500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac101420 0x7f61ac199500 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9f0 tx=0x7f619c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 shutdown_connections 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f61980a3240 0x7f61980a5700 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61ac199f80 0x7f61ac1a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61ac106a80 0x7f61ac199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 --2- 172.21.15.37:0/3299316351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61ac101420 0x7f61ac199500 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 >> 172.21.15.37:0/3299316351 conn(0x7f61ac0fc7a0 msgr2=0x7f61ac0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:28.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 shutdown_connections 2024-09-19T06:39:28.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:28.867+0000 7f61b0c7b640 1 -- 172.21.15.37:0/3299316351 wait complete. 2024-09-19T06:39:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6836/3741792814,v1:172.21.15.151:6837/3741792814] up:boot 2024-09-19T06:39:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: fsmap 7 up:standby 2024-09-19T06:39:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:39:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:29.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: Deploying daemon mds.h on smithi096 2024-09-19T06:39:29.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:29 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3299316351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6836/3741792814,v1:172.21.15.151:6837/3741792814] up:boot 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: fsmap 7 up:standby 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:29.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:29.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: Deploying daemon mds.h on smithi096 2024-09-19T06:39:29.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:29 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3299316351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6836/3741792814,v1:172.21.15.151:6837/3741792814] up:boot 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: fsmap 7 up:standby 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: Deploying daemon mds.h on smithi096 2024-09-19T06:39:29.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:29 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3299316351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:30.274 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 systemd[1]: Starting Ceph mds.h for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:30.730 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 podman[64691]: 2024-09-19 06:39:30.320609694 +0000 UTC m=+0.015802819 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:30.730 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 podman[64691]: 2024-09-19 06:39:30.442883411 +0000 UTC m=+0.138076526 container create 1cfd244352b2bb005166f81a181d922563ad6ce6e0b026e1c536e30167794b1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:39:30.730 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 podman[64691]: 2024-09-19 06:39:30.72939191 +0000 UTC m=+0.424585022 container init 1cfd244352b2bb005166f81a181d922563ad6ce6e0b026e1c536e30167794b1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-19T06:39:31.024 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 podman[64691]: 2024-09-19 06:39:30.732151413 +0000 UTC m=+0.427344536 container start 1cfd244352b2bb005166f81a181d922563ad6ce6e0b026e1c536e30167794b1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T06:39:31.025 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 ceph-mds[64710]: -- 172.21.15.96:0/3348535698 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f644ded860 con 0x55f644dcb000 2024-09-19T06:39:31.025 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h[64706]: starting mds.h at 2024-09-19T06:39:31.025 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 bash[64691]: 1cfd244352b2bb005166f81a181d922563ad6ce6e0b026e1c536e30167794b1c 2024-09-19T06:39:31.025 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:39:30 smithi096 systemd[1]: Started Ceph mds.h for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:31.595 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:31.864 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:32.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:32.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:32.157 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:32.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:32.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: Deploying daemon mds.i on smithi151 2024-09-19T06:39:32.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] up:boot 2024-09-19T06:39:32.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: fsmap 8 up:standby 2024-09-19T06:39:32.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:31 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:39:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:32.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: Deploying daemon mds.i on smithi151 2024-09-19T06:39:32.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] up:boot 2024-09-19T06:39:32.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: fsmap 8 up:standby 2024-09-19T06:39:32.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:31 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:39:32.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:32.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:32.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:32.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:32.345 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:32.346 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: Deploying daemon mds.i on smithi151 2024-09-19T06:39:32.346 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] up:boot 2024-09-19T06:39:32.346 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: fsmap 8 up:standby 2024-09-19T06:39:32.346 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:31 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:39:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/3976420608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 msgr2=0x7f2ae41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/3976420608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41021c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0009900 tx=0x7f2ae002a410 comp rx=0 tx=0).stop 2024-09-19T06:39:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/3976420608 shutdown_connections 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/3976420608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ae4106f10 0x7f2ae410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/3976420608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae4102790 0x7f2ae41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/3976420608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41021c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/3976420608 >> 172.21.15.37:0/3976420608 conn(0x7f2ae40fd8c0 msgr2=0x7f2ae40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/3976420608 shutdown_connections 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.367+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/3976420608 wait complete. 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 Processor -- start 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 -- start start 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ae4102790 0x7f2ae4198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae4106f10 0x7f2ae419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4112060 con 0x7f2ae4101de0 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae4111ee0 con 0x7f2ae4106f10 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2aeb1c4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ae41121e0 con 0x7f2ae4102790 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47392/0 (socket says 172.21.15.37:47392) 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 -- 172.21.15.37:0/862370743 learned_addr learned my addr 172.21.15.37:0/862370743 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2adbfff640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ae4102790 0x7f2ae4198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 -- 172.21.15.37:0/862370743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ae4102790 msgr2=0x7f2ae4198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ae4102790 0x7f2ae4198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 -- 172.21.15.37:0/862370743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae4106f10 msgr2=0x7f2ae419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae9761640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae4106f10 0x7f2ae419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae4106f10 0x7f2ae419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 -- 172.21.15.37:0/862370743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ae419d570 con 0x7f2ae4101de0 2024-09-19T06:39:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.368+0000 7f2ae8f60640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41985c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0006010 tx=0x7f2ae0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.369+0000 7f2ad9ffb640 1 -- 172.21.15.37:0/862370743 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae003f070 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.369+0000 7f2ad9ffb640 1 -- 172.21.15.37:0/862370743 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ae0031c90 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.369+0000 7f2ad9ffb640 1 -- 172.21.15.37:0/862370743 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0035030 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.369+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ae419d800 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.369+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ae419dc90 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.370+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ae4102240 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.371+0000 7f2ad9ffb640 1 -- 172.21.15.37:0/862370743 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2ae0042070 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.372+0000 7f2ad9ffb640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2acc0a3240 0x7f2acc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.372+0000 7f2adbfff640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2acc0a3240 0x7f2acc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.372+0000 7f2ad9ffb640 1 -- 172.21.15.37:0/862370743 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f2ae00efea0 con 0x7f2ae4101de0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.373+0000 7f2adbfff640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2acc0a3240 0x7f2acc0a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40fdce0 tx=0x7f2ad4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.376+0000 7f2ad9ffb640 1 -- 172.21.15.37:0/862370743 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2ae00bcfc0 con 0x7f2ae4101de0 2024-09-19T06:39:32.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.590+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2ae4109b60 con 0x7f2ae4101de0 2024-09-19T06:39:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.590+0000 7f2ad9ffb640 1 -- 172.21.15.37:0/862370743 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7207 (secure 0 0 0) 0x7f2ae00bc8c0 con 0x7f2ae4101de0 2024-09-19T06:39:32.592 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:32.592 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":9,"btime":"2024-09-19T06:39:31:239118+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T06:39:32.593 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 9 2024-09-19T06:39:32.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2acc0a3240 msgr2=0x7f2acc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2acc0a3240 0x7f2acc0a5700 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f2ae40fdce0 tx=0x7f2ad4008040 comp rx=0 tx=0).stop 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 msgr2=0x7f2ae41985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41985c0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ae0006010 tx=0x7f2ae0003ba0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 shutdown_connections 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2acc0a3240 0x7f2acc0a5700 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ae4106f10 0x7f2ae419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ae4102790 0x7f2ae4198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 --2- 172.21.15.37:0/862370743 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ae4101de0 0x7f2ae41985c0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.592+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 >> 172.21.15.37:0/862370743 conn(0x7f2ae40fd8c0 msgr2=0x7f2ae40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.593+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 shutdown_connections 2024-09-19T06:39:32.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:32.593+0000 7f2aeb1c4640 1 -- 172.21.15.37:0/862370743 wait complete. 2024-09-19T06:39:32.897 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:32 smithi151 systemd[1]: Starting Ceph mds.i for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:33.168 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:32 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/862370743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:33.168 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 podman[66251]: 2024-09-19 06:39:32.934259602 +0000 UTC m=+0.016123317 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:33.168 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 podman[66251]: 2024-09-19 06:39:33.04697226 +0000 UTC m=+0.128836023 container create b5479bcc48463c4a5ffb7905b3b86934b507b9cc444cedc6b87fe678b730c089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-19T06:39:33.234 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:32 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/862370743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:33.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:32 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/862370743' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 podman[66251]: 2024-09-19 06:39:33.291294048 +0000 UTC m=+0.373157762 container init b5479bcc48463c4a5ffb7905b3b86934b507b9cc444cedc6b87fe678b730c089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i, ceph=True, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T06:39:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 podman[66251]: 2024-09-19 06:39:33.294264796 +0000 UTC m=+0.376128508 container start b5479bcc48463c4a5ffb7905b3b86934b507b9cc444cedc6b87fe678b730c089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T06:39:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mds[66270]: -- 172.21.15.151:0/2940210275 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c274597860 con 0x55c274575000 2024-09-19T06:39:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: starting mds.i at 2024-09-19T06:39:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 bash[66251]: b5479bcc48463c4a5ffb7905b3b86934b507b9cc444cedc6b87fe678b730c089 2024-09-19T06:39:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:39:33 smithi151 systemd[1]: Started Ceph mds.i for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:34.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:33 smithi037 ceph-mon[31211]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:34.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:34.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:34.255 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:33 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:33 smithi096 ceph-mon[35004]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:33 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mon[35236]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:34.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:33 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:35.202 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:34 smithi037 ceph-mon[31211]: Deploying daemon mds.g on smithi037 2024-09-19T06:39:35.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:34 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6838/3546164673,v1:172.21.15.151:6839/3546164673] up:boot 2024-09-19T06:39:35.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:34 smithi037 ceph-mon[31211]: fsmap 9 up:standby 2024-09-19T06:39:35.203 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:34 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:39:35.267 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:35.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:34 smithi096 ceph-mon[35004]: Deploying daemon mds.g on smithi037 2024-09-19T06:39:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:34 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6838/3546164673,v1:172.21.15.151:6839/3546164673] up:boot 2024-09-19T06:39:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:34 smithi096 ceph-mon[35004]: fsmap 9 up:standby 2024-09-19T06:39:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:34 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:39:35.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:34 smithi151 ceph-mon[35236]: Deploying daemon mds.g on smithi037 2024-09-19T06:39:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:34 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6838/3546164673,v1:172.21.15.151:6839/3546164673] up:boot 2024-09-19T06:39:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:34 smithi151 ceph-mon[35236]: fsmap 9 up:standby 2024-09-19T06:39:35.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:34 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:39:35.525 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 systemd[1]: Starting Ceph mds.g for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:35.882 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 podman[70996]: 2024-09-19 06:39:35.512110601 +0000 UTC m=+0.018555522 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:35.882 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 podman[70996]: 2024-09-19 06:39:35.636314443 +0000 UTC m=+0.142759345 container create 5bf2c8397de1a3cd63c0bb3e71360400edb254bb9e3c2129ffffa3cf27b798e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:39:35.882 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 podman[70996]: 2024-09-19 06:39:35.88117296 +0000 UTC m=+0.387617875 container init 5bf2c8397de1a3cd63c0bb3e71360400edb254bb9e3c2129ffffa3cf27b798e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:39:36.081 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:36.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:35 smithi096 ceph-mon[35004]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:36.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:35 smithi037 ceph-mon[31211]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:36.204 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 podman[70996]: 2024-09-19 06:39:35.884684997 +0000 UTC m=+0.391129899 container start 5bf2c8397de1a3cd63c0bb3e71360400edb254bb9e3c2129ffffa3cf27b798e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-19T06:39:36.204 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 ceph-mds[71028]: -- 172.21.15.37:0/2060117943 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5646b3493860 con 0x5646b3471000 2024-09-19T06:39:36.204 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g[71024]: starting mds.g at 2024-09-19T06:39:36.204 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 bash[70996]: 5bf2c8397de1a3cd63c0bb3e71360400edb254bb9e3c2129ffffa3cf27b798e8 2024-09-19T06:39:36.204 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:39:35 smithi037 systemd[1]: Started Ceph mds.g for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:35 smithi151 ceph-mon[35236]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 -- 172.21.15.37:0/3096198482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 msgr2=0x7f83e81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 --2- 172.21.15.37:0/3096198482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81021c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009900 tx=0x7f83e402a410 comp rx=0 tx=0).stop 2024-09-19T06:39:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 -- 172.21.15.37:0/3096198482 shutdown_connections 2024-09-19T06:39:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 --2- 172.21.15.37:0/3096198482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83e8106f10 0x7f83e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 --2- 172.21.15.37:0/3096198482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e8102790 0x7f83e81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 --2- 172.21.15.37:0/3096198482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81021c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 -- 172.21.15.37:0/3096198482 >> 172.21.15.37:0/3096198482 conn(0x7f83e80fd8c0 msgr2=0x7f83e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 -- 172.21.15.37:0/3096198482 shutdown_connections 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.568+0000 7f83f056a640 1 -- 172.21.15.37:0/3096198482 wait complete. 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.569+0000 7f83f056a640 1 Processor -- start 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.569+0000 7f83f056a640 1 -- start start 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83f056a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83f056a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83e8102790 0x7f83e8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83f056a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e8106f10 0x7f83e819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83f056a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8111f70 con 0x7f83e8101de0 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83f056a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f83e8111df0 con 0x7f83e8106f10 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83f056a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f83e81120f0 con 0x7f83e8102790 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47426/0 (socket says 172.21.15.37:47426) 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 -- 172.21.15.37:0/2753386572 learned_addr learned my addr 172.21.15.37:0/2753386572 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:36.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83eeb07640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e8106f10 0x7f83e819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83edb05640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83e8102790 0x7f83e8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 -- 172.21.15.37:0/2753386572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83e8102790 msgr2=0x7f83e8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83e8102790 0x7f83e8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 -- 172.21.15.37:0/2753386572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e8106f10 msgr2=0x7f83e819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e8106f10 0x7f83e819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 -- 172.21.15.37:0/2753386572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83e819d480 con 0x7f83e8101de0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83edb05640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83e8102790 0x7f83e8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83eeb07640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e8106f10 0x7f83e819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83ee306640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81984d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83e4006010 tx=0x7f83e4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.570+0000 7f83df7fe640 1 -- 172.21.15.37:0/2753386572 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e403f030 con 0x7f83e8101de0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.571+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83e819d710 con 0x7f83e8101de0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.571+0000 7f83df7fe640 1 -- 172.21.15.37:0/2753386572 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f83e4004440 con 0x7f83e8101de0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.571+0000 7f83df7fe640 1 -- 172.21.15.37:0/2753386572 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e4035070 con 0x7f83e8101de0 2024-09-19T06:39:36.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.571+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83e81a5050 con 0x7f83e8101de0 2024-09-19T06:39:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.572+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83e81021c0 con 0x7f83e8101de0 2024-09-19T06:39:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.573+0000 7f83df7fe640 1 -- 172.21.15.37:0/2753386572 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f83e4042030 con 0x7f83e8101de0 2024-09-19T06:39:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.574+0000 7f83df7fe640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83d40a3240 0x7f83d40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.574+0000 7f83edb05640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83d40a3240 0x7f83d40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.574+0000 7f83df7fe640 1 -- 172.21.15.37:0/2753386572 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f83e40ef880 con 0x7f83e8101de0 2024-09-19T06:39:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.574+0000 7f83edb05640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83d40a3240 0x7f83d40a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83e8108900 tx=0x7f83d8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:36.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.578+0000 7f83df7fe640 1 -- 172.21.15.37:0/2753386572 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f83e40bca40 con 0x7f83e8101de0 2024-09-19T06:39:36.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.804+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83e810a0f0 con 0x7f83e8101de0 2024-09-19T06:39:36.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.805+0000 7f83df7fe640 1 -- 172.21.15.37:0/2753386572 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8037 (secure 0 0 0) 0x7f83e40bc340 con 0x7f83e8101de0 2024-09-19T06:39:36.807 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:36.807 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":10,"btime":"2024-09-19T06:39:33:895162+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T06:39:36.807 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 10 2024-09-19T06:39:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83d40a3240 msgr2=0x7f83d40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83d40a3240 0x7f83d40a5700 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83e8108900 tx=0x7f83d8005f90 comp rx=0 tx=0).stop 2024-09-19T06:39:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 msgr2=0x7f83e81984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81984d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f83e4006010 tx=0x7f83e4004040 comp rx=0 tx=0).stop 2024-09-19T06:39:36.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 shutdown_connections 2024-09-19T06:39:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f83d40a3240 0x7f83d40a5700 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f83e8106f10 0x7f83e819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f83e8102790 0x7f83e8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 --2- 172.21.15.37:0/2753386572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f83e8101de0 0x7f83e81984d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 >> 172.21.15.37:0/2753386572 conn(0x7f83e80fd8c0 msgr2=0x7f83e80fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 shutdown_connections 2024-09-19T06:39:36.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:36.807+0000 7f83f056a640 1 -- 172.21.15.37:0/2753386572 wait complete. 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: Deploying daemon mds.a on smithi037 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2753386572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.37:6838/2512161212,v1:172.21.15.37:6839/2512161212] up:boot 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: fsmap 10 up:standby 2024-09-19T06:39:37.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:37 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:39:37.403 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.403 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.403 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.403 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:37.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:37.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:37.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: Deploying daemon mds.a on smithi037 2024-09-19T06:39:37.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2753386572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:37.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.37:6838/2512161212,v1:172.21.15.37:6839/2512161212] up:boot 2024-09-19T06:39:37.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: fsmap 10 up:standby 2024-09-19T06:39:37.405 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:37 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:39:37.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:37.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:37.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:37.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:37.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: Deploying daemon mds.a on smithi037 2024-09-19T06:39:37.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2753386572' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:37.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.37:6838/2512161212,v1:172.21.15.37:6839/2512161212] up:boot 2024-09-19T06:39:37.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: fsmap 10 up:standby 2024-09-19T06:39:37.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:37 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:39:38.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:38 smithi037 ceph-mon[31211]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:38.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:38 smithi151 ceph-mon[35236]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:38 smithi096 ceph-mon[35004]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:38.525 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:38 smithi037 systemd[1]: Starting Ceph mds.a for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:38.967 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:38 smithi037 podman[71395]: 2024-09-19 06:39:38.609118186 +0000 UTC m=+0.017250757 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:38.967 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:38 smithi037 podman[71395]: 2024-09-19 06:39:38.709945992 +0000 UTC m=+0.118078528 container create df1b81e71043325d19cedf1f3563481210d455512a5b62f121e7ced855e95b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T06:39:38.967 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:38 smithi037 podman[71395]: 2024-09-19 06:39:38.937709746 +0000 UTC m=+0.345842294 container init df1b81e71043325d19cedf1f3563481210d455512a5b62f121e7ced855e95b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T06:39:38.968 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:38 smithi037 podman[71395]: 2024-09-19 06:39:38.940918752 +0000 UTC m=+0.349051289 container start df1b81e71043325d19cedf1f3563481210d455512a5b62f121e7ced855e95b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-19T06:39:38.968 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:38 smithi037 ceph-mds[71414]: -- 172.21.15.37:0/3932689371 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfb5307860 con 0x55dfb52e5000 2024-09-19T06:39:39.275 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:38 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a[71410]: starting mds.a at 2024-09-19T06:39:39.275 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:39 smithi037 bash[71395]: df1b81e71043325d19cedf1f3563481210d455512a5b62f121e7ced855e95b94 2024-09-19T06:39:39.275 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:39:39 smithi037 systemd[1]: Started Ceph mds.a for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.37:6840/2995019412,v1:172.21.15.37:6841/2995019412] up:boot 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: fsmap 11 up:standby 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:40.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:40.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:40.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:40 smithi151 ceph-mon[35236]: Deploying daemon mds.b on smithi096 2024-09-19T06:39:40.475 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:40.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:40.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.37:6840/2995019412,v1:172.21.15.37:6841/2995019412] up:boot 2024-09-19T06:39:40.506 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: fsmap 11 up:standby 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:40.507 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:40 smithi037 ceph-mon[31211]: Deploying daemon mds.b on smithi096 2024-09-19T06:39:40.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:40.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.37:6840/2995019412,v1:172.21.15.37:6841/2995019412] up:boot 2024-09-19T06:39:40.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: fsmap 11 up:standby 2024-09-19T06:39:40.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:39:40.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:40.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-19T06:39:40.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-19T06:39:40.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:40.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:40 smithi096 ceph-mon[35004]: Deploying daemon mds.b on smithi096 2024-09-19T06:39:40.749 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:41.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.224+0000 7faaddafe640 1 -- 172.21.15.37:0/1106323471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8106f10 msgr2=0x7faad810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:41.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.224+0000 7faaddafe640 1 --2- 172.21.15.37:0/1106323471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8106f10 0x7faad810ea90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faad0009ca0 tx=0x7faad002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 -- 172.21.15.37:0/1106323471 shutdown_connections 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 --2- 172.21.15.37:0/1106323471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8106f10 0x7faad810ea90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 --2- 172.21.15.37:0/1106323471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faad8102790 0x7faad81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 --2- 172.21.15.37:0/1106323471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faad8101de0 0x7faad81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 -- 172.21.15.37:0/1106323471 >> 172.21.15.37:0/1106323471 conn(0x7faad80fd8c0 msgr2=0x7faad80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 -- 172.21.15.37:0/1106323471 shutdown_connections 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 -- 172.21.15.37:0/1106323471 wait complete. 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 Processor -- start 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.225+0000 7faaddafe640 1 -- start start 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faaddafe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8101de0 0x7faad81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faaddafe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faad8102790 0x7faad8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faaddafe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faad8106f10 0x7faad819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faaddafe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faad8111f70 con 0x7faad8101de0 2024-09-19T06:39:41.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faaddafe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faad8111df0 con 0x7faad8102790 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faaddafe640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faad81120f0 con 0x7faad8106f10 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8101de0 0x7faad81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8101de0 0x7faad81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59980/0 (socket says 172.21.15.37:59980) 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 -- 172.21.15.37:0/2999184103 learned_addr learned my addr 172.21.15.37:0/2999184103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad7fff640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faad8106f10 0x7faad819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faacffff640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faad8102790 0x7faad8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 -- 172.21.15.37:0/2999184103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faad8106f10 msgr2=0x7faad819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faad8106f10 0x7faad819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 -- 172.21.15.37:0/2999184103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faad8102790 msgr2=0x7faad8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faad8102790 0x7faad8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 -- 172.21.15.37:0/2999184103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad819d3a0 con 0x7faad8101de0 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad7fff640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faad8106f10 0x7faad819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:41.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faacffff640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faad8102790 0x7faad8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad77fe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8101de0 0x7faad81983f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faac800c890 tx=0x7faac800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad57fa640 1 -- 172.21.15.37:0/2999184103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8004220 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad819d690 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad57fa640 1 -- 172.21.15.37:0/2999184103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faac80043c0 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.226+0000 7faad57fa640 1 -- 172.21.15.37:0/2999184103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faac8004a00 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.227+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad81a4f70 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.227+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad81021c0 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.229+0000 7faad57fa640 1 -- 172.21.15.37:0/2999184103 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7faac80079b0 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.229+0000 7faad57fa640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faac40a3240 0x7faac40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.229+0000 7faad57fa640 1 -- 172.21.15.37:0/2999184103 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7faac80cf220 con 0x7faad8101de0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.229+0000 7faacffff640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faac40a3240 0x7faac40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.230+0000 7faacffff640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faac40a3240 0x7faac40a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faad80fdce0 tx=0x7faac0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:41.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.233+0000 7faad57fa640 1 -- 172.21.15.37:0/2999184103 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faac8014030 con 0x7faad8101de0 2024-09-19T06:39:41.274 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 systemd[1]: Starting Ceph mds.b for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:39:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.445+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faad8109bc0 con 0x7faad8101de0 2024-09-19T06:39:41.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.446+0000 7faad57fa640 1 -- 172.21.15.37:0/2999184103 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9689 (secure 0 0 0) 0x7faac809c3e0 con 0x7faad8101de0 2024-09-19T06:39:41.447 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:41.448 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":12,"btime":"2024-09-19T06:39:39:064404+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-19T06:39:41.448 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 12 2024-09-19T06:39:41.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faac40a3240 msgr2=0x7faac40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faac40a3240 0x7faac40a5700 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7faad80fdce0 tx=0x7faac0005f90 comp rx=0 tx=0).stop 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8101de0 msgr2=0x7faad81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8101de0 0x7faad81983f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7faac800c890 tx=0x7faac800cd50 comp rx=0 tx=0).stop 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 shutdown_connections 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faac40a3240 0x7faac40a5700 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faad8106f10 0x7faad819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faad8102790 0x7faad8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 --2- 172.21.15.37:0/2999184103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faad8101de0 0x7faad81983f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 >> 172.21.15.37:0/2999184103 conn(0x7faad80fd8c0 msgr2=0x7faad80feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 shutdown_connections 2024-09-19T06:39:41.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:41.448+0000 7faaddafe640 1 -- 172.21.15.37:0/2999184103 wait complete. 2024-09-19T06:39:41.574 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 podman[65008]: 2024-09-19 06:39:41.216247288 +0000 UTC m=+0.015597054 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:39:41.574 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 podman[65008]: 2024-09-19 06:39:41.321445405 +0000 UTC m=+0.120795162 container create a47f4a9cf11ade7e02038632d804f2635382e584001edf63fae0e0ba0b38e4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:39:41.575 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 podman[65008]: 2024-09-19 06:39:41.574237212 +0000 UTC m=+0.373586977 container init a47f4a9cf11ade7e02038632d804f2635382e584001edf63fae0e0ba0b38e4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T06:39:42.024 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 podman[65008]: 2024-09-19 06:39:41.577092906 +0000 UTC m=+0.376442663 container start a47f4a9cf11ade7e02038632d804f2635382e584001edf63fae0e0ba0b38e4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T06:39:42.024 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 ceph-mds[65027]: -- 172.21.15.96:0/3578397747 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e4d994d860 con 0x55e4d992b000 2024-09-19T06:39:42.024 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: starting mds.b at 2024-09-19T06:39:42.024 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 bash[65008]: a47f4a9cf11ade7e02038632d804f2635382e584001edf63fae0e0ba0b38e4ec 2024-09-19T06:39:42.024 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:41 smithi096 systemd[1]: Started Ceph mds.b for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:39:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2999184103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:42 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2999184103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.474 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:42 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2999184103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:42 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:43.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:43 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] up:boot 2024-09-19T06:39:43.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:43 smithi151 ceph-mon[35236]: fsmap 12 up:standby 2024-09-19T06:39:43.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:43 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:39:43.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:43 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] up:boot 2024-09-19T06:39:43.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:43 smithi096 ceph-mon[35004]: fsmap 12 up:standby 2024-09-19T06:39:43.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:43 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:39:43.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:43 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] up:boot 2024-09-19T06:39:43.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:43 smithi037 ceph-mon[31211]: fsmap 12 up:standby 2024-09-19T06:39:43.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:43 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:39:44.139 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:39:44.353 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:44.353 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.353 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.353 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:44.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:44.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.354 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:44 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:44.384 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:44.384 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.384 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.384 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.384 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.384 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.385 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.385 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:44.385 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:44.385 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.385 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:44 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:44.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:44.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.388 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.389 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.389 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:44.389 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:44.389 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:44.389 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:44 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:39:44.874 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:45.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 -- 172.21.15.37:0/876321669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74102790 msgr2=0x7efe741067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:45.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 --2- 172.21.15.37:0/876321669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74102790 0x7efe741067e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe6802a410 comp rx=0 tx=0).stop 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 -- 172.21.15.37:0/876321669 shutdown_connections 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 --2- 172.21.15.37:0/876321669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74106f10 0x7efe7410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 --2- 172.21.15.37:0/876321669 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74102790 0x7efe741067e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 --2- 172.21.15.37:0/876321669 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe74101de0 0x7efe741021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 -- 172.21.15.37:0/876321669 >> 172.21.15.37:0/876321669 conn(0x7efe740fd8c0 msgr2=0x7efe740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 -- 172.21.15.37:0/876321669 shutdown_connections 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.401+0000 7efe789d9640 1 -- 172.21.15.37:0/876321669 wait complete. 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 Processor -- start 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 -- start start 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74101de0 0x7efe740744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:45.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe74102790 0x7efe74074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74106f10 0x7efe74074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe74111fb0 con 0x7efe74101de0 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe74111e30 con 0x7efe74106f10 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe789d9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe74112130 con 0x7efe74102790 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe7259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74101de0 0x7efe740744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe7259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74101de0 0x7efe740744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59998/0 (socket says 172.21.15.37:59998) 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.402+0000 7efe7259c640 1 -- 172.21.15.37:0/3259026193 learned_addr learned my addr 172.21.15.37:0/3259026193 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe71d9b640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe74102790 0x7efe74074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe72d9d640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74106f10 0x7efe74074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe7259c640 1 -- 172.21.15.37:0/3259026193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe74102790 msgr2=0x7efe74074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe7259c640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe74102790 0x7efe74074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe7259c640 1 -- 172.21.15.37:0/3259026193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74106f10 msgr2=0x7efe74074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:45.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe7259c640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74106f10 0x7efe74074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe7259c640 1 -- 172.21.15.37:0/3259026193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe741a6970 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe72d9d640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74106f10 0x7efe74074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe7259c640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74101de0 0x7efe740744f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c970 tx=0x7efe6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe637fe640 1 -- 172.21.15.37:0/3259026193 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe64007bb0 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe637fe640 1 -- 172.21.15.37:0/3259026193 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe64007d50 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe741a6c60 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe637fe640 1 -- 172.21.15.37:0/3259026193 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6400f450 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.403+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe741a7150 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.404+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe741021c0 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.406+0000 7efe637fe640 1 -- 172.21.15.37:0/3259026193 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efe64004fe0 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.406+0000 7efe637fe640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efe5c0a3240 0x7efe5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.407+0000 7efe71d9b640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efe5c0a3240 0x7efe5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.407+0000 7efe637fe640 1 -- 172.21.15.37:0/3259026193 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7efe640cf6c0 con 0x7efe74101de0 2024-09-19T06:39:45.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.407+0000 7efe71d9b640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efe5c0a3240 0x7efe5c0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe74075f80 tx=0x7efe68031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:45.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.410+0000 7efe637fe640 1 -- 172.21.15.37:0/3259026193 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efe64014030 con 0x7efe74101de0 2024-09-19T06:39:45.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.648+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe74075a20 con 0x7efe74101de0 2024-09-19T06:39:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.649+0000 7efe637fe640 1 -- 172.21.15.37:0/3259026193 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10515 (secure 0 0 0) 0x7efe6409c860 con 0x7efe74101de0 2024-09-19T06:39:45.651 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:45.651 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":13,"btime":"2024-09-19T06:39:42:072807+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24428,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24430,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24445,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24449,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24470,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24497,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-19T06:39:45.651 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 13 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efe5c0a3240 msgr2=0x7efe5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efe5c0a3240 0x7efe5c0a5700 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efe74075f80 tx=0x7efe68031490 comp rx=0 tx=0).stop 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74101de0 msgr2=0x7efe740744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74101de0 0x7efe740744f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c970 tx=0x7efe6400ce30 comp rx=0 tx=0).stop 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 shutdown_connections 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7efe5c0a3240 0x7efe5c0a5700 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe74106f10 0x7efe74074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe74102790 0x7efe74074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 --2- 172.21.15.37:0/3259026193 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe74101de0 0x7efe740744f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 >> 172.21.15.37:0/3259026193 conn(0x7efe740fd8c0 msgr2=0x7efe740feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 shutdown_connections 2024-09-19T06:39:45.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:45.652+0000 7efe789d9640 1 -- 172.21.15.37:0/3259026193 wait complete. 2024-09-19T06:39:46.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:46 smithi037 ceph-mon[31211]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:46.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:46 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:46 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:46 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:46 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:46 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3259026193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:46.319 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-19T06:39:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:46 smithi151 ceph-mon[35236]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:46 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:46 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:46 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:46 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:46 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3259026193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:46.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:46 smithi096 ceph-mon[35004]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:46.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:46 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:46 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:46 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:46 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:46.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:46 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3259026193' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:39:46.591 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:47.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 -- 172.21.15.37:0/3033637138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a04102d30 msgr2=0x7f6a04103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:47.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 --2- 172.21.15.37:0/3033637138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a04102d30 0x7f6a04103110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f69f4009cd0 tx=0x7f69f402a910 comp rx=0 tx=0).stop 2024-09-19T06:39:47.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 -- 172.21.15.37:0/3033637138 shutdown_connections 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 --2- 172.21.15.37:0/3033637138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a041082a0 0x7f6a0410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 --2- 172.21.15.37:0/3033637138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a041036e0 0x7f6a04107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 --2- 172.21.15.37:0/3033637138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a04102d30 0x7f6a04103110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 -- 172.21.15.37:0/3033637138 >> 172.21.15.37:0/3033637138 conn(0x7f6a040fc7d0 msgr2=0x7f6a040febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 -- 172.21.15.37:0/3033637138 shutdown_connections 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 -- 172.21.15.37:0/3033637138 wait complete. 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.051+0000 7f6a09786640 1 Processor -- start 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a09786640 1 -- start start 2024-09-19T06:39:47.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a09786640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a04102d30 0x7f6a04100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a09786640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a041036e0 0x7f6a04100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a09786640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a041082a0 0x7f6a04101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a09786640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a041120e0 con 0x7f6a041082a0 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a09786640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04111f60 con 0x7f6a04102d30 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a09786640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a04112260 con 0x7f6a041036e0 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a041082a0 0x7f6a04101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a041082a0 0x7f6a04101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60026/0 (socket says 172.21.15.37:60026) 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 -- 172.21.15.37:0/1127977077 learned_addr learned my addr 172.21.15.37:0/1127977077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a02ffd640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a04102d30 0x7f6a04100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a027fc640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a041036e0 0x7f6a04100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 -- 172.21.15.37:0/1127977077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a041036e0 msgr2=0x7f6a04100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:47.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a041036e0 0x7f6a04100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 -- 172.21.15.37:0/1127977077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a04102d30 msgr2=0x7f6a04100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a04102d30 0x7f6a04100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 -- 172.21.15.37:0/1127977077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a041a6ae0 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a027fc640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a041036e0 0x7f6a04100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.052+0000 7f6a037fe640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a041082a0 0x7f6a04101410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00d950 tx=0x7f69fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.053+0000 7f69ebfff640 1 -- 172.21.15.37:0/1127977077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc017030 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.053+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a041a6dd0 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.053+0000 7f69ebfff640 1 -- 172.21.15.37:0/1127977077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69fc004020 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.053+0000 7f69ebfff640 1 -- 172.21.15.37:0/1127977077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69fc003c00 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.053+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a041a7290 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.054+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a0406a850 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.055+0000 7f69ebfff640 1 -- 172.21.15.37:0/1127977077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f69fc01a070 con 0x7f6a041082a0 2024-09-19T06:39:47.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.055+0000 7f69ebfff640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f69ec0a3240 0x7f69ec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:47.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.056+0000 7f6a02ffd640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f69ec0a3240 0x7f69ec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:47.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.056+0000 7f69ebfff640 1 -- 172.21.15.37:0/1127977077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f69fc0cedd0 con 0x7f6a041082a0 2024-09-19T06:39:47.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.056+0000 7f6a02ffd640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f69ec0a3240 0x7f69ec0a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005fa0 tx=0x7f69f4005ef0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:47.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.060+0000 7f69ebfff640 1 -- 172.21.15.37:0/1127977077 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f69fc021070 con 0x7f6a041082a0 2024-09-19T06:39:47.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.239+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f6a0410a2b0 con 0x7f6a041082a0 2024-09-19T06:39:47.241 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:47 smithi037 ceph-mon[31211]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:47.241 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.241 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.241 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:47.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:47.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:47 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.239+0000 7f69ebfff640 1 -- 172.21.15.37:0/1127977077 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) ==== 84+0+65 (secure 0 0 0) 0x7f69fc09bf90 con 0x7f6a041082a0 2024-09-19T06:39:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:47.242 INFO:teuthology.orchestra.run.smithi037.stdout:[ 2024-09-19T06:39:47.242 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:39:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "poolnum": 1, 2024-09-19T06:39:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: "poolname": ".mgr" 2024-09-19T06:39:47.243 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:39:47.243 INFO:teuthology.orchestra.run.smithi037.stdout:] 2024-09-19T06:39:47.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f69ec0a3240 msgr2=0x7f69ec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f69ec0a3240 0x7f69ec0a5700 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f69f4005fa0 tx=0x7f69f4005ef0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a041082a0 msgr2=0x7f6a04101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a041082a0 0x7f6a04101410 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f69fc00d950 tx=0x7f69fc00de10 comp rx=0 tx=0).stop 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 shutdown_connections 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f69ec0a3240 0x7f69ec0a5700 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a041082a0 0x7f6a04101410 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a041036e0 0x7f6a04100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 --2- 172.21.15.37:0/1127977077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a04102d30 0x7f6a04100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 >> 172.21.15.37:0/1127977077 conn(0x7f6a040fc7d0 msgr2=0x7f6a04105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 shutdown_connections 2024-09-19T06:39:47.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:47.242+0000 7f6a09786640 1 -- 172.21.15.37:0/1127977077 wait complete. 2024-09-19T06:39:47.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:47 smithi096 ceph-mon[35004]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:47.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:47.475 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:47.476 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:47 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:47 smithi151 ceph-mon[35236]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:47.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:39:47.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:39:47.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:47 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:39:47.922 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-19T06:39:47.922 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-19T06:39:48.178 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:48.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:48 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1127977077' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:39:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:48 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1127977077' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:39:48.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:48 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1127977077' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-19T06:39:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 -- 172.21.15.37:0/972587199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528100a80 msgr2=0x7fc528100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 --2- 172.21.15.37:0/972587199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528100a80 0x7fc528100ee0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc518009930 tx=0x7fc51802a420 comp rx=0 tx=0).stop 2024-09-19T06:39:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 -- 172.21.15.37:0/972587199 shutdown_connections 2024-09-19T06:39:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 --2- 172.21.15.37:0/972587199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc528101420 0x7fc52810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 --2- 172.21.15.37:0/972587199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528100a80 0x7fc528100ee0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 --2- 172.21.15.37:0/972587199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528106a80 0x7fc528106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 -- 172.21.15.37:0/972587199 >> 172.21.15.37:0/972587199 conn(0x7fc5280fc7a0 msgr2=0x7fc5280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 -- 172.21.15.37:0/972587199 shutdown_connections 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 -- 172.21.15.37:0/972587199 wait complete. 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 Processor -- start 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.629+0000 7fc52eef2640 1 -- start start 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52eef2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc528100a80 0x7fc528102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52eef2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528101420 0x7fc528102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52eef2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528106a80 0x7fc5281030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52eef2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc528110f20 con 0x7fc528101420 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52eef2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc528110da0 con 0x7fc528100a80 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52eef2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc5281110a0 con 0x7fc528106a80 2024-09-19T06:39:48.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528101420 0x7fc528102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528101420 0x7fc528102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60042/0 (socket says 172.21.15.37:60042) 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 -- 172.21.15.37:0/659506580 learned_addr learned my addr 172.21.15.37:0/659506580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52d48f640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528106a80 0x7fc5281030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52cc8e640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc528100a80 0x7fc528102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 -- 172.21.15.37:0/659506580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528106a80 msgr2=0x7fc5281030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528106a80 0x7fc5281030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 -- 172.21.15.37:0/659506580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc528100a80 msgr2=0x7fc528102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc528100a80 0x7fc528102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 -- 172.21.15.37:0/659506580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5281a6a80 con 0x7fc528101420 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52cc8e640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc528100a80 0x7fc528102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:39:48.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc52d48f640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528106a80 0x7fc5281030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.630+0000 7fc51ffff640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528101420 0x7fc528102b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc518009a50 tx=0x7fc518004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.631+0000 7fc51dffb640 1 -- 172.21.15.37:0/659506580 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51803f030 con 0x7fc528101420 2024-09-19T06:39:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.631+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5281a6d10 con 0x7fc528101420 2024-09-19T06:39:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.631+0000 7fc51dffb640 1 -- 172.21.15.37:0/659506580 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc518003ce0 con 0x7fc528101420 2024-09-19T06:39:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.631+0000 7fc51dffb640 1 -- 172.21.15.37:0/659506580 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518035070 con 0x7fc528101420 2024-09-19T06:39:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.631+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5281a71a0 con 0x7fc528101420 2024-09-19T06:39:48.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.632+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc528100f60 con 0x7fc528101420 2024-09-19T06:39:48.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.633+0000 7fc51dffb640 1 -- 172.21.15.37:0/659506580 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc518042030 con 0x7fc528101420 2024-09-19T06:39:48.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.634+0000 7fc51dffb640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5100a3240 0x7fc5100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:48.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.634+0000 7fc51dffb640 1 -- 172.21.15.37:0/659506580 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7fc5180efab0 con 0x7fc528101420 2024-09-19T06:39:48.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.634+0000 7fc52cc8e640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5100a3240 0x7fc5100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:48.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.634+0000 7fc52cc8e640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5100a3240 0x7fc5100a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc524006fd0 tx=0x7fc524008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:48.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.638+0000 7fc51dffb640 1 -- 172.21.15.37:0/659506580 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc5180bcc70 con 0x7fc528101420 2024-09-19T06:39:48.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:48.815+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fc5281a73d0 con 0x7fc528101420 2024-09-19T06:39:49.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.298+0000 7fc51dffb640 1 -- 172.21.15.37:0/659506580 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) ==== 138+0+0 (secure 0 0 0) 0x7fc5180dc0b0 con 0x7fc528101420 2024-09-19T06:39:49.300 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' created 2024-09-19T06:39:49.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5100a3240 msgr2=0x7fc5100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:49.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5100a3240 0x7fc5100a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc524006fd0 tx=0x7fc524008040 comp rx=0 tx=0).stop 2024-09-19T06:39:49.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528101420 msgr2=0x7fc528102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:49.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528101420 0x7fc528102b90 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc518009a50 tx=0x7fc518004270 comp rx=0 tx=0).stop 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 shutdown_connections 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc5100a3240 0x7fc5100a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc528106a80 0x7fc5281030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc528101420 0x7fc528102b90 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 --2- 172.21.15.37:0/659506580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc528100a80 0x7fc528102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.300+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 >> 172.21.15.37:0/659506580 conn(0x7fc5280fc7a0 msgr2=0x7fc528104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.301+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 shutdown_connections 2024-09-19T06:39:49.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:49.301+0000 7fc52eef2640 1 -- 172.21.15.37:0/659506580 wait complete. 2024-09-19T06:39:49.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:49 smithi151 ceph-mon[35236]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:49.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:49 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/659506580' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:39:49.691 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:49 smithi037 ceph-mon[31211]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:49.691 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:49 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/659506580' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:39:49.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:49 smithi096 ceph-mon[35004]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:49.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:49 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/659506580' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-19T06:39:50.014 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-19T06:39:50.270 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:50.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:50 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/659506580' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:39:50.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:50 smithi151 ceph-mon[35236]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:39:50.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:50 smithi151 ceph-mon[35236]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:39:50.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:50 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/659506580' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:39:50.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:50 smithi037 ceph-mon[31211]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:39:50.662 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:50 smithi037 ceph-mon[31211]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:39:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 -- 172.21.15.37:0/1524230168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 msgr2=0x7f0d80106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 --2- 172.21.15.37:0/1524230168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80106f60 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0d78009900 tx=0x7f0d7802a410 comp rx=0 tx=0).stop 2024-09-19T06:39:50.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 -- 172.21.15.37:0/1524230168 shutdown_connections 2024-09-19T06:39:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 --2- 172.21.15.37:0/1524230168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d80100aa0 0x7f0d8010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 --2- 172.21.15.37:0/1524230168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d80100100 0x7f0d80100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 --2- 172.21.15.37:0/1524230168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80106f60 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 -- 172.21.15.37:0/1524230168 >> 172.21.15.37:0/1524230168 conn(0x7f0d80077230 msgr2=0x7f0d800fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:50.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 -- 172.21.15.37:0/1524230168 shutdown_connections 2024-09-19T06:39:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 -- 172.21.15.37:0/1524230168 wait complete. 2024-09-19T06:39:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 Processor -- start 2024-09-19T06:39:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.751+0000 7f0d8584d640 1 -- start start 2024-09-19T06:39:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d8584d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d80100100 0x7f0d801012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d8584d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d80100aa0 0x7f0d80101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:50.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d8584d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80101d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d8584d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80112110 con 0x7f0d80106b80 2024-09-19T06:39:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d8584d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80111f90 con 0x7f0d80100aa0 2024-09-19T06:39:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d8584d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d80112290 con 0x7f0d80100100 2024-09-19T06:39:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80101d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80101d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39208/0 (socket says 172.21.15.37:39208) 2024-09-19T06:39:50.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 -- 172.21.15.37:0/838767869 learned_addr learned my addr 172.21.15.37:0/838767869 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:50.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7e7fc640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d80100aa0 0x7f0d80101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:50.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7effd640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d80100100 0x7f0d801012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:50.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 -- 172.21.15.37:0/838767869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d80100100 msgr2=0x7f0d801012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:50.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d80100100 0x7f0d801012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:50.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 -- 172.21.15.37:0/838767869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d80100aa0 msgr2=0x7f0d80101820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:50.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d80100aa0 0x7f0d80101820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:50.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 -- 172.21.15.37:0/838767869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d801a6a50 con 0x7f0d80106b80 2024-09-19T06:39:50.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7e7fc640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d80100aa0 0x7f0d80101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:50.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.752+0000 7f0d7f7fe640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80101d60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b9e0 tx=0x7f0d7000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:50.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.753+0000 7f0d8484b640 1 -- 172.21.15.37:0/838767869 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d70019070 con 0x7f0d80106b80 2024-09-19T06:39:50.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.753+0000 7f0d8484b640 1 -- 172.21.15.37:0/838767869 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d70014440 con 0x7f0d80106b80 2024-09-19T06:39:50.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.753+0000 7f0d8484b640 1 -- 172.21.15.37:0/838767869 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d700125c0 con 0x7f0d80106b80 2024-09-19T06:39:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.753+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d801a6d40 con 0x7f0d80106b80 2024-09-19T06:39:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.753+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d801a71d0 con 0x7f0d80106b80 2024-09-19T06:39:50.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.754+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d80100560 con 0x7f0d80106b80 2024-09-19T06:39:50.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.755+0000 7f0d8484b640 1 -- 172.21.15.37:0/838767869 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0d7001c070 con 0x7f0d80106b80 2024-09-19T06:39:50.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.756+0000 7f0d8484b640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0d6c0a32c0 0x7f0d6c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:50.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.756+0000 7f0d7effd640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0d6c0a32c0 0x7f0d6c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:50.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.756+0000 7f0d8484b640 1 -- 172.21.15.37:0/838767869 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7497+0+0 (secure 0 0 0) 0x7f0d700cebf0 con 0x7f0d80106b80 2024-09-19T06:39:50.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.756+0000 7f0d7effd640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0d6c0a32c0 0x7f0d6c0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d780096c0 tx=0x7f0d78031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:50.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.760+0000 7f0d8484b640 1 -- 172.21.15.37:0/838767869 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0d7009bc40 con 0x7f0d80106b80 2024-09-19T06:39:50.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:50 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/659506580' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-19T06:39:50.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:50 smithi096 ceph-mon[35004]: osdmap e77: 12 total, 12 up, 12 in 2024-09-19T06:39:50.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:50 smithi096 ceph-mon[35004]: osdmap e78: 12 total, 12 up, 12 in 2024-09-19T06:39:50.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:50.937+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0d801a75e0 con 0x7f0d80106b80 2024-09-19T06:39:51.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.370+0000 7f0d8484b640 1 -- 172.21.15.37:0/838767869 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v79) ==== 170+0+0 (secure 0 0 0) 0x7f0d7009b540 con 0x7f0d80106b80 2024-09-19T06:39:51.372 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' created 2024-09-19T06:39:51.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0d6c0a32c0 msgr2=0x7f0d6c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:51.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0d6c0a32c0 0x7f0d6c0a5780 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0d780096c0 tx=0x7f0d78031490 comp rx=0 tx=0).stop 2024-09-19T06:39:51.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 msgr2=0x7f0d80101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:51.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80101d60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0d7000b9e0 tx=0x7f0d7000bea0 comp rx=0 tx=0).stop 2024-09-19T06:39:51.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 shutdown_connections 2024-09-19T06:39:51.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0d6c0a32c0 0x7f0d6c0a5780 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:51.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d80106b80 0x7f0d80101d60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:51.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d80100aa0 0x7f0d80101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:51.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 --2- 172.21.15.37:0/838767869 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d80100100 0x7f0d801012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:51.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 >> 172.21.15.37:0/838767869 conn(0x7f0d80077230 msgr2=0x7f0d80104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:51.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 shutdown_connections 2024-09-19T06:39:51.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:51.372+0000 7f0d8584d640 1 -- 172.21.15.37:0/838767869 wait complete. 2024-09-19T06:39:51.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:51 smithi037 ceph-mon[31211]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:39:51.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:51 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/838767869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:39:51.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:51 smithi096 ceph-mon[35004]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:39:51.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:51 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/838767869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:39:51.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:51 smithi151 ceph-mon[35236]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-19T06:39:51.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:51 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/838767869' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-19T06:39:52.074 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-19T06:39:52.334 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:52.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:52 smithi037 ceph-mon[31211]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:52.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:52 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/838767869' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:39:52.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:52 smithi037 ceph-mon[31211]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:39:52.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:52 smithi037 ceph-mon[31211]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:39:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:52 smithi096 ceph-mon[35004]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:52 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/838767869' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:39:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:52 smithi096 ceph-mon[35004]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:39:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:52 smithi096 ceph-mon[35004]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:39:52.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 -- 172.21.15.37:0/4069639718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c101de0 msgr2=0x7fe86c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 --2- 172.21.15.37:0/4069639718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c101de0 0x7fe86c1021c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe85c009900 tx=0x7fe85c02a410 comp rx=0 tx=0).stop 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 -- 172.21.15.37:0/4069639718 shutdown_connections 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 --2- 172.21.15.37:0/4069639718 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe86c106f10 0x7fe86c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 --2- 172.21.15.37:0/4069639718 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c102790 0x7fe86c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 --2- 172.21.15.37:0/4069639718 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c101de0 0x7fe86c1021c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 -- 172.21.15.37:0/4069639718 >> 172.21.15.37:0/4069639718 conn(0x7fe86c0fd8c0 msgr2=0x7fe86c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 -- 172.21.15.37:0/4069639718 shutdown_connections 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.795+0000 7fe87091f640 1 -- 172.21.15.37:0/4069639718 wait complete. 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 Processor -- start 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 -- start start 2024-09-19T06:39:52.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c101de0 0x7fe86c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe86c102790 0x7fe86c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c106f10 0x7fe86c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c112010 con 0x7fe86c106f10 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c111e90 con 0x7fe86c101de0 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe87091f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe86c112190 con 0x7fe86c102790 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe86ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c106f10 0x7fe86c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe86ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c106f10 0x7fe86c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39218/0 (socket says 172.21.15.37:39218) 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe86ad9d640 1 -- 172.21.15.37:0/6473232 learned_addr learned my addr 172.21.15.37:0/6473232 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe861d9b640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe86c102790 0x7fe86c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.796+0000 7fe86a59c640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c101de0 0x7fe86c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:52.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe86ad9d640 1 -- 172.21.15.37:0/6473232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe86c102790 msgr2=0x7fe86c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe86ad9d640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe86c102790 0x7fe86c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe86ad9d640 1 -- 172.21.15.37:0/6473232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c101de0 msgr2=0x7fe86c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe86ad9d640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c101de0 0x7fe86c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe86ad9d640 1 -- 172.21.15.37:0/6473232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe86c1a6a70 con 0x7fe86c106f10 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe861d9b640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe86c102790 0x7fe86c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe86a59c640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c101de0 0x7fe86c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe86ad9d640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c106f10 0x7fe86c074f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b970 tx=0x7fe86400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe863fff640 1 -- 172.21.15.37:0/6473232 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864019070 con 0x7fe86c106f10 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe86c1a6d60 con 0x7fe86c106f10 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe863fff640 1 -- 172.21.15.37:0/6473232 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe86400cb40 con 0x7fe86c106f10 2024-09-19T06:39:52.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe863fff640 1 -- 172.21.15.37:0/6473232 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe864007610 con 0x7fe86c106f10 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.797+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe86c1a72c0 con 0x7fe86c106f10 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.798+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe86c102240 con 0x7fe86c106f10 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.799+0000 7fe863fff640 1 -- 172.21.15.37:0/6473232 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe86401c070 con 0x7fe86c106f10 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.800+0000 7fe863fff640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe8580a3240 0x7fe8580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.800+0000 7fe86a59c640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe8580a3240 0x7fe8580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.800+0000 7fe863fff640 1 -- 172.21.15.37:0/6473232 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7879+0+0 (secure 0 0 0) 0x7fe8640cee20 con 0x7fe86c106f10 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.800+0000 7fe86a59c640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe8580a3240 0x7fe8580a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005ab0 tx=0x7fe85c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:52.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:52.804+0000 7fe863fff640 1 -- 172.21.15.37:0/6473232 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe86409bc50 con 0x7fe86c106f10 2024-09-19T06:39:52.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:52 smithi151 ceph-mon[35236]: pgmap v274: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:52.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:52 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/838767869' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-19T06:39:52.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:52 smithi151 ceph-mon[35236]: osdmap e79: 12 total, 12 up, 12 in 2024-09-19T06:39:52.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:52 smithi151 ceph-mon[35236]: osdmap e80: 12 total, 12 up, 12 in 2024-09-19T06:39:53.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.019+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fe86c109b60 con 0x7fe86c106f10 2024-09-19T06:39:53.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.404+0000 7fe863fff640 1 -- 172.21.15.37:0/6473232 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fe86409b550 con 0x7fe86c106f10 2024-09-19T06:39:53.407 INFO:teuthology.orchestra.run.smithi037.stderr:new fs with metadata pool 2 and data pool 3 2024-09-19T06:39:53.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe8580a3240 msgr2=0x7fe8580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:53.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe8580a3240 0x7fe8580a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe85c005ab0 tx=0x7fe85c005a20 comp rx=0 tx=0).stop 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c106f10 msgr2=0x7fe86c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c106f10 0x7fe86c074f70 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe86400b970 tx=0x7fe86400be30 comp rx=0 tx=0).stop 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 shutdown_connections 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe8580a3240 0x7fe8580a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe86c106f10 0x7fe86c074f70 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe86c102790 0x7fe86c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 --2- 172.21.15.37:0/6473232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe86c101de0 0x7fe86c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.407+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 >> 172.21.15.37:0/6473232 conn(0x7fe86c0fd8c0 msgr2=0x7fe86c0fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.408+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 shutdown_connections 2024-09-19T06:39:53.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:53.408+0000 7fe87091f640 1 -- 172.21.15.37:0/6473232 wait complete. 2024-09-19T06:39:53.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:53 smithi037 ceph-mon[31211]: pgmap v277: 97 pgs: 7 creating+peering, 2 creating+activating, 77 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:53.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:53 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/6473232' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:39:53.733 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:53 smithi037 ceph-mon[31211]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:39:53.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:53 smithi096 ceph-mon[35004]: pgmap v277: 97 pgs: 7 creating+peering, 2 creating+activating, 77 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:53.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:53 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/6473232' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:39:53.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:53 smithi096 ceph-mon[35004]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:39:53.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:53 smithi151 ceph-mon[35236]: pgmap v277: 97 pgs: 7 creating+peering, 2 creating+activating, 77 unknown, 11 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:39:53.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:53 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/6473232' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-19T06:39:53.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:53 smithi151 ceph-mon[35236]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-19T06:39:54.028 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:39:54.291 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:54.663 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:54 smithi037 ceph-mon[31211]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:39:54.663 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:54 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/6473232' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:39:54.663 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:54 smithi037 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:39:54.663 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:54 smithi037 ceph-mon[31211]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:39:54.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.742+0000 7f2f44b56640 1 -- 172.21.15.37:0/1984657461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 msgr2=0x7f2f401006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.742+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1984657461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f401006d0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009930 tx=0x7f2f3002a420 comp rx=0 tx=0).stop 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.742+0000 7f2f44b56640 1 -- 172.21.15.37:0/1984657461 shutdown_connections 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.742+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1984657461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f40106f50 0x7f2f4010eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.742+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1984657461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f40100ca0 0x7f2f40106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.742+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1984657461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f401006d0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.742+0000 7f2f44b56640 1 -- 172.21.15.37:0/1984657461 >> 172.21.15.37:0/1984657461 conn(0x7f2f40077210 msgr2=0x7f2f400fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.743+0000 7f2f44b56640 1 -- 172.21.15.37:0/1984657461 shutdown_connections 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.743+0000 7f2f44b56640 1 -- 172.21.15.37:0/1984657461 wait complete. 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.743+0000 7f2f44b56640 1 Processor -- start 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.743+0000 7f2f44b56640 1 -- start start 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f44b56640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f40198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:54.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f44b56640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f40100ca0 0x7f2f40198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f44b56640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f40106f50 0x7f2f4019cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f44b56640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40112000 con 0x7f2f401002f0 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f44b56640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40111e80 con 0x7f2f40106f50 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f44b56640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f40112180 con 0x7f2f40100ca0 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f40198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f40198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39244/0 (socket says 172.21.15.37:39244) 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 -- 172.21.15.37:0/1572064681 learned_addr learned my addr 172.21.15.37:0/1572064681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3dd9b640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f40100ca0 0x7f2f40198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3ed9d640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f40106f50 0x7f2f4019cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 -- 172.21.15.37:0/1572064681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f40100ca0 msgr2=0x7f2f40198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f40100ca0 0x7f2f40198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 -- 172.21.15.37:0/1572064681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f40106f50 msgr2=0x7f2f4019cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:54.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f40106f50 0x7f2f4019cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 -- 172.21.15.37:0/1572064681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4019d4d0 con 0x7f2f401002f0 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.744+0000 7f2f3e59c640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f40198520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009a50 tx=0x7f2f30004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.745+0000 7f2f2f7fe640 1 -- 172.21.15.37:0/1572064681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3003f030 con 0x7f2f401002f0 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.745+0000 7f2f2f7fe640 1 -- 172.21.15.37:0/1572064681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f30003ce0 con 0x7f2f401002f0 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.745+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4019d760 con 0x7f2f401002f0 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.745+0000 7f2f2f7fe640 1 -- 172.21.15.37:0/1572064681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f30035070 con 0x7f2f401002f0 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.745+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f401a50a0 con 0x7f2f401002f0 2024-09-19T06:39:54.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.746+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f401033b0 con 0x7f2f401002f0 2024-09-19T06:39:54.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.748+0000 7f2f2f7fe640 1 -- 172.21.15.37:0/1572064681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2f30042030 con 0x7f2f401002f0 2024-09-19T06:39:54.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.748+0000 7f2f2f7fe640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2f280a3240 0x7f2f280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:54.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.748+0000 7f2f3dd9b640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2f280a3240 0x7f2f280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:54.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.748+0000 7f2f2f7fe640 1 -- 172.21.15.37:0/1572064681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f2f300f0260 con 0x7f2f401002f0 2024-09-19T06:39:54.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.749+0000 7f2f3dd9b640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2f280a3240 0x7f2f280a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005e10 tx=0x7f2f34005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:54.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.752+0000 7f2f2f7fe640 1 -- 172.21.15.37:0/1572064681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2f300bd0d0 con 0x7f2f401002f0 2024-09-19T06:39:54.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:54 smithi096 ceph-mon[35004]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:39:54.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:54 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/6473232' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:39:54.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:54 smithi096 ceph-mon[35004]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:39:54.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:54 smithi096 ceph-mon[35004]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:39:54.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:54 smithi151 ceph-mon[35236]: osdmap e81: 12 total, 12 up, 12 in 2024-09-19T06:39:54.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:54 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/6473232' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-19T06:39:54.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:54 smithi151 ceph-mon[35236]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-19T06:39:54.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:54 smithi151 ceph-mon[35236]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-19T06:39:54.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.929+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f4010a120 con 0x7f2f401002f0 2024-09-19T06:39:54.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.931+0000 7f2f2f7fe640 1 -- 172.21.15.37:0/1572064681 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22268 (secure 0 0 0) 0x7f2f300bc9d0 con 0x7f2f401002f0 2024-09-19T06:39:54.933 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:54.934 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:39:53.021912+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:39:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2f280a3240 msgr2=0x7f2f280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2f280a3240 0x7f2f280a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2f34005e10 tx=0x7f2f34005da0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 msgr2=0x7f2f40198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f40198520 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009a50 tx=0x7f2f30004270 comp rx=0 tx=0).stop 2024-09-19T06:39:54.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 shutdown_connections 2024-09-19T06:39:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2f280a3240 0x7f2f280a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f40106f50 0x7f2f4019cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f40100ca0 0x7f2f40198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 --2- 172.21.15.37:0/1572064681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f401002f0 0x7f2f40198520 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 >> 172.21.15.37:0/1572064681 conn(0x7f2f40077210 msgr2=0x7f2f40105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 shutdown_connections 2024-09-19T06:39:54.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:54.933+0000 7f2f44b56640 1 -- 172.21.15.37:0/1572064681 wait complete. 2024-09-19T06:39:55.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:55 smithi037 ceph-mon[31211]: pgmap v280: 97 pgs: 7 creating+peering, 2 creating+activating, 10 unknown, 78 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-19T06:39:55.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:55 smithi037 ceph-mon[31211]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:39:55.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:55 smithi037 ceph-mon[31211]: Cluster is now healthy 2024-09-19T06:39:55.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:55 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] up:active 2024-09-19T06:39:55.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:55 smithi037 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:55.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:55 smithi037 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:55.619 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:55 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1572064681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:55.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:39:55.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:55 smithi096 ceph-mon[35004]: pgmap v280: 97 pgs: 7 creating+peering, 2 creating+activating, 10 unknown, 78 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-19T06:39:55.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:55 smithi096 ceph-mon[35004]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:39:55.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:55 smithi096 ceph-mon[35004]: Cluster is now healthy 2024-09-19T06:39:55.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:55 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] up:active 2024-09-19T06:39:55.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:55 smithi096 ceph-mon[35004]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:55.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:55 smithi096 ceph-mon[35004]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:55.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:55 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1572064681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:55 smithi151 ceph-mon[35236]: pgmap v280: 97 pgs: 7 creating+peering, 2 creating+activating, 10 unknown, 78 active+clean; 577 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 1 op/s 2024-09-19T06:39:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:55 smithi151 ceph-mon[35236]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-19T06:39:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:55 smithi151 ceph-mon[35236]: Cluster is now healthy 2024-09-19T06:39:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:55 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] up:active 2024-09-19T06:39:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:55 smithi151 ceph-mon[35236]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:55 smithi151 ceph-mon[35236]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:55.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:55 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1572064681' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:55.910 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/3257583387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4100e20 msgr2=0x7fa0b4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/3257583387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4100e20 0x7fa0b4101200 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4009900 tx=0x7fa0a402a480 comp rx=0 tx=0).stop 2024-09-19T06:39:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/3257583387 shutdown_connections 2024-09-19T06:39:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/3257583387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b4102ef0 0x7fa0b410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/3257583387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0b4102570 0x7fa0b41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/3257583387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4100e20 0x7fa0b4101200 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/3257583387 >> 172.21.15.37:0/3257583387 conn(0x7fa0b40fc7d0 msgr2=0x7fa0b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/3257583387 shutdown_connections 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.397+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/3257583387 wait complete. 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 Processor -- start 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 -- start start 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b4100e20 0x7fa0b4103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4102570 0x7fa0b4103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4110f80 con 0x7fa0b4102570 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4110e00 con 0x7fa0b4100e20 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bc7eb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0b4111100 con 0x7fa0b4102ef0 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0b9d86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4102570 0x7fa0b4103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0b9d86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4102570 0x7fa0b4103c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39264/0 (socket says 172.21.15.37:39264) 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0b9d86640 1 -- 172.21.15.37:0/906421412 learned_addr learned my addr 172.21.15.37:0/906421412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0bad88640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0ba587640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b4100e20 0x7fa0b4103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.398+0000 7fa0b9d86640 1 -- 172.21.15.37:0/906421412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0b4102ef0 msgr2=0x7fa0b41041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b9d86640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b9d86640 1 -- 172.21.15.37:0/906421412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b4100e20 msgr2=0x7fa0b4103750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b9d86640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b4100e20 0x7fa0b4103750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b9d86640 1 -- 172.21.15.37:0/906421412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0b41a6920 con 0x7fa0b4102570 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0ba587640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b4100e20 0x7fa0b4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0bad88640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b9d86640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4102570 0x7fa0b4103c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800eab0 tx=0x7fa0a800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b37fe640 1 -- 172.21.15.37:0/906421412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a800ce40 con 0x7fa0b4102570 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b37fe640 1 -- 172.21.15.37:0/906421412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0a8011950 con 0x7fa0b4102570 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0b41a6c10 con 0x7fa0b4102570 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0b37fe640 1 -- 172.21.15.37:0/906421412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a8010ae0 con 0x7fa0b4102570 2024-09-19T06:39:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.399+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0b41a71a0 con 0x7fa0b4102570 2024-09-19T06:39:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.400+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0b4101200 con 0x7fa0b4102570 2024-09-19T06:39:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.402+0000 7fa0b37fe640 1 -- 172.21.15.37:0/906421412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fa0a8011b00 con 0x7fa0b4102570 2024-09-19T06:39:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.403+0000 7fa0b37fe640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa09c0a32c0 0x7fa09c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.403+0000 7fa0ba587640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa09c0a32c0 0x7fa09c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.403+0000 7fa0b37fe640 1 -- 172.21.15.37:0/906421412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa0a80cfe80 con 0x7fa0b4102570 2024-09-19T06:39:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.403+0000 7fa0ba587640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa09c0a32c0 0x7fa09c0a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4000c00 tx=0x7fa0a4004100 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.405+0000 7fa0b37fe640 1 -- 172.21.15.37:0/906421412 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0a8016030 con 0x7fa0b4102570 2024-09-19T06:39:56.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.586+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0b41a73d0 con 0x7fa0b4102570 2024-09-19T06:39:56.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.588+0000 7fa0b37fe640 1 -- 172.21.15.37:0/906421412 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22268 (secure 0 0 0) 0x7fa0a809cd70 con 0x7fa0b4102570 2024-09-19T06:39:56.590 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:39:56.590 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:39:53.021912+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:39:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa09c0a32c0 msgr2=0x7fa09c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa09c0a32c0 0x7fa09c0a5780 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa0a4000c00 tx=0x7fa0a4004100 comp rx=0 tx=0).stop 2024-09-19T06:39:56.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4102570 msgr2=0x7fa0b4103c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4102570 0x7fa0b4103c90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa0a800eab0 tx=0x7fa0a800ef70 comp rx=0 tx=0).stop 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 shutdown_connections 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa09c0a32c0 0x7fa09c0a5780 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0b4102ef0 0x7fa0b41041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0b4102570 0x7fa0b4103c90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 --2- 172.21.15.37:0/906421412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0b4100e20 0x7fa0b4103750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 >> 172.21.15.37:0/906421412 conn(0x7fa0b40fc7d0 msgr2=0x7fa0b406a180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 shutdown_connections 2024-09-19T06:39:56.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:56.592+0000 7fa0bc7eb640 1 -- 172.21.15.37:0/906421412 wait complete. 2024-09-19T06:39:57.247 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-19T06:39:57.409 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:39:57 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: 2024-09-19T06:39:57.082+0000 7fa3af5ee640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-19T06:39:57.506 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:57.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:57 smithi037 ceph-mon[31211]: pgmap v281: 97 pgs: 7 creating+peering, 90 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 869 B/s wr, 6 op/s 2024-09-19T06:39:57.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:57 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/906421412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:57.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:57 smithi096 ceph-mon[35004]: pgmap v281: 97 pgs: 7 creating+peering, 90 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 869 B/s wr, 6 op/s 2024-09-19T06:39:57.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:57 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/906421412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:57.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:57 smithi151 ceph-mon[35236]: pgmap v281: 97 pgs: 7 creating+peering, 90 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 869 B/s wr, 6 op/s 2024-09-19T06:39:57.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:57 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/906421412' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:39:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 -- 172.21.15.37:0/1420545382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 msgr2=0x7f20101067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 --2- 172.21.15.37:0/1420545382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f20101067e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2000009900 tx=0x7f200002a410 comp rx=0 tx=0).stop 2024-09-19T06:39:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 -- 172.21.15.37:0/1420545382 shutdown_connections 2024-09-19T06:39:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 --2- 172.21.15.37:0/1420545382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2010106f10 0x7f201010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 --2- 172.21.15.37:0/1420545382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f20101067e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 --2- 172.21.15.37:0/1420545382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2010101de0 0x7f20101021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 -- 172.21.15.37:0/1420545382 >> 172.21.15.37:0/1420545382 conn(0x7f20100fd8c0 msgr2=0x7f20100ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 -- 172.21.15.37:0/1420545382 shutdown_connections 2024-09-19T06:39:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.008+0000 7f2017840640 1 -- 172.21.15.37:0/1420545382 wait complete. 2024-09-19T06:39:58.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 Processor -- start 2024-09-19T06:39:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 -- start start 2024-09-19T06:39:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2010101de0 0x7f2010198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f2010198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2010106f10 0x7f201019cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111f30 con 0x7f2010102790 2024-09-19T06:39:58.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2010111db0 con 0x7f2010106f10 2024-09-19T06:39:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2017840640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f20101120b0 con 0x7f2010101de0 2024-09-19T06:39:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.009+0000 7f2014ddb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f2010198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f2010198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39284/0 (socket says 172.21.15.37:39284) 2024-09-19T06:39:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 -- 172.21.15.37:0/3275223144 learned_addr learned my addr 172.21.15.37:0/3275223144 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f20155dc640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2010101de0 0x7f2010198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2015ddd640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2010106f10 0x7f201019cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 -- 172.21.15.37:0/3275223144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2010101de0 msgr2=0x7f2010198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2010101de0 0x7f2010198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 -- 172.21.15.37:0/3275223144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2010106f10 msgr2=0x7f201019cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2010106f10 0x7f201019cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 -- 172.21.15.37:0/3275223144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f201019d2c0 con 0x7f2010102790 2024-09-19T06:39:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f20155dc640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2010101de0 0x7f2010198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2014ddb640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f2010198850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2000005ab0 tx=0x7f20000042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f20067fc640 1 -- 172.21.15.37:0/3275223144 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f200003f030 con 0x7f2010102790 2024-09-19T06:39:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f20067fc640 1 -- 172.21.15.37:0/3275223144 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2000003ce0 con 0x7f2010102790 2024-09-19T06:39:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f201019d550 con 0x7f2010102790 2024-09-19T06:39:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f20067fc640 1 -- 172.21.15.37:0/3275223144 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2000035070 con 0x7f2010102790 2024-09-19T06:39:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.010+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20101a4e90 con 0x7f2010102790 2024-09-19T06:39:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.011+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20101032c0 con 0x7f2010102790 2024-09-19T06:39:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.013+0000 7f20067fc640 1 -- 172.21.15.37:0/3275223144 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2000042030 con 0x7f2010102790 2024-09-19T06:39:58.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.014+0000 7f20067fc640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1ffc0a32c0 0x7f1ffc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:58.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.014+0000 7f20155dc640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1ffc0a32c0 0x7f1ffc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.014+0000 7f20067fc640 1 -- 172.21.15.37:0/3275223144 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f20000f0050 con 0x7f2010102790 2024-09-19T06:39:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.014+0000 7f20155dc640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1ffc0a32c0 0x7f1ffc0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f200c00abe0 tx=0x7f200c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.018+0000 7f20067fc640 1 -- 172.21.15.37:0/3275223144 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f20000bcec0 con 0x7f2010102790 2024-09-19T06:39:58.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.232+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2010199670 con 0x7f2010102790 2024-09-19T06:39:58.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.415+0000 7f20067fc640 1 -- 172.21.15.37:0/3275223144 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f20000bc7c0 con 0x7f2010102790 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.416+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1ffc0a32c0 msgr2=0x7f1ffc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.416+0000 7f2017840640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1ffc0a32c0 0x7f1ffc0a5780 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f200c00abe0 tx=0x7f200c00a5c0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.416+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 msgr2=0x7f2010198850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.416+0000 7f2017840640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f2010198850 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2000005ab0 tx=0x7f20000042f0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 shutdown_connections 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1ffc0a32c0 0x7f1ffc0a5780 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2010106f10 0x7f201019cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2010102790 0x7f2010198850 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 --2- 172.21.15.37:0/3275223144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2010101de0 0x7f2010198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 >> 172.21.15.37:0/3275223144 conn(0x7f20100fd8c0 msgr2=0x7f20100febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 shutdown_connections 2024-09-19T06:39:58.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:58.417+0000 7f2017840640 1 -- 172.21.15.37:0/3275223144 wait complete. 2024-09-19T06:39:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:58 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3275223144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:39:58.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:58 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3275223144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:39:58.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:58 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3275223144' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-19T06:39:59.112 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-19T06:39:59.112 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-19T06:39:59.375 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:39:59.748 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:59 smithi037 ceph-mon[31211]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-19T06:39:59.748 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:59 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3275223144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:39:59.748 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:39:59 smithi037 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:59 smithi096 ceph-mon[35004]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-19T06:39:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:59 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3275223144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:39:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:39:59 smithi096 ceph-mon[35004]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:39:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 -- 172.21.15.37:0/4268090491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c102790 msgr2=0x7f754c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 --2- 172.21.15.37:0/4268090491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c102790 0x7f754c1067e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7534009900 tx=0x7f753402a410 comp rx=0 tx=0).stop 2024-09-19T06:39:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 -- 172.21.15.37:0/4268090491 shutdown_connections 2024-09-19T06:39:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 --2- 172.21.15.37:0/4268090491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f754c106f10 0x7f754c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 --2- 172.21.15.37:0/4268090491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c102790 0x7f754c1067e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 --2- 172.21.15.37:0/4268090491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f754c101de0 0x7f754c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:59.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 -- 172.21.15.37:0/4268090491 >> 172.21.15.37:0/4268090491 conn(0x7f754c0fd8c0 msgr2=0x7f754c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 -- 172.21.15.37:0/4268090491 shutdown_connections 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.831+0000 7f7550b71640 1 -- 172.21.15.37:0/4268090491 wait complete. 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 Processor -- start 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 -- start start 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f754c101de0 0x7f754c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f754c102790 0x7f754c1988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c106f10 0x7f754c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f754c111f70 con 0x7f754c106f10 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f754c111df0 con 0x7f754c102790 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f7550b71640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f754c1120f0 con 0x7f754c101de0 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f754ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c106f10 0x7f754c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:59.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f754ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c106f10 0x7f754c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59716/0 (socket says 172.21.15.37:59716) 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.832+0000 7f754ad9d640 1 -- 172.21.15.37:0/3360317962 learned_addr learned my addr 172.21.15.37:0/3360317962 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f754a59c640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f754c101de0 0x7f754c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f7543fff640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f754c102790 0x7f754c1988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f754ad9d640 1 -- 172.21.15.37:0/3360317962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f754c101de0 msgr2=0x7f754c198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f754ad9d640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f754c101de0 0x7f754c198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f754ad9d640 1 -- 172.21.15.37:0/3360317962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f754c102790 msgr2=0x7f754c1988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f754ad9d640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f754c102790 0x7f754c1988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f754ad9d640 1 -- 172.21.15.37:0/3360317962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c19d310 con 0x7f754c106f10 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f7543fff640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f754c102790 0x7f754c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f754ad9d640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c106f10 0x7f754c19cc30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f754400eaa0 tx=0x7f754400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f75437fe640 1 -- 172.21.15.37:0/3360317962 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7544004d20 con 0x7f754c106f10 2024-09-19T06:39:59.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f75437fe640 1 -- 172.21.15.37:0/3360317962 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7544005020 con 0x7f754c106f10 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c19d600 con 0x7f754c106f10 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f75437fe640 1 -- 172.21.15.37:0/3360317962 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75440116d0 con 0x7f754c106f10 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.833+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c1a4ee0 con 0x7f754c106f10 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.834+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c1021c0 con 0x7f754c106f10 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.835+0000 7f75437fe640 1 -- 172.21.15.37:0/3360317962 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f75440051d0 con 0x7f754c106f10 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.836+0000 7f75437fe640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75380a32c0 0x7f75380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.836+0000 7f754a59c640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75380a32c0 0x7f75380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.836+0000 7f75437fe640 1 -- 172.21.15.37:0/3360317962 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f75440cfcd0 con 0x7f754c106f10 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.837+0000 7f754a59c640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75380a32c0 0x7f75380a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f753c00abe0 tx=0x7f753c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:39:59.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:39:59.839+0000 7f75437fe640 1 -- 172.21.15.37:0/3360317962 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7544016030 con 0x7f754c106f10 2024-09-19T06:39:59.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:59 smithi151 ceph-mon[35236]: pgmap v282: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-19T06:39:59.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:59 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3275223144' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-19T06:39:59.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:39:59 smithi151 ceph-mon[35236]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:40:00.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.052+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f754c10a150 con 0x7f754c106f10 2024-09-19T06:40:00.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.420+0000 7f75437fe640 1 -- 172.21.15.37:0/3360317962 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f754409cbc0 con 0x7f754c106f10 2024-09-19T06:40:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75380a32c0 msgr2=0x7f75380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75380a32c0 0x7f75380a5780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f753c00abe0 tx=0x7f753c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c106f10 msgr2=0x7f754c19cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c106f10 0x7f754c19cc30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f754400eaa0 tx=0x7f754400eea0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 shutdown_connections 2024-09-19T06:40:00.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75380a32c0 0x7f75380a5780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f754c106f10 0x7f754c19cc30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f754c102790 0x7f754c1988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 --2- 172.21.15.37:0/3360317962 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f754c101de0 0x7f754c198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 >> 172.21.15.37:0/3360317962 conn(0x7f754c0fd8c0 msgr2=0x7f754c0fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 shutdown_connections 2024-09-19T06:40:00.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:00.423+0000 7f7550b71640 1 -- 172.21.15.37:0/3360317962 wait complete. 2024-09-19T06:40:00.744 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:00 smithi037 ceph-mon[31211]: overall HEALTH_OK 2024-09-19T06:40:00.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:00 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3360317962' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T06:40:00.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:00 smithi096 ceph-mon[35004]: overall HEALTH_OK 2024-09-19T06:40:00.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:00 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3360317962' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T06:40:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:00 smithi151 ceph-mon[35236]: overall HEALTH_OK 2024-09-19T06:40:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:00 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3360317962' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-19T06:40:01.060 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-19T06:40:01.321 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:01.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:01 smithi037 ceph-mon[31211]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T06:40:01.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:01 smithi037 ceph-mon[31211]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:40:01.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:01 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3360317962' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T06:40:01.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:01 smithi037 ceph-mon[31211]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:40:01.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:01 smithi037 ceph-mon[31211]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:40:01.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:01 smithi037 ceph-mon[31211]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T06:40:01.716 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:01 smithi037 ceph-mon[31211]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T06:40:01.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:01 smithi096 ceph-mon[35004]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T06:40:01.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:01 smithi096 ceph-mon[35004]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:40:01.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:01 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3360317962' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T06:40:01.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:01 smithi096 ceph-mon[35004]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:40:01.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:01 smithi096 ceph-mon[35004]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:40:01.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:01 smithi096 ceph-mon[35004]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T06:40:01.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:01 smithi096 ceph-mon[35004]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 -- 172.21.15.37:0/2570343703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4106a80 msgr2=0x7f30d4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 --2- 172.21.15.37:0/2570343703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4106a80 0x7f30d4106e60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009930 tx=0x7f30c402a420 comp rx=0 tx=0).stop 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 -- 172.21.15.37:0/2570343703 shutdown_connections 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 --2- 172.21.15.37:0/2570343703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30d4101420 0x7f30d410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 --2- 172.21.15.37:0/2570343703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d4100a80 0x7f30d4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 --2- 172.21.15.37:0/2570343703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4106a80 0x7f30d4106e60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 -- 172.21.15.37:0/2570343703 >> 172.21.15.37:0/2570343703 conn(0x7f30d40fc7a0 msgr2=0x7f30d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 -- 172.21.15.37:0/2570343703 shutdown_connections 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 -- 172.21.15.37:0/2570343703 wait complete. 2024-09-19T06:40:01.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.801+0000 7f30d8cae640 1 Processor -- start 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d8cae640 1 -- start start 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d8cae640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4100a80 0x7f30d4197040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d8cae640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30d4101420 0x7f30d4197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d8cae640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d4106a80 0x7f30d4197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d8cae640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4110e80 con 0x7f30d4100a80 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d8cae640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4110d00 con 0x7f30d4101420 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d8cae640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f30d4111000 con 0x7f30d4106a80 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4100a80 0x7f30d4197040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4100a80 0x7f30d4197040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59744/0 (socket says 172.21.15.37:59744) 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 -- 172.21.15.37:0/1648418423 learned_addr learned my addr 172.21.15.37:0/1648418423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d1d9b640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30d4101420 0x7f30d4197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d2d9d640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d4106a80 0x7f30d4197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 -- 172.21.15.37:0/1648418423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d4106a80 msgr2=0x7f30d4197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:01.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d4106a80 0x7f30d4197ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 -- 172.21.15.37:0/1648418423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30d4101420 msgr2=0x7f30d4197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30d4101420 0x7f30d4197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 -- 172.21.15.37:0/1648418423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d41a68b0 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d1d9b640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30d4101420 0x7f30d4197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.802+0000 7f30d259c640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4100a80 0x7f30d4197040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009a50 tx=0x7f30c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.803+0000 7f30c37fe640 1 -- 172.21.15.37:0/1648418423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c403f030 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.803+0000 7f30c37fe640 1 -- 172.21.15.37:0/1648418423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c4003ce0 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.803+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d41a6b40 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.803+0000 7f30c37fe640 1 -- 172.21.15.37:0/1648418423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4035070 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.803+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a7020 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.804+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d4100ee0 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.806+0000 7f30c37fe640 1 -- 172.21.15.37:0/1648418423 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f30c4042030 con 0x7f30d4100a80 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.807+0000 7f30c37fe640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f30bc0a3240 0x7f30bc0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:01.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.807+0000 7f30c37fe640 1 -- 172.21.15.37:0/1648418423 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f30c40f0260 con 0x7f30d4100a80 2024-09-19T06:40:01.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.807+0000 7f30d1d9b640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f30bc0a3240 0x7f30bc0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:01.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.808+0000 7f30d1d9b640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f30bc0a3240 0x7f30bc0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f30d41991a0 tx=0x7f30c800a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:01.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:01.810+0000 7f30c37fe640 1 -- 172.21.15.37:0/1648418423 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f30c40bd0d0 con 0x7f30d4100a80 2024-09-19T06:40:01.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:01 smithi151 ceph-mon[35236]: pgmap v283: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-19T06:40:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:01 smithi151 ceph-mon[35236]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-19T06:40:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:01 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3360317962' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-19T06:40:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:01 smithi151 ceph-mon[35236]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-19T06:40:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:01 smithi151 ceph-mon[35236]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-19T06:40:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:01 smithi151 ceph-mon[35236]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-19T06:40:01.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:01 smithi151 ceph-mon[35236]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-19T06:40:02.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.028+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f30d402d7c0 con 0x7f30d4100a80 2024-09-19T06:40:02.434 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: 2024-09-19T06:40:02.083+0000 7fa3af5ee640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-19T06:40:02.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.441+0000 7f30c37fe640 1 -- 172.21.15.37:0/1648418423 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f30c40bc9d0 con 0x7f30d4100a80 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f30bc0a3240 msgr2=0x7f30bc0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f30bc0a3240 0x7f30bc0a5700 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f30d41991a0 tx=0x7f30c800a5c0 comp rx=0 tx=0).stop 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4100a80 msgr2=0x7f30d4197040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4100a80 0x7f30d4197040 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009a50 tx=0x7f30c4004270 comp rx=0 tx=0).stop 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 shutdown_connections 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f30bc0a3240 0x7f30bc0a5700 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f30d4106a80 0x7f30d4197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f30d4101420 0x7f30d4197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 --2- 172.21.15.37:0/1648418423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f30d4100a80 0x7f30d4197040 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 >> 172.21.15.37:0/1648418423 conn(0x7f30d40fc7a0 msgr2=0x7f30d40fdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 shutdown_connections 2024-09-19T06:40:02.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:02.444+0000 7f30d8cae640 1 -- 172.21.15.37:0/1648418423 wait complete. 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] up:active 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: Cluster is now healthy 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-09-19T06:40:02.745 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:02 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1648418423' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:40:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] up:active 2024-09-19T06:40:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:40:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:40:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: Cluster is now healthy 2024-09-19T06:40:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T06:40:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-09-19T06:40:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-09-19T06:40:02.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:02 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1648418423' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] up:active 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: Cluster is now healthy 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-09-19T06:40:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:02 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1648418423' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-19T06:40:03.161 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-19T06:40:03.421 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:03.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:03 smithi037 ceph-mon[31211]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T06:40:03.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:03 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1648418423' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:40:03.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:03 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] up:active 2024-09-19T06:40:03.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:03 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-09-19T06:40:03.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:03 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:40:03.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:03 smithi096 ceph-mon[35004]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T06:40:03.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:03 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1648418423' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:40:03.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:03 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] up:active 2024-09-19T06:40:03.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:03 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-09-19T06:40:03.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:03 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:40:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 -- 172.21.15.37:0/3853073952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 msgr2=0x7f670c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 --2- 172.21.15.37:0/3853073952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c106e60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009930 tx=0x7f66fc02a420 comp rx=0 tx=0).stop 2024-09-19T06:40:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 -- 172.21.15.37:0/3853073952 shutdown_connections 2024-09-19T06:40:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 --2- 172.21.15.37:0/3853073952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f670c101420 0x7f670c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 --2- 172.21.15.37:0/3853073952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670c100a80 0x7f670c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 --2- 172.21.15.37:0/3853073952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c106e60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 -- 172.21.15.37:0/3853073952 >> 172.21.15.37:0/3853073952 conn(0x7f670c0fc7a0 msgr2=0x7f670c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 -- 172.21.15.37:0/3853073952 shutdown_connections 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.850+0000 7f67119dc640 1 -- 172.21.15.37:0/3853073952 wait complete. 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 Processor -- start 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 -- start start 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f670c100a80 0x7f670c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670c101420 0x7f670c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f670c110e40 con 0x7f670c106a80 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f670c110cc0 con 0x7f670c100a80 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f67119dc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f670c110fc0 con 0x7f670c101420 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f670b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:03.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f670b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c19cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59754/0 (socket says 172.21.15.37:59754) 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f670b7fe640 1 -- 172.21.15.37:0/1830869054 learned_addr learned my addr 172.21.15.37:0/1830869054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f670a7fc640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670c101420 0x7f670c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.851+0000 7f670affd640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f670c100a80 0x7f670c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f670b7fe640 1 -- 172.21.15.37:0/1830869054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670c101420 msgr2=0x7f670c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f670b7fe640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670c101420 0x7f670c198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f670b7fe640 1 -- 172.21.15.37:0/1830869054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f670c100a80 msgr2=0x7f670c1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f670b7fe640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f670c100a80 0x7f670c1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f670b7fe640 1 -- 172.21.15.37:0/1830869054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f670c19d370 con 0x7f670c106a80 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f670b7fe640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c19cc90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f670400eaa0 tx=0x7f670400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f67109da640 1 -- 172.21.15.37:0/1830869054 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6704004d20 con 0x7f670c106a80 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f670c19d660 con 0x7f670c106a80 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f67109da640 1 -- 172.21.15.37:0/1830869054 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6704005020 con 0x7f670c106a80 2024-09-19T06:40:03.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f67109da640 1 -- 172.21.15.37:0/1830869054 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67040116d0 con 0x7f670c106a80 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.852+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f670c1a4f40 con 0x7f670c106a80 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.853+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f670c100ee0 con 0x7f670c106a80 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.854+0000 7f67109da640 1 -- 172.21.15.37:0/1830869054 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f67040051d0 con 0x7f670c106a80 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.855+0000 7f67109da640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f66f40a3240 0x7f66f40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.855+0000 7f670affd640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f66f40a3240 0x7f66f40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.855+0000 7f67109da640 1 -- 172.21.15.37:0/1830869054 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f67040cf970 con 0x7f670c106a80 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.855+0000 7f670affd640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f66f40a3240 0x7f66f40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009900 tx=0x7f66fc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:03.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:03.859+0000 7f67109da640 1 -- 172.21.15.37:0/1830869054 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6704016030 con 0x7f670c106a80 2024-09-19T06:40:03.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:03 smithi151 ceph-mon[35236]: pgmap v284: 97 pgs: 97 active+clean; 579 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 5 op/s 2024-09-19T06:40:03.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:03 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1648418423' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-19T06:40:03.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:03 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] up:active 2024-09-19T06:40:03.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:03 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-09-19T06:40:03.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:03 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:40:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.074+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f670c02d7c0 con 0x7f670c106a80 2024-09-19T06:40:04.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.451+0000 7f67109da640 1 -- 172.21.15.37:0/1830869054 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7f670409c740 con 0x7f670c106a80 2024-09-19T06:40:04.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f66f40a3240 msgr2=0x7f66f40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:04.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f66f40a3240 0x7f66f40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f66fc009900 tx=0x7f66fc0058d0 comp rx=0 tx=0).stop 2024-09-19T06:40:04.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 msgr2=0x7f670c19cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:04.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c19cc90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f670400eaa0 tx=0x7f670400eea0 comp rx=0 tx=0).stop 2024-09-19T06:40:04.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 shutdown_connections 2024-09-19T06:40:04.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f66f40a3240 0x7f66f40a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:04.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f670c106a80 0x7f670c19cc90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:04.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f670c101420 0x7f670c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:04.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 --2- 172.21.15.37:0/1830869054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f670c100a80 0x7f670c1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:04.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.454+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 >> 172.21.15.37:0/1830869054 conn(0x7f670c0fc7a0 msgr2=0x7f670c06a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:04.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.455+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 shutdown_connections 2024-09-19T06:40:04.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:04.455+0000 7f67119dc640 1 -- 172.21.15.37:0/1830869054 wait complete. 2024-09-19T06:40:04.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:04 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1830869054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:40:04.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:04 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1830869054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:40:04.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:04 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1830869054' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-19T06:40:05.147 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-19T06:40:05.147 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-19T06:40:05.410 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:05 smithi037 ceph-mon[31211]: pgmap v285: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T06:40:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:05 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1830869054' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:40:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:05 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:40:05.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:05 smithi096 ceph-mon[35004]: pgmap v285: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T06:40:05.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:05 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1830869054' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:40:05.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:05 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:40:05.879 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:05 smithi151 ceph-mon[35236]: pgmap v285: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T06:40:05.880 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:05 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1830869054' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-19T06:40:05.880 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:05 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:40:05.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 -- 172.21.15.37:0/608457824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88069f00 msgr2=0x7f8c88100480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:05.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 --2- 172.21.15.37:0/608457824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88069f00 0x7f8c88100480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009900 tx=0x7f8c8402a410 comp rx=0 tx=0).stop 2024-09-19T06:40:05.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 -- 172.21.15.37:0/608457824 shutdown_connections 2024-09-19T06:40:05.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 --2- 172.21.15.37:0/608457824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c88104e80 0x7f8c88105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:05.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 --2- 172.21.15.37:0/608457824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c8806a370 0x7f8c881009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 --2- 172.21.15.37:0/608457824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88069f00 0x7f8c88100480 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 -- 172.21.15.37:0/608457824 >> 172.21.15.37:0/608457824 conn(0x7f8c88077210 msgr2=0x7f8c880fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 -- 172.21.15.37:0/608457824 shutdown_connections 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 -- 172.21.15.37:0/608457824 wait complete. 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.892+0000 7f8c8f858640 1 Processor -- start 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8f858640 1 -- start start 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8f858640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c88069f00 0x7f8c88102ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8f858640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c8806a370 0x7f8c88102ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8f858640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88104e80 0x7f8c88103530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8f858640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88110e70 con 0x7f8c88104e80 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8f858640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88110cf0 con 0x7f8c88069f00 2024-09-19T06:40:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8f858640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c88110ff0 con 0x7f8c8806a370 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88104e80 0x7f8c88103530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88104e80 0x7f8c88103530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59758/0 (socket says 172.21.15.37:59758) 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 -- 172.21.15.37:0/789376796 learned_addr learned my addr 172.21.15.37:0/789376796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8cdf3640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c8806a370 0x7f8c88102ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8d5f4640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c88069f00 0x7f8c88102ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 -- 172.21.15.37:0/789376796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c8806a370 msgr2=0x7f8c88102ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c8806a370 0x7f8c88102ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 -- 172.21.15.37:0/789376796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c88069f00 msgr2=0x7f8c88102ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c88069f00 0x7f8c88102ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 -- 172.21.15.37:0/789376796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c881a68e0 con 0x7f8c88104e80 2024-09-19T06:40:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8cdf3640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c8806a370 0x7f8c88102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.893+0000 7f8c8ddf5640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88104e80 0x7f8c88103530 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000eb10 tx=0x7f8c8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.894+0000 7f8c7e7fc640 1 -- 172.21.15.37:0/789376796 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80004b20 con 0x7f8c88104e80 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.894+0000 7f8c7e7fc640 1 -- 172.21.15.37:0/789376796 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c80004cc0 con 0x7f8c88104e80 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.894+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c881a6bd0 con 0x7f8c88104e80 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.894+0000 7f8c7e7fc640 1 -- 172.21.15.37:0/789376796 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80010760 con 0x7f8c88104e80 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.894+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c881a7110 con 0x7f8c88104e80 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.895+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c88105330 con 0x7f8c88104e80 2024-09-19T06:40:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.896+0000 7f8c7e7fc640 1 -- 172.21.15.37:0/789376796 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8c80005020 con 0x7f8c88104e80 2024-09-19T06:40:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.896+0000 7f8c7e7fc640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8c740a32c0 0x7f8c740a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.897+0000 7f8c8d5f4640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8c740a32c0 0x7f8c740a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.897+0000 7f8c7e7fc640 1 -- 172.21.15.37:0/789376796 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f8c800cfe70 con 0x7f8c88104e80 2024-09-19T06:40:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.897+0000 7f8c8d5f4640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8c740a32c0 0x7f8c740a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8c84005ab0 tx=0x7f8c84005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:05.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:05.901+0000 7f8c7e7fc640 1 -- 172.21.15.37:0/789376796 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8c8009cc40 con 0x7f8c88104e80 2024-09-19T06:40:06.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.102+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f8c880fccd0 con 0x7f8c740a32c0 2024-09-19T06:40:06.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.127+0000 7f8c7e7fc640 1 -- 172.21.15.37:0/789376796 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f8c880fccd0 con 0x7f8c740a32c0 2024-09-19T06:40:06.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8c740a32c0 msgr2=0x7f8c740a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8c740a32c0 0x7f8c740a5780 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8c84005ab0 tx=0x7f8c84005a20 comp rx=0 tx=0).stop 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88104e80 msgr2=0x7f8c88103530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88104e80 0x7f8c88103530 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c8000eb10 tx=0x7f8c8000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 shutdown_connections 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f8c740a32c0 0x7f8c740a5780 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c88104e80 0x7f8c88103530 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c8806a370 0x7f8c88102ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 --2- 172.21.15.37:0/789376796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c88069f00 0x7f8c88102ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:06.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.129+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 >> 172.21.15.37:0/789376796 conn(0x7f8c88077210 msgr2=0x7f8c880fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:06.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.130+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 shutdown_connections 2024-09-19T06:40:06.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:06.130+0000 7f8c8f858640 1 -- 172.21.15.37:0/789376796 wait complete. 2024-09-19T06:40:06.436 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:40:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:40:06.107+0000 7f023741e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:40:06.437 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:40:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:40:06.107+0000 7f023741e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:40:06.437 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:40:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:40:06.107+0000 7f023741e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:40:06.437 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:40:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:40:06.107+0000 7f023741e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:40:06.437 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:40:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:40:06.107+0000 7f023741e640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:40:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:06 smithi096 ceph-mon[35004]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:06.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:06 smithi037 ceph-mon[31211]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:06.808 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-19T06:40:06.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:06 smithi151 ceph-mon[35236]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:07.065 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 -- 172.21.15.37:0/3552084893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 msgr2=0x7fb20c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:07.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 --2- 172.21.15.37:0/3552084893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c103120 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb208009900 tx=0x7fb20802a480 comp rx=0 tx=0).stop 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 -- 172.21.15.37:0/3552084893 shutdown_connections 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 --2- 172.21.15.37:0/3552084893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c10c170 0x7fb20c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 --2- 172.21.15.37:0/3552084893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb20c103660 0x7fb20c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 --2- 172.21.15.37:0/3552084893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c103120 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 -- 172.21.15.37:0/3552084893 >> 172.21.15.37:0/3552084893 conn(0x7fb20c0fc7a0 msgr2=0x7fb20c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 -- 172.21.15.37:0/3552084893 shutdown_connections 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.495+0000 7fb214782640 1 -- 172.21.15.37:0/3552084893 wait complete. 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 Processor -- start 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 -- start start 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c103660 0x7fb20c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:07.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb20c10c170 0x7fb20c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c111f70 con 0x7fb20c102d40 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c111df0 con 0x7fb20c103660 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb214782640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1120f0 con 0x7fb20c10c170 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb21251e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb21251e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59784/0 (socket says 172.21.15.37:59784) 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb21251e640 1 -- 172.21.15.37:0/3710243113 learned_addr learned my addr 172.21.15.37:0/3710243113 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.496+0000 7fb211d1d640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c103660 0x7fb20c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb212d1f640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb20c10c170 0x7fb20c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb21251e640 1 -- 172.21.15.37:0/3710243113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb20c10c170 msgr2=0x7fb20c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb21251e640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb20c10c170 0x7fb20c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb21251e640 1 -- 172.21.15.37:0/3710243113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c103660 msgr2=0x7fb20c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb21251e640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c103660 0x7fb20c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb21251e640 1 -- 172.21.15.37:0/3710243113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c19d420 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb21251e640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c198470 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb208006010 tx=0x7fb2080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb2037fe640 1 -- 172.21.15.37:0/3710243113 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20803f030 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c19d6b0 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb2037fe640 1 -- 172.21.15.37:0/3710243113 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb208003ce0 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb2037fe640 1 -- 172.21.15.37:0/3710243113 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb208035070 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.497+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c1a4ff0 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.498+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c06b070 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.499+0000 7fb2037fe640 1 -- 172.21.15.37:0/3710243113 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb208042030 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.500+0000 7fb2037fe640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb1f80a32d0 0x7fb1f80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.500+0000 7fb211d1d640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb1f80a32d0 0x7fb1f80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.500+0000 7fb2037fe640 1 -- 172.21.15.37:0/3710243113 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb2080effd0 con 0x7fb20c102d40 2024-09-19T06:40:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.501+0000 7fb211d1d640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb1f80a32d0 0x7fb1f80a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb20c10d5b0 tx=0x7fb1fc005f50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.505+0000 7fb2037fe640 1 -- 172.21.15.37:0/3710243113 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb2080bcd00 con 0x7fb20c102d40 2024-09-19T06:40:07.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.711+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fb20c062630 con 0x7fb1f80a32d0 2024-09-19T06:40:07.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:07 smithi037 ceph-mon[31211]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T06:40:07.712 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:07 smithi037 ceph-mon[31211]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:40:07.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.733+0000 7fb2037fe640 1 -- 172.21.15.37:0/3710243113 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fb20c062630 con 0x7fb1f80a32d0 2024-09-19T06:40:07.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.735+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb1f80a32d0 msgr2=0x7fb1f80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:07.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.735+0000 7fb214782640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb1f80a32d0 0x7fb1f80a5790 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb20c10d5b0 tx=0x7fb1fc005f50 comp rx=0 tx=0).stop 2024-09-19T06:40:07.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.735+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 msgr2=0x7fb20c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.735+0000 7fb214782640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c198470 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb208006010 tx=0x7fb2080042f0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.735+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 shutdown_connections 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.736+0000 7fb214782640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb1f80a32d0 0x7fb1f80a5790 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.736+0000 7fb214782640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb20c10c170 0x7fb20c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.736+0000 7fb214782640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c103660 0x7fb20c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.736+0000 7fb214782640 1 --2- 172.21.15.37:0/3710243113 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb20c102d40 0x7fb20c198470 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.736+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 >> 172.21.15.37:0/3710243113 conn(0x7fb20c0fc7a0 msgr2=0x7fb20c0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.736+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 shutdown_connections 2024-09-19T06:40:07.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:07.736+0000 7fb214782640 1 -- 172.21.15.37:0/3710243113 wait complete. 2024-09-19T06:40:07.764 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:07 smithi151 ceph-mon[35236]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T06:40:07.764 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:07 smithi151 ceph-mon[35236]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:40:07.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:07 smithi096 ceph-mon[35004]: pgmap v286: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-09-19T06:40:07.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:07 smithi096 ceph-mon[35004]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-19T06:40:08.503 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-19T06:40:08.761 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:08.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:08 smithi096 ceph-mon[35004]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:08.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:08 smithi037 ceph-mon[31211]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:08.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:08 smithi151 ceph-mon[35236]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.203+0000 7fda2cf1f640 1 -- 172.21.15.37:0/3201120145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28106f10 msgr2=0x7fda2810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.203+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/3201120145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28106f10 0x7fda2810ea90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fda20009ca0 tx=0x7fda2002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 -- 172.21.15.37:0/3201120145 shutdown_connections 2024-09-19T06:40:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/3201120145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28106f10 0x7fda2810ea90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/3201120145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda28102790 0x7fda281067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/3201120145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda28101de0 0x7fda281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 -- 172.21.15.37:0/3201120145 >> 172.21.15.37:0/3201120145 conn(0x7fda280fd8c0 msgr2=0x7fda280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 -- 172.21.15.37:0/3201120145 shutdown_connections 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 -- 172.21.15.37:0/3201120145 wait complete. 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.204+0000 7fda2cf1f640 1 Processor -- start 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2cf1f640 1 -- start start 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2cf1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28101de0 0x7fda28198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2cf1f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda28102790 0x7fda28198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2cf1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda28106f10 0x7fda2819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2cf1f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fda28111f70 con 0x7fda28101de0 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2cf1f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda28111df0 con 0x7fda28106f10 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2cf1f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda281120f0 con 0x7fda28102790 2024-09-19T06:40:09.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28101de0 0x7fda28198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28101de0 0x7fda28198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59800/0 (socket says 172.21.15.37:59800) 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 -- 172.21.15.37:0/78438649 learned_addr learned my addr 172.21.15.37:0/78438649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda25d9b640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda28102790 0x7fda28198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda26d9d640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda28106f10 0x7fda2819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 -- 172.21.15.37:0/78438649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda28102790 msgr2=0x7fda28198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda28102790 0x7fda28198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 -- 172.21.15.37:0/78438649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda28106f10 msgr2=0x7fda2819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda28106f10 0x7fda2819cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 -- 172.21.15.37:0/78438649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2819d2e0 con 0x7fda28101de0 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda25d9b640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda28102790 0x7fda28198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:40:09.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.205+0000 7fda2659c640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28101de0 0x7fda28198330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c920 tx=0x7fda1800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.206+0000 7fda177fe640 1 -- 172.21.15.37:0/78438649 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18007c70 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.206+0000 7fda177fe640 1 -- 172.21.15.37:0/78438649 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda18007e10 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.206+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2819d5d0 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.206+0000 7fda177fe640 1 -- 172.21.15.37:0/78438649 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda18018890 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.206+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda281a4eb0 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.207+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda281021c0 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.208+0000 7fda177fe640 1 -- 172.21.15.37:0/78438649 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fda18018a50 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.209+0000 7fda177fe640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fda100a32d0 0x7fda100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.209+0000 7fda25d9b640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fda100a32d0 0x7fda100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.209+0000 7fda177fe640 1 -- 172.21.15.37:0/78438649 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fda180cfe90 con 0x7fda28101de0 2024-09-19T06:40:09.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.209+0000 7fda25d9b640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fda100a32d0 0x7fda100a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005fd0 tx=0x7fda1c0094d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:09.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.212+0000 7fda177fe640 1 -- 172.21.15.37:0/78438649 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fda18014030 con 0x7fda28101de0 2024-09-19T06:40:09.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.416+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fda28062630 con 0x7fda100a32d0 2024-09-19T06:40:09.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.420+0000 7fda177fe640 1 -- 172.21.15.37:0/78438649 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fda28062630 con 0x7fda100a32d0 2024-09-19T06:40:09.421 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_0/94e81f12-a112-494d-8993-326c86d112a2 2024-09-19T06:40:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fda100a32d0 msgr2=0x7fda100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fda100a32d0 0x7fda100a5790 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fda1c005fd0 tx=0x7fda1c0094d0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28101de0 msgr2=0x7fda28198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28101de0 0x7fda28198330 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fda1800c920 tx=0x7fda1800cde0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 shutdown_connections 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fda100a32d0 0x7fda100a5790 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda28106f10 0x7fda2819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda28102790 0x7fda28198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 --2- 172.21.15.37:0/78438649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda28101de0 0x7fda28198330 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 >> 172.21.15.37:0/78438649 conn(0x7fda280fd8c0 msgr2=0x7fda280febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 shutdown_connections 2024-09-19T06:40:09.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:09.423+0000 7fda2cf1f640 1 -- 172.21.15.37:0/78438649 wait complete. 2024-09-19T06:40:09.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:09 smithi096 ceph-mon[35004]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T06:40:09.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:09 smithi037 ceph-mon[31211]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T06:40:09.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:09 smithi151 ceph-mon[35236]: pgmap v287: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-09-19T06:40:10.115 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-19T06:40:10.377 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:10.757 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:10 smithi037 ceph-mon[31211]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 -- 172.21.15.37:0/1738292382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101240 msgr2=0x7ffa08101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1738292382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101240 0x7ffa08101680 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc009900 tx=0x7ff9fc02a3f0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 -- 172.21.15.37:0/1738292382 shutdown_connections 2024-09-19T06:40:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1738292382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08101bc0 0x7ffa0810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1738292382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101240 0x7ffa08101680 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1738292382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa08105a90 0x7ffa08105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 -- 172.21.15.37:0/1738292382 >> 172.21.15.37:0/1738292382 conn(0x7ffa08077210 msgr2=0x7ffa080fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 -- 172.21.15.37:0/1738292382 shutdown_connections 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 -- 172.21.15.37:0/1738292382 wait complete. 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.756+0000 7ffa0d538640 1 Processor -- start 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa0d538640 1 -- start start 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa0d538640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08101240 0x7ffa081985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa0d538640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101bc0 0x7ffa08198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa0d538640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa08105a90 0x7ffa0819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa0d538640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08111030 con 0x7ffa08101bc0 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa0d538640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08110eb0 con 0x7ffa08105a90 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa0d538640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081111b0 con 0x7ffa08101240 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101bc0 0x7ffa08198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:10.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101bc0 0x7ffa08198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51260/0 (socket says 172.21.15.37:51260) 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 -- 172.21.15.37:0/1844841220 learned_addr learned my addr 172.21.15.37:0/1844841220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa06ffd640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08101240 0x7ffa081985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa077fe640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa08105a90 0x7ffa0819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 -- 172.21.15.37:0/1844841220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08101240 msgr2=0x7ffa081985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08101240 0x7ffa081985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 -- 172.21.15.37:0/1844841220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa08105a90 msgr2=0x7ffa0819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa08105a90 0x7ffa0819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.757+0000 7ffa067fc640 1 -- 172.21.15.37:0/1844841220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa0819d550 con 0x7ffa08101bc0 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.758+0000 7ffa067fc640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101bc0 0x7ffa08198ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006010 tx=0x7ff9fc003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:10.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.758+0000 7ff9effff640 1 -- 172.21.15.37:0/1844841220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc03f070 con 0x7ffa08101bc0 2024-09-19T06:40:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.758+0000 7ff9effff640 1 -- 172.21.15.37:0/1844841220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff9fc031e30 con 0x7ffa08101bc0 2024-09-19T06:40:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.758+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa0819d7e0 con 0x7ffa08101bc0 2024-09-19T06:40:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.758+0000 7ff9effff640 1 -- 172.21.15.37:0/1844841220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc035030 con 0x7ffa08101bc0 2024-09-19T06:40:10.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.758+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081a5120 con 0x7ffa08101bc0 2024-09-19T06:40:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.759+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa08101680 con 0x7ffa08101bc0 2024-09-19T06:40:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.760+0000 7ff9effff640 1 -- 172.21.15.37:0/1844841220 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff9fc042070 con 0x7ffa08101bc0 2024-09-19T06:40:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.761+0000 7ff9effff640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff9f00a3350 0x7ff9f00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.761+0000 7ffa06ffd640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff9f00a3350 0x7ff9f00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.761+0000 7ff9effff640 1 -- 172.21.15.37:0/1844841220 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff9fc0f0160 con 0x7ffa08101bc0 2024-09-19T06:40:10.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.762+0000 7ffa06ffd640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff9f00a3350 0x7ff9f00a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009990 tx=0x7ff9f8009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:10.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.766+0000 7ff9effff640 1 -- 172.21.15.37:0/1844841220 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff9fc0bced0 con 0x7ffa08101bc0 2024-09-19T06:40:10.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.965+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ffa081048d0 con 0x7ff9f00a3350 2024-09-19T06:40:10.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.991+0000 7ff9effff640 1 -- 172.21.15.37:0/1844841220 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ffa081048d0 con 0x7ff9f00a3350 2024-09-19T06:40:10.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff9f00a3350 msgr2=0x7ff9f00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff9f00a3350 0x7ff9f00a5810 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8009990 tx=0x7ff9f8009110 comp rx=0 tx=0).stop 2024-09-19T06:40:10.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101bc0 msgr2=0x7ffa08198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:10.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101bc0 0x7ffa08198ae0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc006010 tx=0x7ff9fc003c80 comp rx=0 tx=0).stop 2024-09-19T06:40:10.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 shutdown_connections 2024-09-19T06:40:10.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff9f00a3350 0x7ff9f00a5810 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa08105a90 0x7ffa0819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa08101bc0 0x7ffa08198ae0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 --2- 172.21.15.37:0/1844841220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa08101240 0x7ffa081985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:10.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 >> 172.21.15.37:0/1844841220 conn(0x7ffa08077210 msgr2=0x7ffa08100270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:10.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 shutdown_connections 2024-09-19T06:40:10.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:10.993+0000 7ffa0d538640 1 -- 172.21.15.37:0/1844841220 wait complete. 2024-09-19T06:40:11.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:10 smithi096 ceph-mon[35004]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:11.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:10 smithi151 ceph-mon[35236]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:11.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-19T06:40:12.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:11 smithi096 ceph-mon[35004]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-19T06:40:12.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:11 smithi096 ceph-mon[35004]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:12.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:11 smithi037 ceph-mon[31211]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-19T06:40:12.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:11 smithi037 ceph-mon[31211]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:12.038 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:12.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:11 smithi151 ceph-mon[35236]: pgmap v288: 97 pgs: 97 active+clean; 582 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-09-19T06:40:12.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:11 smithi151 ceph-mon[35236]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 -- 172.21.15.37:0/1571417734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c10c160 msgr2=0x7fa29c10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1571417734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c10c160 0x7fa29c10e600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa28c009ca0 tx=0x7fa28c02a950 comp rx=0 tx=0).stop 2024-09-19T06:40:12.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 -- 172.21.15.37:0/1571417734 shutdown_connections 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1571417734 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c10c160 0x7fa29c10e600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1571417734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa29c103650 0x7fa29c10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1571417734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa29c102d30 0x7fa29c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 -- 172.21.15.37:0/1571417734 >> 172.21.15.37:0/1571417734 conn(0x7fa29c0fc7d0 msgr2=0x7fa29c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 -- 172.21.15.37:0/1571417734 shutdown_connections 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 -- 172.21.15.37:0/1571417734 wait complete. 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 Processor -- start 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.493+0000 7fa2a418f640 1 -- start start 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a418f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c102d30 0x7fa29c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a418f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa29c103650 0x7fa29c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a418f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa29c10c160 0x7fa29c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a418f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c111f40 con 0x7fa29c102d30 2024-09-19T06:40:12.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a418f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c111dc0 con 0x7fa29c103650 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a418f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa29c1120c0 con 0x7fa29c10c160 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c102d30 0x7fa29c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c102d30 0x7fa29c198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51284/0 (socket says 172.21.15.37:51284) 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 -- 172.21.15.37:0/1960269826 learned_addr learned my addr 172.21.15.37:0/1960269826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a172a640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa29c103650 0x7fa29c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 -- 172.21.15.37:0/1960269826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa29c10c160 msgr2=0x7fa29c19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa29c10c160 0x7fa29c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 -- 172.21.15.37:0/1960269826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa29c103650 msgr2=0x7fa29c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa29c103650 0x7fa29c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 -- 172.21.15.37:0/1960269826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa29c19d3e0 con 0x7fa29c102d30 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa2a1f2b640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c102d30 0x7fa29c198430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa29400d950 tx=0x7fa29400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:12.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa29affd640 1 -- 172.21.15.37:0/1960269826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa294017070 con 0x7fa29c102d30 2024-09-19T06:40:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa29affd640 1 -- 172.21.15.37:0/1960269826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa294009c30 con 0x7fa29c102d30 2024-09-19T06:40:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.494+0000 7fa29affd640 1 -- 172.21.15.37:0/1960269826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa29400f6b0 con 0x7fa29c102d30 2024-09-19T06:40:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.495+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa29c19d6d0 con 0x7fa29c102d30 2024-09-19T06:40:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.495+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa29c1a4fb0 con 0x7fa29c102d30 2024-09-19T06:40:12.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.496+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa29c103110 con 0x7fa29c102d30 2024-09-19T06:40:12.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.497+0000 7fa29affd640 1 -- 172.21.15.37:0/1960269826 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa2940109d0 con 0x7fa29c102d30 2024-09-19T06:40:12.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.497+0000 7fa29affd640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa2840a3350 0x7fa2840a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:12.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.498+0000 7fa2a172a640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa2840a3350 0x7fa2840a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:12.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.498+0000 7fa29affd640 1 -- 172.21.15.37:0/1960269826 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa2940cf530 con 0x7fa29c102d30 2024-09-19T06:40:12.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.498+0000 7fa2a172a640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa2840a3350 0x7fa2840a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2900058c0 tx=0x7fa290005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:12.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.502+0000 7fa29affd640 1 -- 172.21.15.37:0/1960269826 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa294009de0 con 0x7fa29c102d30 2024-09-19T06:40:12.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.704+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fa29c062630 con 0x7fa2840a3350 2024-09-19T06:40:12.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.708+0000 7fa29affd640 1 -- 172.21.15.37:0/1960269826 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fa29c062630 con 0x7fa2840a3350 2024-09-19T06:40:12.709 INFO:teuthology.orchestra.run.smithi037.stdout:/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4 2024-09-19T06:40:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa2840a3350 msgr2=0x7fa2840a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa2840a3350 0x7fa2840a5810 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fa2900058c0 tx=0x7fa290005850 comp rx=0 tx=0).stop 2024-09-19T06:40:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c102d30 msgr2=0x7fa29c198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c102d30 0x7fa29c198430 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fa29400d950 tx=0x7fa29400de10 comp rx=0 tx=0).stop 2024-09-19T06:40:12.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 shutdown_connections 2024-09-19T06:40:12.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa2840a3350 0x7fa2840a5810 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa29c10c160 0x7fa29c19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa29c103650 0x7fa29c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 --2- 172.21.15.37:0/1960269826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa29c102d30 0x7fa29c198430 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:12.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.710+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 >> 172.21.15.37:0/1960269826 conn(0x7fa29c0fc7d0 msgr2=0x7fa29c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:12.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.711+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 shutdown_connections 2024-09-19T06:40:12.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:12.711+0000 7fa2a418f640 1 -- 172.21.15.37:0/1960269826 wait complete. 2024-09-19T06:40:13.424 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:40:13.685 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:13.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:13 smithi037 ceph-mon[31211]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-09-19T06:40:13.955 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:13 smithi037 ceph-mon[31211]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:13 smithi096 ceph-mon[35004]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-09-19T06:40:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:13 smithi096 ceph-mon[35004]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:14.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:13 smithi151 ceph-mon[35236]: pgmap v289: 97 pgs: 97 active+clean; 621 KiB data, 351 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 5.8 KiB/s wr, 13 op/s 2024-09-19T06:40:14.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:13 smithi151 ceph-mon[35236]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:40:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 -- 172.21.15.37:0/3812689270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8102790 msgr2=0x7f5db81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/3812689270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8102790 0x7f5db81067e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5db4009900 tx=0x7f5db402a3f0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 -- 172.21.15.37:0/3812689270 shutdown_connections 2024-09-19T06:40:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/3812689270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5db8106f10 0x7f5db810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/3812689270 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8102790 0x7f5db81067e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/3812689270 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5db8101de0 0x7f5db81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 -- 172.21.15.37:0/3812689270 >> 172.21.15.37:0/3812689270 conn(0x7f5db80fd8c0 msgr2=0x7f5db80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 -- 172.21.15.37:0/3812689270 shutdown_connections 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.134+0000 7f5dc03d3640 1 -- 172.21.15.37:0/3812689270 wait complete. 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 Processor -- start 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 -- start start 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5db8101de0 0x7f5db81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5db8102790 0x7f5db8198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8106f10 0x7f5db819cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8111f70 con 0x7f5db8106f10 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8111df0 con 0x7f5db8101de0 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dc03d3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5db81120f0 con 0x7f5db8102790 2024-09-19T06:40:14.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dbe970640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8106f10 0x7f5db819cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dbe970640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8106f10 0x7f5db819cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51298/0 (socket says 172.21.15.37:51298) 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.135+0000 7f5dbe970640 1 -- 172.21.15.37:0/2669436978 learned_addr learned my addr 172.21.15.37:0/2669436978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbe16f640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5db8101de0 0x7f5db81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbd96e640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5db8102790 0x7f5db8198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbe970640 1 -- 172.21.15.37:0/2669436978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5db8102790 msgr2=0x7f5db8198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbe970640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5db8102790 0x7f5db8198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbe970640 1 -- 172.21.15.37:0/2669436978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5db8101de0 msgr2=0x7f5db81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbe970640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5db8101de0 0x7f5db81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbe970640 1 -- 172.21.15.37:0/2669436978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db819d370 con 0x7f5db8106f10 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dbe970640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8106f10 0x7f5db819cc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5db000cce0 tx=0x7f5db000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5daf7fe640 1 -- 172.21.15.37:0/2669436978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db000f450 con 0x7f5db8106f10 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5daf7fe640 1 -- 172.21.15.37:0/2669436978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5db000fad0 con 0x7f5db8106f10 2024-09-19T06:40:14.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db819d660 con 0x7f5db8106f10 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5daf7fe640 1 -- 172.21.15.37:0/2669436978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db0018b60 con 0x7f5db8106f10 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.136+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db81a4f40 con 0x7f5db8106f10 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.137+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db81021c0 con 0x7f5db8106f10 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.139+0000 7f5daf7fe640 1 -- 172.21.15.37:0/2669436978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5db000fc80 con 0x7f5db8106f10 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.140+0000 7f5daf7fe640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5da40a3350 0x7f5da40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.140+0000 7f5dbe16f640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5da40a3350 0x7f5da40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.140+0000 7f5daf7fe640 1 -- 172.21.15.37:0/2669436978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f5db00d0110 con 0x7f5db8106f10 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.140+0000 7f5dbe16f640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5da40a3350 0x7f5da40a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5da800abe0 tx=0x7f5da800a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:14.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.143+0000 7f5daf7fe640 1 -- 172.21.15.37:0/2669436978 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5db0014030 con 0x7f5db8106f10 2024-09-19T06:40:14.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.358+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5db8199720 con 0x7f5db8106f10 2024-09-19T06:40:14.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.358+0000 7f5daf7fe640 1 -- 172.21.15.37:0/2669436978 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11903 (secure 0 0 0) 0x7f5db009ce00 con 0x7f5db8106f10 2024-09-19T06:40:14.360 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:14.360 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"btime":"2024-09-19T06:40:04:443772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:40:04.075579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:40:14.360 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5da40a3350 msgr2=0x7f5da40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5da40a3350 0x7f5da40a5810 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5da800abe0 tx=0x7f5da800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8106f10 msgr2=0x7f5db819cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8106f10 0x7f5db819cc90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5db000cce0 tx=0x7f5db000a620 comp rx=0 tx=0).stop 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 shutdown_connections 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f5da40a3350 0x7f5da40a5810 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5db8106f10 0x7f5db819cc90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5db8102790 0x7f5db8198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 --2- 172.21.15.37:0/2669436978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5db8101de0 0x7f5db81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:14.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 >> 172.21.15.37:0/2669436978 conn(0x7f5db80fd8c0 msgr2=0x7f5db80febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:14.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 shutdown_connections 2024-09-19T06:40:14.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:14.361+0000 7f5dc03d3640 1 -- 172.21.15.37:0/2669436978 wait complete. 2024-09-19T06:40:14.946 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:14 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2669436978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:40:14.978 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:40:15.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:14 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2669436978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:40:15.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:14 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2669436978' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:40:15.235 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 -- 172.21.15.37:0/3342590072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f0102d30 msgr2=0x7f52f0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 --2- 172.21.15.37:0/3342590072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f0102d30 0x7f52f0103110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009930 tx=0x7f52ec02a440 comp rx=0 tx=0).stop 2024-09-19T06:40:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 -- 172.21.15.37:0/3342590072 shutdown_connections 2024-09-19T06:40:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 --2- 172.21.15.37:0/3342590072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52f01082a0 0x7f52f010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 --2- 172.21.15.37:0/3342590072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52f01036e0 0x7f52f0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 --2- 172.21.15.37:0/3342590072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f0102d30 0x7f52f0103110 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 -- 172.21.15.37:0/3342590072 >> 172.21.15.37:0/3342590072 conn(0x7f52f00fc7d0 msgr2=0x7f52f00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 -- 172.21.15.37:0/3342590072 shutdown_connections 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.673+0000 7f52f8317640 1 -- 172.21.15.37:0/3342590072 wait complete. 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 Processor -- start 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 -- start start 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52f0102d30 0x7f52f0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f01036e0 0x7f52f0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52f01082a0 0x7f52f019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0112160 con 0x7f52f01036e0 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52f0111fe0 con 0x7f52f01082a0 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f8317640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f52f01122e0 con 0x7f52f0102d30 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f58b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f01036e0 0x7f52f0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f58b2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f01036e0 0x7f52f0198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51326/0 (socket says 172.21.15.37:51326) 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f58b2640 1 -- 172.21.15.37:0/2260577332 learned_addr learned my addr 172.21.15.37:0/2260577332 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:15.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f60b3640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52f0102d30 0x7f52f0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.674+0000 7f52f68b4640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52f01082a0 0x7f52f019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f58b2640 1 -- 172.21.15.37:0/2260577332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52f0102d30 msgr2=0x7f52f0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f58b2640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52f0102d30 0x7f52f0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f58b2640 1 -- 172.21.15.37:0/2260577332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52f01082a0 msgr2=0x7f52f019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f58b2640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52f01082a0 0x7f52f019cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f58b2640 1 -- 172.21.15.37:0/2260577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f019d6d0 con 0x7f52f01036e0 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f68b4640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52f01082a0 0x7f52f019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f60b3640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52f0102d30 0x7f52f0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f58b2640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f01036e0 0x7f52f0198c60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52e0006fd0 tx=0x7f52e000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52e77fe640 1 -- 172.21.15.37:0/2260577332 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e0011830 con 0x7f52f01036e0 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52e77fe640 1 -- 172.21.15.37:0/2260577332 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52e0016070 con 0x7f52f01036e0 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f019d9c0 con 0x7f52f01036e0 2024-09-19T06:40:15.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52e77fe640 1 -- 172.21.15.37:0/2260577332 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52e000f2e0 con 0x7f52f01036e0 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.675+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f01a52a0 con 0x7f52f01036e0 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.676+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f006a850 con 0x7f52f01036e0 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.677+0000 7f52e77fe640 1 -- 172.21.15.37:0/2260577332 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f52e001b460 con 0x7f52f01036e0 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.678+0000 7f52e77fe640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f52dc0a3350 0x7f52dc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.678+0000 7f52f60b3640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f52dc0a3350 0x7f52dc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.678+0000 7f52e77fe640 1 -- 172.21.15.37:0/2260577332 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f52e00cee80 con 0x7f52f01036e0 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.678+0000 7f52f60b3640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f52dc0a3350 0x7f52dc0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52ec000c00 tx=0x7f52ec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:15.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.682+0000 7f52e77fe640 1 -- 172.21.15.37:0/2260577332 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f52e009bd00 con 0x7f52f01036e0 2024-09-19T06:40:15.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.861+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f52f010a2b0 con 0x7f52f01036e0 2024-09-19T06:40:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.863+0000 7f52e77fe640 1 -- 172.21.15.37:0/2260577332 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22268 (secure 0 0 0) 0x7f52e009b600 con 0x7f52f01036e0 2024-09-19T06:40:15.864 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:15.864 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:39:53.021912+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:40:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f52dc0a3350 msgr2=0x7f52dc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f52dc0a3350 0x7f52dc0a5810 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f52ec000c00 tx=0x7f52ec005a20 comp rx=0 tx=0).stop 2024-09-19T06:40:15.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f01036e0 msgr2=0x7f52f0198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f01036e0 0x7f52f0198c60 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f52e0006fd0 tx=0x7f52e000b8e0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 shutdown_connections 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f52dc0a3350 0x7f52dc0a5810 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52f01082a0 0x7f52f019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f52f01036e0 0x7f52f0198c60 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 --2- 172.21.15.37:0/2260577332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f52f0102d30 0x7f52f0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 >> 172.21.15.37:0/2260577332 conn(0x7f52f00fc7d0 msgr2=0x7f52f0105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 shutdown_connections 2024-09-19T06:40:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:15.865+0000 7f52f8317640 1 -- 172.21.15.37:0/2260577332 wait complete. 2024-09-19T06:40:16.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:15 smithi096 ceph-mon[35004]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-19T06:40:16.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:15 smithi037 ceph-mon[31211]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-19T06:40:16.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:15 smithi151 ceph-mon[35236]: pgmap v290: 97 pgs: 97 active+clean; 621 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.8 KiB/s wr, 14 op/s 2024-09-19T06:40:16.568 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:40:16.568 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:40:16.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:16 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2260577332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:16.823 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:17.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:16 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2260577332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:17.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:16 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2260577332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:17.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 -- 172.21.15.37:0/4217714913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c1082a0 msgr2=0x7f6b0c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 --2- 172.21.15.37:0/4217714913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c10ec40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6b04009cd0 tx=0x7f6b0402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 -- 172.21.15.37:0/4217714913 shutdown_connections 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 --2- 172.21.15.37:0/4217714913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c10ec40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 --2- 172.21.15.37:0/4217714913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 --2- 172.21.15.37:0/4217714913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b0c102d30 0x7f6b0c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 -- 172.21.15.37:0/4217714913 >> 172.21.15.37:0/4217714913 conn(0x7f6b0c0fc7d0 msgr2=0x7f6b0c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 -- 172.21.15.37:0/4217714913 shutdown_connections 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 -- 172.21.15.37:0/4217714913 wait complete. 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 Processor -- start 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.207+0000 7f6b14532640 1 -- start start 2024-09-19T06:40:17.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b14532640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c102d30 0x7f6b0c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b14532640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b14532640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b14532640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c112100 con 0x7f6b0c102d30 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b14532640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c111f80 con 0x7f6b0c1036e0 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b14532640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6b0c112280 con 0x7f6b0c1082a0 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c102d30 0x7f6b0c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c102d30 0x7f6b0c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51352/0 (socket says 172.21.15.37:51352) 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 -- 172.21.15.37:0/3119226430 learned_addr learned my addr 172.21.15.37:0/3119226430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b11acd640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b12acf640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 -- 172.21.15.37:0/3119226430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b0c1082a0 msgr2=0x7f6b0c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:17.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 -- 172.21.15.37:0/3119226430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b0c1036e0 msgr2=0x7f6b0c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 -- 172.21.15.37:0/3119226430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b0c1a6a10 con 0x7f6b0c102d30 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.208+0000 7f6b122ce640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c102d30 0x7f6b0c100990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800b4a0 tx=0x7f6b0800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.209+0000 7f6b037fe640 1 -- 172.21.15.37:0/3119226430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b08015020 con 0x7f6b0c102d30 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.209+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b0c1a6d00 con 0x7f6b0c102d30 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.209+0000 7f6b037fe640 1 -- 172.21.15.37:0/3119226430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b08011d10 con 0x7f6b0c102d30 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.209+0000 7f6b037fe640 1 -- 172.21.15.37:0/3119226430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800f2e0 con 0x7f6b0c102d30 2024-09-19T06:40:17.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.209+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b0c1a7190 con 0x7f6b0c102d30 2024-09-19T06:40:17.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.210+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b0c06a850 con 0x7f6b0c102d30 2024-09-19T06:40:17.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.211+0000 7f6b037fe640 1 -- 172.21.15.37:0/3119226430 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6b08018020 con 0x7f6b0c102d30 2024-09-19T06:40:17.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.212+0000 7f6b037fe640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6af80a32d0 0x7f6af80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.212+0000 7f6b11acd640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6af80a32d0 0x7f6af80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.212+0000 7f6b037fe640 1 -- 172.21.15.37:0/3119226430 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f6b080cf120 con 0x7f6b0c102d30 2024-09-19T06:40:17.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.212+0000 7f6b11acd640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6af80a32d0 0x7f6af80a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c102420 tx=0x7f6afc00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:17.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.216+0000 7f6b037fe640 1 -- 172.21.15.37:0/3119226430 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6b0801f020 con 0x7f6b0c102d30 2024-09-19T06:40:17.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.388+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6b0c10a320 con 0x7f6af80a32d0 2024-09-19T06:40:17.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.398+0000 7f6b037fe640 1 -- 172.21.15.37:0/3119226430 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7f6afc00fba0 con 0x7f6af80a32d0 2024-09-19T06:40:17.401 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:17.403 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:40:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6af80a32d0 msgr2=0x7f6af80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6af80a32d0 0x7f6af80a5790 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c102420 tx=0x7f6afc00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c102d30 msgr2=0x7f6b0c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c102d30 0x7f6b0c100990 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800b4a0 tx=0x7f6b0800b8a0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 shutdown_connections 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f6af80a32d0 0x7f6af80a5790 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b0c1082a0 0x7f6b0c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6b0c1036e0 0x7f6b0c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 --2- 172.21.15.37:0/3119226430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6b0c102d30 0x7f6b0c100990 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.400+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 >> 172.21.15.37:0/3119226430 conn(0x7f6b0c0fc7d0 msgr2=0x7f6b0c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.401+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 shutdown_connections 2024-09-19T06:40:17.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:17.401+0000 7f6b14532640 1 -- 172.21.15.37:0/3119226430 wait complete. 2024-09-19T06:40:17.976 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:17 smithi096 ceph-mon[35004]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-19T06:40:18.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:17 smithi037 ceph-mon[31211]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-19T06:40:18.094 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-19T06:40:16.196532+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641407,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":238,"num_read_kb":226,"num_write":335,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359964,"kb_used_data":10716,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581860,"statfs":{"total":1151940427776,"available":1151571824640,"internally_reserved":0,"allocated":10973184,"data_stored":6956625,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45951,"num_objects":14,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":85,"num_read_kb":93,"num_write":56,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003007"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403605+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:39:53.403605+0000","last_peered":"2024-09-19T06:39:53.403605+0000","last_clean":"2024-09-19T06:39:53.403605+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:39:53.403605+0000","last_undegraded":"2024-09-19T06:39:53.403605+0000","last_fullsized":"2024-09-19T06:39:53.403605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419068+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:39:53.419068+0000","last_peered":"2024-09-19T06:39:53.419068+0000","last_clean":"2024-09-19T06:39:53.419068+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:39:53.419068+0000","last_undegraded":"2024-09-19T06:39:53.419068+0000","last_fullsized":"2024-09-19T06:39:53.419068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419285+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:39:53.419285+0000","last_peered":"2024-09-19T06:39:53.419285+0000","last_clean":"2024-09-19T06:39:53.419285+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:39:53.419285+0000","last_undegraded":"2024-09-19T06:39:53.419285+0000","last_fullsized":"2024-09-19T06:39:53.419285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418707+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:39:53.418707+0000","last_peered":"2024-09-19T06:39:53.418707+0000","last_clean":"2024-09-19T06:39:53.418707+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:39:53.418707+0000","last_undegraded":"2024-09-19T06:39:53.418707+0000","last_fullsized":"2024-09-19T06:39:53.418707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400788+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:39:53.400788+0000","last_peered":"2024-09-19T06:39:53.400788+0000","last_clean":"2024-09-19T06:39:53.400788+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:39:53.400788+0000","last_undegraded":"2024-09-19T06:39:53.400788+0000","last_fullsized":"2024-09-19T06:39:53.400788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417175+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:39:53.417175+0000","last_peered":"2024-09-19T06:39:53.417175+0000","last_clean":"2024-09-19T06:39:53.417175+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:39:53.417175+0000","last_undegraded":"2024-09-19T06:39:53.417175+0000","last_fullsized":"2024-09-19T06:39:53.417175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415134+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:39:53.415134+0000","last_peered":"2024-09-19T06:39:53.415134+0000","last_clean":"2024-09-19T06:39:53.415134+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:39:53.415134+0000","last_undegraded":"2024-09-19T06:39:53.415134+0000","last_fullsized":"2024-09-19T06:39:53.415134+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414333+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:39:53.414333+0000","last_peered":"2024-09-19T06:39:53.414333+0000","last_clean":"2024-09-19T06:39:53.414333+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:39:53.414333+0000","last_undegraded":"2024-09-19T06:39:53.414333+0000","last_fullsized":"2024-09-19T06:39:53.414333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.401273+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:39:53.401273+0000","last_peered":"2024-09-19T06:39:53.401273+0000","last_clean":"2024-09-19T06:39:53.401273+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:39:53.401273+0000","last_undegraded":"2024-09-19T06:39:53.401273+0000","last_fullsized":"2024-09-19T06:39:53.401273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417564+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:39:53.417564+0000","last_peered":"2024-09-19T06:39:53.417564+0000","last_clean":"2024-09-19T06:39:53.417564+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:39:53.417564+0000","last_undegraded":"2024-09-19T06:39:53.417564+0000","last_fullsized":"2024-09-19T06:39:53.417564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415382+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:39:53.415382+0000","last_peered":"2024-09-19T06:39:53.415382+0000","last_clean":"2024-09-19T06:39:53.415382+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:39:53.415382+0000","last_undegraded":"2024-09-19T06:39:53.415382+0000","last_fullsized":"2024-09-19T06:39:53.415382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400663+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:39:53.400663+0000","last_peered":"2024-09-19T06:39:53.400663+0000","last_clean":"2024-09-19T06:39:53.400663+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:39:53.400663+0000","last_undegraded":"2024-09-19T06:39:53.400663+0000","last_fullsized":"2024-09-19T06:39:53.400663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419338+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:39:53.419338+0000","last_peered":"2024-09-19T06:39:53.419338+0000","last_clean":"2024-09-19T06:39:53.419338+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:39:53.419338+0000","last_undegraded":"2024-09-19T06:39:53.419338+0000","last_fullsized":"2024-09-19T06:39:53.419338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418306+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:39:53.418306+0000","last_peered":"2024-09-19T06:39:53.418306+0000","last_clean":"2024-09-19T06:39:53.418306+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:39:53.418306+0000","last_undegraded":"2024-09-19T06:39:53.418306+0000","last_fullsized":"2024-09-19T06:39:53.418306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413939+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:39:53.413939+0000","last_peered":"2024-09-19T06:39:53.413939+0000","last_clean":"2024-09-19T06:39:53.413939+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:39:53.413939+0000","last_undegraded":"2024-09-19T06:39:53.413939+0000","last_fullsized":"2024-09-19T06:39:53.413939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403646+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:39:53.403646+0000","last_peered":"2024-09-19T06:39:53.403646+0000","last_clean":"2024-09-19T06:39:53.403646+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:39:53.403646+0000","last_undegraded":"2024-09-19T06:39:53.403646+0000","last_fullsized":"2024-09-19T06:39:53.403646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414294+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:39:53.414294+0000","last_peered":"2024-09-19T06:39:53.414294+0000","last_clean":"2024-09-19T06:39:53.414294+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:39:53.414294+0000","last_undegraded":"2024-09-19T06:39:53.414294+0000","last_fullsized":"2024-09-19T06:39:53.414294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415375+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:39:53.415375+0000","last_peered":"2024-09-19T06:39:53.415375+0000","last_clean":"2024-09-19T06:39:53.415375+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:39:53.415375+0000","last_undegraded":"2024-09-19T06:39:53.415375+0000","last_fullsized":"2024-09-19T06:39:53.415375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403279+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:39:53.403279+0000","last_peered":"2024-09-19T06:39:53.403279+0000","last_clean":"2024-09-19T06:39:53.403279+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:39:53.403279+0000","last_undegraded":"2024-09-19T06:39:53.403279+0000","last_fullsized":"2024-09-19T06:39:53.403279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400705+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:39:53.400705+0000","last_peered":"2024-09-19T06:39:53.400705+0000","last_clean":"2024-09-19T06:39:53.400705+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:39:53.400705+0000","last_undegraded":"2024-09-19T06:39:53.400705+0000","last_fullsized":"2024-09-19T06:39:53.400705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414105+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:39:53.414105+0000","last_peered":"2024-09-19T06:39:53.414105+0000","last_clean":"2024-09-19T06:39:53.414105+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:39:53.414105+0000","last_undegraded":"2024-09-19T06:39:53.414105+0000","last_fullsized":"2024-09-19T06:39:53.414105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415846+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:39:53.415846+0000","last_peered":"2024-09-19T06:39:53.415846+0000","last_clean":"2024-09-19T06:39:53.415846+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:39:53.415846+0000","last_undegraded":"2024-09-19T06:39:53.415846+0000","last_fullsized":"2024-09-19T06:39:53.415846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414231+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:39:53.414231+0000","last_peered":"2024-09-19T06:39:53.414231+0000","last_clean":"2024-09-19T06:39:53.414231+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:39:53.414231+0000","last_undegraded":"2024-09-19T06:39:53.414231+0000","last_fullsized":"2024-09-19T06:39:53.414231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416096+0000","last_change":"2024-09-19T06:39:52.399797+0000","last_active":"2024-09-19T06:39:53.416096+0000","last_peered":"2024-09-19T06:39:53.416096+0000","last_clean":"2024-09-19T06:39:53.416096+0000","last_became_active":"2024-09-19T06:39:52.399452+0000","last_became_peered":"2024-09-19T06:39:52.399452+0000","last_unstale":"2024-09-19T06:39:53.416096+0000","last_undegraded":"2024-09-19T06:39:53.416096+0000","last_fullsized":"2024-09-19T06:39:53.416096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:33.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415925+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:39:53.415925+0000","last_peered":"2024-09-19T06:39:53.415925+0000","last_clean":"2024-09-19T06:39:53.415925+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:39:53.415925+0000","last_undegraded":"2024-09-19T06:39:53.415925+0000","last_fullsized":"2024-09-19T06:39:53.415925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403645+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:39:53.403645+0000","last_peered":"2024-09-19T06:39:53.403645+0000","last_clean":"2024-09-19T06:39:53.403645+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:39:53.403645+0000","last_undegraded":"2024-09-19T06:39:53.403645+0000","last_fullsized":"2024-09-19T06:39:53.403645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400700+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:39:53.400700+0000","last_peered":"2024-09-19T06:39:53.400700+0000","last_clean":"2024-09-19T06:39:53.400700+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:39:53.400700+0000","last_undegraded":"2024-09-19T06:39:53.400700+0000","last_fullsized":"2024-09-19T06:39:53.400700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419814+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:39:53.419814+0000","last_peered":"2024-09-19T06:39:53.419814+0000","last_clean":"2024-09-19T06:39:53.419814+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:39:53.419814+0000","last_undegraded":"2024-09-19T06:39:53.419814+0000","last_fullsized":"2024-09-19T06:39:53.419814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414873+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:39:53.414873+0000","last_peered":"2024-09-19T06:39:53.414873+0000","last_clean":"2024-09-19T06:39:53.414873+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:39:53.414873+0000","last_undegraded":"2024-09-19T06:39:53.414873+0000","last_fullsized":"2024-09-19T06:39:53.414873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414716+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:39:53.414716+0000","last_peered":"2024-09-19T06:39:53.414716+0000","last_clean":"2024-09-19T06:39:53.414716+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:39:53.414716+0000","last_undegraded":"2024-09-19T06:39:53.414716+0000","last_fullsized":"2024-09-19T06:39:53.414716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403217+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:39:53.403217+0000","last_peered":"2024-09-19T06:39:53.403217+0000","last_clean":"2024-09-19T06:39:53.403217+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:39:53.403217+0000","last_undegraded":"2024-09-19T06:39:53.403217+0000","last_fullsized":"2024-09-19T06:39:53.403217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":36,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:11.466424+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:40:11.466424+0000","last_peered":"2024-09-19T06:40:11.466424+0000","last_clean":"2024-09-19T06:40:11.466424+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:40:11.466424+0000","last_undegraded":"2024-09-19T06:40:11.466424+0000","last_fullsized":"2024-09-19T06:40:11.466424+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414773+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:39:53.414773+0000","last_peered":"2024-09-19T06:39:53.414773+0000","last_clean":"2024-09-19T06:39:53.414773+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:39:53.414773+0000","last_undegraded":"2024-09-19T06:39:53.414773+0000","last_fullsized":"2024-09-19T06:39:53.414773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.446851+0000","last_change":"2024-09-19T06:39:50.318132+0000","last_active":"2024-09-19T06:40:01.446851+0000","last_peered":"2024-09-19T06:40:01.446851+0000","last_clean":"2024-09-19T06:40:01.446851+0000","last_became_active":"2024-09-19T06:39:50.317774+0000","last_became_peered":"2024-09-19T06:39:50.317774+0000","last_unstale":"2024-09-19T06:40:01.446851+0000","last_undegraded":"2024-09-19T06:40:01.446851+0000","last_fullsized":"2024-09-19T06:40:01.446851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:58.290128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.445718+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:40:01.445718+0000","last_peered":"2024-09-19T06:40:01.445718+0000","last_clean":"2024-09-19T06:40:01.445718+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:40:01.445718+0000","last_undegraded":"2024-09-19T06:40:01.445718+0000","last_fullsized":"2024-09-19T06:40:01.445718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415204+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:39:53.415204+0000","last_peered":"2024-09-19T06:39:53.415204+0000","last_clean":"2024-09-19T06:39:53.415204+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:39:53.415204+0000","last_undegraded":"2024-09-19T06:39:53.415204+0000","last_fullsized":"2024-09-19T06:39:53.415204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414736+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:39:53.414736+0000","last_peered":"2024-09-19T06:39:53.414736+0000","last_clean":"2024-09-19T06:39:53.414736+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:39:53.414736+0000","last_undegraded":"2024-09-19T06:39:53.414736+0000","last_fullsized":"2024-09-19T06:39:53.414736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413749+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:39:53.413749+0000","last_peered":"2024-09-19T06:39:53.413749+0000","last_clean":"2024-09-19T06:39:53.413749+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:39:53.413749+0000","last_undegraded":"2024-09-19T06:39:53.413749+0000","last_fullsized":"2024-09-19T06:39:53.413749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:14.470266+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:40:14.470266+0000","last_peered":"2024-09-19T06:40:14.470266+0000","last_clean":"2024-09-19T06:40:14.470266+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:40:14.470266+0000","last_undegraded":"2024-09-19T06:40:14.470266+0000","last_fullsized":"2024-09-19T06:40:14.470266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414187+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:39:53.414187+0000","last_peered":"2024-09-19T06:39:53.414187+0000","last_clean":"2024-09-19T06:39:53.414187+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:39:53.414187+0000","last_undegraded":"2024-09-19T06:39:53.414187+0000","last_fullsized":"2024-09-19T06:39:53.414187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438327+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:40:00.438327+0000","last_peered":"2024-09-19T06:40:00.438327+0000","last_clean":"2024-09-19T06:40:00.438327+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:40:00.438327+0000","last_undegraded":"2024-09-19T06:40:00.438327+0000","last_fullsized":"2024-09-19T06:40:00.438327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419760+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:39:53.419760+0000","last_peered":"2024-09-19T06:39:53.419760+0000","last_clean":"2024-09-19T06:39:53.419760+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:39:53.419760+0000","last_undegraded":"2024-09-19T06:39:53.419760+0000","last_fullsized":"2024-09-19T06:39:53.419760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437339+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:40:00.437339+0000","last_peered":"2024-09-19T06:40:00.437339+0000","last_clean":"2024-09-19T06:40:00.437339+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:40:00.437339+0000","last_undegraded":"2024-09-19T06:40:00.437339+0000","last_fullsized":"2024-09-19T06:40:00.437339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414395+0000","last_change":"2024-09-19T06:39:52.391059+0000","last_active":"2024-09-19T06:39:53.414395+0000","last_peered":"2024-09-19T06:39:53.414395+0000","last_clean":"2024-09-19T06:39:53.414395+0000","last_became_active":"2024-09-19T06:39:52.389703+0000","last_became_peered":"2024-09-19T06:39:52.389703+0000","last_unstale":"2024-09-19T06:39:53.414395+0000","last_undegraded":"2024-09-19T06:39:53.414395+0000","last_fullsized":"2024-09-19T06:39:53.414395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:32.021109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'22","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:15.473350+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:40:15.473350+0000","last_peered":"2024-09-19T06:40:15.473350+0000","last_clean":"2024-09-19T06:40:15.473350+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:40:15.473350+0000","last_undegraded":"2024-09-19T06:40:15.473350+0000","last_fullsized":"2024-09-19T06:40:15.473350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":62,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415675+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:39:53.415675+0000","last_peered":"2024-09-19T06:39:53.415675+0000","last_clean":"2024-09-19T06:39:53.415675+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:39:53.415675+0000","last_undegraded":"2024-09-19T06:39:53.415675+0000","last_fullsized":"2024-09-19T06:39:53.415675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447875+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:40:01.447875+0000","last_peered":"2024-09-19T06:40:01.447875+0000","last_clean":"2024-09-19T06:40:01.447875+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:40:01.447875+0000","last_undegraded":"2024-09-19T06:40:01.447875+0000","last_fullsized":"2024-09-19T06:40:01.447875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437843+0000","last_change":"2024-09-19T06:39:50.312876+0000","last_active":"2024-09-19T06:40:00.437843+0000","last_peered":"2024-09-19T06:40:00.437843+0000","last_clean":"2024-09-19T06:40:00.437843+0000","last_became_active":"2024-09-19T06:39:50.312500+0000","last_became_peered":"2024-09-19T06:39:50.312500+0000","last_unstale":"2024-09-19T06:40:00.437843+0000","last_undegraded":"2024-09-19T06:40:00.437843+0000","last_fullsized":"2024-09-19T06:40:00.437843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:00.113022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417798+0000","last_change":"2024-09-19T06:39:52.399395+0000","last_active":"2024-09-19T06:39:53.417798+0000","last_peered":"2024-09-19T06:39:53.417798+0000","last_clean":"2024-09-19T06:39:53.417798+0000","last_became_active":"2024-09-19T06:39:52.399042+0000","last_became_peered":"2024-09-19T06:39:52.399042+0000","last_unstale":"2024-09-19T06:39:53.417798+0000","last_undegraded":"2024-09-19T06:39:53.417798+0000","last_fullsized":"2024-09-19T06:39:53.417798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:10.010156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:12.467192+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:40:12.467192+0000","last_peered":"2024-09-19T06:40:12.467192+0000","last_clean":"2024-09-19T06:40:12.467192+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:40:12.467192+0000","last_undegraded":"2024-09-19T06:40:12.467192+0000","last_fullsized":"2024-09-19T06:40:12.467192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416672+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:39:53.416672+0000","last_peered":"2024-09-19T06:39:53.416672+0000","last_clean":"2024-09-19T06:39:53.416672+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:39:53.416672+0000","last_undegraded":"2024-09-19T06:39:53.416672+0000","last_fullsized":"2024-09-19T06:39:53.416672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448340+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:40:01.448340+0000","last_peered":"2024-09-19T06:40:01.448340+0000","last_clean":"2024-09-19T06:40:01.448340+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:40:01.448340+0000","last_undegraded":"2024-09-19T06:40:01.448340+0000","last_fullsized":"2024-09-19T06:40:01.448340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415210+0000","last_change":"2024-09-19T06:39:52.390919+0000","last_active":"2024-09-19T06:39:53.415210+0000","last_peered":"2024-09-19T06:39:53.415210+0000","last_clean":"2024-09-19T06:39:53.415210+0000","last_became_active":"2024-09-19T06:39:52.390585+0000","last_became_peered":"2024-09-19T06:39:52.390585+0000","last_unstale":"2024-09-19T06:39:53.415210+0000","last_undegraded":"2024-09-19T06:39:53.415210+0000","last_fullsized":"2024-09-19T06:39:53.415210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.164497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.451484+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:40:02.451484+0000","last_peered":"2024-09-19T06:40:02.451484+0000","last_clean":"2024-09-19T06:40:02.451484+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:40:02.451484+0000","last_undegraded":"2024-09-19T06:40:02.451484+0000","last_fullsized":"2024-09-19T06:40:02.451484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403462+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:39:53.403462+0000","last_peered":"2024-09-19T06:39:53.403462+0000","last_clean":"2024-09-19T06:39:53.403462+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:39:53.403462+0000","last_undegraded":"2024-09-19T06:39:53.403462+0000","last_fullsized":"2024-09-19T06:39:53.403462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418357+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:39:53.418357+0000","last_peered":"2024-09-19T06:39:53.418357+0000","last_clean":"2024-09-19T06:39:53.418357+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:39:53.418357+0000","last_undegraded":"2024-09-19T06:39:53.418357+0000","last_fullsized":"2024-09-19T06:39:53.418357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418035+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:39:53.418035+0000","last_peered":"2024-09-19T06:39:53.418035+0000","last_clean":"2024-09-19T06:39:53.418035+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:39:53.418035+0000","last_undegraded":"2024-09-19T06:39:53.418035+0000","last_fullsized":"2024-09-19T06:39:53.418035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419293+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:39:53.419293+0000","last_peered":"2024-09-19T06:39:53.419293+0000","last_clean":"2024-09-19T06:39:53.419293+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:39:53.419293+0000","last_undegraded":"2024-09-19T06:39:53.419293+0000","last_fullsized":"2024-09-19T06:39:53.419293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144684+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:39:54.144684+0000","last_peered":"2024-09-19T06:39:54.144684+0000","last_clean":"2024-09-19T06:39:54.144684+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:39:54.144684+0000","last_undegraded":"2024-09-19T06:39:54.144684+0000","last_fullsized":"2024-09-19T06:39:54.144684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447446+0000","last_change":"2024-09-19T06:39:50.507312+0000","last_active":"2024-09-19T06:40:01.447446+0000","last_peered":"2024-09-19T06:40:01.447446+0000","last_clean":"2024-09-19T06:40:01.447446+0000","last_became_active":"2024-09-19T06:39:50.506976+0000","last_became_peered":"2024-09-19T06:39:50.506976+0000","last_unstale":"2024-09-19T06:40:01.447446+0000","last_undegraded":"2024-09-19T06:40:01.447446+0000","last_fullsized":"2024-09-19T06:40:01.447446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:44.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417964+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:39:53.417964+0000","last_peered":"2024-09-19T06:39:53.417964+0000","last_clean":"2024-09-19T06:39:53.417964+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:39:53.417964+0000","last_undegraded":"2024-09-19T06:39:53.417964+0000","last_fullsized":"2024-09-19T06:39:53.417964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417272+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:39:53.417272+0000","last_peered":"2024-09-19T06:39:53.417272+0000","last_clean":"2024-09-19T06:39:53.417272+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:39:53.417272+0000","last_undegraded":"2024-09-19T06:39:53.417272+0000","last_fullsized":"2024-09-19T06:39:53.417272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.450445+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:40:02.450445+0000","last_peered":"2024-09-19T06:40:02.450445+0000","last_clean":"2024-09-19T06:40:02.450445+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:40:02.450445+0000","last_undegraded":"2024-09-19T06:40:02.450445+0000","last_fullsized":"2024-09-19T06:40:02.450445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415347+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:39:53.415347+0000","last_peered":"2024-09-19T06:39:53.415347+0000","last_clean":"2024-09-19T06:39:53.415347+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:39:53.415347+0000","last_undegraded":"2024-09-19T06:39:53.415347+0000","last_fullsized":"2024-09-19T06:39:53.415347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447538+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:40:01.447538+0000","last_peered":"2024-09-19T06:40:01.447538+0000","last_clean":"2024-09-19T06:40:01.447538+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:40:01.447538+0000","last_undegraded":"2024-09-19T06:40:01.447538+0000","last_fullsized":"2024-09-19T06:40:01.447538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400695+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:39:53.400695+0000","last_peered":"2024-09-19T06:39:53.400695+0000","last_clean":"2024-09-19T06:39:53.400695+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:39:53.400695+0000","last_undegraded":"2024-09-19T06:39:53.400695+0000","last_fullsized":"2024-09-19T06:39:53.400695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:12.466648+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:40:12.466648+0000","last_peered":"2024-09-19T06:40:12.466648+0000","last_clean":"2024-09-19T06:40:12.466648+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:40:12.466648+0000","last_undegraded":"2024-09-19T06:40:12.466648+0000","last_fullsized":"2024-09-19T06:40:12.466648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414405+0000","last_change":"2024-09-19T06:39:52.396795+0000","last_active":"2024-09-19T06:39:53.414405+0000","last_peered":"2024-09-19T06:39:53.414405+0000","last_clean":"2024-09-19T06:39:53.414405+0000","last_became_active":"2024-09-19T06:39:52.396481+0000","last_became_peered":"2024-09-19T06:39:52.396481+0000","last_unstale":"2024-09-19T06:39:53.414405+0000","last_undegraded":"2024-09-19T06:39:53.414405+0000","last_fullsized":"2024-09-19T06:39:53.414405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:05.637504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144665+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:39:54.144665+0000","last_peered":"2024-09-19T06:39:54.144665+0000","last_clean":"2024-09-19T06:39:54.144665+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:39:54.144665+0000","last_undegraded":"2024-09-19T06:39:54.144665+0000","last_fullsized":"2024-09-19T06:39:54.144665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400964+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:39:53.400964+0000","last_peered":"2024-09-19T06:39:53.400964+0000","last_clean":"2024-09-19T06:39:53.400964+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:39:53.400964+0000","last_undegraded":"2024-09-19T06:39:53.400964+0000","last_fullsized":"2024-09-19T06:39:53.400964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414089+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:39:53.414089+0000","last_peered":"2024-09-19T06:39:53.414089+0000","last_clean":"2024-09-19T06:39:53.414089+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:39:53.414089+0000","last_undegraded":"2024-09-19T06:39:53.414089+0000","last_fullsized":"2024-09-19T06:39:53.414089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:56.016828+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:39:56.016828+0000","last_peered":"2024-09-19T06:39:56.016828+0000","last_clean":"2024-09-19T06:39:56.016828+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:39:56.016828+0000","last_undegraded":"2024-09-19T06:39:56.016828+0000","last_fullsized":"2024-09-19T06:39:56.016828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437914+0000","last_change":"2024-09-19T06:39:50.506741+0000","last_active":"2024-09-19T06:40:00.437914+0000","last_peered":"2024-09-19T06:40:00.437914+0000","last_clean":"2024-09-19T06:40:00.437914+0000","last_became_active":"2024-09-19T06:39:50.506437+0000","last_became_peered":"2024-09-19T06:39:50.506437+0000","last_unstale":"2024-09-19T06:40:00.437914+0000","last_undegraded":"2024-09-19T06:40:00.437914+0000","last_fullsized":"2024-09-19T06:40:00.437914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:42.304929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:13.415103+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:40:13.415103+0000","last_peered":"2024-09-19T06:40:13.415103+0000","last_clean":"2024-09-19T06:40:13.415103+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:40:13.415103+0000","last_undegraded":"2024-09-19T06:40:13.415103+0000","last_fullsized":"2024-09-19T06:40:13.415103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416102+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:39:53.416102+0000","last_peered":"2024-09-19T06:39:53.416102+0000","last_clean":"2024-09-19T06:39:53.416102+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:39:53.416102+0000","last_undegraded":"2024-09-19T06:39:53.416102+0000","last_fullsized":"2024-09-19T06:39:53.416102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.436752+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:40:00.436752+0000","last_peered":"2024-09-19T06:40:00.436752+0000","last_clean":"2024-09-19T06:40:00.436752+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:40:00.436752+0000","last_undegraded":"2024-09-19T06:40:00.436752+0000","last_fullsized":"2024-09-19T06:40:00.436752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419205+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:39:53.419205+0000","last_peered":"2024-09-19T06:39:53.419205+0000","last_clean":"2024-09-19T06:39:53.419205+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:39:53.419205+0000","last_undegraded":"2024-09-19T06:39:53.419205+0000","last_fullsized":"2024-09-19T06:39:53.419205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437748+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:40:00.437748+0000","last_peered":"2024-09-19T06:40:00.437748+0000","last_clean":"2024-09-19T06:40:00.437748+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:40:00.437748+0000","last_undegraded":"2024-09-19T06:40:00.437748+0000","last_fullsized":"2024-09-19T06:40:00.437748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414233+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:39:53.414233+0000","last_peered":"2024-09-19T06:39:53.414233+0000","last_clean":"2024-09-19T06:39:53.414233+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:39:53.414233+0000","last_undegraded":"2024-09-19T06:39:53.414233+0000","last_fullsized":"2024-09-19T06:39:53.414233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438533+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:40:00.438533+0000","last_peered":"2024-09-19T06:40:00.438533+0000","last_clean":"2024-09-19T06:40:00.438533+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:40:00.438533+0000","last_undegraded":"2024-09-19T06:40:00.438533+0000","last_fullsized":"2024-09-19T06:40:00.438533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403355+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:39:53.403355+0000","last_peered":"2024-09-19T06:39:53.403355+0000","last_clean":"2024-09-19T06:39:53.403355+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:39:53.403355+0000","last_undegraded":"2024-09-19T06:39:53.403355+0000","last_fullsized":"2024-09-19T06:39:53.403355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415130+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:39:53.415130+0000","last_peered":"2024-09-19T06:39:53.415130+0000","last_clean":"2024-09-19T06:39:53.415130+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:39:53.415130+0000","last_undegraded":"2024-09-19T06:39:53.415130+0000","last_fullsized":"2024-09-19T06:39:53.415130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415567+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:39:53.415567+0000","last_peered":"2024-09-19T06:39:53.415567+0000","last_clean":"2024-09-19T06:39:53.415567+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:39:53.415567+0000","last_undegraded":"2024-09-19T06:39:53.415567+0000","last_fullsized":"2024-09-19T06:39:53.415567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.439066+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:40:00.439066+0000","last_peered":"2024-09-19T06:40:00.439066+0000","last_clean":"2024-09-19T06:40:00.439066+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:40:00.439066+0000","last_undegraded":"2024-09-19T06:40:00.439066+0000","last_fullsized":"2024-09-19T06:40:00.439066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417596+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:39:53.417596+0000","last_peered":"2024-09-19T06:39:53.417596+0000","last_clean":"2024-09-19T06:39:53.417596+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:39:53.417596+0000","last_undegraded":"2024-09-19T06:39:53.417596+0000","last_fullsized":"2024-09-19T06:39:53.417596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415100+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:39:53.415100+0000","last_peered":"2024-09-19T06:39:53.415100+0000","last_clean":"2024-09-19T06:39:53.415100+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:39:53.415100+0000","last_undegraded":"2024-09-19T06:39:53.415100+0000","last_fullsized":"2024-09-19T06:39:53.415100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.420645+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:39:53.420645+0000","last_peered":"2024-09-19T06:39:53.420645+0000","last_clean":"2024-09-19T06:39:53.420645+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:39:53.420645+0000","last_undegraded":"2024-09-19T06:39:53.420645+0000","last_fullsized":"2024-09-19T06:39:53.420645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414246+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:39:53.414246+0000","last_peered":"2024-09-19T06:39:53.414246+0000","last_clean":"2024-09-19T06:39:53.414246+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:39:53.414246+0000","last_undegraded":"2024-09-19T06:39:53.414246+0000","last_fullsized":"2024-09-19T06:39:53.414246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447690+0000","last_change":"2024-09-19T06:39:50.315109+0000","last_active":"2024-09-19T06:40:01.447690+0000","last_peered":"2024-09-19T06:40:01.447690+0000","last_clean":"2024-09-19T06:40:01.447690+0000","last_became_active":"2024-09-19T06:39:50.314734+0000","last_became_peered":"2024-09-19T06:39:50.314734+0000","last_unstale":"2024-09-19T06:40:01.447690+0000","last_undegraded":"2024-09-19T06:40:01.447690+0000","last_fullsized":"2024-09-19T06:40:01.447690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:25.769548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419909+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:39:53.419909+0000","last_peered":"2024-09-19T06:39:53.419909+0000","last_clean":"2024-09-19T06:39:53.419909+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:39:53.419909+0000","last_undegraded":"2024-09-19T06:39:53.419909+0000","last_fullsized":"2024-09-19T06:39:53.419909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418802+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:39:53.418802+0000","last_peered":"2024-09-19T06:39:53.418802+0000","last_clean":"2024-09-19T06:39:53.418802+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:39:53.418802+0000","last_undegraded":"2024-09-19T06:39:53.418802+0000","last_fullsized":"2024-09-19T06:39:53.418802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415796+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:39:53.415796+0000","last_peered":"2024-09-19T06:39:53.415796+0000","last_clean":"2024-09-19T06:39:53.415796+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:39:53.415796+0000","last_undegraded":"2024-09-19T06:39:53.415796+0000","last_fullsized":"2024-09-19T06:39:53.415796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448401+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:40:01.448401+0000","last_peered":"2024-09-19T06:40:01.448401+0000","last_clean":"2024-09-19T06:40:01.448401+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:40:01.448401+0000","last_undegraded":"2024-09-19T06:40:01.448401+0000","last_fullsized":"2024-09-19T06:40:01.448401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50805,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":102,"num_read_kb":107,"num_write":152,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":152415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281827,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":419949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1058625,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1012037,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913789999,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1011328,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":420404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":419885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":950272,"data_stored":467519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":466169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":420845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":48152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:40:18.098 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:40:18.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:17 smithi151 ceph-mon[35236]: pgmap v291: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 5.4 KiB/s wr, 11 op/s 2024-09-19T06:40:18.362 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:18.752 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:18 smithi037 ceph-mon[31211]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:18.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 -- 172.21.15.37:0/3409185551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894102790 msgr2=0x7f28941067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:18.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 --2- 172.21.15.37:0/3409185551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894102790 0x7f28941067e0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2888009ca0 tx=0x7f288802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 -- 172.21.15.37:0/3409185551 shutdown_connections 2024-09-19T06:40:18.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 --2- 172.21.15.37:0/3409185551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2894106f10 0x7f289410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 --2- 172.21.15.37:0/3409185551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894102790 0x7f28941067e0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 --2- 172.21.15.37:0/3409185551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2894101de0 0x7f28941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 -- 172.21.15.37:0/3409185551 >> 172.21.15.37:0/3409185551 conn(0x7f28940fd8c0 msgr2=0x7f28940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 -- 172.21.15.37:0/3409185551 shutdown_connections 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.833+0000 7f289c31e640 1 -- 172.21.15.37:0/3409185551 wait complete. 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 Processor -- start 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 -- start start 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894101de0 0x7f28940744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2894102790 0x7f2894074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2894106f10 0x7f2894074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111f70 con 0x7f2894101de0 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2894111df0 con 0x7f2894102790 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289c31e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28941120f0 con 0x7f2894106f10 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289a0ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894101de0 0x7f28940744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:18.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289a0ba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894101de0 0x7f28940744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51372/0 (socket says 172.21.15.37:51372) 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289a0ba640 1 -- 172.21.15.37:0/2999539944 learned_addr learned my addr 172.21.15.37:0/2999539944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289a8bb640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2894106f10 0x7f2894074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f28998b9640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2894102790 0x7f2894074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289a0ba640 1 -- 172.21.15.37:0/2999539944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2894106f10 msgr2=0x7f2894074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.834+0000 7f289a0ba640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2894106f10 0x7f2894074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f289a0ba640 1 -- 172.21.15.37:0/2999539944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2894102790 msgr2=0x7f2894074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f289a0ba640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2894102790 0x7f2894074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f289a0ba640 1 -- 172.21.15.37:0/2999539944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28941a6820 con 0x7f2894101de0 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f28998b9640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2894102790 0x7f2894074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f289a0ba640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894101de0 0x7f28940744f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f288400c920 tx=0x7f288400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f28937fe640 1 -- 172.21.15.37:0/2999539944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884007c70 con 0x7f2894101de0 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28941a6b10 con 0x7f2894101de0 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f28937fe640 1 -- 172.21.15.37:0/2999539944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2884007e10 con 0x7f2894101de0 2024-09-19T06:40:18.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f28937fe640 1 -- 172.21.15.37:0/2999539944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884018890 con 0x7f2894101de0 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.835+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a70a0 con 0x7f2894101de0 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.836+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28941021c0 con 0x7f2894101de0 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.837+0000 7f28937fe640 1 -- 172.21.15.37:0/2999539944 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2884018a70 con 0x7f2894101de0 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.838+0000 7f28937fe640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f287c0a32d0 0x7f287c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.838+0000 7f28998b9640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f287c0a32d0 0x7f287c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.838+0000 7f28937fe640 1 -- 172.21.15.37:0/2999539944 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f28840cfb40 con 0x7f2894101de0 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.838+0000 7f28998b9640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f287c0a32d0 0x7f287c0a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2894075f80 tx=0x7f2888000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:18.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:18.842+0000 7f28937fe640 1 -- 172.21.15.37:0/2999539944 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2884014030 con 0x7f2894101de0 2024-09-19T06:40:18.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:18 smithi151 ceph-mon[35236]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:19.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.014+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2894062630 con 0x7f287c0a32d0 2024-09-19T06:40:19.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.021+0000 7f28937fe640 1 -- 172.21.15.37:0/2999539944 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258861 (secure 0 0 0) 0x7f2894062630 con 0x7f287c0a32d0 2024-09-19T06:40:19.024 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:19.025 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:40:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f287c0a32d0 msgr2=0x7f287c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f287c0a32d0 0x7f287c0a5790 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2894075f80 tx=0x7f2888000f40 comp rx=0 tx=0).stop 2024-09-19T06:40:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894101de0 msgr2=0x7f28940744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:19.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894101de0 0x7f28940744f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f288400c920 tx=0x7f288400cde0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 shutdown_connections 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f287c0a32d0 0x7f287c0a5790 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2894106f10 0x7f2894074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2894102790 0x7f2894074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 --2- 172.21.15.37:0/2999539944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2894101de0 0x7f28940744f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 >> 172.21.15.37:0/2999539944 conn(0x7f28940fd8c0 msgr2=0x7f28940fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 shutdown_connections 2024-09-19T06:40:19.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:19.024+0000 7f289c31e640 1 -- 172.21.15.37:0/2999539944 wait complete. 2024-09-19T06:40:19.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:18 smithi096 ceph-mon[35004]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:19.801 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-19T06:40:18.196942+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641407,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":243,"num_read_kb":226,"num_write":335,"num_write_kb":3032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":359972,"kb_used_data":10724,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581852,"statfs":{"total":1151940427776,"available":1151571816448,"internally_reserved":0,"allocated":10981376,"data_stored":6964777,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":45817,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":70,"num_read_kb":77,"num_write":29,"num_write_kb":56,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002674"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403605+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:39:53.403605+0000","last_peered":"2024-09-19T06:39:53.403605+0000","last_clean":"2024-09-19T06:39:53.403605+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:39:53.403605+0000","last_undegraded":"2024-09-19T06:39:53.403605+0000","last_fullsized":"2024-09-19T06:39:53.403605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419068+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:39:53.419068+0000","last_peered":"2024-09-19T06:39:53.419068+0000","last_clean":"2024-09-19T06:39:53.419068+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:39:53.419068+0000","last_undegraded":"2024-09-19T06:39:53.419068+0000","last_fullsized":"2024-09-19T06:39:53.419068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419285+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:39:53.419285+0000","last_peered":"2024-09-19T06:39:53.419285+0000","last_clean":"2024-09-19T06:39:53.419285+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:39:53.419285+0000","last_undegraded":"2024-09-19T06:39:53.419285+0000","last_fullsized":"2024-09-19T06:39:53.419285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418707+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:39:53.418707+0000","last_peered":"2024-09-19T06:39:53.418707+0000","last_clean":"2024-09-19T06:39:53.418707+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:39:53.418707+0000","last_undegraded":"2024-09-19T06:39:53.418707+0000","last_fullsized":"2024-09-19T06:39:53.418707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400788+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:39:53.400788+0000","last_peered":"2024-09-19T06:39:53.400788+0000","last_clean":"2024-09-19T06:39:53.400788+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:39:53.400788+0000","last_undegraded":"2024-09-19T06:39:53.400788+0000","last_fullsized":"2024-09-19T06:39:53.400788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417175+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:39:53.417175+0000","last_peered":"2024-09-19T06:39:53.417175+0000","last_clean":"2024-09-19T06:39:53.417175+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:39:53.417175+0000","last_undegraded":"2024-09-19T06:39:53.417175+0000","last_fullsized":"2024-09-19T06:39:53.417175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415134+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:39:53.415134+0000","last_peered":"2024-09-19T06:39:53.415134+0000","last_clean":"2024-09-19T06:39:53.415134+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:39:53.415134+0000","last_undegraded":"2024-09-19T06:39:53.415134+0000","last_fullsized":"2024-09-19T06:39:53.415134+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414333+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:39:53.414333+0000","last_peered":"2024-09-19T06:39:53.414333+0000","last_clean":"2024-09-19T06:39:53.414333+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:39:53.414333+0000","last_undegraded":"2024-09-19T06:39:53.414333+0000","last_fullsized":"2024-09-19T06:39:53.414333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.401273+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:39:53.401273+0000","last_peered":"2024-09-19T06:39:53.401273+0000","last_clean":"2024-09-19T06:39:53.401273+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:39:53.401273+0000","last_undegraded":"2024-09-19T06:39:53.401273+0000","last_fullsized":"2024-09-19T06:39:53.401273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417564+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:39:53.417564+0000","last_peered":"2024-09-19T06:39:53.417564+0000","last_clean":"2024-09-19T06:39:53.417564+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:39:53.417564+0000","last_undegraded":"2024-09-19T06:39:53.417564+0000","last_fullsized":"2024-09-19T06:39:53.417564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415382+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:39:53.415382+0000","last_peered":"2024-09-19T06:39:53.415382+0000","last_clean":"2024-09-19T06:39:53.415382+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:39:53.415382+0000","last_undegraded":"2024-09-19T06:39:53.415382+0000","last_fullsized":"2024-09-19T06:39:53.415382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400663+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:39:53.400663+0000","last_peered":"2024-09-19T06:39:53.400663+0000","last_clean":"2024-09-19T06:39:53.400663+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:39:53.400663+0000","last_undegraded":"2024-09-19T06:39:53.400663+0000","last_fullsized":"2024-09-19T06:39:53.400663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419338+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:39:53.419338+0000","last_peered":"2024-09-19T06:39:53.419338+0000","last_clean":"2024-09-19T06:39:53.419338+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:39:53.419338+0000","last_undegraded":"2024-09-19T06:39:53.419338+0000","last_fullsized":"2024-09-19T06:39:53.419338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418306+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:39:53.418306+0000","last_peered":"2024-09-19T06:39:53.418306+0000","last_clean":"2024-09-19T06:39:53.418306+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:39:53.418306+0000","last_undegraded":"2024-09-19T06:39:53.418306+0000","last_fullsized":"2024-09-19T06:39:53.418306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413939+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:39:53.413939+0000","last_peered":"2024-09-19T06:39:53.413939+0000","last_clean":"2024-09-19T06:39:53.413939+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:39:53.413939+0000","last_undegraded":"2024-09-19T06:39:53.413939+0000","last_fullsized":"2024-09-19T06:39:53.413939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403646+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:39:53.403646+0000","last_peered":"2024-09-19T06:39:53.403646+0000","last_clean":"2024-09-19T06:39:53.403646+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:39:53.403646+0000","last_undegraded":"2024-09-19T06:39:53.403646+0000","last_fullsized":"2024-09-19T06:39:53.403646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414294+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:39:53.414294+0000","last_peered":"2024-09-19T06:39:53.414294+0000","last_clean":"2024-09-19T06:39:53.414294+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:39:53.414294+0000","last_undegraded":"2024-09-19T06:39:53.414294+0000","last_fullsized":"2024-09-19T06:39:53.414294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415375+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:39:53.415375+0000","last_peered":"2024-09-19T06:39:53.415375+0000","last_clean":"2024-09-19T06:39:53.415375+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:39:53.415375+0000","last_undegraded":"2024-09-19T06:39:53.415375+0000","last_fullsized":"2024-09-19T06:39:53.415375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403279+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:39:53.403279+0000","last_peered":"2024-09-19T06:39:53.403279+0000","last_clean":"2024-09-19T06:39:53.403279+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:39:53.403279+0000","last_undegraded":"2024-09-19T06:39:53.403279+0000","last_fullsized":"2024-09-19T06:39:53.403279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400705+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:39:53.400705+0000","last_peered":"2024-09-19T06:39:53.400705+0000","last_clean":"2024-09-19T06:39:53.400705+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:39:53.400705+0000","last_undegraded":"2024-09-19T06:39:53.400705+0000","last_fullsized":"2024-09-19T06:39:53.400705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414105+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:39:53.414105+0000","last_peered":"2024-09-19T06:39:53.414105+0000","last_clean":"2024-09-19T06:39:53.414105+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:39:53.414105+0000","last_undegraded":"2024-09-19T06:39:53.414105+0000","last_fullsized":"2024-09-19T06:39:53.414105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415846+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:39:53.415846+0000","last_peered":"2024-09-19T06:39:53.415846+0000","last_clean":"2024-09-19T06:39:53.415846+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:39:53.415846+0000","last_undegraded":"2024-09-19T06:39:53.415846+0000","last_fullsized":"2024-09-19T06:39:53.415846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414231+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:39:53.414231+0000","last_peered":"2024-09-19T06:39:53.414231+0000","last_clean":"2024-09-19T06:39:53.414231+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:39:53.414231+0000","last_undegraded":"2024-09-19T06:39:53.414231+0000","last_fullsized":"2024-09-19T06:39:53.414231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416096+0000","last_change":"2024-09-19T06:39:52.399797+0000","last_active":"2024-09-19T06:39:53.416096+0000","last_peered":"2024-09-19T06:39:53.416096+0000","last_clean":"2024-09-19T06:39:53.416096+0000","last_became_active":"2024-09-19T06:39:52.399452+0000","last_became_peered":"2024-09-19T06:39:52.399452+0000","last_unstale":"2024-09-19T06:39:53.416096+0000","last_undegraded":"2024-09-19T06:39:53.416096+0000","last_fullsized":"2024-09-19T06:39:53.416096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:33.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415925+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:39:53.415925+0000","last_peered":"2024-09-19T06:39:53.415925+0000","last_clean":"2024-09-19T06:39:53.415925+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:39:53.415925+0000","last_undegraded":"2024-09-19T06:39:53.415925+0000","last_fullsized":"2024-09-19T06:39:53.415925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403645+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:39:53.403645+0000","last_peered":"2024-09-19T06:39:53.403645+0000","last_clean":"2024-09-19T06:39:53.403645+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:39:53.403645+0000","last_undegraded":"2024-09-19T06:39:53.403645+0000","last_fullsized":"2024-09-19T06:39:53.403645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400700+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:39:53.400700+0000","last_peered":"2024-09-19T06:39:53.400700+0000","last_clean":"2024-09-19T06:39:53.400700+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:39:53.400700+0000","last_undegraded":"2024-09-19T06:39:53.400700+0000","last_fullsized":"2024-09-19T06:39:53.400700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419814+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:39:53.419814+0000","last_peered":"2024-09-19T06:39:53.419814+0000","last_clean":"2024-09-19T06:39:53.419814+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:39:53.419814+0000","last_undegraded":"2024-09-19T06:39:53.419814+0000","last_fullsized":"2024-09-19T06:39:53.419814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414873+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:39:53.414873+0000","last_peered":"2024-09-19T06:39:53.414873+0000","last_clean":"2024-09-19T06:39:53.414873+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:39:53.414873+0000","last_undegraded":"2024-09-19T06:39:53.414873+0000","last_fullsized":"2024-09-19T06:39:53.414873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414716+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:39:53.414716+0000","last_peered":"2024-09-19T06:39:53.414716+0000","last_clean":"2024-09-19T06:39:53.414716+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:39:53.414716+0000","last_undegraded":"2024-09-19T06:39:53.414716+0000","last_fullsized":"2024-09-19T06:39:53.414716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403217+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:39:53.403217+0000","last_peered":"2024-09-19T06:39:53.403217+0000","last_clean":"2024-09-19T06:39:53.403217+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:39:53.403217+0000","last_undegraded":"2024-09-19T06:39:53.403217+0000","last_fullsized":"2024-09-19T06:39:53.403217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":41,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:16.474099+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:40:16.474099+0000","last_peered":"2024-09-19T06:40:16.474099+0000","last_clean":"2024-09-19T06:40:16.474099+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:40:16.474099+0000","last_undegraded":"2024-09-19T06:40:16.474099+0000","last_fullsized":"2024-09-19T06:40:16.474099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414773+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:39:53.414773+0000","last_peered":"2024-09-19T06:39:53.414773+0000","last_clean":"2024-09-19T06:39:53.414773+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:39:53.414773+0000","last_undegraded":"2024-09-19T06:39:53.414773+0000","last_fullsized":"2024-09-19T06:39:53.414773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.446851+0000","last_change":"2024-09-19T06:39:50.318132+0000","last_active":"2024-09-19T06:40:01.446851+0000","last_peered":"2024-09-19T06:40:01.446851+0000","last_clean":"2024-09-19T06:40:01.446851+0000","last_became_active":"2024-09-19T06:39:50.317774+0000","last_became_peered":"2024-09-19T06:39:50.317774+0000","last_unstale":"2024-09-19T06:40:01.446851+0000","last_undegraded":"2024-09-19T06:40:01.446851+0000","last_fullsized":"2024-09-19T06:40:01.446851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:58.290128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.445718+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:40:01.445718+0000","last_peered":"2024-09-19T06:40:01.445718+0000","last_clean":"2024-09-19T06:40:01.445718+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:40:01.445718+0000","last_undegraded":"2024-09-19T06:40:01.445718+0000","last_fullsized":"2024-09-19T06:40:01.445718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415204+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:39:53.415204+0000","last_peered":"2024-09-19T06:39:53.415204+0000","last_clean":"2024-09-19T06:39:53.415204+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:39:53.415204+0000","last_undegraded":"2024-09-19T06:39:53.415204+0000","last_fullsized":"2024-09-19T06:39:53.415204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414736+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:39:53.414736+0000","last_peered":"2024-09-19T06:39:53.414736+0000","last_clean":"2024-09-19T06:39:53.414736+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:39:53.414736+0000","last_undegraded":"2024-09-19T06:39:53.414736+0000","last_fullsized":"2024-09-19T06:39:53.414736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413749+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:39:53.413749+0000","last_peered":"2024-09-19T06:39:53.413749+0000","last_clean":"2024-09-19T06:39:53.413749+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:39:53.413749+0000","last_undegraded":"2024-09-19T06:39:53.413749+0000","last_fullsized":"2024-09-19T06:39:53.413749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":35,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:14.470266+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:40:14.470266+0000","last_peered":"2024-09-19T06:40:14.470266+0000","last_clean":"2024-09-19T06:40:14.470266+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:40:14.470266+0000","last_undegraded":"2024-09-19T06:40:14.470266+0000","last_fullsized":"2024-09-19T06:40:14.470266+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414187+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:39:53.414187+0000","last_peered":"2024-09-19T06:39:53.414187+0000","last_clean":"2024-09-19T06:39:53.414187+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:39:53.414187+0000","last_undegraded":"2024-09-19T06:39:53.414187+0000","last_fullsized":"2024-09-19T06:39:53.414187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438327+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:40:00.438327+0000","last_peered":"2024-09-19T06:40:00.438327+0000","last_clean":"2024-09-19T06:40:00.438327+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:40:00.438327+0000","last_undegraded":"2024-09-19T06:40:00.438327+0000","last_fullsized":"2024-09-19T06:40:00.438327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419760+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:39:53.419760+0000","last_peered":"2024-09-19T06:39:53.419760+0000","last_clean":"2024-09-19T06:39:53.419760+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:39:53.419760+0000","last_undegraded":"2024-09-19T06:39:53.419760+0000","last_fullsized":"2024-09-19T06:39:53.419760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437339+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:40:00.437339+0000","last_peered":"2024-09-19T06:40:00.437339+0000","last_clean":"2024-09-19T06:40:00.437339+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:40:00.437339+0000","last_undegraded":"2024-09-19T06:40:00.437339+0000","last_fullsized":"2024-09-19T06:40:00.437339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414395+0000","last_change":"2024-09-19T06:39:52.391059+0000","last_active":"2024-09-19T06:39:53.414395+0000","last_peered":"2024-09-19T06:39:53.414395+0000","last_clean":"2024-09-19T06:39:53.414395+0000","last_became_active":"2024-09-19T06:39:52.389703+0000","last_became_peered":"2024-09-19T06:39:52.389703+0000","last_unstale":"2024-09-19T06:39:53.414395+0000","last_undegraded":"2024-09-19T06:39:53.414395+0000","last_fullsized":"2024-09-19T06:39:53.414395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:32.021109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'22","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:15.473350+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:40:15.473350+0000","last_peered":"2024-09-19T06:40:15.473350+0000","last_clean":"2024-09-19T06:40:15.473350+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:40:15.473350+0000","last_undegraded":"2024-09-19T06:40:15.473350+0000","last_fullsized":"2024-09-19T06:40:15.473350+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46802,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":62,"num_write":23,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415675+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:39:53.415675+0000","last_peered":"2024-09-19T06:39:53.415675+0000","last_clean":"2024-09-19T06:39:53.415675+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:39:53.415675+0000","last_undegraded":"2024-09-19T06:39:53.415675+0000","last_fullsized":"2024-09-19T06:39:53.415675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447875+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:40:01.447875+0000","last_peered":"2024-09-19T06:40:01.447875+0000","last_clean":"2024-09-19T06:40:01.447875+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:40:01.447875+0000","last_undegraded":"2024-09-19T06:40:01.447875+0000","last_fullsized":"2024-09-19T06:40:01.447875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437843+0000","last_change":"2024-09-19T06:39:50.312876+0000","last_active":"2024-09-19T06:40:00.437843+0000","last_peered":"2024-09-19T06:40:00.437843+0000","last_clean":"2024-09-19T06:40:00.437843+0000","last_became_active":"2024-09-19T06:39:50.312500+0000","last_became_peered":"2024-09-19T06:39:50.312500+0000","last_unstale":"2024-09-19T06:40:00.437843+0000","last_undegraded":"2024-09-19T06:40:00.437843+0000","last_fullsized":"2024-09-19T06:40:00.437843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:00.113022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417798+0000","last_change":"2024-09-19T06:39:52.399395+0000","last_active":"2024-09-19T06:39:53.417798+0000","last_peered":"2024-09-19T06:39:53.417798+0000","last_clean":"2024-09-19T06:39:53.417798+0000","last_became_active":"2024-09-19T06:39:52.399042+0000","last_became_peered":"2024-09-19T06:39:52.399042+0000","last_unstale":"2024-09-19T06:39:53.417798+0000","last_undegraded":"2024-09-19T06:39:53.417798+0000","last_fullsized":"2024-09-19T06:39:53.417798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:10.010156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":37,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:12.467192+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:40:12.467192+0000","last_peered":"2024-09-19T06:40:12.467192+0000","last_clean":"2024-09-19T06:40:12.467192+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:40:12.467192+0000","last_undegraded":"2024-09-19T06:40:12.467192+0000","last_fullsized":"2024-09-19T06:40:12.467192+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416672+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:39:53.416672+0000","last_peered":"2024-09-19T06:39:53.416672+0000","last_clean":"2024-09-19T06:39:53.416672+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:39:53.416672+0000","last_undegraded":"2024-09-19T06:39:53.416672+0000","last_fullsized":"2024-09-19T06:39:53.416672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448340+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:40:01.448340+0000","last_peered":"2024-09-19T06:40:01.448340+0000","last_clean":"2024-09-19T06:40:01.448340+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:40:01.448340+0000","last_undegraded":"2024-09-19T06:40:01.448340+0000","last_fullsized":"2024-09-19T06:40:01.448340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415210+0000","last_change":"2024-09-19T06:39:52.390919+0000","last_active":"2024-09-19T06:39:53.415210+0000","last_peered":"2024-09-19T06:39:53.415210+0000","last_clean":"2024-09-19T06:39:53.415210+0000","last_became_active":"2024-09-19T06:39:52.390585+0000","last_became_peered":"2024-09-19T06:39:52.390585+0000","last_unstale":"2024-09-19T06:39:53.415210+0000","last_undegraded":"2024-09-19T06:39:53.415210+0000","last_fullsized":"2024-09-19T06:39:53.415210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.164497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.451484+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:40:02.451484+0000","last_peered":"2024-09-19T06:40:02.451484+0000","last_clean":"2024-09-19T06:40:02.451484+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:40:02.451484+0000","last_undegraded":"2024-09-19T06:40:02.451484+0000","last_fullsized":"2024-09-19T06:40:02.451484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403462+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:39:53.403462+0000","last_peered":"2024-09-19T06:39:53.403462+0000","last_clean":"2024-09-19T06:39:53.403462+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:39:53.403462+0000","last_undegraded":"2024-09-19T06:39:53.403462+0000","last_fullsized":"2024-09-19T06:39:53.403462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418357+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:39:53.418357+0000","last_peered":"2024-09-19T06:39:53.418357+0000","last_clean":"2024-09-19T06:39:53.418357+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:39:53.418357+0000","last_undegraded":"2024-09-19T06:39:53.418357+0000","last_fullsized":"2024-09-19T06:39:53.418357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418035+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:39:53.418035+0000","last_peered":"2024-09-19T06:39:53.418035+0000","last_clean":"2024-09-19T06:39:53.418035+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:39:53.418035+0000","last_undegraded":"2024-09-19T06:39:53.418035+0000","last_fullsized":"2024-09-19T06:39:53.418035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419293+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:39:53.419293+0000","last_peered":"2024-09-19T06:39:53.419293+0000","last_clean":"2024-09-19T06:39:53.419293+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:39:53.419293+0000","last_undegraded":"2024-09-19T06:39:53.419293+0000","last_fullsized":"2024-09-19T06:39:53.419293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144684+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:39:54.144684+0000","last_peered":"2024-09-19T06:39:54.144684+0000","last_clean":"2024-09-19T06:39:54.144684+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:39:54.144684+0000","last_undegraded":"2024-09-19T06:39:54.144684+0000","last_fullsized":"2024-09-19T06:39:54.144684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447446+0000","last_change":"2024-09-19T06:39:50.507312+0000","last_active":"2024-09-19T06:40:01.447446+0000","last_peered":"2024-09-19T06:40:01.447446+0000","last_clean":"2024-09-19T06:40:01.447446+0000","last_became_active":"2024-09-19T06:39:50.506976+0000","last_became_peered":"2024-09-19T06:39:50.506976+0000","last_unstale":"2024-09-19T06:40:01.447446+0000","last_undegraded":"2024-09-19T06:40:01.447446+0000","last_fullsized":"2024-09-19T06:40:01.447446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:44.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417964+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:39:53.417964+0000","last_peered":"2024-09-19T06:39:53.417964+0000","last_clean":"2024-09-19T06:39:53.417964+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:39:53.417964+0000","last_undegraded":"2024-09-19T06:39:53.417964+0000","last_fullsized":"2024-09-19T06:39:53.417964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417272+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:39:53.417272+0000","last_peered":"2024-09-19T06:39:53.417272+0000","last_clean":"2024-09-19T06:39:53.417272+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:39:53.417272+0000","last_undegraded":"2024-09-19T06:39:53.417272+0000","last_fullsized":"2024-09-19T06:39:53.417272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.450445+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:40:02.450445+0000","last_peered":"2024-09-19T06:40:02.450445+0000","last_clean":"2024-09-19T06:40:02.450445+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:40:02.450445+0000","last_undegraded":"2024-09-19T06:40:02.450445+0000","last_fullsized":"2024-09-19T06:40:02.450445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415347+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:39:53.415347+0000","last_peered":"2024-09-19T06:39:53.415347+0000","last_clean":"2024-09-19T06:39:53.415347+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:39:53.415347+0000","last_undegraded":"2024-09-19T06:39:53.415347+0000","last_fullsized":"2024-09-19T06:39:53.415347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447538+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:40:01.447538+0000","last_peered":"2024-09-19T06:40:01.447538+0000","last_clean":"2024-09-19T06:40:01.447538+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:40:01.447538+0000","last_undegraded":"2024-09-19T06:40:01.447538+0000","last_fullsized":"2024-09-19T06:40:01.447538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400695+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:39:53.400695+0000","last_peered":"2024-09-19T06:39:53.400695+0000","last_clean":"2024-09-19T06:39:53.400695+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:39:53.400695+0000","last_undegraded":"2024-09-19T06:39:53.400695+0000","last_fullsized":"2024-09-19T06:39:53.400695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:12.466648+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:40:12.466648+0000","last_peered":"2024-09-19T06:40:12.466648+0000","last_clean":"2024-09-19T06:40:12.466648+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:40:12.466648+0000","last_undegraded":"2024-09-19T06:40:12.466648+0000","last_fullsized":"2024-09-19T06:40:12.466648+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414405+0000","last_change":"2024-09-19T06:39:52.396795+0000","last_active":"2024-09-19T06:39:53.414405+0000","last_peered":"2024-09-19T06:39:53.414405+0000","last_clean":"2024-09-19T06:39:53.414405+0000","last_became_active":"2024-09-19T06:39:52.396481+0000","last_became_peered":"2024-09-19T06:39:52.396481+0000","last_unstale":"2024-09-19T06:39:53.414405+0000","last_undegraded":"2024-09-19T06:39:53.414405+0000","last_fullsized":"2024-09-19T06:39:53.414405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:05.637504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144665+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:39:54.144665+0000","last_peered":"2024-09-19T06:39:54.144665+0000","last_clean":"2024-09-19T06:39:54.144665+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:39:54.144665+0000","last_undegraded":"2024-09-19T06:39:54.144665+0000","last_fullsized":"2024-09-19T06:39:54.144665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400964+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:39:53.400964+0000","last_peered":"2024-09-19T06:39:53.400964+0000","last_clean":"2024-09-19T06:39:53.400964+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:39:53.400964+0000","last_undegraded":"2024-09-19T06:39:53.400964+0000","last_fullsized":"2024-09-19T06:39:53.400964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414089+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:39:53.414089+0000","last_peered":"2024-09-19T06:39:53.414089+0000","last_clean":"2024-09-19T06:39:53.414089+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:39:53.414089+0000","last_undegraded":"2024-09-19T06:39:53.414089+0000","last_fullsized":"2024-09-19T06:39:53.414089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:56.016828+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:39:56.016828+0000","last_peered":"2024-09-19T06:39:56.016828+0000","last_clean":"2024-09-19T06:39:56.016828+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:39:56.016828+0000","last_undegraded":"2024-09-19T06:39:56.016828+0000","last_fullsized":"2024-09-19T06:39:56.016828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437914+0000","last_change":"2024-09-19T06:39:50.506741+0000","last_active":"2024-09-19T06:40:00.437914+0000","last_peered":"2024-09-19T06:40:00.437914+0000","last_clean":"2024-09-19T06:40:00.437914+0000","last_became_active":"2024-09-19T06:39:50.506437+0000","last_became_peered":"2024-09-19T06:39:50.506437+0000","last_unstale":"2024-09-19T06:40:00.437914+0000","last_undegraded":"2024-09-19T06:40:00.437914+0000","last_fullsized":"2024-09-19T06:40:00.437914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:42.304929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'9","reported_seq":27,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:13.415103+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:40:13.415103+0000","last_peered":"2024-09-19T06:40:13.415103+0000","last_clean":"2024-09-19T06:40:13.415103+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:40:13.415103+0000","last_undegraded":"2024-09-19T06:40:13.415103+0000","last_fullsized":"2024-09-19T06:40:13.415103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416102+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:39:53.416102+0000","last_peered":"2024-09-19T06:39:53.416102+0000","last_clean":"2024-09-19T06:39:53.416102+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:39:53.416102+0000","last_undegraded":"2024-09-19T06:39:53.416102+0000","last_fullsized":"2024-09-19T06:39:53.416102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.436752+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:40:00.436752+0000","last_peered":"2024-09-19T06:40:00.436752+0000","last_clean":"2024-09-19T06:40:00.436752+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:40:00.436752+0000","last_undegraded":"2024-09-19T06:40:00.436752+0000","last_fullsized":"2024-09-19T06:40:00.436752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419205+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:39:53.419205+0000","last_peered":"2024-09-19T06:39:53.419205+0000","last_clean":"2024-09-19T06:39:53.419205+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:39:53.419205+0000","last_undegraded":"2024-09-19T06:39:53.419205+0000","last_fullsized":"2024-09-19T06:39:53.419205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437748+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:40:00.437748+0000","last_peered":"2024-09-19T06:40:00.437748+0000","last_clean":"2024-09-19T06:40:00.437748+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:40:00.437748+0000","last_undegraded":"2024-09-19T06:40:00.437748+0000","last_fullsized":"2024-09-19T06:40:00.437748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414233+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:39:53.414233+0000","last_peered":"2024-09-19T06:39:53.414233+0000","last_clean":"2024-09-19T06:39:53.414233+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:39:53.414233+0000","last_undegraded":"2024-09-19T06:39:53.414233+0000","last_fullsized":"2024-09-19T06:39:53.414233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438533+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:40:00.438533+0000","last_peered":"2024-09-19T06:40:00.438533+0000","last_clean":"2024-09-19T06:40:00.438533+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:40:00.438533+0000","last_undegraded":"2024-09-19T06:40:00.438533+0000","last_fullsized":"2024-09-19T06:40:00.438533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403355+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:39:53.403355+0000","last_peered":"2024-09-19T06:39:53.403355+0000","last_clean":"2024-09-19T06:39:53.403355+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:39:53.403355+0000","last_undegraded":"2024-09-19T06:39:53.403355+0000","last_fullsized":"2024-09-19T06:39:53.403355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415130+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:39:53.415130+0000","last_peered":"2024-09-19T06:39:53.415130+0000","last_clean":"2024-09-19T06:39:53.415130+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:39:53.415130+0000","last_undegraded":"2024-09-19T06:39:53.415130+0000","last_fullsized":"2024-09-19T06:39:53.415130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415567+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:39:53.415567+0000","last_peered":"2024-09-19T06:39:53.415567+0000","last_clean":"2024-09-19T06:39:53.415567+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:39:53.415567+0000","last_undegraded":"2024-09-19T06:39:53.415567+0000","last_fullsized":"2024-09-19T06:39:53.415567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.439066+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:40:00.439066+0000","last_peered":"2024-09-19T06:40:00.439066+0000","last_clean":"2024-09-19T06:40:00.439066+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:40:00.439066+0000","last_undegraded":"2024-09-19T06:40:00.439066+0000","last_fullsized":"2024-09-19T06:40:00.439066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417596+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:39:53.417596+0000","last_peered":"2024-09-19T06:39:53.417596+0000","last_clean":"2024-09-19T06:39:53.417596+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:39:53.417596+0000","last_undegraded":"2024-09-19T06:39:53.417596+0000","last_fullsized":"2024-09-19T06:39:53.417596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415100+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:39:53.415100+0000","last_peered":"2024-09-19T06:39:53.415100+0000","last_clean":"2024-09-19T06:39:53.415100+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:39:53.415100+0000","last_undegraded":"2024-09-19T06:39:53.415100+0000","last_fullsized":"2024-09-19T06:39:53.415100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.420645+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:39:53.420645+0000","last_peered":"2024-09-19T06:39:53.420645+0000","last_clean":"2024-09-19T06:39:53.420645+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:39:53.420645+0000","last_undegraded":"2024-09-19T06:39:53.420645+0000","last_fullsized":"2024-09-19T06:39:53.420645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414246+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:39:53.414246+0000","last_peered":"2024-09-19T06:39:53.414246+0000","last_clean":"2024-09-19T06:39:53.414246+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:39:53.414246+0000","last_undegraded":"2024-09-19T06:39:53.414246+0000","last_fullsized":"2024-09-19T06:39:53.414246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447690+0000","last_change":"2024-09-19T06:39:50.315109+0000","last_active":"2024-09-19T06:40:01.447690+0000","last_peered":"2024-09-19T06:40:01.447690+0000","last_clean":"2024-09-19T06:40:01.447690+0000","last_became_active":"2024-09-19T06:39:50.314734+0000","last_became_peered":"2024-09-19T06:39:50.314734+0000","last_unstale":"2024-09-19T06:40:01.447690+0000","last_undegraded":"2024-09-19T06:40:01.447690+0000","last_fullsized":"2024-09-19T06:40:01.447690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:25.769548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419909+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:39:53.419909+0000","last_peered":"2024-09-19T06:39:53.419909+0000","last_clean":"2024-09-19T06:39:53.419909+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:39:53.419909+0000","last_undegraded":"2024-09-19T06:39:53.419909+0000","last_fullsized":"2024-09-19T06:39:53.419909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418802+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:39:53.418802+0000","last_peered":"2024-09-19T06:39:53.418802+0000","last_clean":"2024-09-19T06:39:53.418802+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:39:53.418802+0000","last_undegraded":"2024-09-19T06:39:53.418802+0000","last_fullsized":"2024-09-19T06:39:53.418802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415796+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:39:53.415796+0000","last_peered":"2024-09-19T06:39:53.415796+0000","last_clean":"2024-09-19T06:39:53.415796+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:39:53.415796+0000","last_undegraded":"2024-09-19T06:39:53.415796+0000","last_fullsized":"2024-09-19T06:39:53.415796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448401+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:40:01.448401+0000","last_peered":"2024-09-19T06:40:01.448401+0000","last_clean":"2024-09-19T06:40:01.448401+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:40:01.448401+0000","last_undegraded":"2024-09-19T06:40:01.448401+0000","last_fullsized":"2024-09-19T06:40:01.448401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":50805,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":152,"num_write_kb":92,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389120,"data_stored":160567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281827,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":419949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776150,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1011712,"data_stored":1058625,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939676,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168609,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397541,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1012037,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561067,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790000,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1011328,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953525,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":420404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182458,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":419885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411392,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32120,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713032,"statfs":{"total":95995035648,"available":95962144768,"internally_reserved":0,"allocated":958464,"data_stored":475671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":466169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":420845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":46802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":56304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":48773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:40:19.805 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:40:19.806 INFO:tasks.cephadm:Setting up client nodes... 2024-09-19T06:40:19.806 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-19T06:40:20.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:19 smithi096 ceph-mon[35004]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-19T06:40:20.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:19 smithi096 ceph-mon[35004]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:20.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:19 smithi037 ceph-mon[31211]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-19T06:40:20.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:19 smithi037 ceph-mon[31211]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:20.066 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:20.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:19 smithi151 ceph-mon[35236]: pgmap v292: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-19T06:40:20.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:19 smithi151 ceph-mon[35236]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:20.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 -- 172.21.15.37:0/3860603112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c101c00 msgr2=0x7f7a2c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 --2- 172.21.15.37:0/3860603112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c101c00 0x7f7a2c102000 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c009900 tx=0x7f7a1c02a410 comp rx=0 tx=0).stop 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 -- 172.21.15.37:0/3860603112 shutdown_connections 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 --2- 172.21.15.37:0/3860603112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a2c1037c0 0x7f7a2c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 --2- 172.21.15.37:0/3860603112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a2c102e00 0x7f7a2c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 --2- 172.21.15.37:0/3860603112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c101c00 0x7f7a2c102000 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 -- 172.21.15.37:0/3860603112 >> 172.21.15.37:0/3860603112 conn(0x7f7a2c0fd390 msgr2=0x7f7a2c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 -- 172.21.15.37:0/3860603112 shutdown_connections 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.539+0000 7f7a325c8640 1 -- 172.21.15.37:0/3860603112 wait complete. 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 Processor -- start 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 -- start start 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a2c101c00 0x7f7a2c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c102e00 0x7f7a2c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:20.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a2c1037c0 0x7f7a2c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c10d890 con 0x7f7a2c102e00 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c10d710 con 0x7f7a2c1037c0 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a325c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7a2c10da10 con 0x7f7a2c101c00 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a2b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c102e00 0x7f7a2c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a2b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c102e00 0x7f7a2c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49714/0 (socket says 172.21.15.37:49714) 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.540+0000 7f7a2b7fe640 1 -- 172.21.15.37:0/44389983 learned_addr learned my addr 172.21.15.37:0/44389983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a2bfff640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a2c101c00 0x7f7a2c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a30b65640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a2c1037c0 0x7f7a2c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a2b7fe640 1 -- 172.21.15.37:0/44389983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a2c101c00 msgr2=0x7f7a2c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a2b7fe640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a2c101c00 0x7f7a2c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a2b7fe640 1 -- 172.21.15.37:0/44389983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a2c1037c0 msgr2=0x7f7a2c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a2b7fe640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a2c1037c0 0x7f7a2c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a2b7fe640 1 -- 172.21.15.37:0/44389983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a2c0753e0 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a2b7fe640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c102e00 0x7f7a2c0745e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000d950 tx=0x7f7a2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a297fa640 1 -- 172.21.15.37:0/44389983 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a20017070 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a2c1a2060 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a297fa640 1 -- 172.21.15.37:0/44389983 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a20009c30 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a297fa640 1 -- 172.21.15.37:0/44389983 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2000f6b0 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.541+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a2c1a2570 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.542+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a2c108fa0 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.543+0000 7f7a297fa640 1 -- 172.21.15.37:0/44389983 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7a200109d0 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.544+0000 7f7a297fa640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7a140a32d0 0x7f7a140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.544+0000 7f7a2bfff640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7a140a32d0 0x7f7a140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.544+0000 7f7a297fa640 1 -- 172.21.15.37:0/44389983 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f7a200cf5e0 con 0x7f7a2c102e00 2024-09-19T06:40:20.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.545+0000 7f7a2bfff640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7a140a32d0 0x7f7a140a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006010 tx=0x7f7a1c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:20.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.549+0000 7f7a297fa640 1 -- 172.21.15.37:0/44389983 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7a20009de0 con 0x7f7a2c102e00 2024-09-19T06:40:20.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.794+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7a2c062df0 con 0x7f7a2c102e00 2024-09-19T06:40:20.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.796+0000 7f7a297fa640 1 -- 172.21.15.37:0/44389983 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7a2009c350 con 0x7f7a2c102e00 2024-09-19T06:40:20.798 INFO:teuthology.orchestra.run.smithi037.stdout:[client.0] 2024-09-19T06:40:20.798 INFO:teuthology.orchestra.run.smithi037.stdout: key = AQBUx+tm3C5kLxAAF/fXfTOGRQ6rSg46Sbz/hw== 2024-09-19T06:40:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.798+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7a140a32d0 msgr2=0x7f7a140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.798+0000 7f7a325c8640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7a140a32d0 0x7f7a140a5790 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c006010 tx=0x7f7a1c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.798+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c102e00 msgr2=0x7f7a2c0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.798+0000 7f7a325c8640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c102e00 0x7f7a2c0745e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000d950 tx=0x7f7a2000de10 comp rx=0 tx=0).stop 2024-09-19T06:40:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 shutdown_connections 2024-09-19T06:40:20.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7a140a32d0 0x7f7a140a5790 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7a2c1037c0 0x7f7a2c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7a2c102e00 0x7f7a2c0745e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 --2- 172.21.15.37:0/44389983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7a2c101c00 0x7f7a2c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:20.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 >> 172.21.15.37:0/44389983 conn(0x7f7a2c0fd390 msgr2=0x7f7a2c106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:20.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 shutdown_connections 2024-09-19T06:40:20.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:20.799+0000 7f7a325c8640 1 -- 172.21.15.37:0/44389983 wait complete. 2024-09-19T06:40:21.596 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:40:21.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-19T06:40:21.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-19T06:40:21.637 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-19T06:40:21.637 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-19T06:40:21.637 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-19T06:40:21.935 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:22.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:21 smithi096 ceph-mon[35004]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-19T06:40:22.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:21 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/44389983' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:40:22.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:21 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/44389983' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:40:22.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:21 smithi037 ceph-mon[31211]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-19T06:40:22.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:21 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/44389983' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:40:22.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:21 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/44389983' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:40:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:21 smithi151 ceph-mon[35236]: pgmap v293: 97 pgs: 97 active+clean; 626 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2024-09-19T06:40:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:21 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/44389983' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-19T06:40:22.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:21 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/44389983' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 -- 172.21.15.37:0/659921547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8102790 msgr2=0x7f0ac81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 --2- 172.21.15.37:0/659921547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8102790 0x7f0ac81067e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009900 tx=0x7f0ab802a410 comp rx=0 tx=0).stop 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 -- 172.21.15.37:0/659921547 shutdown_connections 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 --2- 172.21.15.37:0/659921547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac8106f10 0x7f0ac810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 --2- 172.21.15.37:0/659921547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8102790 0x7f0ac81067e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 --2- 172.21.15.37:0/659921547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac8101de0 0x7f0ac81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 -- 172.21.15.37:0/659921547 >> 172.21.15.37:0/659921547 conn(0x7f0ac80fd8c0 msgr2=0x7f0ac80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 -- 172.21.15.37:0/659921547 shutdown_connections 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.389+0000 7f0ad0197640 1 -- 172.21.15.37:0/659921547 wait complete. 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 Processor -- start 2024-09-19T06:40:22.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 -- start start 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac8101de0 0x7f0ac81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac8102790 0x7f0ac81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8106f10 0x7f0ac819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8111fe0 con 0x7f0ac8106f10 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8111e60 con 0x7f0ac8101de0 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ad0197640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0ac8112160 con 0x7f0ac8102790 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ace734640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8106f10 0x7f0ac819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ace734640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8106f10 0x7f0ac819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49740/0 (socket says 172.21.15.37:49740) 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0acdf33640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac8101de0 0x7f0ac81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.390+0000 7f0ace734640 1 -- 172.21.15.37:0/3951625046 learned_addr learned my addr 172.21.15.37:0/3951625046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0acd732640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac8102790 0x7f0ac81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ace734640 1 -- 172.21.15.37:0/3951625046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac8102790 msgr2=0x7f0ac81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ace734640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac8102790 0x7f0ac81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ace734640 1 -- 172.21.15.37:0/3951625046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac8101de0 msgr2=0x7f0ac81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ace734640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac8101de0 0x7f0ac81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ace734640 1 -- 172.21.15.37:0/3951625046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ac819d450 con 0x7f0ac8106f10 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ace734640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8106f10 0x7f0ac819cd70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000eaa0 tx=0x7f0ac000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0abeffd640 1 -- 172.21.15.37:0/3951625046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0004d20 con 0x7f0ac8106f10 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0abeffd640 1 -- 172.21.15.37:0/3951625046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ac0005020 con 0x7f0ac8106f10 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ac819d740 con 0x7f0ac8106f10 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0abeffd640 1 -- 172.21.15.37:0/3951625046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00116d0 con 0x7f0ac8106f10 2024-09-19T06:40:22.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.391+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ac81a5020 con 0x7f0ac8106f10 2024-09-19T06:40:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.392+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ac81021c0 con 0x7f0ac8106f10 2024-09-19T06:40:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.393+0000 7f0abeffd640 1 -- 172.21.15.37:0/3951625046 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0ac00051d0 con 0x7f0ac8106f10 2024-09-19T06:40:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.394+0000 7f0abeffd640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0ab40a32d0 0x7f0ab40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.394+0000 7f0acdf33640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0ab40a32d0 0x7f0ab40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.394+0000 7f0abeffd640 1 -- 172.21.15.37:0/3951625046 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f0ac00cfb80 con 0x7f0ac8106f10 2024-09-19T06:40:22.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.394+0000 7f0acdf33640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0ab40a32d0 0x7f0ab40a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009990 tx=0x7f0ac4009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:22.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.398+0000 7f0abeffd640 1 -- 172.21.15.37:0/3951625046 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0ac0016030 con 0x7f0ac8106f10 2024-09-19T06:40:22.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.624+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0ac810a150 con 0x7f0ac8106f10 2024-09-19T06:40:22.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.628+0000 7f0abeffd640 1 -- 172.21.15.37:0/3951625046 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264046 (secure 0 0 0) 0x7f0ac009c8f0 con 0x7f0ac8106f10 2024-09-19T06:40:22.632 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0ab40a32d0 msgr2=0x7f0ab40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0ab40a32d0 0x7f0ab40a5790 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009990 tx=0x7f0ac4009110 comp rx=0 tx=0).stop 2024-09-19T06:40:22.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8106f10 msgr2=0x7f0ac819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8106f10 0x7f0ac819cd70 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0ac000eaa0 tx=0x7f0ac000eea0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 shutdown_connections 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f0ab40a32d0 0x7f0ab40a5790 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0ac8106f10 0x7f0ac819cd70 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0ac8102790 0x7f0ac81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 --2- 172.21.15.37:0/3951625046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ac8101de0 0x7f0ac81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 >> 172.21.15.37:0/3951625046 conn(0x7f0ac80fd8c0 msgr2=0x7f0ac80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 shutdown_connections 2024-09-19T06:40:22.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:22.633+0000 7f0ad0197640 1 -- 172.21.15.37:0/3951625046 wait complete. 2024-09-19T06:40:22.952 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:22 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3951625046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:40:23.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:22 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3951625046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:40:23.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:22 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3951625046' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-19T06:40:23.406 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6800","nonce":871019432},{"type":"v1","addr":"172.21.15.37:6801","nonce":871019432}]},"active_addr":"172.21.15.37:6801/871019432","active_change":"2024-09-19T06:32:56.109718+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.37:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":3406854118}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1436144832}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":711456420}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":1644258429}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.37:0","nonce":2588143564}]}]} 2024-09-19T06:40:23.412 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-19T06:40:23.412 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-19T06:40:23.412 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:40:23.671 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:24.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:23 smithi096 ceph-mon[35004]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-19T06:40:24.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:23 smithi037 ceph-mon[31211]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-19T06:40:24.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:23 smithi151 ceph-mon[35236]: pgmap v294: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 5.4 KiB/s wr, 9 op/s 2024-09-19T06:40:24.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 -- 172.21.15.37:0/659690594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc102790 msgr2=0x7f86fc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:24.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 --2- 172.21.15.37:0/659690594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc102790 0x7f86fc1067e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f86e4009900 tx=0x7f86e402a410 comp rx=0 tx=0).stop 2024-09-19T06:40:24.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 -- 172.21.15.37:0/659690594 shutdown_connections 2024-09-19T06:40:24.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 --2- 172.21.15.37:0/659690594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86fc106f10 0x7f86fc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 --2- 172.21.15.37:0/659690594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc102790 0x7f86fc1067e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 --2- 172.21.15.37:0/659690594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86fc101de0 0x7f86fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 -- 172.21.15.37:0/659690594 >> 172.21.15.37:0/659690594 conn(0x7f86fc0fd8c0 msgr2=0x7f86fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 -- 172.21.15.37:0/659690594 shutdown_connections 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 -- 172.21.15.37:0/659690594 wait complete. 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.140+0000 7f8700c9b640 1 Processor -- start 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f8700c9b640 1 -- start start 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f8700c9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86fc101de0 0x7f86fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f8700c9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86fc102790 0x7f86fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f8700c9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc106f10 0x7f86fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f8700c9b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc111f90 con 0x7f86fc106f10 2024-09-19T06:40:24.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f8700c9b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc111e10 con 0x7f86fc102790 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f8700c9b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f86fc112110 con 0x7f86fc101de0 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc106f10 0x7f86fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc106f10 0x7f86fc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49754/0 (socket says 172.21.15.37:49754) 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 -- 172.21.15.37:0/1205998662 learned_addr learned my addr 172.21.15.37:0/1205998662 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fa59c640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86fc101de0 0x7f86fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86f1d9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86fc102790 0x7f86fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 -- 172.21.15.37:0/1205998662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86fc101de0 msgr2=0x7f86fc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86fc101de0 0x7f86fc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 -- 172.21.15.37:0/1205998662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86fc102790 msgr2=0x7f86fc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86fc102790 0x7f86fc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.141+0000 7f86fad9d640 1 -- 172.21.15.37:0/1205998662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86fc1a6930 con 0x7f86fc106f10 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.142+0000 7f86f1d9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86fc102790 0x7f86fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:24.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.142+0000 7f86fad9d640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc106f10 0x7f86fc074f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f86f400b970 tx=0x7f86f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.142+0000 7f86f3fff640 1 -- 172.21.15.37:0/1205998662 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4019070 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.142+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86fc1a6c20 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.142+0000 7f86f3fff640 1 -- 172.21.15.37:0/1205998662 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86f400cb40 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.142+0000 7f86f3fff640 1 -- 172.21.15.37:0/1205998662 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4007610 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.142+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86fc1a71b0 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.143+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86fc1021c0 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.144+0000 7f86f3fff640 1 -- 172.21.15.37:0/1205998662 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f86f401c070 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.145+0000 7f86f3fff640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f86e80a32d0 0x7f86e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.145+0000 7f86fa59c640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f86e80a32d0 0x7f86e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.145+0000 7f86f3fff640 1 -- 172.21.15.37:0/1205998662 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f86f40cee50 con 0x7f86fc106f10 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.145+0000 7f86fa59c640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f86e80a32d0 0x7f86e80a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0058c0 tx=0x7f86ec00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:24.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.149+0000 7f86f3fff640 1 -- 172.21.15.37:0/1205998662 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86f409b580 con 0x7f86fc106f10 2024-09-19T06:40:24.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.326+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f86fc10a0f0 con 0x7f86fc106f10 2024-09-19T06:40:24.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.327+0000 7f86f3fff640 1 -- 172.21.15.37:0/1205998662 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22268 (secure 0 0 0) 0x7f86f4080710 con 0x7f86fc106f10 2024-09-19T06:40:24.329 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:24.329 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:39:53.021912+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:40:24.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f86e80a32d0 msgr2=0x7f86e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f86e80a32d0 0x7f86e80a5790 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f86ec0058c0 tx=0x7f86ec00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc106f10 msgr2=0x7f86fc074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc106f10 0x7f86fc074f70 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f86f400b970 tx=0x7f86f400be30 comp rx=0 tx=0).stop 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 shutdown_connections 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f86e80a32d0 0x7f86e80a5790 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86fc106f10 0x7f86fc074f70 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86fc102790 0x7f86fc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 --2- 172.21.15.37:0/1205998662 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86fc101de0 0x7f86fc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:24.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 >> 172.21.15.37:0/1205998662 conn(0x7f86fc0fd8c0 msgr2=0x7f86fc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:24.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 shutdown_connections 2024-09-19T06:40:24.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:24.330+0000 7f8700c9b640 1 -- 172.21.15.37:0/1205998662 wait complete. 2024-09-19T06:40:25.015 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:24 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1205998662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:25.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:24 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1205998662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:25.045 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-19T06:40:25.045 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:40:25.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:24 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1205998662' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:25.299 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:25.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 -- 172.21.15.37:0/1038258382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 msgr2=0x7ffae810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/1038258382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae810ea90 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009ca0 tx=0x7ffad802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 -- 172.21.15.37:0/1038258382 shutdown_connections 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/1038258382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae810ea90 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/1038258382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffae8102790 0x7ffae81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/1038258382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffae8101de0 0x7ffae81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 -- 172.21.15.37:0/1038258382 >> 172.21.15.37:0/1038258382 conn(0x7ffae80fd8c0 msgr2=0x7ffae80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 -- 172.21.15.37:0/1038258382 shutdown_connections 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.767+0000 7ffaed7e4640 1 -- 172.21.15.37:0/1038258382 wait complete. 2024-09-19T06:40:25.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 Processor -- start 2024-09-19T06:40:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 -- start start 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffae8101de0 0x7ffae8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffae8102790 0x7ffae8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8111fe0 con 0x7ffae8106f10 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8111e60 con 0x7ffae8101de0 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffaed7e4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffae8112160 con 0x7ffae8102790 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.768+0000 7ffae77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae819cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49770/0 (socket says 172.21.15.37:49770) 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 -- 172.21.15.37:0/4216727234 learned_addr learned my addr 172.21.15.37:0/4216727234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae6ffd640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffae8101de0 0x7ffae8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae67fc640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffae8102790 0x7ffae8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 -- 172.21.15.37:0/4216727234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffae8102790 msgr2=0x7ffae8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffae8102790 0x7ffae8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 -- 172.21.15.37:0/4216727234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffae8101de0 msgr2=0x7ffae8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffae8101de0 0x7ffae8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 -- 172.21.15.37:0/4216727234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffae819d4b0 con 0x7ffae8106f10 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae67fc640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffae8102790 0x7ffae8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffae77fe640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae819cdd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffad8003f10 tx=0x7ffad8003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffacffff640 1 -- 172.21.15.37:0/4216727234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad8031650 con 0x7ffae8106f10 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffacffff640 1 -- 172.21.15.37:0/4216727234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffad80317f0 con 0x7ffae8106f10 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffae819d740 con 0x7ffae8106f10 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffacffff640 1 -- 172.21.15.37:0/4216727234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffad802fab0 con 0x7ffae8106f10 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.769+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffae81a5080 con 0x7ffae8106f10 2024-09-19T06:40:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.770+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffae81021c0 con 0x7ffae8106f10 2024-09-19T06:40:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.771+0000 7ffacffff640 1 -- 172.21.15.37:0/4216727234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ffad802fc90 con 0x7ffae8106f10 2024-09-19T06:40:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.772+0000 7ffacffff640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffad00a32d0 0x7ffad00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.773+0000 7ffae6ffd640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffad00a32d0 0x7ffad00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.773+0000 7ffacffff640 1 -- 172.21.15.37:0/4216727234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ffad80f0d00 con 0x7ffae8106f10 2024-09-19T06:40:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.773+0000 7ffae6ffd640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffad00a32d0 0x7ffad00a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009990 tx=0x7ffae0009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.777+0000 7ffacffff640 1 -- 172.21.15.37:0/4216727234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffad80f4030 con 0x7ffae8106f10 2024-09-19T06:40:25.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.953+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ffae810a0f0 con 0x7ffae8106f10 2024-09-19T06:40:25.954 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:25 smithi037 ceph-mon[31211]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:40:25.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.955+0000 7ffacffff640 1 -- 172.21.15.37:0/4216727234 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+22268 (secure 0 0 0) 0x7ffad80bdb80 con 0x7ffae8106f10 2024-09-19T06:40:25.956 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:25.956 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":81,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:39:53.021912+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffad00a32d0 msgr2=0x7ffad00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffad00a32d0 0x7ffad00a5790 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009990 tx=0x7ffae0009110 comp rx=0 tx=0).stop 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 msgr2=0x7ffae819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae819cdd0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffad8003f10 tx=0x7ffad8003f40 comp rx=0 tx=0).stop 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 shutdown_connections 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffad00a32d0 0x7ffad00a5790 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffae8106f10 0x7ffae819cdd0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffae8102790 0x7ffae8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 --2- 172.21.15.37:0/4216727234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffae8101de0 0x7ffae8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:25.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 >> 172.21.15.37:0/4216727234 conn(0x7ffae80fd8c0 msgr2=0x7ffae80febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 shutdown_connections 2024-09-19T06:40:25.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:25.957+0000 7ffaed7e4640 1 -- 172.21.15.37:0/4216727234 wait complete. 2024-09-19T06:40:26.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:25 smithi096 ceph-mon[35004]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:40:26.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:25 smithi151 ceph-mon[35236]: pgmap v295: 97 pgs: 97 active+clean; 634 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:40:26.641 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-19T06:40:26.642 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-19T06:40:26.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-19T06:40:26.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-19T06:40:26.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-19T06:40:26.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-19T06:40:26.644 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-19T06:40:26.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-19T06:40:26.645 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-19T06:40:26.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-19T06:40:26.646 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-19T06:40:26.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-19T06:40:27.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:26 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4216727234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:27.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:26 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4216727234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:27.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:26 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4216727234' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:40:27.332 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.462 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.463 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.463 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.469 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.470 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.483 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.488 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.491 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.501 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:27.915 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:27 smithi037 ceph-mon[31211]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-19T06:40:28.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:27 smithi096 ceph-mon[35004]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-19T06:40:28.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:27 smithi151 ceph-mon[35236]: pgmap v296: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.0 KiB/s wr, 8 op/s 2024-09-19T06:40:28.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 -- 172.21.15.37:0/2064797057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f810e500 msgr2=0x7f74f8110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 --2- 172.21.15.37:0/2064797057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f810e500 0x7f74f8110910 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f74f0009ca0 tx=0x7f74f002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 -- 172.21.15.37:0/2064797057 shutdown_connections 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 --2- 172.21.15.37:0/2064797057 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f810e500 0x7f74f8110910 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 --2- 172.21.15.37:0/2064797057 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74f8101440 0x7f74f810df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 --2- 172.21.15.37:0/2064797057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f8100b20 0x7f74f8100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 -- 172.21.15.37:0/2064797057 >> 172.21.15.37:0/2064797057 conn(0x7f74f80fc7d0 msgr2=0x7f74f80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 -- 172.21.15.37:0/2064797057 shutdown_connections 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.498+0000 7f74ff2b4640 1 -- 172.21.15.37:0/2064797057 wait complete. 2024-09-19T06:40:28.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 Processor -- start 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 -- start start 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f8100b20 0x7f74f819a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8101440 0x7f74f819ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74f810e500 0x7f74f819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74f81142d0 con 0x7f74f8101440 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8114150 con 0x7f74f8100b20 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74ff2b4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74f8114450 con 0x7f74f810e500 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74fc84f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8101440 0x7f74f819ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:28.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74fc84f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8101440 0x7f74f819ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49794/0 (socket says 172.21.15.37:49794) 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.499+0000 7f74fc84f640 1 -- 172.21.15.37:0/4221497955 learned_addr learned my addr 172.21.15.37:0/4221497955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fd050640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f8100b20 0x7f74f819a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fd851640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74f810e500 0x7f74f819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fc84f640 1 -- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74f810e500 msgr2=0x7f74f819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fc84f640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74f810e500 0x7f74f819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fc84f640 1 -- 172.21.15.37:0/4221497955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f8100b20 msgr2=0x7f74f819a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fc84f640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74f8100b20 0x7f74f819a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fc84f640 1 -- 172.21.15.37:0/4221497955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74f819f6a0 con 0x7f74f8101440 2024-09-19T06:40:28.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fd851640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74f810e500 0x7f74f819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74fc84f640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8101440 0x7f74f819ac30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b450 tx=0x7f74e800b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e8015020 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74f819f990 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74e8011d10 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74e800f2e0 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.500+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74f81a7270 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.501+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f74f810fae0 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.503+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f74e8018020 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.503+0000 7f74ee7fc640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f74e40a32d0 0x7f74e40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.503+0000 7f74fd050640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f74e40a32d0 0x7f74e40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.503+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f74e80cf760 con 0x7f74f8101440 2024-09-19T06:40:28.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.503+0000 7f74ee7fc640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] conn(0x7f74e40acb20 0x7f74e40aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.503+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 --> [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74e8013ec0 con 0x7f74e40acb20 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.503+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f74e80cfc40 con 0x7f74f8101440 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.504+0000 7f74fd050640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f74e40a32d0 0x7f74e40a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74f400abe0 tx=0x7f74f400a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.504+0000 7f74fd851640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] conn(0x7f74e40acb20 0x7f74e40aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.504+0000 7f74fd851640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] conn(0x7f74e40acb20 0x7f74e40aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.505+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== osd.11 v2:172.21.15.151:6824/3357779771 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74e8013ec0 con 0x7f74e40acb20 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.519+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 --> [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f74f806a850 con 0x7f74e40acb20 2024-09-19T06:40:28.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.519+0000 7f74ee7fc640 1 -- 172.21.15.37:0/4221497955 <== osd.11 v2:172.21.15.151:6824/3357779771 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f74f806a850 con 0x7f74e40acb20 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] conn(0x7f74e40acb20 msgr2=0x7f74e40aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] conn(0x7f74e40acb20 0x7f74e40aef40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f74e40a32d0 msgr2=0x7f74e40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f74e40a32d0 0x7f74e40a5790 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74f400abe0 tx=0x7f74f400a2d0 comp rx=0 tx=0).stop 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8101440 msgr2=0x7f74f819ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 --2- 172.21.15.37:0/4221497955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74f8101440 0x7f74f819ac30 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f74e800b450 tx=0x7f74e800b850 comp rx=0 tx=0).stop 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74fd851640 1 -- 172.21.15.37:0/4221497955 reap_dead start 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 shutdown_connections 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 >> 172.21.15.37:0/4221497955 conn(0x7f74f80fc7d0 msgr2=0x7f74f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 shutdown_connections 2024-09-19T06:40:28.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:28.520+0000 7f74ff2b4640 1 -- 172.21.15.37:0/4221497955 wait complete. 2024-09-19T06:40:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.498+0000 7fb319597640 1 -- 172.21.15.37:0/1229295480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314106a80 msgr2=0x7fb314106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.498+0000 7fb319597640 1 --2- 172.21.15.37:0/1229295480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314106a80 0x7fb314106e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb304009cd0 tx=0x7fb30402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.498+0000 7fb319597640 1 -- 172.21.15.37:0/1229295480 shutdown_connections 2024-09-19T06:40:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.498+0000 7fb319597640 1 --2- 172.21.15.37:0/1229295480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb314101420 0x7fb31410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.498+0000 7fb319597640 1 --2- 172.21.15.37:0/1229295480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb314100a80 0x7fb314100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.498+0000 7fb319597640 1 --2- 172.21.15.37:0/1229295480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314106a80 0x7fb314106e60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.498+0000 7fb319597640 1 -- 172.21.15.37:0/1229295480 >> 172.21.15.37:0/1229295480 conn(0x7fb3140fc7a0 msgr2=0x7fb3140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.499+0000 7fb319597640 1 -- 172.21.15.37:0/1229295480 shutdown_connections 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.499+0000 7fb319597640 1 -- 172.21.15.37:0/1229295480 wait complete. 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.499+0000 7fb319597640 1 Processor -- start 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb319597640 1 -- start start 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb319597640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314100a80 0x7fb3141984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb319597640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb314101420 0x7fb314198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb319597640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb314106a80 0x7fb31419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb319597640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb314110e80 con 0x7fb314100a80 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb319597640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb314110d00 con 0x7fb314101420 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb319597640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb314111000 con 0x7fb314106a80 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314100a80 0x7fb3141984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314100a80 0x7fb3141984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47906/0 (socket says 172.21.15.37:47906) 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 -- 172.21.15.37:0/3564443860 learned_addr learned my addr 172.21.15.37:0/3564443860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb3137fe640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb314106a80 0x7fb31419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb3127fc640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb314101420 0x7fb314198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 -- 172.21.15.37:0/3564443860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb314106a80 msgr2=0x7fb31419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb314106a80 0x7fb31419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 -- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb314101420 msgr2=0x7fb314198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb314101420 0x7fb314198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.500+0000 7fb312ffd640 1 -- 172.21.15.37:0/3564443860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb31419d4a0 con 0x7fb314100a80 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.501+0000 7fb312ffd640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314100a80 0x7fb3141984f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb30402ae00 tx=0x7fb304003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.501+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304004040 con 0x7fb314100a80 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.501+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3040046c0 con 0x7fb314100a80 2024-09-19T06:40:29.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.501+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb31419d730 con 0x7fb314100a80 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.501+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb304046480 con 0x7fb314100a80 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.501+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3141a5070 con 0x7fb314100a80 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.502+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb3140628c0 con 0x7fb314100a80 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.503+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb3040305a0 con 0x7fb314100a80 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.504+0000 7fb2fbfff640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2fc0a3350 0x7fb2fc0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.504+0000 7fb3127fc640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2fc0a3350 0x7fb2fc0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.504+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fb30402e030 con 0x7fb314100a80 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.504+0000 7fb2fbfff640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] conn(0x7fb2fc0acba0 0x7fb2fc0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.504+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 --> [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb2fc0af670 con 0x7fb2fc0acba0 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.504+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7fb3040f1540 con 0x7fb314100a80 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.504+0000 7fb3127fc640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2fc0a3350 0x7fb2fc0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3141051d0 tx=0x7fb30800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.505+0000 7fb3137fe640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] conn(0x7fb2fc0acba0 0x7fb2fc0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.505+0000 7fb3137fe640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] conn(0x7fb2fc0acba0 0x7fb2fc0aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.506+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== osd.1 v2:172.21.15.96:6800/515532313 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb2fc0af670 con 0x7fb2fc0acba0 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.519+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 --> [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb314062cc0 con 0x7fb2fc0acba0 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb2fbfff640 1 -- 172.21.15.37:0/3564443860 <== osd.1 v2:172.21.15.96:6800/515532313 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb314062cc0 con 0x7fb2fc0acba0 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] conn(0x7fb2fc0acba0 msgr2=0x7fb2fc0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] conn(0x7fb2fc0acba0 0x7fb2fc0aefc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2fc0a3350 msgr2=0x7fb2fc0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2fc0a3350 0x7fb2fc0a5810 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb3141051d0 tx=0x7fb30800b390 comp rx=0 tx=0).stop 2024-09-19T06:40:29.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314100a80 msgr2=0x7fb3141984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 --2- 172.21.15.37:0/3564443860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb314100a80 0x7fb3141984f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fb30402ae00 tx=0x7fb304003970 comp rx=0 tx=0).stop 2024-09-19T06:40:29.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb3137fe640 1 -- 172.21.15.37:0/3564443860 reap_dead start 2024-09-19T06:40:29.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 shutdown_connections 2024-09-19T06:40:29.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 >> 172.21.15.37:0/3564443860 conn(0x7fb3140fc7a0 msgr2=0x7fb31406aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:29.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 shutdown_connections 2024-09-19T06:40:29.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.520+0000 7fb319597640 1 -- 172.21.15.37:0/3564443860 wait complete. 2024-09-19T06:40:29.751 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:29 smithi037 ceph-mon[31211]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 -- 172.21.15.37:0/748258050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1884102db0 msgr2=0x7f1884103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 --2- 172.21.15.37:0/748258050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1884102db0 0x7f1884103190 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f1874009930 tx=0x7f187402a420 comp rx=0 tx=0).stop 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 -- 172.21.15.37:0/748258050 shutdown_connections 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 --2- 172.21.15.37:0/748258050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18841082e0 0x7f188410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 --2- 172.21.15.37:0/748258050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1884103760 0x7f1884107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 --2- 172.21.15.37:0/748258050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1884102db0 0x7f1884103190 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 -- 172.21.15.37:0/748258050 >> 172.21.15.37:0/748258050 conn(0x7f18840fc7d0 msgr2=0x7f18840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 -- 172.21.15.37:0/748258050 shutdown_connections 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.797+0000 7f188a4ef640 1 -- 172.21.15.37:0/748258050 wait complete. 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 Processor -- start 2024-09-19T06:40:29.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 -- start start 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1884102db0 0x7f1884198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1884103760 0x7f18841989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18841082e0 0x7f188419cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1884112090 con 0x7f18841082e0 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1884111f10 con 0x7f1884103760 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f188a4ef640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1884112210 con 0x7f1884102db0 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f1888a8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18841082e0 0x7f188419cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f1888a8c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18841082e0 0x7f188419cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47934/0 (socket says 172.21.15.37:47934) 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.798+0000 7f1888a8c640 1 -- 172.21.15.37:0/3645483511 learned_addr learned my addr 172.21.15.37:0/3645483511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f18837fe640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1884103760 0x7f18841989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f1883fff640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1884102db0 0x7f1884198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f1888a8c640 1 -- 172.21.15.37:0/3645483511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1884102db0 msgr2=0x7f1884198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f1888a8c640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1884102db0 0x7f1884198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f1888a8c640 1 -- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1884103760 msgr2=0x7f18841989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f1888a8c640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1884103760 0x7f18841989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f1888a8c640 1 -- 172.21.15.37:0/3645483511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f188419d430 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f1888a8c640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18841082e0 0x7f188419cd50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f187c00cce0 tx=0x7f187c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c00f450 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f188419d720 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f187c00fad0 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f187c018b60 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.799+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18841a5000 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.800+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f18840fedf0 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.801+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f187c00fc80 con 0x7f18841082e0 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.802+0000 7f18817fa640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f186c0a3350 0x7f186c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.802+0000 7f1883fff640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f186c0a3350 0x7f186c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.802+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f187c0d00f0 con 0x7f18841082e0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.802+0000 7f18817fa640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] conn(0x7f186c0acba0 0x7f186c0aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.802+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 --> [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f186c0af670 con 0x7f186c0acba0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.802+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f187c0d05d0 con 0x7f18841082e0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.803+0000 7f1883fff640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f186c0a3350 0x7f186c0a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1874006010 tx=0x7f18740058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.803+0000 7f18837fe640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] conn(0x7f186c0acba0 0x7f186c0aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.803+0000 7f18837fe640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] conn(0x7f186c0acba0 0x7f186c0aefc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.804+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== osd.4 v2:172.21.15.96:6808/3832301595 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f186c0af670 con 0x7f186c0acba0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.817+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 --> [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f188406a7d0 con 0x7f186c0acba0 2024-09-19T06:40:29.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f18817fa640 1 -- 172.21.15.37:0/3645483511 <== osd.4 v2:172.21.15.96:6808/3832301595 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f188406a7d0 con 0x7f186c0acba0 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] conn(0x7f186c0acba0 msgr2=0x7f186c0aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f188a4ef640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] conn(0x7f186c0acba0 0x7f186c0aefc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f186c0a3350 msgr2=0x7f186c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f188a4ef640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f186c0a3350 0x7f186c0a5810 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1874006010 tx=0x7f18740058d0 comp rx=0 tx=0).stop 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18841082e0 msgr2=0x7f188419cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f188a4ef640 1 --2- 172.21.15.37:0/3645483511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18841082e0 0x7f188419cd50 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f187c00cce0 tx=0x7f187c00a620 comp rx=0 tx=0).stop 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.818+0000 7f1888a8c640 1 -- 172.21.15.37:0/3645483511 reap_dead start 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.819+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 shutdown_connections 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.819+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 >> 172.21.15.37:0/3645483511 conn(0x7f18840fc7d0 msgr2=0x7f18840fe390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.819+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 shutdown_connections 2024-09-19T06:40:29.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.819+0000 7f188a4ef640 1 -- 172.21.15.37:0/3645483511 wait complete. 2024-09-19T06:40:30.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:29 smithi096 ceph-mon[35004]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T06:40:30.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:29 smithi151 ceph-mon[35236]: pgmap v297: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T06:40:30.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 -- 172.21.15.37:0/3479474793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102570 msgr2=0x7f1c701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 --2- 172.21.15.37:0/3479474793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102570 0x7f1c701029b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f1c60009900 tx=0x7f1c6002a480 comp rx=0 tx=0).stop 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 -- 172.21.15.37:0/3479474793 shutdown_connections 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 --2- 172.21.15.37:0/3479474793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102ef0 0x7f1c7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 --2- 172.21.15.37:0/3479474793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102570 0x7f1c701029b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 --2- 172.21.15.37:0/3479474793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c70100e20 0x7f1c70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 -- 172.21.15.37:0/3479474793 >> 172.21.15.37:0/3479474793 conn(0x7f1c700fc7d0 msgr2=0x7f1c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 -- 172.21.15.37:0/3479474793 shutdown_connections 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 -- 172.21.15.37:0/3479474793 wait complete. 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 Processor -- start 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 -- start start 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.215+0000 7f1c77b34640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c70100e20 0x7f1c701993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c77b34640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102570 0x7f1c70199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c77b34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102ef0 0x7f1c701a4450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c77b34640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70110fa0 con 0x7f1c70102ef0 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c77b34640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70110e20 con 0x7f1c70102570 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c77b34640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70111120 con 0x7f1c70100e20 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102ef0 0x7f1c701a4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102ef0 0x7f1c701a4450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47956/0 (socket says 172.21.15.37:47956) 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 -- 172.21.15.37:0/1551778139 learned_addr learned my addr 172.21.15.37:0/1551778139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c750cf640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102570 0x7f1c70199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c758d0640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c70100e20 0x7f1c701993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 -- 172.21.15.37:0/1551778139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c70100e20 msgr2=0x7f1c701993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c70100e20 0x7f1c701993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 -- 172.21.15.37:0/1551778139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102570 msgr2=0x7f1c70199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102570 0x7f1c70199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 -- 172.21.15.37:0/1551778139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c701a4990 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c760d1640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102ef0 0x7f1c701a4450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b970 tx=0x7f1c6800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68019070 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c701a4b60 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c6800cb40 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c68007650 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.216+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c701a5070 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.217+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1c70062900 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.218+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1c6801c070 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.219+0000 7f1c66ffd640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1c5c0a32d0 0x7f1c5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.219+0000 7f1c758d0640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1c5c0a32d0 0x7f1c5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.219+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1c680cf760 con 0x7f1c70102ef0 2024-09-19T06:40:30.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.219+0000 7f1c66ffd640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] conn(0x7f1c5c0acb20 0x7f1c5c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.219+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 --> [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1c680047d0 con 0x7f1c5c0acb20 2024-09-19T06:40:30.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.219+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1c680cfb10 con 0x7f1c70102ef0 2024-09-19T06:40:30.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.220+0000 7f1c750cf640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] conn(0x7f1c5c0acb20 0x7f1c5c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.220+0000 7f1c758d0640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1c5c0a32d0 0x7f1c5c0a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005fd0 tx=0x7f1c6c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.220+0000 7f1c750cf640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] conn(0x7f1c5c0acb20 0x7f1c5c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.221+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== osd.6 v2:172.21.15.37:6818/3128436148 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1c680047d0 con 0x7f1c5c0acb20 2024-09-19T06:40:30.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.234+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 --> [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1c70062d80 con 0x7f1c5c0acb20 2024-09-19T06:40:30.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c66ffd640 1 -- 172.21.15.37:0/1551778139 <== osd.6 v2:172.21.15.37:6818/3128436148 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1c70062d80 con 0x7f1c5c0acb20 2024-09-19T06:40:30.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] conn(0x7f1c5c0acb20 msgr2=0x7f1c5c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] conn(0x7f1c5c0acb20 0x7f1c5c0aef40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1c5c0a32d0 msgr2=0x7f1c5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1c5c0a32d0 0x7f1c5c0a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005fd0 tx=0x7f1c6c005850 comp rx=0 tx=0).stop 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102ef0 msgr2=0x7f1c701a4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 --2- 172.21.15.37:0/1551778139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c70102ef0 0x7f1c701a4450 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800b970 tx=0x7f1c6800be30 comp rx=0 tx=0).stop 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c760d1640 1 -- 172.21.15.37:0/1551778139 reap_dead start 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 shutdown_connections 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 >> 172.21.15.37:0/1551778139 conn(0x7f1c700fc7d0 msgr2=0x7f1c700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 shutdown_connections 2024-09-19T06:40:30.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:30.235+0000 7f1c77b34640 1 -- 172.21.15.37:0/1551778139 wait complete. 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 -- 172.21.15.37:0/2995801943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 msgr2=0x7f6538106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 --2- 172.21.15.37:0/2995801943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 0x7f6538106e50 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6528009930 tx=0x7f652802a420 comp rx=0 tx=0).stop 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 -- 172.21.15.37:0/2995801943 shutdown_connections 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 --2- 172.21.15.37:0/2995801943 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6538101410 0x7f653810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 --2- 172.21.15.37:0/2995801943 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6538100a70 0x7f6538100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 --2- 172.21.15.37:0/2995801943 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 0x7f6538106e50 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 -- 172.21.15.37:0/2995801943 >> 172.21.15.37:0/2995801943 conn(0x7f65380fc790 msgr2=0x7f65380febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:30.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 -- 172.21.15.37:0/2995801943 shutdown_connections 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.719+0000 7f653c878640 1 -- 172.21.15.37:0/2995801943 wait complete. 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 Processor -- start 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 -- start start 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6538100a70 0x7f6538199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6538101410 0x7f65381999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 0x7f65381a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6538110f10 con 0x7f6538106a70 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6538110d90 con 0x7f6538101410 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f653c878640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6538111090 con 0x7f6538100a70 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f6536d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 0x7f65381a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f6536d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 0x7f65381a4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47916/0 (socket says 172.21.15.37:47916) 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.720+0000 7f6536d9d640 1 -- 172.21.15.37:0/2327409101 learned_addr learned my addr 172.21.15.37:0/2327409101 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f653659c640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6538100a70 0x7f6538199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f652dd9b640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6538101410 0x7f65381999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f6536d9d640 1 -- 172.21.15.37:0/2327409101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6538100a70 msgr2=0x7f6538199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f6536d9d640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6538100a70 0x7f6538199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f6536d9d640 1 -- 172.21.15.37:0/2327409101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6538101410 msgr2=0x7f65381999c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f6536d9d640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6538101410 0x7f65381999c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f6536d9d640 1 -- 172.21.15.37:0/2327409101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65381a4a40 con 0x7f6538106a70 2024-09-19T06:40:30.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f6536d9d640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 0x7f65381a4500 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f653000b970 tx=0x7f653000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530019070 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6538197270 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6530014440 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65300075d0 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.721+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6538197760 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.722+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6538062900 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.724+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f653001c070 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.724+0000 7f652ffff640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f65240a3350 0x7f65240a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.724+0000 7f653659c640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f65240a3350 0x7f65240a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.724+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f65300cf4d0 con 0x7f6538106a70 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.725+0000 7f652ffff640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] conn(0x7f65240acba0 0x7f65240aefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.725+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 --> [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f65240af670 con 0x7f65240acba0 2024-09-19T06:40:30.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.725+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f65300cf9b0 con 0x7f6538106a70 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.725+0000 7f652dd9b640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] conn(0x7f65240acba0 0x7f65240aefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.725+0000 7f653659c640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f65240a3350 0x7f65240a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6528009900 tx=0x7f65280058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.725+0000 7f652dd9b640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] conn(0x7f65240acba0 0x7f65240aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.726+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== osd.0 v2:172.21.15.37:6802/2764918113 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f65240af670 con 0x7f65240acba0 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 --> [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6538062d80 con 0x7f65240acba0 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f652ffff640 1 -- 172.21.15.37:0/2327409101 <== osd.0 v2:172.21.15.37:6802/2764918113 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6538062d80 con 0x7f65240acba0 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] conn(0x7f65240acba0 msgr2=0x7f65240aefc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] conn(0x7f65240acba0 0x7f65240aefc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f65240a3350 msgr2=0x7f65240a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f65240a3350 0x7f65240a5810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6528009900 tx=0x7f65280058d0 comp rx=0 tx=0).stop 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 msgr2=0x7f65381a4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 --2- 172.21.15.37:0/2327409101 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6538106a70 0x7f65381a4500 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f653000b970 tx=0x7f653000bd70 comp rx=0 tx=0).stop 2024-09-19T06:40:30.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f6536d9d640 1 -- 172.21.15.37:0/2327409101 reap_dead start 2024-09-19T06:40:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 shutdown_connections 2024-09-19T06:40:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.740+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 >> 172.21.15.37:0/2327409101 conn(0x7f65380fc790 msgr2=0x7f65380fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.741+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 shutdown_connections 2024-09-19T06:40:30.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:29.741+0000 7f653c878640 1 -- 172.21.15.37:0/2327409101 wait complete. 2024-09-19T06:40:31.310 INFO:teuthology.orchestra.run.smithi037.stdout:137438953529 2024-09-19T06:40:31.311 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-19T06:40:31.683 INFO:teuthology.orchestra.run.smithi037.stdout:188978561071 2024-09-19T06:40:31.684 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-19T06:40:31.906 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:31 smithi037 ceph-mon[31211]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T06:40:32.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:31 smithi096 ceph-mon[35004]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T06:40:32.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:31 smithi151 ceph-mon[35236]: pgmap v298: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/377402337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 msgr2=0x7ff1bc103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 --2- 172.21.15.37:0/377402337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 0x7ff1bc103190 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac009930 tx=0x7ff1ac02a420 comp rx=0 tx=0).stop 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/377402337 shutdown_connections 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 --2- 172.21.15.37:0/377402337 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1bc1082e0 0x7ff1bc10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 --2- 172.21.15.37:0/377402337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1bc103760 0x7ff1bc107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 --2- 172.21.15.37:0/377402337 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 0x7ff1bc103190 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/377402337 >> 172.21.15.37:0/377402337 conn(0x7ff1bc0fc7d0 msgr2=0x7ff1bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/377402337 shutdown_connections 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.283+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/377402337 wait complete. 2024-09-19T06:40:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 Processor -- start 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 -- start start 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 0x7ff1bc100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1bc103760 0x7ff1bc100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1bc1082e0 0x7ff1bc101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc112110 con 0x7ff1bc102db0 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc111f90 con 0x7ff1bc1082e0 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1c0ed9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff1bc112290 con 0x7ff1bc103760 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 0x7ff1bc100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 0x7ff1bc100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47966/0 (socket says 172.21.15.37:47966) 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 -- 172.21.15.37:0/1959388941 learned_addr learned my addr 172.21.15.37:0/1959388941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1bad9d640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1bc1082e0 0x7ff1bc101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1b3fff640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1bc103760 0x7ff1bc100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 -- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1bc103760 msgr2=0x7ff1bc100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1bc103760 0x7ff1bc100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 -- 172.21.15.37:0/1959388941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1bc1082e0 msgr2=0x7ff1bc101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1bc1082e0 0x7ff1bc101490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1ba59c640 1 -- 172.21.15.37:0/1959388941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1bc1a6ae0 con 0x7ff1bc102db0 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1b3fff640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff1bc103760 0x7ff1bc100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.284+0000 7ff1bad9d640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1bc1082e0 0x7ff1bc101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.285+0000 7ff1ba59c640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 0x7ff1bc100a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac004770 tx=0x7ff1ac003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.285+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac03f070 con 0x7ff1bc102db0 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.285+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff1ac031c20 con 0x7ff1bc102db0 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.285+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1bc1a6d70 con 0x7ff1bc102db0 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.285+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1ac035030 con 0x7ff1bc102db0 2024-09-19T06:40:32.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.285+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1bc1a71f0 con 0x7ff1bc102db0 2024-09-19T06:40:32.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.286+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff1bc0628c0 con 0x7ff1bc102db0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.287+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff1ac042070 con 0x7ff1bc102db0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1b37fe640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff1a80a32d0 0x7ff1a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1b3fff640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff1a80a32d0 0x7ff1a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ff1ac0f0b50 con 0x7ff1bc102db0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1b37fe640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] conn(0x7ff1a80acb20 0x7ff1a80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 --> [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1ac038e90 con 0x7ff1a80acb20 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7ff1ac0f0f00 con 0x7ff1bc102db0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1b3fff640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff1a80a32d0 0x7ff1a80a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40058c0 tx=0x7ff1a4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.288+0000 7ff1bad9d640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] conn(0x7ff1a80acb20 0x7ff1a80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.289+0000 7ff1bad9d640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] conn(0x7ff1a80acb20 0x7ff1a80aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:32.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.290+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== osd.5 v2:172.21.15.151:6808/4035837098 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff1ac038e90 con 0x7ff1a80acb20 2024-09-19T06:40:32.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.303+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 --> [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff1bc062cc0 con 0x7ff1a80acb20 2024-09-19T06:40:32.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1b37fe640 1 -- 172.21.15.37:0/1959388941 <== osd.5 v2:172.21.15.151:6808/4035837098 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff1bc062cc0 con 0x7ff1a80acb20 2024-09-19T06:40:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] conn(0x7ff1a80acb20 msgr2=0x7ff1a80aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] conn(0x7ff1a80acb20 0x7ff1a80aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff1a80a32d0 msgr2=0x7ff1a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff1a80a32d0 0x7ff1a80a5790 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40058c0 tx=0x7ff1a4005850 comp rx=0 tx=0).stop 2024-09-19T06:40:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 msgr2=0x7ff1bc100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 --2- 172.21.15.37:0/1959388941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff1bc102db0 0x7ff1bc100a10 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac004770 tx=0x7ff1ac003b90 comp rx=0 tx=0).stop 2024-09-19T06:40:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1bad9d640 1 -- 172.21.15.37:0/1959388941 reap_dead start 2024-09-19T06:40:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 shutdown_connections 2024-09-19T06:40:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 >> 172.21.15.37:0/1959388941 conn(0x7ff1bc0fc7d0 msgr2=0x7ff1bc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 shutdown_connections 2024-09-19T06:40:32.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:32.304+0000 7ff1c0ed9640 1 -- 172.21.15.37:0/1959388941 wait complete. 2024-09-19T06:40:33.066 INFO:teuthology.orchestra.run.smithi037.stdout:158913790003 2024-09-19T06:40:33.066 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-19T06:40:34.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:33 smithi096 ceph-mon[35004]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-19T06:40:34.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:33 smithi037 ceph-mon[31211]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-19T06:40:34.094 INFO:teuthology.orchestra.run.smithi037.stdout:55834574920 2024-09-19T06:40:34.094 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-19T06:40:34.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:33 smithi151 ceph-mon[35236]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 -- 172.21.15.37:0/3040104829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420108290 msgr2=0x7f542010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 --2- 172.21.15.37:0/3040104829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420108290 0x7f542010ec30 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f5418009cd0 tx=0x7f541802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 -- 172.21.15.37:0/3040104829 shutdown_connections 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 --2- 172.21.15.37:0/3040104829 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420108290 0x7f542010ec30 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 --2- 172.21.15.37:0/3040104829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5420103740 0x7f5420107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 --2- 172.21.15.37:0/3040104829 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5420102d90 0x7f5420103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 -- 172.21.15.37:0/3040104829 >> 172.21.15.37:0/3040104829 conn(0x7f54200fc7d0 msgr2=0x7f54200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 -- 172.21.15.37:0/3040104829 shutdown_connections 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 -- 172.21.15.37:0/3040104829 wait complete. 2024-09-19T06:40:34.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 Processor -- start 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.264+0000 7f54278ea640 1 -- start start 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f54278ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420102d90 0x7f54201009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f54278ea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5420103740 0x7f5420100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f54278ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5420108290 0x7f5420101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f54278ea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54201120d0 con 0x7f5420102d90 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f54278ea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5420111f50 con 0x7f5420108290 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f54278ea640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5420112250 con 0x7f5420103740 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420102d90 0x7f54201009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420102d90 0x7f54201009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47990/0 (socket says 172.21.15.37:47990) 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 -- 172.21.15.37:0/573739282 learned_addr learned my addr 172.21.15.37:0/573739282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5424e85640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5420103740 0x7f5420100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425e87640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5420108290 0x7f5420101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:34.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 -- 172.21.15.37:0/573739282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5420103740 msgr2=0x7f5420100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5420103740 0x7f5420100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 -- 172.21.15.37:0/573739282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5420108290 msgr2=0x7f5420101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5420108290 0x7f5420101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 -- 172.21.15.37:0/573739282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54201a6a10 con 0x7f5420102d90 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425e87640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5420108290 0x7f5420101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f5425686640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420102d90 0x7f54201009f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c920 tx=0x7f541c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.265+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c007c70 con 0x7f5420102d90 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.266+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54201a6d00 con 0x7f5420102d90 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.266+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f541c007e10 con 0x7f5420102d90 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.266+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541c018890 con 0x7f5420102d90 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.266+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54201a71e0 con 0x7f5420102d90 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.267+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f54200628c0 con 0x7f5420102d90 2024-09-19T06:40:34.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.268+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f541c018a70 con 0x7f5420102d90 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.268+0000 7f54167fc640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f540c0a32d0 0x7f540c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f5424e85640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f540c0a32d0 0x7f540c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f541c0cfaf0 con 0x7f5420102d90 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f54167fc640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] conn(0x7f540c0acb20 0x7f540c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 --> [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f541c012ec0 con 0x7f540c0acb20 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f541c0d4260 con 0x7f5420102d90 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f5425e87640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] conn(0x7f540c0acb20 0x7f540c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f5424e85640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f540c0a32d0 0x7f540c0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5420102480 tx=0x7f5410008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.269+0000 7f5425e87640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] conn(0x7f540c0acb20 0x7f540c0aef40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:34.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.270+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== osd.3 v2:172.21.15.37:6810/1041463019 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f541c012ec0 con 0x7f540c0acb20 2024-09-19T06:40:34.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.294+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 --> [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5420062d40 con 0x7f540c0acb20 2024-09-19T06:40:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.294+0000 7f54167fc640 1 -- 172.21.15.37:0/573739282 <== osd.3 v2:172.21.15.37:6810/1041463019 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5420062d40 con 0x7f540c0acb20 2024-09-19T06:40:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] conn(0x7f540c0acb20 msgr2=0x7f540c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:34.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] conn(0x7f540c0acb20 0x7f540c0aef40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f540c0a32d0 msgr2=0x7f540c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f540c0a32d0 0x7f540c0a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5420102480 tx=0x7f5410008040 comp rx=0 tx=0).stop 2024-09-19T06:40:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420102d90 msgr2=0x7f54201009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:34.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 --2- 172.21.15.37:0/573739282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5420102d90 0x7f54201009f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f541c00c920 tx=0x7f541c00cde0 comp rx=0 tx=0).stop 2024-09-19T06:40:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f5425e87640 1 -- 172.21.15.37:0/573739282 reap_dead start 2024-09-19T06:40:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 shutdown_connections 2024-09-19T06:40:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.295+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 >> 172.21.15.37:0/573739282 conn(0x7f54200fc7d0 msgr2=0x7f54200fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.296+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 shutdown_connections 2024-09-19T06:40:34.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:34.296+0000 7f54278ea640 1 -- 172.21.15.37:0/573739282 wait complete. 2024-09-19T06:40:34.822 INFO:teuthology.orchestra.run.smithi037.stdout:34359738445 2024-09-19T06:40:34.823 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-19T06:40:35.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.003+0000 7faf68c98640 1 -- 172.21.15.37:0/2140050129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf64103660 msgr2=0x7faf6410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.003+0000 7faf68c98640 1 --2- 172.21.15.37:0/2140050129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf64103660 0x7faf6410bb50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7faf58009900 tx=0x7faf5802a480 comp rx=0 tx=0).stop 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.004+0000 7faf68c98640 1 -- 172.21.15.37:0/2140050129 shutdown_connections 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.004+0000 7faf68c98640 1 --2- 172.21.15.37:0/2140050129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf6410c170 0x7faf6410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.004+0000 7faf68c98640 1 --2- 172.21.15.37:0/2140050129 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf64103660 0x7faf6410bb50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.004+0000 7faf68c98640 1 --2- 172.21.15.37:0/2140050129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf64102d40 0x7faf64103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.004+0000 7faf68c98640 1 -- 172.21.15.37:0/2140050129 >> 172.21.15.37:0/2140050129 conn(0x7faf640fc7a0 msgr2=0x7faf640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.004+0000 7faf68c98640 1 -- 172.21.15.37:0/2140050129 shutdown_connections 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.004+0000 7faf68c98640 1 -- 172.21.15.37:0/2140050129 wait complete. 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 Processor -- start 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 -- start start 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf64102d40 0x7faf641984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64103660 0x7faf64198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6410c170 0x7faf6419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111f70 con 0x7faf6410c170 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf64111df0 con 0x7faf64102d40 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf68c98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf641120f0 con 0x7faf64103660 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6410c170 0x7faf6419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6410c170 0x7faf6419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48008/0 (socket says 172.21.15.37:48008) 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 -- 172.21.15.37:0/4085560992 learned_addr learned my addr 172.21.15.37:0/4085560992 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf61d9b640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64103660 0x7faf64198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf6259c640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf64102d40 0x7faf641984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 -- 172.21.15.37:0/4085560992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64103660 msgr2=0x7faf64198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf64103660 0x7faf64198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 -- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf64102d40 msgr2=0x7faf641984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf64102d40 0x7faf641984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.005+0000 7faf62d9d640 1 -- 172.21.15.37:0/4085560992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf6419d490 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.006+0000 7faf62d9d640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6410c170 0x7faf6419cdb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00cad0 tx=0x7faf5c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.006+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00ae20 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.006+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf6419d780 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.006+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf5c005020 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.006+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c00f6b0 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.006+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf641a5060 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.007+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7faf64062900 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.008+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7faf5c0051d0 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.009+0000 7faf537fe640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faf4c0a32d0 0x7faf4c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.009+0000 7faf6259c640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faf4c0a32d0 0x7faf4c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.009+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7faf5c0cfdc0 con 0x7faf6410c170 2024-09-19T06:40:35.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.009+0000 7faf537fe640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] conn(0x7faf4c0acb20 0x7faf4c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.009+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 --> [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faf5c023e50 con 0x7faf4c0acb20 2024-09-19T06:40:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.010+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7faf5c0d0170 con 0x7faf6410c170 2024-09-19T06:40:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.010+0000 7faf6259c640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faf4c0a32d0 0x7faf4c0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf5400bbe0 tx=0x7faf5400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.010+0000 7faf61d9b640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] conn(0x7faf4c0acb20 0x7faf4c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.010+0000 7faf61d9b640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] conn(0x7faf4c0acb20 0x7faf4c0aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.011+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== osd.10 v2:172.21.15.96:6824/2313306979 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faf5c023e50 con 0x7faf4c0acb20 2024-09-19T06:40:35.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.031+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 --> [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7faf64062d80 con 0x7faf4c0acb20 2024-09-19T06:40:35.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.031+0000 7faf537fe640 1 -- 172.21.15.37:0/4085560992 <== osd.10 v2:172.21.15.96:6824/2313306979 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7faf64062d80 con 0x7faf4c0acb20 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] conn(0x7faf4c0acb20 msgr2=0x7faf4c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] conn(0x7faf4c0acb20 0x7faf4c0aef40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faf4c0a32d0 msgr2=0x7faf4c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7faf4c0a32d0 0x7faf4c0a5790 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7faf5400bbe0 tx=0x7faf5400b390 comp rx=0 tx=0).stop 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6410c170 msgr2=0x7faf6419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 --2- 172.21.15.37:0/4085560992 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf6410c170 0x7faf6419cdb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7faf5c00cad0 tx=0x7faf5c00cf90 comp rx=0 tx=0).stop 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf62d9d640 1 -- 172.21.15.37:0/4085560992 reap_dead start 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 shutdown_connections 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 >> 172.21.15.37:0/4085560992 conn(0x7faf640fc7a0 msgr2=0x7faf640fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:35.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 shutdown_connections 2024-09-19T06:40:35.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.032+0000 7faf68c98640 1 -- 172.21.15.37:0/4085560992 wait complete. 2024-09-19T06:40:35.756 INFO:teuthology.orchestra.run.smithi037.stdout:322122547221 2024-09-19T06:40:35.756 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 -- 172.21.15.37:0/1787565548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c102d30 msgr2=0x7f756c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 --2- 172.21.15.37:0/1787565548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c102d30 0x7f756c103110 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f755c009930 tx=0x7f755c02a420 comp rx=0 tx=0).stop 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 -- 172.21.15.37:0/1787565548 shutdown_connections 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 --2- 172.21.15.37:0/1787565548 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c1082a0 0x7f756c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 --2- 172.21.15.37:0/1787565548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c1036e0 0x7f756c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 --2- 172.21.15.37:0/1787565548 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c102d30 0x7f756c103110 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 -- 172.21.15.37:0/1787565548 >> 172.21.15.37:0/1787565548 conn(0x7f756c0fc7d0 msgr2=0x7f756c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 -- 172.21.15.37:0/1787565548 shutdown_connections 2024-09-19T06:40:35.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.901+0000 7f7571121640 1 -- 172.21.15.37:0/1787565548 wait complete. 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.902+0000 7f7571121640 1 Processor -- start 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.902+0000 7f7571121640 1 -- start start 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f7571121640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c102d30 0x7f756c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f7571121640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c1036e0 0x7f756c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f7571121640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c1082a0 0x7f756c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f7571121640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1120e0 con 0x7f756c1082a0 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f7571121640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f756c111f60 con 0x7f756c102d30 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f7571121640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f756c112260 con 0x7f756c1036e0 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c1082a0 0x7f756c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c1082a0 0x7f756c19cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48022/0 (socket says 172.21.15.37:48022) 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 -- 172.21.15.37:0/612522277 learned_addr learned my addr 172.21.15.37:0/612522277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c102d30 0x7f756c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756a59c640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c1036e0 0x7f756c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 -- 172.21.15.37:0/612522277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c1036e0 msgr2=0x7f756c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c1036e0 0x7f756c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 -- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c102d30 msgr2=0x7f756c198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c102d30 0x7f756c198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 -- 172.21.15.37:0/612522277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c19d4d0 con 0x7f756c1082a0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756a59c640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c1036e0 0x7f756c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756ad9d640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c102d30 0x7f756c198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f756b59e640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c1082a0 0x7f756c19cdf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f756400cce0 tx=0x7f756400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.903+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400f450 con 0x7f756c1082a0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.904+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c19d7c0 con 0x7f756c1082a0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.904+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756400fad0 con 0x7f756c1082a0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.904+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564018c10 con 0x7f756c1082a0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.904+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c1a50a0 con 0x7f756c1082a0 2024-09-19T06:40:35.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.905+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f756c107710 con 0x7f756c1082a0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.906+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f756400f5f0 con 0x7f756c1082a0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f7553fff640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75540a32d0 0x7f75540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f756ad9d640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75540a32d0 0x7f75540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f75640d02b0 con 0x7f756c1082a0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f7553fff640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] conn(0x7f75540acb20 0x7f75540aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 --> [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7564015e70 con 0x7f75540acb20 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f75640d0660 con 0x7f756c1082a0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f756ad9d640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75540a32d0 0x7f75540a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f755c006010 tx=0x7f755c031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.907+0000 7f756a59c640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] conn(0x7f75540acb20 0x7f75540aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.908+0000 7f756a59c640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] conn(0x7f75540acb20 0x7f75540aef40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:35.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.909+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== osd.7 v2:172.21.15.96:6816/4105644959 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7564015e70 con 0x7f75540acb20 2024-09-19T06:40:35.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.923+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 --> [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f756c06a850 con 0x7f75540acb20 2024-09-19T06:40:35.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.923+0000 7f7553fff640 1 -- 172.21.15.37:0/612522277 <== osd.7 v2:172.21.15.96:6816/4105644959 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f756c06a850 con 0x7f75540acb20 2024-09-19T06:40:35.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] conn(0x7f75540acb20 msgr2=0x7f75540aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] conn(0x7f75540acb20 0x7f75540aef40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75540a32d0 msgr2=0x7f75540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f75540a32d0 0x7f75540a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f755c006010 tx=0x7f755c031510 comp rx=0 tx=0).stop 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c1082a0 msgr2=0x7f756c19cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 --2- 172.21.15.37:0/612522277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c1082a0 0x7f756c19cdf0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f756400cce0 tx=0x7f756400a620 comp rx=0 tx=0).stop 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f756b59e640 1 -- 172.21.15.37:0/612522277 reap_dead start 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 shutdown_connections 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 >> 172.21.15.37:0/612522277 conn(0x7f756c0fc7d0 msgr2=0x7f756c105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 shutdown_connections 2024-09-19T06:40:35.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:35.924+0000 7f7571121640 1 -- 172.21.15.37:0/612522277 wait complete. 2024-09-19T06:40:36.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:35 smithi096 ceph-mon[35004]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:40:36.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:35 smithi037 ceph-mon[31211]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:40:36.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:35 smithi151 ceph-mon[35236]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:40:36.756 INFO:teuthology.orchestra.run.smithi037.stdout:210453397546 2024-09-19T06:40:36.757 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-19T06:40:37.342 INFO:teuthology.orchestra.run.smithi037.stdout:107374182462 2024-09-19T06:40:37.343 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-19T06:40:37.554 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:37.559 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:38.023 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:37 smithi037 ceph-mon[31211]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:40:38.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:37 smithi096 ceph-mon[35004]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:40:38.025 INFO:teuthology.orchestra.run.smithi037.stdout:292057776154 2024-09-19T06:40:38.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-19T06:40:38.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:37 smithi151 ceph-mon[35236]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:40:38.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 -- 172.21.15.37:0/1280348189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f4810e4f0 msgr2=0x7f9f48110900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:38.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 --2- 172.21.15.37:0/1280348189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f4810e4f0 0x7f9f48110900 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9f40009ca0 tx=0x7f9f4002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 -- 172.21.15.37:0/1280348189 shutdown_connections 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 --2- 172.21.15.37:0/1280348189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f4810e4f0 0x7f9f48110900 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 --2- 172.21.15.37:0/1280348189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f48101380 0x7f9f4810dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 --2- 172.21.15.37:0/1280348189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f481009d0 0x7f9f48100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 -- 172.21.15.37:0/1280348189 >> 172.21.15.37:0/1280348189 conn(0x7f9f480fc7d0 msgr2=0x7f9f480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 -- 172.21.15.37:0/1280348189 shutdown_connections 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.472+0000 7f9f4d482640 1 -- 172.21.15.37:0/1280348189 wait complete. 2024-09-19T06:40:38.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 Processor -- start 2024-09-19T06:40:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 -- start start 2024-09-19T06:40:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f481009d0 0x7f9f481a03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f48101380 0x7f9f481a0920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4810e4f0 0x7f9f4819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9f481142b0 con 0x7f9f48101380 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48114130 con 0x7f9f481009d0 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.473+0000 7f9f4d482640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48114430 con 0x7f9f4810e4f0 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f48101380 0x7f9f481a0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f48101380 0x7f9f481a0920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48044/0 (socket says 172.21.15.37:48044) 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 -- 172.21.15.37:0/724862413 learned_addr learned my addr 172.21.15.37:0/724862413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f46ffd640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f481009d0 0x7f9f481a03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f477fe640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4810e4f0 0x7f9f4819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 -- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4810e4f0 msgr2=0x7f9f4819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:38.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4810e4f0 0x7f9f4819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 -- 172.21.15.37:0/724862413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f481009d0 msgr2=0x7f9f481a03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f481009d0 0x7f9f481a03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 -- 172.21.15.37:0/724862413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4819acd0 con 0x7f9f48101380 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f477fe640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9f4810e4f0 0x7f9f4819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f3e7fc640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f48101380 0x7f9f481a0920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d950 tx=0x7f9f3000dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f30017070 con 0x7f9f48101380 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4819afc0 con 0x7f9f48101380 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9f3000de80 con 0x7f9f48101380 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3000f620 con 0x7f9f48101380 2024-09-19T06:40:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.474+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f481a72b0 con 0x7f9f48101380 2024-09-19T06:40:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.475+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9f48062980 con 0x7f9f48101380 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.477+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9f30009c30 con 0x7f9f48101380 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.477+0000 7f9f44ff9640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9f340a32d0 0x7f9f340a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.477+0000 7f9f46ffd640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9f340a32d0 0x7f9f340a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.477+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f9f300cf9f0 con 0x7f9f48101380 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.478+0000 7f9f44ff9640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] conn(0x7f9f340acb20 0x7f9f340aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.478+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 --> [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9f340af5f0 con 0x7f9f340acb20 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.478+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f9f300cfed0 con 0x7f9f48101380 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.478+0000 7f9f46ffd640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9f340a32d0 0x7f9f340a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009990 tx=0x7f9f38009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.478+0000 7f9f477fe640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] conn(0x7f9f340acb20 0x7f9f340aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:38.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.478+0000 7f9f477fe640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] conn(0x7f9f340acb20 0x7f9f340aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:38.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.479+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== osd.2 v2:172.21.15.151:6800/1198314257 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9f340af5f0 con 0x7f9f340acb20 2024-09-19T06:40:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.493+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 --> [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9f48062d80 con 0x7f9f340acb20 2024-09-19T06:40:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.493+0000 7f9f44ff9640 1 -- 172.21.15.37:0/724862413 <== osd.2 v2:172.21.15.151:6800/1198314257 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9f48062d80 con 0x7f9f340acb20 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] conn(0x7f9f340acb20 msgr2=0x7f9f340aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] conn(0x7f9f340acb20 0x7f9f340aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9f340a32d0 msgr2=0x7f9f340a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f9f340a32d0 0x7f9f340a5790 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009990 tx=0x7f9f38009110 comp rx=0 tx=0).stop 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f48101380 msgr2=0x7f9f481a0920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 --2- 172.21.15.37:0/724862413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9f48101380 0x7f9f481a0920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9f3000d950 tx=0x7f9f3000dd50 comp rx=0 tx=0).stop 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f477fe640 1 -- 172.21.15.37:0/724862413 reap_dead start 2024-09-19T06:40:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 shutdown_connections 2024-09-19T06:40:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 >> 172.21.15.37:0/724862413 conn(0x7f9f480fc7d0 msgr2=0x7f9f480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 shutdown_connections 2024-09-19T06:40:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:38.494+0000 7f9f4d482640 1 -- 172.21.15.37:0/724862413 wait complete. 2024-09-19T06:40:39.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.084+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1089308999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 msgr2=0x7f10e010b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.084+0000 7f10e4ba0640 1 --2- 172.21.15.37:0/1089308999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 0x7f10e010b9a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009900 tx=0x7f10d402a480 comp rx=0 tx=0).stop 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1089308999 shutdown_connections 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 --2- 172.21.15.37:0/1089308999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e010bfc0 0x7f10e010e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 --2- 172.21.15.37:0/1089308999 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 0x7f10e010b9a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 --2- 172.21.15.37:0/1089308999 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e0102b90 0x7f10e0102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1089308999 >> 172.21.15.37:0/1089308999 conn(0x7f10e00fc610 msgr2=0x7f10e00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1089308999 shutdown_connections 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1089308999 wait complete. 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 Processor -- start 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.085+0000 7f10e4ba0640 1 -- start start 2024-09-19T06:40:39.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10e4ba0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e0102b90 0x7f10e0198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10e4ba0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 0x7f10e0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10e4ba0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e010bfc0 0x7f10e019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10e4ba0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0111e10 con 0x7f10e01034b0 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10e4ba0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0111c90 con 0x7f10e0102b90 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10e4ba0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10e0111f90 con 0x7f10e010bfc0 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 0x7f10e0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 0x7f10e0198870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48072/0 (socket says 172.21.15.37:48072) 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 -- 172.21.15.37:0/1820390942 learned_addr learned my addr 172.21.15.37:0/1820390942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10df7fe640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e0102b90 0x7f10e0198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10dffff640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e010bfc0 0x7f10e019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 -- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e010bfc0 msgr2=0x7f10e019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:39.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e010bfc0 0x7f10e019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 -- 172.21.15.37:0/1820390942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e0102b90 msgr2=0x7f10e0198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e0102b90 0x7f10e0198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 -- 172.21.15.37:0/1820390942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e019d2e0 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10deffd640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 0x7f10e0198870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10d4004770 tx=0x7f10d4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.086+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d403f070 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.087+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e019d570 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.087+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e01a4eb0 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.087+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10d4031ca0 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.087+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4049060 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.087+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f10cc001030 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.089+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f10d4042070 con 0x7f10e01034b0 2024-09-19T06:40:39.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.089+0000 7f10dcff9640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f10c80a32d0 0x7f10c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.090+0000 7f10df7fe640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f10c80a32d0 0x7f10c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.090+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f10d40f04a0 con 0x7f10e01034b0 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.090+0000 7f10dcff9640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] conn(0x7f10c80acb20 0x7f10c80aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.090+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 --> [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f10c80af5f0 con 0x7f10c80acb20 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.090+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f10d40f0980 con 0x7f10e01034b0 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.090+0000 7f10df7fe640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f10c80a32d0 0x7f10c80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f10d000bbe0 tx=0x7f10d000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.090+0000 7f10dffff640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] conn(0x7f10c80acb20 0x7f10c80aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.091+0000 7f10dffff640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] conn(0x7f10c80acb20 0x7f10c80aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:39.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.092+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== osd.8 v2:172.21.15.151:6816/2450296865 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f10c80af5f0 con 0x7f10c80acb20 2024-09-19T06:40:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.105+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 --> [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f10cc002dd0 con 0x7f10c80acb20 2024-09-19T06:40:39.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10dcff9640 1 -- 172.21.15.37:0/1820390942 <== osd.8 v2:172.21.15.151:6816/2450296865 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f10cc002dd0 con 0x7f10c80acb20 2024-09-19T06:40:39.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] conn(0x7f10c80acb20 msgr2=0x7f10c80aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10e4ba0640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] conn(0x7f10c80acb20 0x7f10c80aef40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f10c80a32d0 msgr2=0x7f10c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10e4ba0640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f10c80a32d0 0x7f10c80a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f10d000bbe0 tx=0x7f10d000b390 comp rx=0 tx=0).stop 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 msgr2=0x7f10e0198870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10e4ba0640 1 --2- 172.21.15.37:0/1820390942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e01034b0 0x7f10e0198870 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f10d4004770 tx=0x7f10d4003c10 comp rx=0 tx=0).stop 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.106+0000 7f10dffff640 1 -- 172.21.15.37:0/1820390942 reap_dead start 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.107+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 shutdown_connections 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.107+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 >> 172.21.15.37:0/1820390942 conn(0x7f10e00fc610 msgr2=0x7f10e00fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.107+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 shutdown_connections 2024-09-19T06:40:39.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:39.107+0000 7f10e4ba0640 1 -- 172.21.15.37:0/1820390942 wait complete. 2024-09-19T06:40:39.738 INFO:teuthology.orchestra.run.smithi037.stdout:240518168614 2024-09-19T06:40:39.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-19T06:40:39.971 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:39 smithi037 ceph-mon[31211]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:40:39.981 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:40.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:39 smithi096 ceph-mon[35004]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:40:40.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:39 smithi151 ceph-mon[35236]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:40:40.472 INFO:teuthology.orchestra.run.smithi037.stdout:77309411397 2024-09-19T06:40:40.472 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-19T06:40:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 -- 172.21.15.37:0/3677465448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70102570 msgr2=0x7f1d701029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 --2- 172.21.15.37:0/3677465448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70102570 0x7f1d701029b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1d58009900 tx=0x7f1d5802a480 comp rx=0 tx=0).stop 2024-09-19T06:40:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 -- 172.21.15.37:0/3677465448 shutdown_connections 2024-09-19T06:40:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 --2- 172.21.15.37:0/3677465448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70102ef0 0x7f1d7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 --2- 172.21.15.37:0/3677465448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70102570 0x7f1d701029b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 --2- 172.21.15.37:0/3677465448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d70100e20 0x7f1d70101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 -- 172.21.15.37:0/3677465448 >> 172.21.15.37:0/3677465448 conn(0x7f1d700fc7d0 msgr2=0x7f1d700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 -- 172.21.15.37:0/3677465448 shutdown_connections 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.630+0000 7f1d76592640 1 -- 172.21.15.37:0/3677465448 wait complete. 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 Processor -- start 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 -- start start 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70100e20 0x7f1d70199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d70102570 0x7f1d701997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70102ef0 0x7f1d701a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70110f50 con 0x7f1d70100e20 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1d70110dd0 con 0x7f1d70102ef0 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d76592640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1d701110d0 con 0x7f1d70102570 2024-09-19T06:40:40.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70100e20 0x7f1d70199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70100e20 0x7f1d70199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45296/0 (socket says 172.21.15.37:45296) 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 -- 172.21.15.37:0/875969234 learned_addr learned my addr 172.21.15.37:0/875969234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d74b2f640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70102ef0 0x7f1d701a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d67fff640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d70102570 0x7f1d701997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 -- 172.21.15.37:0/875969234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d70102570 msgr2=0x7f1d701997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1d70102570 0x7f1d701997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 -- 172.21.15.37:0/875969234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70102ef0 msgr2=0x7f1d701a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:40.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1d70102ef0 0x7f1d701a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.631+0000 7f1d6ffff640 1 -- 172.21.15.37:0/875969234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d701a4840 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.632+0000 7f1d6ffff640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70100e20 0x7f1d70199280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000c920 tx=0x7f1d6000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.632+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60007c70 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.632+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d701a4a10 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.632+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d60007e10 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.632+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d60018890 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.632+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d701a4f50 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.633+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1d70107260 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.634+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1d60018a30 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.635+0000 7f1d6dffb640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1d5c0a32d0 0x7f1d5c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.635+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f1d600cff60 con 0x7f1d70100e20 2024-09-19T06:40:40.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.635+0000 7f1d67fff640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1d5c0a32d0 0x7f1d5c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.635+0000 7f1d6dffb640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] conn(0x7f1d5c0acb20 0x7f1d5c0aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.635+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 --> [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1d5c0af5f0 con 0x7f1d5c0acb20 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.635+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_get_version_reply(handle=1 version=81) ==== 24+0+0 (secure 0 0 0) 0x7f1d600d0310 con 0x7f1d70100e20 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.635+0000 7f1d74b2f640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] conn(0x7f1d5c0acb20 0x7f1d5c0aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.636+0000 7f1d67fff640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1d5c0a32d0 0x7f1d5c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d580096c0 tx=0x7f1d58000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.636+0000 7f1d74b2f640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] conn(0x7f1d5c0acb20 0x7f1d5c0aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.636+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== osd.9 v2:172.21.15.37:6826/3305545849 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1d5c0af5f0 con 0x7f1d5c0acb20 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.650+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 --> [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1d701021b0 con 0x7f1d5c0acb20 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d6dffb640 1 -- 172.21.15.37:0/875969234 <== osd.9 v2:172.21.15.37:6826/3305545849 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1d701021b0 con 0x7f1d5c0acb20 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] conn(0x7f1d5c0acb20 msgr2=0x7f1d5c0aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] conn(0x7f1d5c0acb20 0x7f1d5c0aef40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1d5c0a32d0 msgr2=0x7f1d5c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1d5c0a32d0 0x7f1d5c0a5790 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1d580096c0 tx=0x7f1d58000bc0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70100e20 msgr2=0x7f1d70199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:40.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 --2- 172.21.15.37:0/875969234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1d70100e20 0x7f1d70199280 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1d6000c920 tx=0x7f1d6000cde0 comp rx=0 tx=0).stop 2024-09-19T06:40:40.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d74b2f640 1 -- 172.21.15.37:0/875969234 reap_dead start 2024-09-19T06:40:40.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 shutdown_connections 2024-09-19T06:40:40.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 >> 172.21.15.37:0/875969234 conn(0x7f1d700fc7d0 msgr2=0x7f1d700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:40.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.651+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 shutdown_connections 2024-09-19T06:40:40.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:40.652+0000 7f1d76592640 1 -- 172.21.15.37:0/875969234 wait complete. 2024-09-19T06:40:40.904 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:41.347 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:41.353 INFO:teuthology.orchestra.run.smithi037.stdout:270582939681 2024-09-19T06:40:41.353 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-19T06:40:41.378 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:41.484 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:41.844 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:41 smithi037 ceph-mon[31211]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:40:42.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:41 smithi096 ceph-mon[35004]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:40:42.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:41 smithi151 ceph-mon[35236]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:40:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.448+0000 7fcb09848640 1 -- 172.21.15.37:0/211998379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 msgr2=0x7fcb0410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.448+0000 7fcb09848640 1 --2- 172.21.15.37:0/211998379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb0410d910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009ca0 tx=0x7fcafc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 -- 172.21.15.37:0/211998379 shutdown_connections 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 --2- 172.21.15.37:0/211998379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb0410d910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 --2- 172.21.15.37:0/211998379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb04101240 0x7fcb04101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 --2- 172.21.15.37:0/211998379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb04105a90 0x7fcb04105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 -- 172.21.15.37:0/211998379 >> 172.21.15.37:0/211998379 conn(0x7fcb04077210 msgr2=0x7fcb040fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 -- 172.21.15.37:0/211998379 shutdown_connections 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 -- 172.21.15.37:0/211998379 wait complete. 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 Processor -- start 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.449+0000 7fcb09848640 1 -- start start 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb09848640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb04101240 0x7fcb04198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb09848640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb041989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb09848640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb04105a90 0x7fcb0419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb09848640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcb04110f70 con 0x7fcb04101bc0 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb09848640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcb04110df0 con 0x7fcb04101240 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb09848640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcb041110f0 con 0x7fcb04105a90 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb041989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb041989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45318/0 (socket says 172.21.15.37:45318) 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 -- 172.21.15.37:0/4108279135 learned_addr learned my addr 172.21.15.37:0/4108279135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb02ffd640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb04101240 0x7fcb04198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 -- 172.21.15.37:0/4108279135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb04105a90 msgr2=0x7fcb0419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb04105a90 0x7fcb0419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 -- 172.21.15.37:0/4108279135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb04101240 msgr2=0x7fcb04198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb04101240 0x7fcb04198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 -- 172.21.15.37:0/4108279135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb0419d410 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb02ffd640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb04101240 0x7fcb04198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb027fc640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb041989a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00d950 tx=0x7fcaec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb08846640 1 -- 172.21.15.37:0/4108279135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec017070 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb08846640 1 -- 172.21.15.37:0/4108279135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcaec009c30 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb0419d700 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.450+0000 7fcb08846640 1 -- 172.21.15.37:0/4108279135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec00f6b0 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.451+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb041a4fe0 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.452+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb04101680 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.453+0000 7fcb08846640 1 -- 172.21.15.37:0/4108279135 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcaec0109d0 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.454+0000 7fcb08846640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcaf00a32d0 0x7fcaf00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.454+0000 7fcb02ffd640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcaf00a32d0 0x7fcaf00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.454+0000 7fcb08846640 1 -- 172.21.15.37:0/4108279135 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fcaec0cf6e0 con 0x7fcb04101bc0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.455+0000 7fcb02ffd640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcaf00a32d0 0x7fcaf00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009b40 tx=0x7fcaf4005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:42.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.458+0000 7fcb08846640 1 -- 172.21.15.37:0/4108279135 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcaec009de0 con 0x7fcb04101bc0 2024-09-19T06:40:42.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.635+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fcb04109fc0 con 0x7fcb04101bc0 2024-09-19T06:40:42.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.636+0000 7fcb08846640 1 -- 172.21.15.37:0/4108279135 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcaec09c350 con 0x7fcb04101bc0 2024-09-19T06:40:42.638 INFO:teuthology.orchestra.run.smithi037.stdout:137438953531 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcaf00a32d0 msgr2=0x7fcaf00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcaf00a32d0 0x7fcaf00a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4009b40 tx=0x7fcaf4005e50 comp rx=0 tx=0).stop 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 msgr2=0x7fcb041989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb041989a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00d950 tx=0x7fcaec00de10 comp rx=0 tx=0).stop 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 shutdown_connections 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcaf00a32d0 0x7fcaf00a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcb04105a90 0x7fcb0419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcb04101bc0 0x7fcb041989a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 --2- 172.21.15.37:0/4108279135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcb04101240 0x7fcb04198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 >> 172.21.15.37:0/4108279135 conn(0x7fcb04077210 msgr2=0x7fcb04100350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:42.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 shutdown_connections 2024-09-19T06:40:42.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:42.638+0000 7fcb09848640 1 -- 172.21.15.37:0/4108279135 wait complete. 2024-09-19T06:40:43.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:42 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4108279135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:40:43.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:42 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4108279135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:40:43.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:42 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4108279135' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-19T06:40:43.264 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953529 got 137438953531 for osd.4 2024-09-19T06:40:43.264 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:43.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 -- 172.21.15.37:0/1727002332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102790 msgr2=0x7fc1a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 --2- 172.21.15.37:0/1727002332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102790 0x7fc1a41067e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc194009ca0 tx=0x7fc19402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 -- 172.21.15.37:0/1727002332 shutdown_connections 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 --2- 172.21.15.37:0/1727002332 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4106f10 0x7fc1a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 --2- 172.21.15.37:0/1727002332 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102790 0x7fc1a41067e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 --2- 172.21.15.37:0/1727002332 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4101de0 0x7fc1a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 -- 172.21.15.37:0/1727002332 >> 172.21.15.37:0/1727002332 conn(0x7fc1a40fd8c0 msgr2=0x7fc1a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 -- 172.21.15.37:0/1727002332 shutdown_connections 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.427+0000 7fc1ab224640 1 -- 172.21.15.37:0/1727002332 wait complete. 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 Processor -- start 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 -- start start 2024-09-19T06:40:43.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4101de0 0x7fc1a41984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4102790 0x7fc1a41989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4106f10 0x7fc1a419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4111f70 con 0x7fc1a4106f10 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4111df0 con 0x7fc1a4101de0 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1ab224640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41120f0 con 0x7fc1a4102790 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1a97c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4106f10 0x7fc1a419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1a97c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4106f10 0x7fc1a419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45348/0 (socket says 172.21.15.37:45348) 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1a97c1640 1 -- 172.21.15.37:0/2557170276 learned_addr learned my addr 172.21.15.37:0/2557170276 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1a8fc0640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4101de0 0x7fc1a41984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.428+0000 7fc1a3fff640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4102790 0x7fc1a41989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a97c1640 1 -- 172.21.15.37:0/2557170276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4102790 msgr2=0x7fc1a41989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a97c1640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4102790 0x7fc1a41989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a97c1640 1 -- 172.21.15.37:0/2557170276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4101de0 msgr2=0x7fc1a41984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a97c1640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4101de0 0x7fc1a41984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a97c1640 1 -- 172.21.15.37:0/2557170276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a419d460 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a97c1640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4106f10 0x7fc1a419cd80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00d750 tx=0x7fc19c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a1ffb640 1 -- 172.21.15.37:0/2557170276 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c017070 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a1ffb640 1 -- 172.21.15.37:0/2557170276 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc19c005500 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a419d750 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1a1ffb640 1 -- 172.21.15.37:0/2557170276 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc19c004d90 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.429+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a41a5030 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.430+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a41021c0 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.431+0000 7fc1a1ffb640 1 -- 172.21.15.37:0/2557170276 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc19c004f30 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.432+0000 7fc1a1ffb640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc1900a3350 0x7fc1900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.432+0000 7fc1a8fc0640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc1900a3350 0x7fc1900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.432+0000 7fc1a1ffb640 1 -- 172.21.15.37:0/2557170276 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fc19c0cf2b0 con 0x7fc1a4106f10 2024-09-19T06:40:43.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.433+0000 7fc1a8fc0640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc1900a3350 0x7fc1900a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005fd0 tx=0x7fc18c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:43.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.436+0000 7fc1a1ffb640 1 -- 172.21.15.37:0/2557170276 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc19c09c020 con 0x7fc1a4106f10 2024-09-19T06:40:43.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.617+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fc1a410a1b0 con 0x7fc1a4106f10 2024-09-19T06:40:43.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.617+0000 7fc1a1ffb640 1 -- 172.21.15.37:0/2557170276 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc19c09b920 con 0x7fc1a4106f10 2024-09-19T06:40:43.619 INFO:teuthology.orchestra.run.smithi037.stdout:240518168615 2024-09-19T06:40:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc1900a3350 msgr2=0x7fc1900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc1900a3350 0x7fc1900a5810 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc18c005fd0 tx=0x7fc18c005850 comp rx=0 tx=0).stop 2024-09-19T06:40:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4106f10 msgr2=0x7fc1a419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4106f10 0x7fc1a419cd80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc19c00d750 tx=0x7fc19c00dc10 comp rx=0 tx=0).stop 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 shutdown_connections 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc1900a3350 0x7fc1900a5810 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4106f10 0x7fc1a419cd80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4102790 0x7fc1a41989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 --2- 172.21.15.37:0/2557170276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4101de0 0x7fc1a41984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.619+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 >> 172.21.15.37:0/2557170276 conn(0x7fc1a40fd8c0 msgr2=0x7fc1a40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.620+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 shutdown_connections 2024-09-19T06:40:43.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.620+0000 7fc1ab224640 1 -- 172.21.15.37:0/2557170276 wait complete. 2024-09-19T06:40:43.876 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:43 smithi037 ceph-mon[31211]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:43.876 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:43 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2557170276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 -- 172.21.15.37:0/214678311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748106f10 msgr2=0x7f374810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 --2- 172.21.15.37:0/214678311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748106f10 0x7f374810ea90 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f3740009ca0 tx=0x7f374002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 -- 172.21.15.37:0/214678311 shutdown_connections 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 --2- 172.21.15.37:0/214678311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748106f10 0x7f374810ea90 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 --2- 172.21.15.37:0/214678311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3748102790 0x7f37481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 --2- 172.21.15.37:0/214678311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3748101de0 0x7f37481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 -- 172.21.15.37:0/214678311 >> 172.21.15.37:0/214678311 conn(0x7f37480fd8c0 msgr2=0x7f37480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:43.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 -- 172.21.15.37:0/214678311 shutdown_connections 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.874+0000 7f374e8dc640 1 -- 172.21.15.37:0/214678311 wait complete. 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 Processor -- start 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 -- start start 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748101de0 0x7f3748198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3748102790 0x7f37481989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3748106f10 0x7f374819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3748111f90 con 0x7f3748101de0 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3748111e10 con 0x7f3748106f10 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f374e8dc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3748112110 con 0x7f3748102790 2024-09-19T06:40:43.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f3747fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748101de0 0x7f3748198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f3747fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748101de0 0x7f3748198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45354/0 (socket says 172.21.15.37:45354) 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.875+0000 7f3747fff640 1 -- 172.21.15.37:0/1279546405 learned_addr learned my addr 172.21.15.37:0/1279546405 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f37477fe640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3748102790 0x7f37481989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f374ce79640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3748106f10 0x7f374819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f3747fff640 1 -- 172.21.15.37:0/1279546405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3748102790 msgr2=0x7f37481989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f3747fff640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3748102790 0x7f37481989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f3747fff640 1 -- 172.21.15.37:0/1279546405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3748106f10 msgr2=0x7f374819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f3747fff640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3748106f10 0x7f374819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:43.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f3747fff640 1 -- 172.21.15.37:0/1279546405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f374819d420 con 0x7f3748101de0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f374ce79640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3748106f10 0x7f374819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f3747fff640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748101de0 0x7f3748198470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f373800d950 tx=0x7f373800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f37457fa640 1 -- 172.21.15.37:0/1279546405 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3738017070 con 0x7f3748101de0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f37457fa640 1 -- 172.21.15.37:0/1279546405 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3738009c30 con 0x7f3748101de0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f374819d710 con 0x7f3748101de0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f37457fa640 1 -- 172.21.15.37:0/1279546405 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f373800f6b0 con 0x7f3748101de0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.876+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37481a4ff0 con 0x7f3748101de0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.877+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37481021c0 con 0x7f3748101de0 2024-09-19T06:40:43.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.879+0000 7f37457fa640 1 -- 172.21.15.37:0/1279546405 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f37380109d0 con 0x7f3748101de0 2024-09-19T06:40:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.879+0000 7f37457fa640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f37300a32d0 0x7f37300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.879+0000 7f37477fe640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f37300a32d0 0x7f37300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.879+0000 7f37457fa640 1 -- 172.21.15.37:0/1279546405 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f37380cf5f0 con 0x7f3748101de0 2024-09-19T06:40:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.880+0000 7f37477fe640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f37300a32d0 0x7f37300a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37480fdce0 tx=0x7f373c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:43.905 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:43.884+0000 7f37457fa640 1 -- 172.21.15.37:0/1279546405 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3738009de0 con 0x7f3748101de0 2024-09-19T06:40:44.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.062+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f3748109b60 con 0x7f3748101de0 2024-09-19T06:40:44.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.063+0000 7f37457fa640 1 -- 172.21.15.37:0/1279546405 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f373809c360 con 0x7f3748101de0 2024-09-19T06:40:44.064 INFO:teuthology.orchestra.run.smithi037.stdout:188978561073 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f37300a32d0 msgr2=0x7f37300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f37300a32d0 0x7f37300a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f37480fdce0 tx=0x7f373c008040 comp rx=0 tx=0).stop 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748101de0 msgr2=0x7f3748198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748101de0 0x7f3748198470 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f373800d950 tx=0x7f373800de10 comp rx=0 tx=0).stop 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 shutdown_connections 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f37300a32d0 0x7f37300a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3748106f10 0x7f374819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3748102790 0x7f37481989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 --2- 172.21.15.37:0/1279546405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3748101de0 0x7f3748198470 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 >> 172.21.15.37:0/1279546405 conn(0x7f37480fd8c0 msgr2=0x7f37480fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 shutdown_connections 2024-09-19T06:40:44.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.065+0000 7f374e8dc640 1 -- 172.21.15.37:0/1279546405 wait complete. 2024-09-19T06:40:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 -- 172.21.15.37:0/4062701143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 msgr2=0x7f51d410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/4062701143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d410ea90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f51cc009ca0 tx=0x7f51cc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 -- 172.21.15.37:0/4062701143 shutdown_connections 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/4062701143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d410ea90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/4062701143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51d4102790 0x7f51d41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/4062701143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51d4101de0 0x7f51d41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 -- 172.21.15.37:0/4062701143 >> 172.21.15.37:0/4062701143 conn(0x7f51d40fd8c0 msgr2=0x7f51d40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 -- 172.21.15.37:0/4062701143 shutdown_connections 2024-09-19T06:40:44.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.088+0000 7f51d8a5f640 1 -- 172.21.15.37:0/4062701143 wait complete. 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 Processor -- start 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 -- start start 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51d4101de0 0x7f51d41983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51d4102790 0x7f51d4198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111f70 con 0x7f51d4106f10 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51d4111df0 con 0x7f51d4102790 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d8a5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f51d41120f0 con 0x7f51d4101de0 2024-09-19T06:40:44.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45360/0 (socket says 172.21.15.37:45360) 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d2d9d640 1 -- 172.21.15.37:0/1347404506 learned_addr learned my addr 172.21.15.37:0/1347404506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d1d9b640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51d4102790 0x7f51d4198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.089+0000 7f51d259c640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51d4101de0 0x7f51d41983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d2d9d640 1 -- 172.21.15.37:0/1347404506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51d4101de0 msgr2=0x7f51d41983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d2d9d640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51d4101de0 0x7f51d41983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d2d9d640 1 -- 172.21.15.37:0/1347404506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51d4102790 msgr2=0x7f51d4198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d2d9d640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51d4102790 0x7f51d4198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d2d9d640 1 -- 172.21.15.37:0/1347404506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d419d3a0 con 0x7f51d4106f10 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d1d9b640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51d4102790 0x7f51d4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d2d9d640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d419ccc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51cc031a20 tx=0x7f51cc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51c37fe640 1 -- 172.21.15.37:0/1347404506 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc003b70 con 0x7f51d4106f10 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d419d630 con 0x7f51d4106f10 2024-09-19T06:40:44.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51c37fe640 1 -- 172.21.15.37:0/1347404506 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f51cc003d10 con 0x7f51d4106f10 2024-09-19T06:40:44.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51c37fe640 1 -- 172.21.15.37:0/1347404506 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51cc02fba0 con 0x7f51d4106f10 2024-09-19T06:40:44.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.090+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d41a4f70 con 0x7f51d4106f10 2024-09-19T06:40:44.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.092+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51d41021c0 con 0x7f51d4106f10 2024-09-19T06:40:44.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.092+0000 7f51c37fe640 1 -- 172.21.15.37:0/1347404506 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f51cc0046c0 con 0x7f51d4106f10 2024-09-19T06:40:44.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.093+0000 7f51c37fe640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f51bc0a32d0 0x7f51bc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:44.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.093+0000 7f51d259c640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f51bc0a32d0 0x7f51bc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:44.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.093+0000 7f51c37fe640 1 -- 172.21.15.37:0/1347404506 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f51cc0f0a80 con 0x7f51d4106f10 2024-09-19T06:40:44.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.094+0000 7f51d259c640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f51bc0a32d0 0x7f51bc0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009bd0 tx=0x7f51c4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:44.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.098+0000 7f51c37fe640 1 -- 172.21.15.37:0/1347404506 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f51cc0bd990 con 0x7f51d4106f10 2024-09-19T06:40:44.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:43 smithi151 ceph-mon[35236]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:44.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:43 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2557170276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:40:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:43 smithi096 ceph-mon[35004]: pgmap v304: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:43 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2557170276' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-19T06:40:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.277+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f51d410a0f0 con 0x7f51d4106f10 2024-09-19T06:40:44.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.278+0000 7f51c37fe640 1 -- 172.21.15.37:0/1347404506 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f51cc0bd290 con 0x7f51d4106f10 2024-09-19T06:40:44.279 INFO:teuthology.orchestra.run.smithi037.stdout:158913790005 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f51bc0a32d0 msgr2=0x7f51bc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f51bc0a32d0 0x7f51bc0a5790 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009bd0 tx=0x7f51c4005f90 comp rx=0 tx=0).stop 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 msgr2=0x7f51d419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d419ccc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f51cc031a20 tx=0x7f51cc031a50 comp rx=0 tx=0).stop 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 shutdown_connections 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f51bc0a32d0 0x7f51bc0a5790 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f51d4106f10 0x7f51d419ccc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51d4102790 0x7f51d4198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 --2- 172.21.15.37:0/1347404506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f51d4101de0 0x7f51d41983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 >> 172.21.15.37:0/1347404506 conn(0x7f51d40fd8c0 msgr2=0x7f51d40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:44.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 shutdown_connections 2024-09-19T06:40:44.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:44.280+0000 7f51d8a5f640 1 -- 172.21.15.37:0/1347404506 wait complete. 2024-09-19T06:40:44.675 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561071 got 188978561073 for osd.6 2024-09-19T06:40:44.675 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:45.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:44 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1279546405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:40:45.005 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:44 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1347404506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:40:45.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:44 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1279546405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:40:45.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:44 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1347404506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:40:45.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:44 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1279546405' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-19T06:40:45.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:44 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1347404506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-19T06:40:45.287 INFO:tasks.cephadm.ceph_manager.ceph:need seq 240518168614 got 240518168615 for osd.8 2024-09-19T06:40:45.287 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.494+0000 7f3e201ee640 1 -- 172.21.15.37:0/3909204351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 msgr2=0x7f3e1810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.494+0000 7f3e201ee640 1 --2- 172.21.15.37:0/3909204351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e1810bb50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009900 tx=0x7f3e0802a480 comp rx=0 tx=0).stop 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.494+0000 7f3e201ee640 1 -- 172.21.15.37:0/3909204351 shutdown_connections 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.494+0000 7f3e201ee640 1 --2- 172.21.15.37:0/3909204351 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e1810c170 0x7f3e1810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.494+0000 7f3e201ee640 1 --2- 172.21.15.37:0/3909204351 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e1810bb50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.494+0000 7f3e201ee640 1 --2- 172.21.15.37:0/3909204351 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e18102d40 0x7f3e18103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.494+0000 7f3e201ee640 1 -- 172.21.15.37:0/3909204351 >> 172.21.15.37:0/3909204351 conn(0x7f3e180fc7a0 msgr2=0x7f3e180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 -- 172.21.15.37:0/3909204351 shutdown_connections 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 -- 172.21.15.37:0/3909204351 wait complete. 2024-09-19T06:40:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 Processor -- start 2024-09-19T06:40:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 -- start start 2024-09-19T06:40:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e18102d40 0x7f3e181983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e18198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1810c170 0x7f3e1819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18111f70 con 0x7f3e18103660 2024-09-19T06:40:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18111df0 con 0x7f3e1810c170 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.495+0000 7f3e201ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e181120f0 con 0x7f3e18102d40 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e18198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e18198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45382/0 (socket says 172.21.15.37:45382) 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 -- 172.21.15.37:0/303137663 learned_addr learned my addr 172.21.15.37:0/303137663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1df8a640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e18102d40 0x7f3e181983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1e78b640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1810c170 0x7f3e1819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 -- 172.21.15.37:0/303137663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e18102d40 msgr2=0x7f3e181983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e18102d40 0x7f3e181983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 -- 172.21.15.37:0/303137663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1810c170 msgr2=0x7f3e1819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1810c170 0x7f3e1819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 -- 172.21.15.37:0/303137663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1819d3a0 con 0x7f3e18103660 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1df8a640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e18102d40 0x7f3e181983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1e78b640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1810c170 0x7f3e1819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e1d789640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e18198930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005ab0 tx=0x7f3e080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e0effd640 1 -- 172.21.15.37:0/303137663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e0803f030 con 0x7f3e18103660 2024-09-19T06:40:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e0effd640 1 -- 172.21.15.37:0/303137663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e08003ce0 con 0x7f3e18103660 2024-09-19T06:40:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.496+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1819d630 con 0x7f3e18103660 2024-09-19T06:40:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.497+0000 7f3e0effd640 1 -- 172.21.15.37:0/303137663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08035070 con 0x7f3e18103660 2024-09-19T06:40:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.497+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e181a4f70 con 0x7f3e18103660 2024-09-19T06:40:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.498+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e1806b070 con 0x7f3e18103660 2024-09-19T06:40:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.499+0000 7f3e0effd640 1 -- 172.21.15.37:0/303137663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3e08042030 con 0x7f3e18103660 2024-09-19T06:40:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.499+0000 7f3e0effd640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e040a3350 0x7f3e040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.500+0000 7f3e1df8a640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e040a3350 0x7f3e040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.500+0000 7f3e0effd640 1 -- 172.21.15.37:0/303137663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f3e080f0530 con 0x7f3e18103660 2024-09-19T06:40:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.500+0000 7f3e1df8a640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e040a3350 0x7f3e040a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400abe0 tx=0x7f3e1400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.503+0000 7f3e0effd640 1 -- 172.21.15.37:0/303137663 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3e080bd1a0 con 0x7f3e18103660 2024-09-19T06:40:45.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.680+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f3e18103120 con 0x7f3e18103660 2024-09-19T06:40:45.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.680+0000 7f3e0effd640 1 -- 172.21.15.37:0/303137663 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f3e18103120 con 0x7f3e18103660 2024-09-19T06:40:45.682 INFO:teuthology.orchestra.run.smithi037.stdout:292057776156 2024-09-19T06:40:45.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e040a3350 msgr2=0x7f3e040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e040a3350 0x7f3e040a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3e1400abe0 tx=0x7f3e1400a5c0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 msgr2=0x7f3e18198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:45.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e18198930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005ab0 tx=0x7f3e080042f0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 shutdown_connections 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3e040a3350 0x7f3e040a5810 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e1810c170 0x7f3e1819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e18103660 0x7f3e18198930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 --2- 172.21.15.37:0/303137663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e18102d40 0x7f3e181983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 >> 172.21.15.37:0/303137663 conn(0x7f3e180fc7a0 msgr2=0x7f3e180fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 shutdown_connections 2024-09-19T06:40:45.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:45.682+0000 7f3e201ee640 1 -- 172.21.15.37:0/303137663 wait complete. 2024-09-19T06:40:46.069 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:45 smithi037 ceph-mon[31211]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-19T06:40:46.070 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:45 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/303137663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:40:46.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:45 smithi151 ceph-mon[35236]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-19T06:40:46.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:45 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/303137663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:40:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:45 smithi096 ceph-mon[35004]: pgmap v305: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-19T06:40:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:45 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/303137663' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-19T06:40:46.460 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776154 got 292057776156 for osd.10 2024-09-19T06:40:46.460 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 -- 172.21.15.37:0/1959015858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 msgr2=0x7fa68010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 --2- 172.21.15.37:0/1959015858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68010ea90 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa678009ca0 tx=0x7fa67802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 -- 172.21.15.37:0/1959015858 shutdown_connections 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 --2- 172.21.15.37:0/1959015858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68010ea90 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 --2- 172.21.15.37:0/1959015858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa680102790 0x7fa6801067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 --2- 172.21.15.37:0/1959015858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa680101de0 0x7fa6801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 -- 172.21.15.37:0/1959015858 >> 172.21.15.37:0/1959015858 conn(0x7fa6800fd8c0 msgr2=0x7fa6800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 -- 172.21.15.37:0/1959015858 shutdown_connections 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.874+0000 7fa6853ee640 1 -- 172.21.15.37:0/1959015858 wait complete. 2024-09-19T06:40:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.875+0000 7fa6853ee640 1 Processor -- start 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.875+0000 7fa6853ee640 1 -- start start 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa6853ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa680101de0 0x7fa680198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa6853ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa680102790 0x7fa6801988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa6853ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68019cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa6853ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa680111f70 con 0x7fa680106f10 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa6853ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa680111df0 con 0x7fa680101de0 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa6853ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6801120f0 con 0x7fa680102790 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68019cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68019cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45406/0 (socket says 172.21.15.37:45406) 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 -- 172.21.15.37:0/2394419953 learned_addr learned my addr 172.21.15.37:0/2394419953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67effd640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa680101de0 0x7fa680198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67e7fc640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa680102790 0x7fa6801988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 -- 172.21.15.37:0/2394419953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa680102790 msgr2=0x7fa6801988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa680102790 0x7fa6801988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 -- 172.21.15.37:0/2394419953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa680101de0 msgr2=0x7fa680198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa680101de0 0x7fa680198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 -- 172.21.15.37:0/2394419953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa68019d340 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa67f7fe640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68019cc60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa678031a20 tx=0x7fa678031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.876+0000 7fa667fff640 1 -- 172.21.15.37:0/2394419953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678003b70 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.877+0000 7fa667fff640 1 -- 172.21.15.37:0/2394419953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa678003d10 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.877+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa68019d5d0 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.877+0000 7fa667fff640 1 -- 172.21.15.37:0/2394419953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa67802fc40 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.877+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6801a4f10 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.878+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6801021c0 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.879+0000 7fa667fff640 1 -- 172.21.15.37:0/2394419953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa678004b40 con 0x7fa680106f10 2024-09-19T06:40:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.879+0000 7fa667fff640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6680a32d0 0x7fa6680a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.879+0000 7fa67effd640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6680a32d0 0x7fa6680a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.880+0000 7fa667fff640 1 -- 172.21.15.37:0/2394419953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fa6780f09d0 con 0x7fa680106f10 2024-09-19T06:40:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.880+0000 7fa67effd640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6680a32d0 0x7fa6680a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa670009bd0 tx=0x7fa670005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:46.883+0000 7fa667fff640 1 -- 172.21.15.37:0/2394419953 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa6780bd8e0 con 0x7fa680106f10 2024-09-19T06:40:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:46.940 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:46.941 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:47.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:46 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.093+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa6801996f0 con 0x7fa680106f10 2024-09-19T06:40:47.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.094+0000 7fa667fff640 1 -- 172.21.15.37:0/2394419953 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa6801996f0 con 0x7fa680106f10 2024-09-19T06:40:47.096 INFO:teuthology.orchestra.run.smithi037.stdout:107374182464 2024-09-19T06:40:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6680a32d0 msgr2=0x7fa6680a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6680a32d0 0x7fa6680a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa670009bd0 tx=0x7fa670005f90 comp rx=0 tx=0).stop 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 msgr2=0x7fa68019cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68019cc60 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa678031a20 tx=0x7fa678031a50 comp rx=0 tx=0).stop 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 shutdown_connections 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6680a32d0 0x7fa6680a5790 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa680106f10 0x7fa68019cc60 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa680102790 0x7fa6801988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 --2- 172.21.15.37:0/2394419953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa680101de0 0x7fa680198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 >> 172.21.15.37:0/2394419953 conn(0x7fa6800fd8c0 msgr2=0x7fa6800fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 shutdown_connections 2024-09-19T06:40:47.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.096+0000 7fa6853ee640 1 -- 172.21.15.37:0/2394419953 wait complete. 2024-09-19T06:40:47.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:46 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:47.210 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:46 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:40:47.340 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790003 got 158913790005 for osd.5 2024-09-19T06:40:47.341 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:47.885 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:47 smithi037 ceph-mon[31211]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:47.885 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:47 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2394419953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:40:47.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 -- 172.21.15.37:0/323987768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90106f20 msgr2=0x7ffa9010eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/323987768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90106f20 0x7ffa9010eaa0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009900 tx=0x7ffa8802a410 comp rx=0 tx=0).stop 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 -- 172.21.15.37:0/323987768 shutdown_connections 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/323987768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90106f20 0x7ffa9010eaa0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/323987768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa901027a0 0x7ffa901067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/323987768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa90101df0 0x7ffa901021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 -- 172.21.15.37:0/323987768 >> 172.21.15.37:0/323987768 conn(0x7ffa900fd8b0 msgr2=0x7ffa900ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 -- 172.21.15.37:0/323987768 shutdown_connections 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 -- 172.21.15.37:0/323987768 wait complete. 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.883+0000 7ffa95a5f640 1 Processor -- start 2024-09-19T06:40:47.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa95a5f640 1 -- start start 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa95a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90101df0 0x7ffa900744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa95a5f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa901027a0 0x7ffa90074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa95a5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa90106f20 0x7ffa90074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa95a5f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90111f80 con 0x7ffa90101df0 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa95a5f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90111e00 con 0x7ffa901027a0 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa95a5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa90112100 con 0x7ffa90106f20 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90101df0 0x7ffa900744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90101df0 0x7ffa900744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45428/0 (socket says 172.21.15.37:45428) 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 -- 172.21.15.37:0/104932638 learned_addr learned my addr 172.21.15.37:0/104932638 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8e7fc640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa901027a0 0x7ffa90074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8f7fe640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa90106f20 0x7ffa90074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 -- 172.21.15.37:0/104932638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa90106f20 msgr2=0x7ffa90074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa90106f20 0x7ffa90074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 -- 172.21.15.37:0/104932638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa901027a0 msgr2=0x7ffa90074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa901027a0 0x7ffa90074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.884+0000 7ffa8effd640 1 -- 172.21.15.37:0/104932638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa901a68f0 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.885+0000 7ffa8effd640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90101df0 0x7ffa900744f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000ebc0 tx=0x7ffa8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.885+0000 7ffa94a5d640 1 -- 172.21.15.37:0/104932638 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80010630 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.885+0000 7ffa94a5d640 1 -- 172.21.15.37:0/104932638 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa80010cb0 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.885+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa901a6be0 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.885+0000 7ffa94a5d640 1 -- 172.21.15.37:0/104932638 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa80018bd0 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.885+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa901a7120 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.886+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa901021d0 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.887+0000 7ffa94a5d640 1 -- 172.21.15.37:0/104932638 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ffa800107d0 con 0x7ffa90101df0 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.888+0000 7ffa94a5d640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffa7c0a32d0 0x7ffa7c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:47.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.888+0000 7ffa94a5d640 1 -- 172.21.15.37:0/104932638 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7ffa800cffc0 con 0x7ffa90101df0 2024-09-19T06:40:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.888+0000 7ffa8e7fc640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffa7c0a32d0 0x7ffa7c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.888+0000 7ffa8e7fc640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffa7c0a32d0 0x7ffa7c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa90075f80 tx=0x7ffa7800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:47.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:47.892+0000 7ffa94a5d640 1 -- 172.21.15.37:0/104932638 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ffa8009cd30 con 0x7ffa90101df0 2024-09-19T06:40:48.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.089+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7ffa9002d7c0 con 0x7ffa90101df0 2024-09-19T06:40:48.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.089+0000 7ffa94a5d640 1 -- 172.21.15.37:0/104932638 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa8009c630 con 0x7ffa90101df0 2024-09-19T06:40:48.091 INFO:teuthology.orchestra.run.smithi037.stdout:210453397548 2024-09-19T06:40:48.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffa7c0a32d0 msgr2=0x7ffa7c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:48.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffa7c0a32d0 0x7ffa7c0a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ffa90075f80 tx=0x7ffa7800b390 comp rx=0 tx=0).stop 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90101df0 msgr2=0x7ffa900744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90101df0 0x7ffa900744f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa8000ebc0 tx=0x7ffa8000c6a0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 shutdown_connections 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ffa7c0a32d0 0x7ffa7c0a5790 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa90106f20 0x7ffa90074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa901027a0 0x7ffa90074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 --2- 172.21.15.37:0/104932638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa90101df0 0x7ffa900744f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.091+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 >> 172.21.15.37:0/104932638 conn(0x7ffa900fd8b0 msgr2=0x7ffa900feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.092+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 shutdown_connections 2024-09-19T06:40:48.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:48.092+0000 7ffa95a5f640 1 -- 172.21.15.37:0/104932638 wait complete. 2024-09-19T06:40:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:47 smithi151 ceph-mon[35236]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:47 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2394419953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:40:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:47 smithi096 ceph-mon[35004]: pgmap v306: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:47 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2394419953' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-19T06:40:48.668 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:48.694 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182462 got 107374182464 for osd.3 2024-09-19T06:40:48.694 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:49.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1498427867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f81008d0 msgr2=0x7f27f8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1498427867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f81008d0 0x7f27f8100cb0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009900 tx=0x7f27f402a410 comp rx=0 tx=0).stop 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1498427867 shutdown_connections 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1498427867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27f810e480 0x7f27f8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1498427867 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27f8101280 0x7f27f810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1498427867 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f81008d0 0x7f27f8100cb0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1498427867 >> 172.21.15.37:0/1498427867 conn(0x7f27f80fc7a0 msgr2=0x7f27f80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1498427867 shutdown_connections 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.068+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1498427867 wait complete. 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 Processor -- start 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 -- start start 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27f81008d0 0x7f27f81a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27f8101280 0x7f27f81a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f810e480 0x7f27f819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27f8114260 con 0x7f27f810e480 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81140e0 con 0x7f27f8101280 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27ffb7d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27f81143e0 con 0x7f27f81008d0 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27fe11a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f810e480 0x7f27f819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.069+0000 7f27fe11a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f810e480 0x7f27f819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45458/0 (socket says 172.21.15.37:45458) 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fe11a640 1 -- 172.21.15.37:0/1448473801 learned_addr learned my addr 172.21.15.37:0/1448473801 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fd118640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27f8101280 0x7f27f81a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fd919640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27f81008d0 0x7f27f81a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fe11a640 1 -- 172.21.15.37:0/1448473801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27f81008d0 msgr2=0x7f27f81a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fe11a640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27f81008d0 0x7f27f81a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fe11a640 1 -- 172.21.15.37:0/1448473801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27f8101280 msgr2=0x7f27f81a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fe11a640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27f8101280 0x7f27f81a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fe11a640 1 -- 172.21.15.37:0/1448473801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f819aec0 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27fe11a640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f810e480 0x7f27f819a650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f27f000b9e0 tx=0x7f27f000bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27eeffd640 1 -- 172.21.15.37:0/1448473801 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f0019070 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27eeffd640 1 -- 172.21.15.37:0/1448473801 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27f0014440 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27eeffd640 1 -- 172.21.15.37:0/1448473801 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27f00125a0 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f819b1b0 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.070+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f81a7500 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.071+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f806b070 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.073+0000 7f27eeffd640 1 -- 172.21.15.37:0/1448473801 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f27f001c070 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.073+0000 7f27eeffd640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f27e40a3350 0x7f27e40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.073+0000 7f27fd919640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f27e40a3350 0x7f27e40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.073+0000 7f27eeffd640 1 -- 172.21.15.37:0/1448473801 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f27f00cf1f0 con 0x7f27f810e480 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.074+0000 7f27fd919640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f27e40a3350 0x7f27e40a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f27f4000c00 tx=0x7f27f40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:49.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.077+0000 7f27eeffd640 1 -- 172.21.15.37:0/1448473801 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f27f009ba20 con 0x7f27f810e480 2024-09-19T06:40:49.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.257+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f27f8109610 con 0x7f27f810e480 2024-09-19T06:40:49.259 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:49 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.260 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:49 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/104932638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:40:49.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.258+0000 7f27eeffd640 1 -- 172.21.15.37:0/1448473801 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f27f009fd20 con 0x7f27f810e480 2024-09-19T06:40:49.260 INFO:teuthology.orchestra.run.smithi037.stdout:322122547225 2024-09-19T06:40:49.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f27e40a3350 msgr2=0x7f27e40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f27e40a3350 0x7f27e40a5810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f27f4000c00 tx=0x7f27f40058d0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f810e480 msgr2=0x7f27f819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f810e480 0x7f27f819a650 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f27f000b9e0 tx=0x7f27f000bde0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 shutdown_connections 2024-09-19T06:40:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f27e40a3350 0x7f27e40a5810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27f810e480 0x7f27f819a650 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27f8101280 0x7f27f81a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 --2- 172.21.15.37:0/1448473801 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27f81008d0 0x7f27f81a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 >> 172.21.15.37:0/1448473801 conn(0x7f27f80fc7a0 msgr2=0x7f27f80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 shutdown_connections 2024-09-19T06:40:49.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.260+0000 7f27ffb7d640 1 -- 172.21.15.37:0/1448473801 wait complete. 2024-09-19T06:40:49.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 -- 172.21.15.37:0/948859412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c102ef0 msgr2=0x7f470c10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 --2- 172.21.15.37:0/948859412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c102ef0 0x7f470c10da40 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4704009ca0 tx=0x7f470402a950 comp rx=0 tx=0).stop 2024-09-19T06:40:49.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 -- 172.21.15.37:0/948859412 shutdown_connections 2024-09-19T06:40:49.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 --2- 172.21.15.37:0/948859412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c102ef0 0x7f470c10da40 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 --2- 172.21.15.37:0/948859412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c102570 0x7f470c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 --2- 172.21.15.37:0/948859412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f470c100e20 0x7f470c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 -- 172.21.15.37:0/948859412 >> 172.21.15.37:0/948859412 conn(0x7f470c0fc7d0 msgr2=0x7f470c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:49.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 -- 172.21.15.37:0/948859412 shutdown_connections 2024-09-19T06:40:49.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 -- 172.21.15.37:0/948859412 wait complete. 2024-09-19T06:40:49.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.282+0000 7f4714656640 1 Processor -- start 2024-09-19T06:40:49.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4714656640 1 -- start start 2024-09-19T06:40:49.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4714656640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c100e20 0x7f470c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4714656640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c102570 0x7f470c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4714656640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f470c102ef0 0x7f470c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4714656640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f470c110f80 con 0x7f470c100e20 2024-09-19T06:40:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c100e20 0x7f470c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c100e20 0x7f470c198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45470/0 (socket says 172.21.15.37:45470) 2024-09-19T06:40:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 -- 172.21.15.37:0/635876176 learned_addr learned my addr 172.21.15.37:0/635876176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f470c110e00 con 0x7f470c102570 2024-09-19T06:40:49.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f470c111100 con 0x7f470c102ef0 2024-09-19T06:40:49.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4711bf1640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c102570 0x7f470c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f4712bf3640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f470c102ef0 0x7f470c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 -- 172.21.15.37:0/635876176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f470c102ef0 msgr2=0x7f470c19cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f470c102ef0 0x7f470c19cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 -- 172.21.15.37:0/635876176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c102570 msgr2=0x7f470c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c102570 0x7f470c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.283+0000 7f47123f2640 1 -- 172.21.15.37:0/635876176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470c19d420 con 0x7f470c100e20 2024-09-19T06:40:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.284+0000 7f47123f2640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c100e20 0x7f470c198470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00c890 tx=0x7f46fc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.284+0000 7f470b7fe640 1 -- 172.21.15.37:0/635876176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc004220 con 0x7f470c100e20 2024-09-19T06:40:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.284+0000 7f470b7fe640 1 -- 172.21.15.37:0/635876176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46fc0043c0 con 0x7f470c100e20 2024-09-19T06:40:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.284+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f470c19d710 con 0x7f470c100e20 2024-09-19T06:40:49.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.284+0000 7f470b7fe640 1 -- 172.21.15.37:0/635876176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46fc0049e0 con 0x7f470c100e20 2024-09-19T06:40:49.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.284+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f470c1a4ff0 con 0x7f470c100e20 2024-09-19T06:40:49.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.285+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f470c101200 con 0x7f470c100e20 2024-09-19T06:40:49.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.286+0000 7f470b7fe640 1 -- 172.21.15.37:0/635876176 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f46fc0079b0 con 0x7f470c100e20 2024-09-19T06:40:49.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.287+0000 7f470b7fe640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f40a3350 0x7f46f40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:49.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.288+0000 7f4711bf1640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f40a3350 0x7f46f40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:49.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.288+0000 7f470b7fe640 1 -- 172.21.15.37:0/635876176 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f46fc0cfa00 con 0x7f470c100e20 2024-09-19T06:40:49.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.288+0000 7f4711bf1640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f40a3350 0x7f46f40a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f470c1051c0 tx=0x7f4700005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:49.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.291+0000 7f470b7fe640 1 -- 172.21.15.37:0/635876176 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f46fc014030 con 0x7f470c100e20 2024-09-19T06:40:49.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:49 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:49 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/104932638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:40:49.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.470+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f470c10a0a0 con 0x7f470c100e20 2024-09-19T06:40:49.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.470+0000 7f470b7fe640 1 -- 172.21.15.37:0/635876176 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f46fc09c6f0 con 0x7f470c100e20 2024-09-19T06:40:49.472 INFO:teuthology.orchestra.run.smithi037.stdout:34359738448 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f40a3350 msgr2=0x7f46f40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f40a3350 0x7f46f40a5810 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f470c1051c0 tx=0x7f4700005f90 comp rx=0 tx=0).stop 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c100e20 msgr2=0x7f470c198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c100e20 0x7f470c198470 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f46fc00c890 tx=0x7f46fc00cd50 comp rx=0 tx=0).stop 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 shutdown_connections 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f46f40a3350 0x7f46f40a5810 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f470c102ef0 0x7f470c19cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f470c102570 0x7f470c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 --2- 172.21.15.37:0/635876176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f470c100e20 0x7f470c198470 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:49.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 >> 172.21.15.37:0/635876176 conn(0x7f470c0fc7d0 msgr2=0x7f470c06a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:49.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 shutdown_connections 2024-09-19T06:40:49.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:49.472+0000 7f4714656640 1 -- 172.21.15.37:0/635876176 wait complete. 2024-09-19T06:40:49.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:49 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:49.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:49 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/104932638' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-19T06:40:49.917 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547221 got 322122547225 for osd.11 2024-09-19T06:40:49.917 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:50.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:50 smithi037 ceph-mon[31211]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:40:50.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:50 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1448473801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:40:50.142 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:50 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/635876176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:40:50.338 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:50 smithi151 ceph-mon[35236]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:40:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:50 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1448473801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:40:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:50 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/635876176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:40:50.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 -- 172.21.15.37:0/1640219237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e41036e0 msgr2=0x7fe5e4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/1640219237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e41036e0 0x7fe5e4107b90 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009930 tx=0x7fe5d802a420 comp rx=0 tx=0).stop 2024-09-19T06:40:50.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 -- 172.21.15.37:0/1640219237 shutdown_connections 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/1640219237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5e41082a0 0x7fe5e410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/1640219237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e41036e0 0x7fe5e4107b90 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/1640219237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5e4102d30 0x7fe5e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 -- 172.21.15.37:0/1640219237 >> 172.21.15.37:0/1640219237 conn(0x7fe5e40fc7d0 msgr2=0x7fe5e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 -- 172.21.15.37:0/1640219237 shutdown_connections 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 -- 172.21.15.37:0/1640219237 wait complete. 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.516+0000 7fe5e98b3640 1 Processor -- start 2024-09-19T06:40:50.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e98b3640 1 -- start start 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e98b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4102d30 0x7fe5e4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e98b3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5e41036e0 0x7fe5e4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e98b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5e41082a0 0x7fe5e4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e98b3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e41120a0 con 0x7fe5e4102d30 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e98b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4111f20 con 0x7fe5e41082a0 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e98b3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5e4112220 con 0x7fe5e41036e0 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4102d30 0x7fe5e4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:50.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4102d30 0x7fe5e4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44800/0 (socket says 172.21.15.37:44800) 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e2ffd640 1 -- 172.21.15.37:0/3719909880 learned_addr learned my addr 172.21.15.37:0/3719909880 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e37fe640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5e41082a0 0x7fe5e4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.517+0000 7fe5e27fc640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5e41036e0 0x7fe5e4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e37fe640 1 -- 172.21.15.37:0/3719909880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5e41036e0 msgr2=0x7fe5e4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e37fe640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5e41036e0 0x7fe5e4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e37fe640 1 -- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4102d30 msgr2=0x7fe5e4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e37fe640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4102d30 0x7fe5e4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e37fe640 1 -- 172.21.15.37:0/3719909880 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5e41a6aa0 con 0x7fe5e41082a0 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e2ffd640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4102d30 0x7fe5e4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e37fe640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5e41082a0 0x7fe5e4101410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00ca30 tx=0x7fe5dc00cef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:50.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e88b1640 1 -- 172.21.15.37:0/3719909880 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc007d90 con 0x7fe5e41082a0 2024-09-19T06:40:50.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5e41a6d90 con 0x7fe5e41082a0 2024-09-19T06:40:50.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e88b1640 1 -- 172.21.15.37:0/3719909880 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5dc005020 con 0x7fe5e41082a0 2024-09-19T06:40:50.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e88b1640 1 -- 172.21.15.37:0/3719909880 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5dc00f780 con 0x7fe5e41082a0 2024-09-19T06:40:50.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.518+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5e41a7250 con 0x7fe5e41082a0 2024-09-19T06:40:50.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.519+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5e4103110 con 0x7fe5e41082a0 2024-09-19T06:40:50.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.521+0000 7fe5e88b1640 1 -- 172.21.15.37:0/3719909880 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe5dc0051d0 con 0x7fe5e41082a0 2024-09-19T06:40:50.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.522+0000 7fe5e88b1640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe5cc0a32d0 0x7fe5cc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:50.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.522+0000 7fe5e2ffd640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe5cc0a32d0 0x7fe5cc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:50.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.522+0000 7fe5e88b1640 1 -- 172.21.15.37:0/3719909880 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fe5dc0cfb30 con 0x7fe5e41082a0 2024-09-19T06:40:50.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.522+0000 7fe5e2ffd640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe5cc0a32d0 0x7fe5cc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40058c0 tx=0x7fe5d4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:50.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:50 smithi096 ceph-mon[35004]: pgmap v307: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:40:50.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:50 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1448473801' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-19T06:40:50.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:50 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/635876176' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-19T06:40:50.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.525+0000 7fe5e88b1640 1 -- 172.21.15.37:0/3719909880 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5dc014070 con 0x7fe5e41082a0 2024-09-19T06:40:50.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.707+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fe5e4101ec0 con 0x7fe5e41082a0 2024-09-19T06:40:50.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.708+0000 7fe5e88b1640 1 -- 172.21.15.37:0/3719909880 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe5e4101ec0 con 0x7fe5e41082a0 2024-09-19T06:40:50.710 INFO:teuthology.orchestra.run.smithi037.stdout:270582939683 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe5cc0a32d0 msgr2=0x7fe5cc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe5cc0a32d0 0x7fe5cc0a5790 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40058c0 tx=0x7fe5d4005f90 comp rx=0 tx=0).stop 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5e41082a0 msgr2=0x7fe5e4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5e41082a0 0x7fe5e4101410 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc00ca30 tx=0x7fe5dc00cef0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 shutdown_connections 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fe5cc0a32d0 0x7fe5cc0a5790 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5e41082a0 0x7fe5e4101410 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5e41036e0 0x7fe5e4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 --2- 172.21.15.37:0/3719909880 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5e4102d30 0x7fe5e4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:50.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 >> 172.21.15.37:0/3719909880 conn(0x7fe5e40fc7d0 msgr2=0x7fe5e4105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:50.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 shutdown_connections 2024-09-19T06:40:50.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:50.710+0000 7fe5e98b3640 1 -- 172.21.15.37:0/3719909880 wait complete. 2024-09-19T06:40:51.148 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738445 got 34359738448 for osd.0 2024-09-19T06:40:51.148 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:51.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:51 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3719909880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:40:51.472 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:51 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3719909880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:40:51.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:51 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3719909880' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-19T06:40:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 -- 172.21.15.37:0/730095833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10100e20 msgr2=0x7f7b10101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 --2- 172.21.15.37:0/730095833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10100e20 0x7f7b10101200 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009900 tx=0x7f7b0002a480 comp rx=0 tx=0).stop 2024-09-19T06:40:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 -- 172.21.15.37:0/730095833 shutdown_connections 2024-09-19T06:40:52.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 --2- 172.21.15.37:0/730095833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b10102ef0 0x7f7b1010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 --2- 172.21.15.37:0/730095833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b10102570 0x7f7b101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 --2- 172.21.15.37:0/730095833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10100e20 0x7f7b10101200 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 -- 172.21.15.37:0/730095833 >> 172.21.15.37:0/730095833 conn(0x7f7b100fc7d0 msgr2=0x7f7b100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 -- 172.21.15.37:0/730095833 shutdown_connections 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.097+0000 7f7b150b9640 1 -- 172.21.15.37:0/730095833 wait complete. 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.098+0000 7f7b150b9640 1 Processor -- start 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.098+0000 7f7b150b9640 1 -- start start 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b150b9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b10100e20 0x7f7b100fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b150b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b10102570 0x7f7b100fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b150b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10102ef0 0x7f7b100fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b150b9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10110f80 con 0x7f7b10102ef0 2024-09-19T06:40:52.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b150b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10110e00 con 0x7f7b10102570 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b150b9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10111100 con 0x7f7b10100e20 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10102ef0 0x7f7b100fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10102ef0 0x7f7b100fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44812/0 (socket says 172.21.15.37:44812) 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 -- 172.21.15.37:0/3422196097 learned_addr learned my addr 172.21.15.37:0/3422196097 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0e59c640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b10102570 0x7f7b100fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0ed9d640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b10100e20 0x7f7b100fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 -- 172.21.15.37:0/3422196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b10100e20 msgr2=0x7f7b100fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b10100e20 0x7f7b100fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 -- 172.21.15.37:0/3422196097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b10102570 msgr2=0x7f7b100fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b10102570 0x7f7b100fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 -- 172.21.15.37:0/3422196097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b101a69c0 con 0x7f7b10102ef0 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b0f59e640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10102ef0 0x7f7b100fd700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800eaa0 tx=0x7f7b0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:52.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7af7fff640 1 -- 172.21.15.37:0/3422196097 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b0800cdb0 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7af7fff640 1 -- 172.21.15.37:0/3422196097 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b08005020 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b101a6cb0 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.099+0000 7f7af7fff640 1 -- 172.21.15.37:0/3422196097 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b080116b0 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.100+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b101a71f0 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.101+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b10101200 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.102+0000 7f7af7fff640 1 -- 172.21.15.37:0/3422196097 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7b08005560 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.102+0000 7f7af7fff640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7af80a32d0 0x7f7af80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.103+0000 7f7b0ed9d640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7af80a32d0 0x7f7af80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.103+0000 7f7af7fff640 1 -- 172.21.15.37:0/3422196097 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7f7b080cfa20 con 0x7f7b10102ef0 2024-09-19T06:40:52.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.103+0000 7f7b0ed9d640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7af80a32d0 0x7f7af80a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009230 tx=0x7f7b000058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:52.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.107+0000 7f7af7fff640 1 -- 172.21.15.37:0/3422196097 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b08016030 con 0x7f7b10102ef0 2024-09-19T06:40:52.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.284+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7b1010a0a0 con 0x7f7b10102ef0 2024-09-19T06:40:52.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.285+0000 7f7af7fff640 1 -- 172.21.15.37:0/3422196097 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7b0809c140 con 0x7f7b10102ef0 2024-09-19T06:40:52.287 INFO:teuthology.orchestra.run.smithi037.stdout:77309411399 2024-09-19T06:40:52.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7af80a32d0 msgr2=0x7f7af80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7af80a32d0 0x7f7af80a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009230 tx=0x7f7b000058d0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10102ef0 msgr2=0x7f7b100fd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10102ef0 0x7f7b100fd700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800eaa0 tx=0x7f7b0800ef60 comp rx=0 tx=0).stop 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 shutdown_connections 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7af80a32d0 0x7f7af80a5790 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b10102ef0 0x7f7b100fd700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b10102570 0x7f7b100fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 --2- 172.21.15.37:0/3422196097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b10100e20 0x7f7b100fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:52.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 >> 172.21.15.37:0/3422196097 conn(0x7f7b100fc7d0 msgr2=0x7f7b1006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 shutdown_connections 2024-09-19T06:40:52.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:52.287+0000 7f7b150b9640 1 -- 172.21.15.37:0/3422196097 wait complete. 2024-09-19T06:40:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:52 smithi151 ceph-mon[35236]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:40:52.410 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:52 smithi037 ceph-mon[31211]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:40:52.438 INFO:tasks.cephadm.ceph_manager.ceph:need seq 270582939681 got 270582939683 for osd.9 2024-09-19T06:40:52.438 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:52.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:52 smithi096 ceph-mon[35004]: pgmap v308: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:40:53.095 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397546 got 210453397548 for osd.7 2024-09-19T06:40:53.095 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 -- 172.21.15.37:0/959540938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018100e20 msgr2=0x7fc018101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 --2- 172.21.15.37:0/959540938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018100e20 0x7fc018101200 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc008009900 tx=0x7fc00802a480 comp rx=0 tx=0).stop 2024-09-19T06:40:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 -- 172.21.15.37:0/959540938 shutdown_connections 2024-09-19T06:40:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 --2- 172.21.15.37:0/959540938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc018102ef0 0x7fc01810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 --2- 172.21.15.37:0/959540938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc018102570 0x7fc0181029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 --2- 172.21.15.37:0/959540938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018100e20 0x7fc018101200 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 -- 172.21.15.37:0/959540938 >> 172.21.15.37:0/959540938 conn(0x7fc0180fc7d0 msgr2=0x7fc0180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 -- 172.21.15.37:0/959540938 shutdown_connections 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.261+0000 7fc01e284640 1 -- 172.21.15.37:0/959540938 wait complete. 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 Processor -- start 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 -- start start 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc018100e20 0x7fc018198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc018102570 0x7fc018198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018102ef0 0x7fc01819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0181120a0 con 0x7fc018102ef0 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc018111f20 con 0x7fc018102570 2024-09-19T06:40:53.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01e284640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc018112220 con 0x7fc018100e20 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018102ef0 0x7fc01819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018102ef0 0x7fc01819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44830/0 (socket says 172.21.15.37:44830) 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 -- 172.21.15.37:0/1849432428 learned_addr learned my addr 172.21.15.37:0/1849432428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc00ffff640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc018102570 0x7fc018198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc017fff640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc018100e20 0x7fc018198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 -- 172.21.15.37:0/1849432428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc018100e20 msgr2=0x7fc018198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc018100e20 0x7fc018198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 -- 172.21.15.37:0/1849432428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc018102570 msgr2=0x7fc018198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc018102570 0x7fc018198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 -- 172.21.15.37:0/1849432428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc01819d4f0 con 0x7fc018102ef0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.262+0000 7fc01c821640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018102ef0 0x7fc01819ce10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc01000b970 tx=0x7fc01000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.263+0000 7fc015ffb640 1 -- 172.21.15.37:0/1849432428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010019070 con 0x7fc018102ef0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.263+0000 7fc015ffb640 1 -- 172.21.15.37:0/1849432428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc01000cb40 con 0x7fc018102ef0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.263+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc01819d7e0 con 0x7fc018102ef0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.263+0000 7fc015ffb640 1 -- 172.21.15.37:0/1849432428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc010007650 con 0x7fc018102ef0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.263+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0181a50c0 con 0x7fc018102ef0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.264+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc018101200 con 0x7fc018102ef0 2024-09-19T06:40:53.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.265+0000 7fc015ffb640 1 -- 172.21.15.37:0/1849432428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc01001c070 con 0x7fc018102ef0 2024-09-19T06:40:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.266+0000 7fc015ffb640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc0040a3350 0x7fc0040a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.266+0000 7fc015ffb640 1 -- 172.21.15.37:0/1849432428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fc0100cf1d0 con 0x7fc018102ef0 2024-09-19T06:40:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.266+0000 7fc017fff640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc0040a3350 0x7fc0040a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:53.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.267+0000 7fc017fff640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc0040a3350 0x7fc0040a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc008005ab0 tx=0x7fc008005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:53.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.269+0000 7fc015ffb640 1 -- 172.21.15.37:0/1849432428 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc01009ba00 con 0x7fc018102ef0 2024-09-19T06:40:53.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:53 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3422196097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:40:53.449 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:53 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3422196097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:40:53.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.447+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc018109090 con 0x7fc018102ef0 2024-09-19T06:40:53.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.447+0000 7fc015ffb640 1 -- 172.21.15.37:0/1849432428 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc01009fd00 con 0x7fc018102ef0 2024-09-19T06:40:53.450 INFO:teuthology.orchestra.run.smithi037.stdout:55834574924 2024-09-19T06:40:53.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.449+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc0040a3350 msgr2=0x7fc0040a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:53.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.449+0000 7fc01e284640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc0040a3350 0x7fc0040a5810 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc008005ab0 tx=0x7fc008005a20 comp rx=0 tx=0).stop 2024-09-19T06:40:53.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.449+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018102ef0 msgr2=0x7fc01819ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.449+0000 7fc01e284640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018102ef0 0x7fc01819ce10 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc01000b970 tx=0x7fc01000be30 comp rx=0 tx=0).stop 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 shutdown_connections 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc0040a3350 0x7fc0040a5810 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc018102ef0 0x7fc01819ce10 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc018102570 0x7fc018198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 --2- 172.21.15.37:0/1849432428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc018100e20 0x7fc018198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 >> 172.21.15.37:0/1849432428 conn(0x7fc0180fc7d0 msgr2=0x7fc0180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 shutdown_connections 2024-09-19T06:40:53.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:53.450+0000 7fc01e284640 1 -- 172.21.15.37:0/1849432428 wait complete. 2024-09-19T06:40:53.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:53 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3422196097' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-19T06:40:53.726 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411397 got 77309411399 for osd.2 2024-09-19T06:40:53.726 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:54.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:54 smithi037 ceph-mon[31211]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:54.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:54 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1849432428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:40:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:54 smithi151 ceph-mon[35236]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:54 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1849432428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:40:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:54 smithi096 ceph-mon[35004]: pgmap v309: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:54 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1849432428' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-19T06:40:54.964 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574920 got 55834574924 for osd.1 2024-09-19T06:40:54.964 DEBUG:teuthology.parallel:result is None 2024-09-19T06:40:54.965 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:40:54.965 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:40:55.248 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 -- 172.21.15.37:0/3882604161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102ef0 msgr2=0x7fc62010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 --2- 172.21.15.37:0/3882604161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102ef0 0x7fc62010da40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc618009ca0 tx=0x7fc61802a950 comp rx=0 tx=0).stop 2024-09-19T06:40:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 -- 172.21.15.37:0/3882604161 shutdown_connections 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 --2- 172.21.15.37:0/3882604161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102ef0 0x7fc62010da40 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 --2- 172.21.15.37:0/3882604161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc620102570 0x7fc6201029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 --2- 172.21.15.37:0/3882604161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc620100e20 0x7fc620101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 -- 172.21.15.37:0/3882604161 >> 172.21.15.37:0/3882604161 conn(0x7fc6200fc7d0 msgr2=0x7fc6200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 -- 172.21.15.37:0/3882604161 shutdown_connections 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 -- 172.21.15.37:0/3882604161 wait complete. 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.701+0000 7fc62699c640 1 Processor -- start 2024-09-19T06:40:55.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc62699c640 1 -- start start 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc62699c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc620100e20 0x7fc620198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc62699c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102570 0x7fc620198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc62699c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc620102ef0 0x7fc62019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc62699c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6201120f0 con 0x7fc620102570 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc62699c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc620111f70 con 0x7fc620100e20 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc62699c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc620112270 con 0x7fc620102ef0 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102570 0x7fc620198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:55.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102570 0x7fc620198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44844/0 (socket says 172.21.15.37:44844) 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 -- 172.21.15.37:0/1490572336 learned_addr learned my addr 172.21.15.37:0/1490572336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc624f39640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc620102ef0 0x7fc62019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61ffff640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc620100e20 0x7fc620198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 -- 172.21.15.37:0/1490572336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc620102ef0 msgr2=0x7fc62019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc620102ef0 0x7fc62019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 -- 172.21.15.37:0/1490572336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc620100e20 msgr2=0x7fc620198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc620100e20 0x7fc620198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc61f7fe640 1 -- 172.21.15.37:0/1490572336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc62019d4e0 con 0x7fc620102570 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.702+0000 7fc624f39640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc620102ef0 0x7fc62019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.703+0000 7fc61f7fe640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102570 0x7fc620198a70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc61400d950 tx=0x7fc61400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.703+0000 7fc61d7fa640 1 -- 172.21.15.37:0/1490572336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc614017070 con 0x7fc620102570 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.703+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc62019d7d0 con 0x7fc620102570 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.703+0000 7fc61d7fa640 1 -- 172.21.15.37:0/1490572336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc614009c30 con 0x7fc620102570 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.703+0000 7fc61d7fa640 1 -- 172.21.15.37:0/1490572336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc61400f6b0 con 0x7fc620102570 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.703+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6201a50b0 con 0x7fc620102570 2024-09-19T06:40:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.704+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc620101200 con 0x7fc620102570 2024-09-19T06:40:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.705+0000 7fc61d7fa640 1 -- 172.21.15.37:0/1490572336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc6140109d0 con 0x7fc620102570 2024-09-19T06:40:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.706+0000 7fc61d7fa640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc6080a32d0 0x7fc6080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.706+0000 7fc61ffff640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc6080a32d0 0x7fc6080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.706+0000 7fc61d7fa640 1 -- 172.21.15.37:0/1490572336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7979+0+0 (secure 0 0 0) 0x7fc6140cf610 con 0x7fc620102570 2024-09-19T06:40:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.706+0000 7fc61ffff640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc6080a32d0 0x7fc6080a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc610009890 tx=0x7fc610008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.710+0000 7fc61d7fa640 1 -- 172.21.15.37:0/1490572336 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc614009de0 con 0x7fc620102570 2024-09-19T06:40:55.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.892+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc620106f40 con 0x7fc6080a32d0 2024-09-19T06:40:55.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.898+0000 7fc61d7fa640 1 -- 172.21.15.37:0/1490572336 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7fc620106f40 con 0x7fc6080a32d0 2024-09-19T06:40:55.902 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:55.903 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc6080a32d0 msgr2=0x7fc6080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc6080a32d0 0x7fc6080a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc610009890 tx=0x7fc610008040 comp rx=0 tx=0).stop 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102570 msgr2=0x7fc620198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102570 0x7fc620198a70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc61400d950 tx=0x7fc61400de10 comp rx=0 tx=0).stop 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 shutdown_connections 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc6080a32d0 0x7fc6080a5790 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc620102ef0 0x7fc62019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc620102570 0x7fc620198a70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 --2- 172.21.15.37:0/1490572336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc620100e20 0x7fc620198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 >> 172.21.15.37:0/1490572336 conn(0x7fc6200fc7d0 msgr2=0x7fc620106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 shutdown_connections 2024-09-19T06:40:55.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:55.902+0000 7fc62699c640 1 -- 172.21.15.37:0/1490572336 wait complete. 2024-09-19T06:40:56.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:56 smithi037 ceph-mon[31211]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:40:56.268 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:56.269 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:56 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:56 smithi151 ceph-mon[35236]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:40:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:56.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:56 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:56 smithi096 ceph-mon[35004]: pgmap v310: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:40:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:56 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:56.638 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-19T06:40:54.208963+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653816,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":465,"num_read_kb":349,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360000,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581824,"statfs":{"total":1151940427776,"available":1151571787776,"internally_reserved":0,"allocated":11010048,"data_stored":6993852,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005917"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403605+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:39:53.403605+0000","last_peered":"2024-09-19T06:39:53.403605+0000","last_clean":"2024-09-19T06:39:53.403605+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:39:53.403605+0000","last_undegraded":"2024-09-19T06:39:53.403605+0000","last_fullsized":"2024-09-19T06:39:53.403605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419068+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:39:53.419068+0000","last_peered":"2024-09-19T06:39:53.419068+0000","last_clean":"2024-09-19T06:39:53.419068+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:39:53.419068+0000","last_undegraded":"2024-09-19T06:39:53.419068+0000","last_fullsized":"2024-09-19T06:39:53.419068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419285+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:39:53.419285+0000","last_peered":"2024-09-19T06:39:53.419285+0000","last_clean":"2024-09-19T06:39:53.419285+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:39:53.419285+0000","last_undegraded":"2024-09-19T06:39:53.419285+0000","last_fullsized":"2024-09-19T06:39:53.419285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418707+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:39:53.418707+0000","last_peered":"2024-09-19T06:39:53.418707+0000","last_clean":"2024-09-19T06:39:53.418707+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:39:53.418707+0000","last_undegraded":"2024-09-19T06:39:53.418707+0000","last_fullsized":"2024-09-19T06:39:53.418707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400788+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:39:53.400788+0000","last_peered":"2024-09-19T06:39:53.400788+0000","last_clean":"2024-09-19T06:39:53.400788+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:39:53.400788+0000","last_undegraded":"2024-09-19T06:39:53.400788+0000","last_fullsized":"2024-09-19T06:39:53.400788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417175+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:39:53.417175+0000","last_peered":"2024-09-19T06:39:53.417175+0000","last_clean":"2024-09-19T06:39:53.417175+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:39:53.417175+0000","last_undegraded":"2024-09-19T06:39:53.417175+0000","last_fullsized":"2024-09-19T06:39:53.417175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415134+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:39:53.415134+0000","last_peered":"2024-09-19T06:39:53.415134+0000","last_clean":"2024-09-19T06:39:53.415134+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:39:53.415134+0000","last_undegraded":"2024-09-19T06:39:53.415134+0000","last_fullsized":"2024-09-19T06:39:53.415134+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414333+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:39:53.414333+0000","last_peered":"2024-09-19T06:39:53.414333+0000","last_clean":"2024-09-19T06:39:53.414333+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:39:53.414333+0000","last_undegraded":"2024-09-19T06:39:53.414333+0000","last_fullsized":"2024-09-19T06:39:53.414333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.401273+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:39:53.401273+0000","last_peered":"2024-09-19T06:39:53.401273+0000","last_clean":"2024-09-19T06:39:53.401273+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:39:53.401273+0000","last_undegraded":"2024-09-19T06:39:53.401273+0000","last_fullsized":"2024-09-19T06:39:53.401273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417564+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:39:53.417564+0000","last_peered":"2024-09-19T06:39:53.417564+0000","last_clean":"2024-09-19T06:39:53.417564+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:39:53.417564+0000","last_undegraded":"2024-09-19T06:39:53.417564+0000","last_fullsized":"2024-09-19T06:39:53.417564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415382+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:39:53.415382+0000","last_peered":"2024-09-19T06:39:53.415382+0000","last_clean":"2024-09-19T06:39:53.415382+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:39:53.415382+0000","last_undegraded":"2024-09-19T06:39:53.415382+0000","last_fullsized":"2024-09-19T06:39:53.415382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400663+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:39:53.400663+0000","last_peered":"2024-09-19T06:39:53.400663+0000","last_clean":"2024-09-19T06:39:53.400663+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:39:53.400663+0000","last_undegraded":"2024-09-19T06:39:53.400663+0000","last_fullsized":"2024-09-19T06:39:53.400663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419338+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:39:53.419338+0000","last_peered":"2024-09-19T06:39:53.419338+0000","last_clean":"2024-09-19T06:39:53.419338+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:39:53.419338+0000","last_undegraded":"2024-09-19T06:39:53.419338+0000","last_fullsized":"2024-09-19T06:39:53.419338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418306+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:39:53.418306+0000","last_peered":"2024-09-19T06:39:53.418306+0000","last_clean":"2024-09-19T06:39:53.418306+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:39:53.418306+0000","last_undegraded":"2024-09-19T06:39:53.418306+0000","last_fullsized":"2024-09-19T06:39:53.418306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413939+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:39:53.413939+0000","last_peered":"2024-09-19T06:39:53.413939+0000","last_clean":"2024-09-19T06:39:53.413939+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:39:53.413939+0000","last_undegraded":"2024-09-19T06:39:53.413939+0000","last_fullsized":"2024-09-19T06:39:53.413939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403646+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:39:53.403646+0000","last_peered":"2024-09-19T06:39:53.403646+0000","last_clean":"2024-09-19T06:39:53.403646+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:39:53.403646+0000","last_undegraded":"2024-09-19T06:39:53.403646+0000","last_fullsized":"2024-09-19T06:39:53.403646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414294+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:39:53.414294+0000","last_peered":"2024-09-19T06:39:53.414294+0000","last_clean":"2024-09-19T06:39:53.414294+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:39:53.414294+0000","last_undegraded":"2024-09-19T06:39:53.414294+0000","last_fullsized":"2024-09-19T06:39:53.414294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415375+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:39:53.415375+0000","last_peered":"2024-09-19T06:39:53.415375+0000","last_clean":"2024-09-19T06:39:53.415375+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:39:53.415375+0000","last_undegraded":"2024-09-19T06:39:53.415375+0000","last_fullsized":"2024-09-19T06:39:53.415375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403279+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:39:53.403279+0000","last_peered":"2024-09-19T06:39:53.403279+0000","last_clean":"2024-09-19T06:39:53.403279+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:39:53.403279+0000","last_undegraded":"2024-09-19T06:39:53.403279+0000","last_fullsized":"2024-09-19T06:39:53.403279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400705+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:39:53.400705+0000","last_peered":"2024-09-19T06:39:53.400705+0000","last_clean":"2024-09-19T06:39:53.400705+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:39:53.400705+0000","last_undegraded":"2024-09-19T06:39:53.400705+0000","last_fullsized":"2024-09-19T06:39:53.400705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414105+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:39:53.414105+0000","last_peered":"2024-09-19T06:39:53.414105+0000","last_clean":"2024-09-19T06:39:53.414105+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:39:53.414105+0000","last_undegraded":"2024-09-19T06:39:53.414105+0000","last_fullsized":"2024-09-19T06:39:53.414105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415846+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:39:53.415846+0000","last_peered":"2024-09-19T06:39:53.415846+0000","last_clean":"2024-09-19T06:39:53.415846+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:39:53.415846+0000","last_undegraded":"2024-09-19T06:39:53.415846+0000","last_fullsized":"2024-09-19T06:39:53.415846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414231+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:39:53.414231+0000","last_peered":"2024-09-19T06:39:53.414231+0000","last_clean":"2024-09-19T06:39:53.414231+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:39:53.414231+0000","last_undegraded":"2024-09-19T06:39:53.414231+0000","last_fullsized":"2024-09-19T06:39:53.414231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416096+0000","last_change":"2024-09-19T06:39:52.399797+0000","last_active":"2024-09-19T06:39:53.416096+0000","last_peered":"2024-09-19T06:39:53.416096+0000","last_clean":"2024-09-19T06:39:53.416096+0000","last_became_active":"2024-09-19T06:39:52.399452+0000","last_became_peered":"2024-09-19T06:39:52.399452+0000","last_unstale":"2024-09-19T06:39:53.416096+0000","last_undegraded":"2024-09-19T06:39:53.416096+0000","last_fullsized":"2024-09-19T06:39:53.416096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:33.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415925+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:39:53.415925+0000","last_peered":"2024-09-19T06:39:53.415925+0000","last_clean":"2024-09-19T06:39:53.415925+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:39:53.415925+0000","last_undegraded":"2024-09-19T06:39:53.415925+0000","last_fullsized":"2024-09-19T06:39:53.415925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403645+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:39:53.403645+0000","last_peered":"2024-09-19T06:39:53.403645+0000","last_clean":"2024-09-19T06:39:53.403645+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:39:53.403645+0000","last_undegraded":"2024-09-19T06:39:53.403645+0000","last_fullsized":"2024-09-19T06:39:53.403645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400700+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:39:53.400700+0000","last_peered":"2024-09-19T06:39:53.400700+0000","last_clean":"2024-09-19T06:39:53.400700+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:39:53.400700+0000","last_undegraded":"2024-09-19T06:39:53.400700+0000","last_fullsized":"2024-09-19T06:39:53.400700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419814+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:39:53.419814+0000","last_peered":"2024-09-19T06:39:53.419814+0000","last_clean":"2024-09-19T06:39:53.419814+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:39:53.419814+0000","last_undegraded":"2024-09-19T06:39:53.419814+0000","last_fullsized":"2024-09-19T06:39:53.419814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414873+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:39:53.414873+0000","last_peered":"2024-09-19T06:39:53.414873+0000","last_clean":"2024-09-19T06:39:53.414873+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:39:53.414873+0000","last_undegraded":"2024-09-19T06:39:53.414873+0000","last_fullsized":"2024-09-19T06:39:53.414873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414716+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:39:53.414716+0000","last_peered":"2024-09-19T06:39:53.414716+0000","last_clean":"2024-09-19T06:39:53.414716+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:39:53.414716+0000","last_undegraded":"2024-09-19T06:39:53.414716+0000","last_fullsized":"2024-09-19T06:39:53.414716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403217+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:39:53.403217+0000","last_peered":"2024-09-19T06:39:53.403217+0000","last_clean":"2024-09-19T06:39:53.403217+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:39:53.403217+0000","last_undegraded":"2024-09-19T06:39:53.403217+0000","last_fullsized":"2024-09-19T06:39:53.403217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:51.525759+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:40:51.525759+0000","last_peered":"2024-09-19T06:40:51.525759+0000","last_clean":"2024-09-19T06:40:51.525759+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:40:51.525759+0000","last_undegraded":"2024-09-19T06:40:51.525759+0000","last_fullsized":"2024-09-19T06:40:51.525759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414773+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:39:53.414773+0000","last_peered":"2024-09-19T06:39:53.414773+0000","last_clean":"2024-09-19T06:39:53.414773+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:39:53.414773+0000","last_undegraded":"2024-09-19T06:39:53.414773+0000","last_fullsized":"2024-09-19T06:39:53.414773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.446851+0000","last_change":"2024-09-19T06:39:50.318132+0000","last_active":"2024-09-19T06:40:01.446851+0000","last_peered":"2024-09-19T06:40:01.446851+0000","last_clean":"2024-09-19T06:40:01.446851+0000","last_became_active":"2024-09-19T06:39:50.317774+0000","last_became_peered":"2024-09-19T06:39:50.317774+0000","last_unstale":"2024-09-19T06:40:01.446851+0000","last_undegraded":"2024-09-19T06:40:01.446851+0000","last_fullsized":"2024-09-19T06:40:01.446851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:58.290128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.445718+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:40:01.445718+0000","last_peered":"2024-09-19T06:40:01.445718+0000","last_clean":"2024-09-19T06:40:01.445718+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:40:01.445718+0000","last_undegraded":"2024-09-19T06:40:01.445718+0000","last_fullsized":"2024-09-19T06:40:01.445718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415204+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:39:53.415204+0000","last_peered":"2024-09-19T06:39:53.415204+0000","last_clean":"2024-09-19T06:39:53.415204+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:39:53.415204+0000","last_undegraded":"2024-09-19T06:39:53.415204+0000","last_fullsized":"2024-09-19T06:39:53.415204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414736+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:39:53.414736+0000","last_peered":"2024-09-19T06:39:53.414736+0000","last_clean":"2024-09-19T06:39:53.414736+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:39:53.414736+0000","last_undegraded":"2024-09-19T06:39:53.414736+0000","last_fullsized":"2024-09-19T06:39:53.414736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413749+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:39:53.413749+0000","last_peered":"2024-09-19T06:39:53.413749+0000","last_clean":"2024-09-19T06:39:53.413749+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:39:53.413749+0000","last_undegraded":"2024-09-19T06:39:53.413749+0000","last_fullsized":"2024-09-19T06:39:53.413749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":70,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:49.522254+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:40:49.522254+0000","last_peered":"2024-09-19T06:40:49.522254+0000","last_clean":"2024-09-19T06:40:49.522254+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:40:49.522254+0000","last_undegraded":"2024-09-19T06:40:49.522254+0000","last_fullsized":"2024-09-19T06:40:49.522254+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":49,"num_read_kb":49,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414187+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:39:53.414187+0000","last_peered":"2024-09-19T06:39:53.414187+0000","last_clean":"2024-09-19T06:39:53.414187+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:39:53.414187+0000","last_undegraded":"2024-09-19T06:39:53.414187+0000","last_fullsized":"2024-09-19T06:39:53.414187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438327+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:40:00.438327+0000","last_peered":"2024-09-19T06:40:00.438327+0000","last_clean":"2024-09-19T06:40:00.438327+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:40:00.438327+0000","last_undegraded":"2024-09-19T06:40:00.438327+0000","last_fullsized":"2024-09-19T06:40:00.438327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419760+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:39:53.419760+0000","last_peered":"2024-09-19T06:39:53.419760+0000","last_clean":"2024-09-19T06:39:53.419760+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:39:53.419760+0000","last_undegraded":"2024-09-19T06:39:53.419760+0000","last_fullsized":"2024-09-19T06:39:53.419760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437339+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:40:00.437339+0000","last_peered":"2024-09-19T06:40:00.437339+0000","last_clean":"2024-09-19T06:40:00.437339+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:40:00.437339+0000","last_undegraded":"2024-09-19T06:40:00.437339+0000","last_fullsized":"2024-09-19T06:40:00.437339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414395+0000","last_change":"2024-09-19T06:39:52.391059+0000","last_active":"2024-09-19T06:39:53.414395+0000","last_peered":"2024-09-19T06:39:53.414395+0000","last_clean":"2024-09-19T06:39:53.414395+0000","last_became_active":"2024-09-19T06:39:52.389703+0000","last_became_peered":"2024-09-19T06:39:52.389703+0000","last_unstale":"2024-09-19T06:39:53.414395+0000","last_undegraded":"2024-09-19T06:39:53.414395+0000","last_fullsized":"2024-09-19T06:39:53.414395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:32.021109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'26","reported_seq":150,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:50.523754+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:40:50.523754+0000","last_peered":"2024-09-19T06:40:50.523754+0000","last_clean":"2024-09-19T06:40:50.523754+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:40:50.523754+0000","last_undegraded":"2024-09-19T06:40:50.523754+0000","last_fullsized":"2024-09-19T06:40:50.523754+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":106,"num_read_kb":110,"num_write":27,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415675+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:39:53.415675+0000","last_peered":"2024-09-19T06:39:53.415675+0000","last_clean":"2024-09-19T06:39:53.415675+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:39:53.415675+0000","last_undegraded":"2024-09-19T06:39:53.415675+0000","last_fullsized":"2024-09-19T06:39:53.415675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447875+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:40:01.447875+0000","last_peered":"2024-09-19T06:40:01.447875+0000","last_clean":"2024-09-19T06:40:01.447875+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:40:01.447875+0000","last_undegraded":"2024-09-19T06:40:01.447875+0000","last_fullsized":"2024-09-19T06:40:01.447875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437843+0000","last_change":"2024-09-19T06:39:50.312876+0000","last_active":"2024-09-19T06:40:00.437843+0000","last_peered":"2024-09-19T06:40:00.437843+0000","last_clean":"2024-09-19T06:40:00.437843+0000","last_became_active":"2024-09-19T06:39:50.312500+0000","last_became_peered":"2024-09-19T06:39:50.312500+0000","last_unstale":"2024-09-19T06:40:00.437843+0000","last_undegraded":"2024-09-19T06:40:00.437843+0000","last_fullsized":"2024-09-19T06:40:00.437843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:00.113022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417798+0000","last_change":"2024-09-19T06:39:52.399395+0000","last_active":"2024-09-19T06:39:53.417798+0000","last_peered":"2024-09-19T06:39:53.417798+0000","last_clean":"2024-09-19T06:39:53.417798+0000","last_became_active":"2024-09-19T06:39:52.399042+0000","last_became_peered":"2024-09-19T06:39:52.399042+0000","last_unstale":"2024-09-19T06:39:53.417798+0000","last_undegraded":"2024-09-19T06:39:53.417798+0000","last_fullsized":"2024-09-19T06:39:53.417798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:10.010156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:52.526527+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:40:52.526527+0000","last_peered":"2024-09-19T06:40:52.526527+0000","last_clean":"2024-09-19T06:40:52.526527+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:40:52.526527+0000","last_undegraded":"2024-09-19T06:40:52.526527+0000","last_fullsized":"2024-09-19T06:40:52.526527+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416672+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:39:53.416672+0000","last_peered":"2024-09-19T06:39:53.416672+0000","last_clean":"2024-09-19T06:39:53.416672+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:39:53.416672+0000","last_undegraded":"2024-09-19T06:39:53.416672+0000","last_fullsized":"2024-09-19T06:39:53.416672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448340+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:40:01.448340+0000","last_peered":"2024-09-19T06:40:01.448340+0000","last_clean":"2024-09-19T06:40:01.448340+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:40:01.448340+0000","last_undegraded":"2024-09-19T06:40:01.448340+0000","last_fullsized":"2024-09-19T06:40:01.448340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415210+0000","last_change":"2024-09-19T06:39:52.390919+0000","last_active":"2024-09-19T06:39:53.415210+0000","last_peered":"2024-09-19T06:39:53.415210+0000","last_clean":"2024-09-19T06:39:53.415210+0000","last_became_active":"2024-09-19T06:39:52.390585+0000","last_became_peered":"2024-09-19T06:39:52.390585+0000","last_unstale":"2024-09-19T06:39:53.415210+0000","last_undegraded":"2024-09-19T06:39:53.415210+0000","last_fullsized":"2024-09-19T06:39:53.415210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.164497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.451484+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:40:02.451484+0000","last_peered":"2024-09-19T06:40:02.451484+0000","last_clean":"2024-09-19T06:40:02.451484+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:40:02.451484+0000","last_undegraded":"2024-09-19T06:40:02.451484+0000","last_fullsized":"2024-09-19T06:40:02.451484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403462+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:39:53.403462+0000","last_peered":"2024-09-19T06:39:53.403462+0000","last_clean":"2024-09-19T06:39:53.403462+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:39:53.403462+0000","last_undegraded":"2024-09-19T06:39:53.403462+0000","last_fullsized":"2024-09-19T06:39:53.403462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418357+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:39:53.418357+0000","last_peered":"2024-09-19T06:39:53.418357+0000","last_clean":"2024-09-19T06:39:53.418357+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:39:53.418357+0000","last_undegraded":"2024-09-19T06:39:53.418357+0000","last_fullsized":"2024-09-19T06:39:53.418357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418035+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:39:53.418035+0000","last_peered":"2024-09-19T06:39:53.418035+0000","last_clean":"2024-09-19T06:39:53.418035+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:39:53.418035+0000","last_undegraded":"2024-09-19T06:39:53.418035+0000","last_fullsized":"2024-09-19T06:39:53.418035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419293+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:39:53.419293+0000","last_peered":"2024-09-19T06:39:53.419293+0000","last_clean":"2024-09-19T06:39:53.419293+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:39:53.419293+0000","last_undegraded":"2024-09-19T06:39:53.419293+0000","last_fullsized":"2024-09-19T06:39:53.419293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144684+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:39:54.144684+0000","last_peered":"2024-09-19T06:39:54.144684+0000","last_clean":"2024-09-19T06:39:54.144684+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:39:54.144684+0000","last_undegraded":"2024-09-19T06:39:54.144684+0000","last_fullsized":"2024-09-19T06:39:54.144684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447446+0000","last_change":"2024-09-19T06:39:50.507312+0000","last_active":"2024-09-19T06:40:01.447446+0000","last_peered":"2024-09-19T06:40:01.447446+0000","last_clean":"2024-09-19T06:40:01.447446+0000","last_became_active":"2024-09-19T06:39:50.506976+0000","last_became_peered":"2024-09-19T06:39:50.506976+0000","last_unstale":"2024-09-19T06:40:01.447446+0000","last_undegraded":"2024-09-19T06:40:01.447446+0000","last_fullsized":"2024-09-19T06:40:01.447446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:44.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417964+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:39:53.417964+0000","last_peered":"2024-09-19T06:39:53.417964+0000","last_clean":"2024-09-19T06:39:53.417964+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:39:53.417964+0000","last_undegraded":"2024-09-19T06:39:53.417964+0000","last_fullsized":"2024-09-19T06:39:53.417964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417272+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:39:53.417272+0000","last_peered":"2024-09-19T06:39:53.417272+0000","last_clean":"2024-09-19T06:39:53.417272+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:39:53.417272+0000","last_undegraded":"2024-09-19T06:39:53.417272+0000","last_fullsized":"2024-09-19T06:39:53.417272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.450445+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:40:02.450445+0000","last_peered":"2024-09-19T06:40:02.450445+0000","last_clean":"2024-09-19T06:40:02.450445+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:40:02.450445+0000","last_undegraded":"2024-09-19T06:40:02.450445+0000","last_fullsized":"2024-09-19T06:40:02.450445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415347+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:39:53.415347+0000","last_peered":"2024-09-19T06:39:53.415347+0000","last_clean":"2024-09-19T06:39:53.415347+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:39:53.415347+0000","last_undegraded":"2024-09-19T06:39:53.415347+0000","last_fullsized":"2024-09-19T06:39:53.415347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447538+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:40:01.447538+0000","last_peered":"2024-09-19T06:40:01.447538+0000","last_clean":"2024-09-19T06:40:01.447538+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:40:01.447538+0000","last_undegraded":"2024-09-19T06:40:01.447538+0000","last_fullsized":"2024-09-19T06:40:01.447538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400695+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:39:53.400695+0000","last_peered":"2024-09-19T06:39:53.400695+0000","last_clean":"2024-09-19T06:39:53.400695+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:39:53.400695+0000","last_undegraded":"2024-09-19T06:39:53.400695+0000","last_fullsized":"2024-09-19T06:39:53.400695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:52.526129+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:40:52.526129+0000","last_peered":"2024-09-19T06:40:52.526129+0000","last_clean":"2024-09-19T06:40:52.526129+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:40:52.526129+0000","last_undegraded":"2024-09-19T06:40:52.526129+0000","last_fullsized":"2024-09-19T06:40:52.526129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414405+0000","last_change":"2024-09-19T06:39:52.396795+0000","last_active":"2024-09-19T06:39:53.414405+0000","last_peered":"2024-09-19T06:39:53.414405+0000","last_clean":"2024-09-19T06:39:53.414405+0000","last_became_active":"2024-09-19T06:39:52.396481+0000","last_became_peered":"2024-09-19T06:39:52.396481+0000","last_unstale":"2024-09-19T06:39:53.414405+0000","last_undegraded":"2024-09-19T06:39:53.414405+0000","last_fullsized":"2024-09-19T06:39:53.414405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:05.637504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144665+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:39:54.144665+0000","last_peered":"2024-09-19T06:39:54.144665+0000","last_clean":"2024-09-19T06:39:54.144665+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:39:54.144665+0000","last_undegraded":"2024-09-19T06:39:54.144665+0000","last_fullsized":"2024-09-19T06:39:54.144665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400964+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:39:53.400964+0000","last_peered":"2024-09-19T06:39:53.400964+0000","last_clean":"2024-09-19T06:39:53.400964+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:39:53.400964+0000","last_undegraded":"2024-09-19T06:39:53.400964+0000","last_fullsized":"2024-09-19T06:39:53.400964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414089+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:39:53.414089+0000","last_peered":"2024-09-19T06:39:53.414089+0000","last_clean":"2024-09-19T06:39:53.414089+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:39:53.414089+0000","last_undegraded":"2024-09-19T06:39:53.414089+0000","last_fullsized":"2024-09-19T06:39:53.414089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:56.016828+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:39:56.016828+0000","last_peered":"2024-09-19T06:39:56.016828+0000","last_clean":"2024-09-19T06:39:56.016828+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:39:56.016828+0000","last_undegraded":"2024-09-19T06:39:56.016828+0000","last_fullsized":"2024-09-19T06:39:56.016828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437914+0000","last_change":"2024-09-19T06:39:50.506741+0000","last_active":"2024-09-19T06:40:00.437914+0000","last_peered":"2024-09-19T06:40:00.437914+0000","last_clean":"2024-09-19T06:40:00.437914+0000","last_became_active":"2024-09-19T06:39:50.506437+0000","last_became_peered":"2024-09-19T06:39:50.506437+0000","last_unstale":"2024-09-19T06:40:00.437914+0000","last_undegraded":"2024-09-19T06:40:00.437914+0000","last_fullsized":"2024-09-19T06:40:00.437914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:42.304929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:16.415629+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:40:16.415629+0000","last_peered":"2024-09-19T06:40:16.415629+0000","last_clean":"2024-09-19T06:40:16.415629+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:40:16.415629+0000","last_undegraded":"2024-09-19T06:40:16.415629+0000","last_fullsized":"2024-09-19T06:40:16.415629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416102+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:39:53.416102+0000","last_peered":"2024-09-19T06:39:53.416102+0000","last_clean":"2024-09-19T06:39:53.416102+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:39:53.416102+0000","last_undegraded":"2024-09-19T06:39:53.416102+0000","last_fullsized":"2024-09-19T06:39:53.416102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.436752+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:40:00.436752+0000","last_peered":"2024-09-19T06:40:00.436752+0000","last_clean":"2024-09-19T06:40:00.436752+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:40:00.436752+0000","last_undegraded":"2024-09-19T06:40:00.436752+0000","last_fullsized":"2024-09-19T06:40:00.436752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419205+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:39:53.419205+0000","last_peered":"2024-09-19T06:39:53.419205+0000","last_clean":"2024-09-19T06:39:53.419205+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:39:53.419205+0000","last_undegraded":"2024-09-19T06:39:53.419205+0000","last_fullsized":"2024-09-19T06:39:53.419205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437748+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:40:00.437748+0000","last_peered":"2024-09-19T06:40:00.437748+0000","last_clean":"2024-09-19T06:40:00.437748+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:40:00.437748+0000","last_undegraded":"2024-09-19T06:40:00.437748+0000","last_fullsized":"2024-09-19T06:40:00.437748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414233+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:39:53.414233+0000","last_peered":"2024-09-19T06:39:53.414233+0000","last_clean":"2024-09-19T06:39:53.414233+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:39:53.414233+0000","last_undegraded":"2024-09-19T06:39:53.414233+0000","last_fullsized":"2024-09-19T06:39:53.414233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438533+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:40:00.438533+0000","last_peered":"2024-09-19T06:40:00.438533+0000","last_clean":"2024-09-19T06:40:00.438533+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:40:00.438533+0000","last_undegraded":"2024-09-19T06:40:00.438533+0000","last_fullsized":"2024-09-19T06:40:00.438533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403355+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:39:53.403355+0000","last_peered":"2024-09-19T06:39:53.403355+0000","last_clean":"2024-09-19T06:39:53.403355+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:39:53.403355+0000","last_undegraded":"2024-09-19T06:39:53.403355+0000","last_fullsized":"2024-09-19T06:39:53.403355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415130+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:39:53.415130+0000","last_peered":"2024-09-19T06:39:53.415130+0000","last_clean":"2024-09-19T06:39:53.415130+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:39:53.415130+0000","last_undegraded":"2024-09-19T06:39:53.415130+0000","last_fullsized":"2024-09-19T06:39:53.415130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415567+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:39:53.415567+0000","last_peered":"2024-09-19T06:39:53.415567+0000","last_clean":"2024-09-19T06:39:53.415567+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:39:53.415567+0000","last_undegraded":"2024-09-19T06:39:53.415567+0000","last_fullsized":"2024-09-19T06:39:53.415567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.439066+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:40:00.439066+0000","last_peered":"2024-09-19T06:40:00.439066+0000","last_clean":"2024-09-19T06:40:00.439066+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:40:00.439066+0000","last_undegraded":"2024-09-19T06:40:00.439066+0000","last_fullsized":"2024-09-19T06:40:00.439066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417596+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:39:53.417596+0000","last_peered":"2024-09-19T06:39:53.417596+0000","last_clean":"2024-09-19T06:39:53.417596+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:39:53.417596+0000","last_undegraded":"2024-09-19T06:39:53.417596+0000","last_fullsized":"2024-09-19T06:39:53.417596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415100+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:39:53.415100+0000","last_peered":"2024-09-19T06:39:53.415100+0000","last_clean":"2024-09-19T06:39:53.415100+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:39:53.415100+0000","last_undegraded":"2024-09-19T06:39:53.415100+0000","last_fullsized":"2024-09-19T06:39:53.415100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.420645+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:39:53.420645+0000","last_peered":"2024-09-19T06:39:53.420645+0000","last_clean":"2024-09-19T06:39:53.420645+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:39:53.420645+0000","last_undegraded":"2024-09-19T06:39:53.420645+0000","last_fullsized":"2024-09-19T06:39:53.420645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414246+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:39:53.414246+0000","last_peered":"2024-09-19T06:39:53.414246+0000","last_clean":"2024-09-19T06:39:53.414246+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:39:53.414246+0000","last_undegraded":"2024-09-19T06:39:53.414246+0000","last_fullsized":"2024-09-19T06:39:53.414246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447690+0000","last_change":"2024-09-19T06:39:50.315109+0000","last_active":"2024-09-19T06:40:01.447690+0000","last_peered":"2024-09-19T06:40:01.447690+0000","last_clean":"2024-09-19T06:40:01.447690+0000","last_became_active":"2024-09-19T06:39:50.314734+0000","last_became_peered":"2024-09-19T06:39:50.314734+0000","last_unstale":"2024-09-19T06:40:01.447690+0000","last_undegraded":"2024-09-19T06:40:01.447690+0000","last_fullsized":"2024-09-19T06:40:01.447690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:25.769548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419909+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:39:53.419909+0000","last_peered":"2024-09-19T06:39:53.419909+0000","last_clean":"2024-09-19T06:39:53.419909+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:39:53.419909+0000","last_undegraded":"2024-09-19T06:39:53.419909+0000","last_fullsized":"2024-09-19T06:39:53.419909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418802+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:39:53.418802+0000","last_peered":"2024-09-19T06:39:53.418802+0000","last_clean":"2024-09-19T06:39:53.418802+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:39:53.418802+0000","last_undegraded":"2024-09-19T06:39:53.418802+0000","last_fullsized":"2024-09-19T06:39:53.418802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415796+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:39:53.415796+0000","last_peered":"2024-09-19T06:39:53.415796+0000","last_clean":"2024-09-19T06:39:53.415796+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:39:53.415796+0000","last_undegraded":"2024-09-19T06:39:53.415796+0000","last_fullsized":"2024-09-19T06:39:53.415796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448401+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:40:01.448401+0000","last_peered":"2024-09-19T06:40:01.448401+0000","last_clean":"2024-09-19T06:40:01.448401+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:40:01.448401+0000","last_undegraded":"2024-09-19T06:40:01.448401+0000","last_fullsized":"2024-09-19T06:40:01.448401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":329,"num_read_kb":230,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281827,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":419949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776158,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1071034,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939684,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168617,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1012037,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1011328,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":420404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":419885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":479928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":478578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":420845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:40:56.643 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:40:56.897 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T06:40:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T06:40:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-09-19T06:40:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-19T06:40:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-19T06:40:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-19T06:40:57.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-19T06:40:57.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T06:40:57.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-19T06:40:57.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T06:40:57.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:40:57.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:40:57.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:57 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:57.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 -- 172.21.15.37:0/511791160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 msgr2=0x7f56c010ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 --2- 172.21.15.37:0/511791160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c010ec30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009cd0 tx=0x7f56b802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 -- 172.21.15.37:0/511791160 shutdown_connections 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 --2- 172.21.15.37:0/511791160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c010ec30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 --2- 172.21.15.37:0/511791160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c0103740 0x7f56c0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 --2- 172.21.15.37:0/511791160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c0102d90 0x7f56c0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 -- 172.21.15.37:0/511791160 >> 172.21.15.37:0/511791160 conn(0x7f56c00fc7d0 msgr2=0x7f56c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 -- 172.21.15.37:0/511791160 shutdown_connections 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 -- 172.21.15.37:0/511791160 wait complete. 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 Processor -- start 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 -- start start 2024-09-19T06:40:57.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c0102d90 0x7f56c019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.490+0000 7f56c6abc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c0103740 0x7f56c019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c6abc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c6abc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0112090 con 0x7f56c0108290 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c6abc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0111f10 con 0x7f56c0103740 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c6abc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56c0112210 con 0x7f56c0102d90 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44862/0 (socket says 172.21.15.37:44862) 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 -- 172.21.15.37:0/3420435681 learned_addr learned my addr 172.21.15.37:0/3420435681 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c4858640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c0102d90 0x7f56c019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 -- 172.21.15.37:0/3420435681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c0102d90 msgr2=0x7f56c019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c0102d90 0x7f56c019a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56b7fff640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c0103740 0x7f56c019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 -- 172.21.15.37:0/3420435681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c0103740 msgr2=0x7f56c019ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c0103740 0x7f56c019ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 -- 172.21.15.37:0/3420435681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c019f670 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56b7fff640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c0103740 0x7f56c019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c5059640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c019ef90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006010 tx=0x7f56b8003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56b5ffb640 1 -- 172.21.15.37:0/3420435681 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b8004450 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c019f900 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56b5ffb640 1 -- 172.21.15.37:0/3420435681 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b8003e10 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.491+0000 7f56b5ffb640 1 -- 172.21.15.37:0/3420435681 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b80466e0 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.492+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c01a7240 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.492+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56c0103170 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.494+0000 7f56b5ffb640 1 -- 172.21.15.37:0/3420435681 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f56b80305a0 con 0x7f56c0108290 2024-09-19T06:40:57.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.494+0000 7f56b5ffb640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f56a80a32d0 0x7f56a80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.495+0000 7f56b5ffb640 1 -- 172.21.15.37:0/3420435681 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f56b802e030 con 0x7f56c0108290 2024-09-19T06:40:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.495+0000 7f56c4858640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f56a80a32d0 0x7f56a80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:57.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.495+0000 7f56c4858640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f56a80a32d0 0x7f56a80a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00ba40 tx=0x7f56bc005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:57.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:57.554 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:40:57.555 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:57 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:57.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.498+0000 7f56b5ffb640 1 -- 172.21.15.37:0/3420435681 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f56b804b060 con 0x7f56c0108290 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='client.14916 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]: dispatch 2024-09-19T06:40:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:40:57.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:40:57.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:57 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' 2024-09-19T06:40:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.675+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56c010dfb0 con 0x7f56a80a32d0 2024-09-19T06:40:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.681+0000 7f56b5ffb640 1 -- 172.21.15.37:0/3420435681 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258864 (secure 0 0 0) 0x7f56c010dfb0 con 0x7f56a80a32d0 2024-09-19T06:40:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:57.692 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f56a80a32d0 msgr2=0x7f56a80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f56a80a32d0 0x7f56a80a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00ba40 tx=0x7f56bc005cc0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 msgr2=0x7f56c019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c019ef90 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f56b8006010 tx=0x7f56b8003c20 comp rx=0 tx=0).stop 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 shutdown_connections 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f56a80a32d0 0x7f56a80a5790 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c0108290 0x7f56c019ef90 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c0103740 0x7f56c019ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 --2- 172.21.15.37:0/3420435681 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c0102d90 0x7f56c019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:57.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 >> 172.21.15.37:0/3420435681 conn(0x7f56c00fc7d0 msgr2=0x7f56c00fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.684+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 shutdown_connections 2024-09-19T06:40:57.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:57.685+0000 7f56c6abc640 1 -- 172.21.15.37:0/3420435681 wait complete. 2024-09-19T06:40:58.330 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T06:40:58.331 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:58 smithi037 ceph-mon[31211]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:40:58.359 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-19T06:40:57.116381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653816,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":480,"num_read_kb":359,"num_write":343,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":360000,"kb_used_data":10752,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124581824,"statfs":{"total":1151940427776,"available":1151571787776,"internally_reserved":0,"allocated":11010048,"data_stored":6993852,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":357414201},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.912283"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403605+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:39:53.403605+0000","last_peered":"2024-09-19T06:39:53.403605+0000","last_clean":"2024-09-19T06:39:53.403605+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:39:53.403605+0000","last_undegraded":"2024-09-19T06:39:53.403605+0000","last_fullsized":"2024-09-19T06:39:53.403605+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419068+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:39:53.419068+0000","last_peered":"2024-09-19T06:39:53.419068+0000","last_clean":"2024-09-19T06:39:53.419068+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:39:53.419068+0000","last_undegraded":"2024-09-19T06:39:53.419068+0000","last_fullsized":"2024-09-19T06:39:53.419068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419285+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:39:53.419285+0000","last_peered":"2024-09-19T06:39:53.419285+0000","last_clean":"2024-09-19T06:39:53.419285+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:39:53.419285+0000","last_undegraded":"2024-09-19T06:39:53.419285+0000","last_fullsized":"2024-09-19T06:39:53.419285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418707+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:39:53.418707+0000","last_peered":"2024-09-19T06:39:53.418707+0000","last_clean":"2024-09-19T06:39:53.418707+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:39:53.418707+0000","last_undegraded":"2024-09-19T06:39:53.418707+0000","last_fullsized":"2024-09-19T06:39:53.418707+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400788+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:39:53.400788+0000","last_peered":"2024-09-19T06:39:53.400788+0000","last_clean":"2024-09-19T06:39:53.400788+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:39:53.400788+0000","last_undegraded":"2024-09-19T06:39:53.400788+0000","last_fullsized":"2024-09-19T06:39:53.400788+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417175+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:39:53.417175+0000","last_peered":"2024-09-19T06:39:53.417175+0000","last_clean":"2024-09-19T06:39:53.417175+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:39:53.417175+0000","last_undegraded":"2024-09-19T06:39:53.417175+0000","last_fullsized":"2024-09-19T06:39:53.417175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415134+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:39:53.415134+0000","last_peered":"2024-09-19T06:39:53.415134+0000","last_clean":"2024-09-19T06:39:53.415134+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:39:53.415134+0000","last_undegraded":"2024-09-19T06:39:53.415134+0000","last_fullsized":"2024-09-19T06:39:53.415134+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414333+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:39:53.414333+0000","last_peered":"2024-09-19T06:39:53.414333+0000","last_clean":"2024-09-19T06:39:53.414333+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:39:53.414333+0000","last_undegraded":"2024-09-19T06:39:53.414333+0000","last_fullsized":"2024-09-19T06:39:53.414333+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.401273+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:39:53.401273+0000","last_peered":"2024-09-19T06:39:53.401273+0000","last_clean":"2024-09-19T06:39:53.401273+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:39:53.401273+0000","last_undegraded":"2024-09-19T06:39:53.401273+0000","last_fullsized":"2024-09-19T06:39:53.401273+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417564+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:39:53.417564+0000","last_peered":"2024-09-19T06:39:53.417564+0000","last_clean":"2024-09-19T06:39:53.417564+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:39:53.417564+0000","last_undegraded":"2024-09-19T06:39:53.417564+0000","last_fullsized":"2024-09-19T06:39:53.417564+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415382+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:39:53.415382+0000","last_peered":"2024-09-19T06:39:53.415382+0000","last_clean":"2024-09-19T06:39:53.415382+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:39:53.415382+0000","last_undegraded":"2024-09-19T06:39:53.415382+0000","last_fullsized":"2024-09-19T06:39:53.415382+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400663+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:39:53.400663+0000","last_peered":"2024-09-19T06:39:53.400663+0000","last_clean":"2024-09-19T06:39:53.400663+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:39:53.400663+0000","last_undegraded":"2024-09-19T06:39:53.400663+0000","last_fullsized":"2024-09-19T06:39:53.400663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419338+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:39:53.419338+0000","last_peered":"2024-09-19T06:39:53.419338+0000","last_clean":"2024-09-19T06:39:53.419338+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:39:53.419338+0000","last_undegraded":"2024-09-19T06:39:53.419338+0000","last_fullsized":"2024-09-19T06:39:53.419338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418306+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:39:53.418306+0000","last_peered":"2024-09-19T06:39:53.418306+0000","last_clean":"2024-09-19T06:39:53.418306+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:39:53.418306+0000","last_undegraded":"2024-09-19T06:39:53.418306+0000","last_fullsized":"2024-09-19T06:39:53.418306+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413939+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:39:53.413939+0000","last_peered":"2024-09-19T06:39:53.413939+0000","last_clean":"2024-09-19T06:39:53.413939+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:39:53.413939+0000","last_undegraded":"2024-09-19T06:39:53.413939+0000","last_fullsized":"2024-09-19T06:39:53.413939+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403646+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:39:53.403646+0000","last_peered":"2024-09-19T06:39:53.403646+0000","last_clean":"2024-09-19T06:39:53.403646+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:39:53.403646+0000","last_undegraded":"2024-09-19T06:39:53.403646+0000","last_fullsized":"2024-09-19T06:39:53.403646+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414294+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:39:53.414294+0000","last_peered":"2024-09-19T06:39:53.414294+0000","last_clean":"2024-09-19T06:39:53.414294+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:39:53.414294+0000","last_undegraded":"2024-09-19T06:39:53.414294+0000","last_fullsized":"2024-09-19T06:39:53.414294+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415375+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:39:53.415375+0000","last_peered":"2024-09-19T06:39:53.415375+0000","last_clean":"2024-09-19T06:39:53.415375+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:39:53.415375+0000","last_undegraded":"2024-09-19T06:39:53.415375+0000","last_fullsized":"2024-09-19T06:39:53.415375+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403279+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:39:53.403279+0000","last_peered":"2024-09-19T06:39:53.403279+0000","last_clean":"2024-09-19T06:39:53.403279+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:39:53.403279+0000","last_undegraded":"2024-09-19T06:39:53.403279+0000","last_fullsized":"2024-09-19T06:39:53.403279+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400705+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:39:53.400705+0000","last_peered":"2024-09-19T06:39:53.400705+0000","last_clean":"2024-09-19T06:39:53.400705+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:39:53.400705+0000","last_undegraded":"2024-09-19T06:39:53.400705+0000","last_fullsized":"2024-09-19T06:39:53.400705+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414105+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:39:53.414105+0000","last_peered":"2024-09-19T06:39:53.414105+0000","last_clean":"2024-09-19T06:39:53.414105+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:39:53.414105+0000","last_undegraded":"2024-09-19T06:39:53.414105+0000","last_fullsized":"2024-09-19T06:39:53.414105+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415846+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:39:53.415846+0000","last_peered":"2024-09-19T06:39:53.415846+0000","last_clean":"2024-09-19T06:39:53.415846+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:39:53.415846+0000","last_undegraded":"2024-09-19T06:39:53.415846+0000","last_fullsized":"2024-09-19T06:39:53.415846+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414231+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:39:53.414231+0000","last_peered":"2024-09-19T06:39:53.414231+0000","last_clean":"2024-09-19T06:39:53.414231+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:39:53.414231+0000","last_undegraded":"2024-09-19T06:39:53.414231+0000","last_fullsized":"2024-09-19T06:39:53.414231+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416096+0000","last_change":"2024-09-19T06:39:52.399797+0000","last_active":"2024-09-19T06:39:53.416096+0000","last_peered":"2024-09-19T06:39:53.416096+0000","last_clean":"2024-09-19T06:39:53.416096+0000","last_became_active":"2024-09-19T06:39:52.399452+0000","last_became_peered":"2024-09-19T06:39:52.399452+0000","last_unstale":"2024-09-19T06:39:53.416096+0000","last_undegraded":"2024-09-19T06:39:53.416096+0000","last_fullsized":"2024-09-19T06:39:53.416096+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:02:33.182396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415925+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:39:53.415925+0000","last_peered":"2024-09-19T06:39:53.415925+0000","last_clean":"2024-09-19T06:39:53.415925+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:39:53.415925+0000","last_undegraded":"2024-09-19T06:39:53.415925+0000","last_fullsized":"2024-09-19T06:39:53.415925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403645+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:39:53.403645+0000","last_peered":"2024-09-19T06:39:53.403645+0000","last_clean":"2024-09-19T06:39:53.403645+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:39:53.403645+0000","last_undegraded":"2024-09-19T06:39:53.403645+0000","last_fullsized":"2024-09-19T06:39:53.403645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400700+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:39:53.400700+0000","last_peered":"2024-09-19T06:39:53.400700+0000","last_clean":"2024-09-19T06:39:53.400700+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:39:53.400700+0000","last_undegraded":"2024-09-19T06:39:53.400700+0000","last_fullsized":"2024-09-19T06:39:53.400700+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419814+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:39:53.419814+0000","last_peered":"2024-09-19T06:39:53.419814+0000","last_clean":"2024-09-19T06:39:53.419814+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:39:53.419814+0000","last_undegraded":"2024-09-19T06:39:53.419814+0000","last_fullsized":"2024-09-19T06:39:53.419814+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414873+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:39:53.414873+0000","last_peered":"2024-09-19T06:39:53.414873+0000","last_clean":"2024-09-19T06:39:53.414873+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:39:53.414873+0000","last_undegraded":"2024-09-19T06:39:53.414873+0000","last_fullsized":"2024-09-19T06:39:53.414873+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414716+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:39:53.414716+0000","last_peered":"2024-09-19T06:39:53.414716+0000","last_clean":"2024-09-19T06:39:53.414716+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:39:53.414716+0000","last_undegraded":"2024-09-19T06:39:53.414716+0000","last_fullsized":"2024-09-19T06:39:53.414716+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403217+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:39:53.403217+0000","last_peered":"2024-09-19T06:39:53.403217+0000","last_clean":"2024-09-19T06:39:53.403217+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:39:53.403217+0000","last_undegraded":"2024-09-19T06:39:53.403217+0000","last_fullsized":"2024-09-19T06:39:53.403217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415182+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:39:53.415182+0000","last_peered":"2024-09-19T06:39:53.415182+0000","last_clean":"2024-09-19T06:39:53.415182+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:39:53.415182+0000","last_undegraded":"2024-09-19T06:39:53.415182+0000","last_fullsized":"2024-09-19T06:39:53.415182+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"81'5","reported_seq":76,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:51.525759+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:40:51.525759+0000","last_peered":"2024-09-19T06:40:51.525759+0000","last_clean":"2024-09-19T06:40:51.525759+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:40:51.525759+0000","last_undegraded":"2024-09-19T06:40:51.525759+0000","last_fullsized":"2024-09-19T06:40:51.525759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414773+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:39:53.414773+0000","last_peered":"2024-09-19T06:39:53.414773+0000","last_clean":"2024-09-19T06:39:53.414773+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:39:53.414773+0000","last_undegraded":"2024-09-19T06:39:53.414773+0000","last_fullsized":"2024-09-19T06:39:53.414773+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.446851+0000","last_change":"2024-09-19T06:39:50.318132+0000","last_active":"2024-09-19T06:40:01.446851+0000","last_peered":"2024-09-19T06:40:01.446851+0000","last_clean":"2024-09-19T06:40:01.446851+0000","last_became_active":"2024-09-19T06:39:50.317774+0000","last_became_peered":"2024-09-19T06:39:50.317774+0000","last_unstale":"2024-09-19T06:40:01.446851+0000","last_undegraded":"2024-09-19T06:40:01.446851+0000","last_fullsized":"2024-09-19T06:40:01.446851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:17:58.290128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.445718+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:40:01.445718+0000","last_peered":"2024-09-19T06:40:01.445718+0000","last_clean":"2024-09-19T06:40:01.445718+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:40:01.445718+0000","last_undegraded":"2024-09-19T06:40:01.445718+0000","last_fullsized":"2024-09-19T06:40:01.445718+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415204+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:39:53.415204+0000","last_peered":"2024-09-19T06:39:53.415204+0000","last_clean":"2024-09-19T06:39:53.415204+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:39:53.415204+0000","last_undegraded":"2024-09-19T06:39:53.415204+0000","last_fullsized":"2024-09-19T06:39:53.415204+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414736+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:39:53.414736+0000","last_peered":"2024-09-19T06:39:53.414736+0000","last_clean":"2024-09-19T06:39:53.414736+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:39:53.414736+0000","last_undegraded":"2024-09-19T06:39:53.414736+0000","last_fullsized":"2024-09-19T06:39:53.414736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.413749+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:39:53.413749+0000","last_peered":"2024-09-19T06:39:53.413749+0000","last_clean":"2024-09-19T06:39:53.413749+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:39:53.413749+0000","last_undegraded":"2024-09-19T06:39:53.413749+0000","last_fullsized":"2024-09-19T06:39:53.413749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"81'3","reported_seq":75,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:54.529544+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:40:54.529544+0000","last_peered":"2024-09-19T06:40:54.529544+0000","last_clean":"2024-09-19T06:40:54.529544+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:40:54.529544+0000","last_undegraded":"2024-09-19T06:40:54.529544+0000","last_fullsized":"2024-09-19T06:40:54.529544+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":54,"num_read_kb":54,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414187+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:39:53.414187+0000","last_peered":"2024-09-19T06:39:53.414187+0000","last_clean":"2024-09-19T06:39:53.414187+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:39:53.414187+0000","last_undegraded":"2024-09-19T06:39:53.414187+0000","last_fullsized":"2024-09-19T06:39:53.414187+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438327+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:40:00.438327+0000","last_peered":"2024-09-19T06:40:00.438327+0000","last_clean":"2024-09-19T06:40:00.438327+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:40:00.438327+0000","last_undegraded":"2024-09-19T06:40:00.438327+0000","last_fullsized":"2024-09-19T06:40:00.438327+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419760+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:39:53.419760+0000","last_peered":"2024-09-19T06:39:53.419760+0000","last_clean":"2024-09-19T06:39:53.419760+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:39:53.419760+0000","last_undegraded":"2024-09-19T06:39:53.419760+0000","last_fullsized":"2024-09-19T06:39:53.419760+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437339+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:40:00.437339+0000","last_peered":"2024-09-19T06:40:00.437339+0000","last_clean":"2024-09-19T06:40:00.437339+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:40:00.437339+0000","last_undegraded":"2024-09-19T06:40:00.437339+0000","last_fullsized":"2024-09-19T06:40:00.437339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414395+0000","last_change":"2024-09-19T06:39:52.391059+0000","last_active":"2024-09-19T06:39:53.414395+0000","last_peered":"2024-09-19T06:39:53.414395+0000","last_clean":"2024-09-19T06:39:53.414395+0000","last_became_active":"2024-09-19T06:39:52.389703+0000","last_became_peered":"2024-09-19T06:39:52.389703+0000","last_unstale":"2024-09-19T06:39:53.414395+0000","last_undegraded":"2024-09-19T06:39:53.414395+0000","last_fullsized":"2024-09-19T06:39:53.414395+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:27:32.021109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"81'26","reported_seq":160,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:55.531054+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:40:55.531054+0000","last_peered":"2024-09-19T06:40:55.531054+0000","last_clean":"2024-09-19T06:40:55.531054+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:40:55.531054+0000","last_undegraded":"2024-09-19T06:40:55.531054+0000","last_fullsized":"2024-09-19T06:40:55.531054+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":59211,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":116,"num_read_kb":115,"num_write":27,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415675+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:39:53.415675+0000","last_peered":"2024-09-19T06:39:53.415675+0000","last_clean":"2024-09-19T06:39:53.415675+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:39:53.415675+0000","last_undegraded":"2024-09-19T06:39:53.415675+0000","last_fullsized":"2024-09-19T06:39:53.415675+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":25,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447875+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:40:01.447875+0000","last_peered":"2024-09-19T06:40:01.447875+0000","last_clean":"2024-09-19T06:40:01.447875+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:40:01.447875+0000","last_undegraded":"2024-09-19T06:40:01.447875+0000","last_fullsized":"2024-09-19T06:40:01.447875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437843+0000","last_change":"2024-09-19T06:39:50.312876+0000","last_active":"2024-09-19T06:40:00.437843+0000","last_peered":"2024-09-19T06:40:00.437843+0000","last_clean":"2024-09-19T06:40:00.437843+0000","last_became_active":"2024-09-19T06:39:50.312500+0000","last_became_peered":"2024-09-19T06:39:50.312500+0000","last_unstale":"2024-09-19T06:40:00.437843+0000","last_undegraded":"2024-09-19T06:40:00.437843+0000","last_fullsized":"2024-09-19T06:40:00.437843+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:41:00.113022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417798+0000","last_change":"2024-09-19T06:39:52.399395+0000","last_active":"2024-09-19T06:39:53.417798+0000","last_peered":"2024-09-19T06:39:53.417798+0000","last_clean":"2024-09-19T06:39:53.417798+0000","last_became_active":"2024-09-19T06:39:52.399042+0000","last_became_peered":"2024-09-19T06:39:52.399042+0000","last_unstale":"2024-09-19T06:39:53.417798+0000","last_undegraded":"2024-09-19T06:39:53.417798+0000","last_fullsized":"2024-09-19T06:39:53.417798+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:10.010156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"81'4","reported_seq":77,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:52.526527+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:40:52.526527+0000","last_peered":"2024-09-19T06:40:52.526527+0000","last_clean":"2024-09-19T06:40:52.526527+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:40:52.526527+0000","last_undegraded":"2024-09-19T06:40:52.526527+0000","last_fullsized":"2024-09-19T06:40:52.526527+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":981,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416672+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:39:53.416672+0000","last_peered":"2024-09-19T06:39:53.416672+0000","last_clean":"2024-09-19T06:39:53.416672+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:39:53.416672+0000","last_undegraded":"2024-09-19T06:39:53.416672+0000","last_fullsized":"2024-09-19T06:39:53.416672+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":34,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448340+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:40:01.448340+0000","last_peered":"2024-09-19T06:40:01.448340+0000","last_clean":"2024-09-19T06:40:01.448340+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:40:01.448340+0000","last_undegraded":"2024-09-19T06:40:01.448340+0000","last_fullsized":"2024-09-19T06:40:01.448340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415210+0000","last_change":"2024-09-19T06:39:52.390919+0000","last_active":"2024-09-19T06:39:53.415210+0000","last_peered":"2024-09-19T06:39:53.415210+0000","last_clean":"2024-09-19T06:39:53.415210+0000","last_became_active":"2024-09-19T06:39:52.390585+0000","last_became_peered":"2024-09-19T06:39:52.390585+0000","last_unstale":"2024-09-19T06:39:53.415210+0000","last_undegraded":"2024-09-19T06:39:53.415210+0000","last_fullsized":"2024-09-19T06:39:53.415210+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:52:02.164497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"81'2","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.451484+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:40:02.451484+0000","last_peered":"2024-09-19T06:40:02.451484+0000","last_clean":"2024-09-19T06:40:02.451484+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:40:02.451484+0000","last_undegraded":"2024-09-19T06:40:02.451484+0000","last_fullsized":"2024-09-19T06:40:02.451484+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403462+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:39:53.403462+0000","last_peered":"2024-09-19T06:39:53.403462+0000","last_clean":"2024-09-19T06:39:53.403462+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:39:53.403462+0000","last_undegraded":"2024-09-19T06:39:53.403462+0000","last_fullsized":"2024-09-19T06:39:53.403462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418357+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:39:53.418357+0000","last_peered":"2024-09-19T06:39:53.418357+0000","last_clean":"2024-09-19T06:39:53.418357+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:39:53.418357+0000","last_undegraded":"2024-09-19T06:39:53.418357+0000","last_fullsized":"2024-09-19T06:39:53.418357+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418035+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:39:53.418035+0000","last_peered":"2024-09-19T06:39:53.418035+0000","last_clean":"2024-09-19T06:39:53.418035+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:39:53.418035+0000","last_undegraded":"2024-09-19T06:39:53.418035+0000","last_fullsized":"2024-09-19T06:39:53.418035+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419293+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:39:53.419293+0000","last_peered":"2024-09-19T06:39:53.419293+0000","last_clean":"2024-09-19T06:39:53.419293+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:39:53.419293+0000","last_undegraded":"2024-09-19T06:39:53.419293+0000","last_fullsized":"2024-09-19T06:39:53.419293+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144684+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:39:54.144684+0000","last_peered":"2024-09-19T06:39:54.144684+0000","last_clean":"2024-09-19T06:39:54.144684+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:39:54.144684+0000","last_undegraded":"2024-09-19T06:39:54.144684+0000","last_fullsized":"2024-09-19T06:39:54.144684+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447446+0000","last_change":"2024-09-19T06:39:50.507312+0000","last_active":"2024-09-19T06:40:01.447446+0000","last_peered":"2024-09-19T06:40:01.447446+0000","last_clean":"2024-09-19T06:40:01.447446+0000","last_became_active":"2024-09-19T06:39:50.506976+0000","last_became_peered":"2024-09-19T06:39:50.506976+0000","last_unstale":"2024-09-19T06:40:01.447446+0000","last_undegraded":"2024-09-19T06:40:01.447446+0000","last_fullsized":"2024-09-19T06:40:01.447446+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:25:44.989718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417964+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:39:53.417964+0000","last_peered":"2024-09-19T06:39:53.417964+0000","last_clean":"2024-09-19T06:39:53.417964+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:39:53.417964+0000","last_undegraded":"2024-09-19T06:39:53.417964+0000","last_fullsized":"2024-09-19T06:39:53.417964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":339,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417272+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:39:53.417272+0000","last_peered":"2024-09-19T06:39:53.417272+0000","last_clean":"2024-09-19T06:39:53.417272+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:39:53.417272+0000","last_undegraded":"2024-09-19T06:39:53.417272+0000","last_fullsized":"2024-09-19T06:39:53.417272+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:02.450445+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:40:02.450445+0000","last_peered":"2024-09-19T06:40:02.450445+0000","last_clean":"2024-09-19T06:40:02.450445+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:40:02.450445+0000","last_undegraded":"2024-09-19T06:40:02.450445+0000","last_fullsized":"2024-09-19T06:40:02.450445+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415347+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:39:53.415347+0000","last_peered":"2024-09-19T06:39:53.415347+0000","last_clean":"2024-09-19T06:39:53.415347+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:39:53.415347+0000","last_undegraded":"2024-09-19T06:39:53.415347+0000","last_fullsized":"2024-09-19T06:39:53.415347+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447538+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:40:01.447538+0000","last_peered":"2024-09-19T06:40:01.447538+0000","last_clean":"2024-09-19T06:40:01.447538+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:40:01.447538+0000","last_undegraded":"2024-09-19T06:40:01.447538+0000","last_fullsized":"2024-09-19T06:40:01.447538+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400695+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:39:53.400695+0000","last_peered":"2024-09-19T06:39:53.400695+0000","last_clean":"2024-09-19T06:39:53.400695+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:39:53.400695+0000","last_undegraded":"2024-09-19T06:39:53.400695+0000","last_fullsized":"2024-09-19T06:39:53.400695+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"81'4","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:52.526129+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:40:52.526129+0000","last_peered":"2024-09-19T06:40:52.526129+0000","last_clean":"2024-09-19T06:40:52.526129+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:40:52.526129+0000","last_undegraded":"2024-09-19T06:40:52.526129+0000","last_fullsized":"2024-09-19T06:40:52.526129+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414405+0000","last_change":"2024-09-19T06:39:52.396795+0000","last_active":"2024-09-19T06:39:53.414405+0000","last_peered":"2024-09-19T06:39:53.414405+0000","last_clean":"2024-09-19T06:39:53.414405+0000","last_became_active":"2024-09-19T06:39:52.396481+0000","last_became_peered":"2024-09-19T06:39:52.396481+0000","last_unstale":"2024-09-19T06:39:53.414405+0000","last_undegraded":"2024-09-19T06:39:53.414405+0000","last_fullsized":"2024-09-19T06:39:53.414405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:39:05.637504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:54.144665+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:39:54.144665+0000","last_peered":"2024-09-19T06:39:54.144665+0000","last_clean":"2024-09-19T06:39:54.144665+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:39:54.144665+0000","last_undegraded":"2024-09-19T06:39:54.144665+0000","last_fullsized":"2024-09-19T06:39:54.144665+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400964+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:39:53.400964+0000","last_peered":"2024-09-19T06:39:53.400964+0000","last_clean":"2024-09-19T06:39:53.400964+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:39:53.400964+0000","last_undegraded":"2024-09-19T06:39:53.400964+0000","last_fullsized":"2024-09-19T06:39:53.400964+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414089+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:39:53.414089+0000","last_peered":"2024-09-19T06:39:53.414089+0000","last_clean":"2024-09-19T06:39:53.414089+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:39:53.414089+0000","last_undegraded":"2024-09-19T06:39:53.414089+0000","last_fullsized":"2024-09-19T06:39:53.414089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"81'1","reported_seq":14,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:56.016828+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:39:56.016828+0000","last_peered":"2024-09-19T06:39:56.016828+0000","last_clean":"2024-09-19T06:39:56.016828+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:39:56.016828+0000","last_undegraded":"2024-09-19T06:39:56.016828+0000","last_fullsized":"2024-09-19T06:39:56.016828+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437914+0000","last_change":"2024-09-19T06:39:50.506741+0000","last_active":"2024-09-19T06:40:00.437914+0000","last_peered":"2024-09-19T06:40:00.437914+0000","last_clean":"2024-09-19T06:40:00.437914+0000","last_became_active":"2024-09-19T06:39:50.506437+0000","last_became_peered":"2024-09-19T06:39:50.506437+0000","last_unstale":"2024-09-19T06:40:00.437914+0000","last_undegraded":"2024-09-19T06:40:00.437914+0000","last_fullsized":"2024-09-19T06:40:00.437914+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:23:42.304929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.400677+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:39:53.400677+0000","last_peered":"2024-09-19T06:39:53.400677+0000","last_clean":"2024-09-19T06:39:53.400677+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:39:53.400677+0000","last_undegraded":"2024-09-19T06:39:53.400677+0000","last_fullsized":"2024-09-19T06:39:53.400677+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"81'11","reported_seq":29,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:16.415629+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:40:16.415629+0000","last_peered":"2024-09-19T06:40:16.415629+0000","last_clean":"2024-09-19T06:40:16.415629+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:40:16.415629+0000","last_undegraded":"2024-09-19T06:40:16.415629+0000","last_fullsized":"2024-09-19T06:40:16.415629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.416102+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:39:53.416102+0000","last_peered":"2024-09-19T06:39:53.416102+0000","last_clean":"2024-09-19T06:39:53.416102+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:39:53.416102+0000","last_undegraded":"2024-09-19T06:39:53.416102+0000","last_fullsized":"2024-09-19T06:39:53.416102+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.436752+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:40:00.436752+0000","last_peered":"2024-09-19T06:40:00.436752+0000","last_clean":"2024-09-19T06:40:00.436752+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:40:00.436752+0000","last_undegraded":"2024-09-19T06:40:00.436752+0000","last_fullsized":"2024-09-19T06:40:00.436752+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419205+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:39:53.419205+0000","last_peered":"2024-09-19T06:39:53.419205+0000","last_clean":"2024-09-19T06:39:53.419205+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:39:53.419205+0000","last_undegraded":"2024-09-19T06:39:53.419205+0000","last_fullsized":"2024-09-19T06:39:53.419205+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.437748+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:40:00.437748+0000","last_peered":"2024-09-19T06:40:00.437748+0000","last_clean":"2024-09-19T06:40:00.437748+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:40:00.437748+0000","last_undegraded":"2024-09-19T06:40:00.437748+0000","last_fullsized":"2024-09-19T06:40:00.437748+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414233+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:39:53.414233+0000","last_peered":"2024-09-19T06:39:53.414233+0000","last_clean":"2024-09-19T06:39:53.414233+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:39:53.414233+0000","last_undegraded":"2024-09-19T06:39:53.414233+0000","last_fullsized":"2024-09-19T06:39:53.414233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":21,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.438533+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:40:00.438533+0000","last_peered":"2024-09-19T06:40:00.438533+0000","last_clean":"2024-09-19T06:40:00.438533+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:40:00.438533+0000","last_undegraded":"2024-09-19T06:40:00.438533+0000","last_fullsized":"2024-09-19T06:40:00.438533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.403355+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:39:53.403355+0000","last_peered":"2024-09-19T06:39:53.403355+0000","last_clean":"2024-09-19T06:39:53.403355+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:39:53.403355+0000","last_undegraded":"2024-09-19T06:39:53.403355+0000","last_fullsized":"2024-09-19T06:39:53.403355+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"81'1","reported_seq":19,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415130+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:39:53.415130+0000","last_peered":"2024-09-19T06:39:53.415130+0000","last_clean":"2024-09-19T06:39:53.415130+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:39:53.415130+0000","last_undegraded":"2024-09-19T06:39:53.415130+0000","last_fullsized":"2024-09-19T06:39:53.415130+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415567+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:39:53.415567+0000","last_peered":"2024-09-19T06:39:53.415567+0000","last_clean":"2024-09-19T06:39:53.415567+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:39:53.415567+0000","last_undegraded":"2024-09-19T06:39:53.415567+0000","last_fullsized":"2024-09-19T06:39:53.415567+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":23,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:00.439066+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:40:00.439066+0000","last_peered":"2024-09-19T06:40:00.439066+0000","last_clean":"2024-09-19T06:40:00.439066+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:40:00.439066+0000","last_undegraded":"2024-09-19T06:40:00.439066+0000","last_fullsized":"2024-09-19T06:40:00.439066+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.417596+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:39:53.417596+0000","last_peered":"2024-09-19T06:39:53.417596+0000","last_clean":"2024-09-19T06:39:53.417596+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:39:53.417596+0000","last_undegraded":"2024-09-19T06:39:53.417596+0000","last_fullsized":"2024-09-19T06:39:53.417596+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415100+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:39:53.415100+0000","last_peered":"2024-09-19T06:39:53.415100+0000","last_clean":"2024-09-19T06:39:53.415100+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:39:53.415100+0000","last_undegraded":"2024-09-19T06:39:53.415100+0000","last_fullsized":"2024-09-19T06:39:53.415100+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.420645+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:39:53.420645+0000","last_peered":"2024-09-19T06:39:53.420645+0000","last_clean":"2024-09-19T06:39:53.420645+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:39:53.420645+0000","last_undegraded":"2024-09-19T06:39:53.420645+0000","last_fullsized":"2024-09-19T06:39:53.420645+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.414246+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:39:53.414246+0000","last_peered":"2024-09-19T06:39:53.414246+0000","last_clean":"2024-09-19T06:39:53.414246+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:39:53.414246+0000","last_undegraded":"2024-09-19T06:39:53.414246+0000","last_fullsized":"2024-09-19T06:39:53.414246+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":22,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.447690+0000","last_change":"2024-09-19T06:39:50.315109+0000","last_active":"2024-09-19T06:40:01.447690+0000","last_peered":"2024-09-19T06:40:01.447690+0000","last_clean":"2024-09-19T06:40:01.447690+0000","last_became_active":"2024-09-19T06:39:50.314734+0000","last_became_peered":"2024-09-19T06:39:50.314734+0000","last_unstale":"2024-09-19T06:40:01.447690+0000","last_undegraded":"2024-09-19T06:40:01.447690+0000","last_fullsized":"2024-09-19T06:40:01.447690+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:59:25.769548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.419909+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:39:53.419909+0000","last_peered":"2024-09-19T06:39:53.419909+0000","last_clean":"2024-09-19T06:39:53.419909+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:39:53.419909+0000","last_undegraded":"2024-09-19T06:39:53.419909+0000","last_fullsized":"2024-09-19T06:39:53.419909+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":20,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.418802+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:39:53.418802+0000","last_peered":"2024-09-19T06:39:53.418802+0000","last_clean":"2024-09-19T06:39:53.418802+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:39:53.418802+0000","last_undegraded":"2024-09-19T06:39:53.418802+0000","last_fullsized":"2024-09-19T06:39:53.418802+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:39:53.415796+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:39:53.415796+0000","last_peered":"2024-09-19T06:39:53.415796+0000","last_clean":"2024-09-19T06:39:53.415796+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:39:53.415796+0000","last_undegraded":"2024-09-19T06:39:53.415796+0000","last_fullsized":"2024-09-19T06:39:53.415796+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":28,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-09-19T06:40:01.448401+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:40:01.448401+0000","last_peered":"2024-09-19T06:40:01.448401+0000","last_clean":"2024-09-19T06:40:01.448401+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:40:01.448401+0000","last_undegraded":"2024-09-19T06:40:01.448401+0000","last_fullsized":"2024-09-19T06:40:01.448401+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63214,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":344,"num_read_kb":240,"num_write":160,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":189642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136,"ondisk_log_size":136,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":281827,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32036,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713116,"statfs":{"total":95995035648,"available":95962230784,"internally_reserved":0,"allocated":872448,"data_stored":419949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776159,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1024000,"data_stored":1071034,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939685,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168618,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32052,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713100,"statfs":{"total":95995035648,"available":95962214400,"internally_reserved":0,"allocated":888832,"data_stored":420105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":1012037,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561076,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32040,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713112,"statfs":{"total":95995035648,"available":95962226688,"internally_reserved":0,"allocated":876544,"data_stored":419574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790008,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":950272,"data_stored":1011328,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":880640,"data_stored":420404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182466,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32056,"kb_used_data":872,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713096,"statfs":{"total":95995035648,"available":95962210304,"internally_reserved":0,"allocated":892928,"data_stored":419885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":962560,"data_stored":479928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":937984,"data_stored":478578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":420845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34204,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":60561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":61182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:40:58.364 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:40:58.364 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-19T06:40:58.364 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-19T06:40:58.364 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T06:40:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T06:40:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T06:40:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-09-19T06:40:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-19T06:40:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-19T06:40:58.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-19T06:40:58.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-19T06:40:58.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T06:40:58.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-19T06:40:58.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T06:40:58.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:58 smithi096 ceph-mon[35004]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: pgmap v311: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 4]}]': finished 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.12", "id": [5, 11]}]': finished 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 1]}]': finished 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [4, 8]}]': finished 2024-09-19T06:40:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-09-19T06:40:58.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-09-19T06:40:58.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: from='mgr.14150 172.21.15.37:0/2524593483' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.27", "id": [3, 9]}]': finished 2024-09-19T06:40:58.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:58 smithi151 ceph-mon[35236]: osdmap e82: 12 total, 12 up, 12 in 2024-09-19T06:40:58.622 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/281867824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 msgr2=0x7fa7c010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/281867824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c010e610 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009ca0 tx=0x7fa7b802a950 comp rx=0 tx=0).stop 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/281867824 shutdown_connections 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/281867824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c010e610 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/281867824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7c0103660 0x7fa7c010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/281867824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c0102d40 0x7fa7c0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/281867824 >> 172.21.15.37:0/281867824 conn(0x7fa7c00fc7a0 msgr2=0x7fa7c00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/281867824 shutdown_connections 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/281867824 wait complete. 2024-09-19T06:40:59.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 Processor -- start 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.065+0000 7fa7c5f0f640 1 -- start start 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7c5f0f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c0102d40 0x7fa7c0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7c5f0f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7c0103660 0x7fa7c01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7c5f0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7c5f0f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0111fb0 con 0x7fa7c010c170 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7c5f0f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0111e30 con 0x7fa7c0103660 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7c5f0f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa7c0112130 con 0x7fa7c0102d40 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c019cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44878/0 (socket says 172.21.15.37:44878) 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 -- 172.21.15.37:0/460265024 learned_addr learned my addr 172.21.15.37:0/460265024 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bf7fe640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c0102d40 0x7fa7c0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 -- 172.21.15.37:0/460265024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c0102d40 msgr2=0x7fa7c0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:59.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c0102d40 0x7fa7c0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 -- 172.21.15.37:0/460265024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7c0103660 msgr2=0x7fa7c01989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7b6ffd640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7c0103660 0x7fa7c01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7c0103660 0x7fa7c01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 -- 172.21.15.37:0/460265024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7c019d420 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bffff640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c019cd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8006010 tx=0x7fa7b802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bd7fa640 1 -- 172.21.15.37:0/460265024 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8034040 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bd7fa640 1 -- 172.21.15.37:0/460265024 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa7b8003e00 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7c019d6b0 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.066+0000 7fa7bd7fa640 1 -- 172.21.15.37:0/460265024 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7b80456e0 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.067+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7c01a4ff0 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.068+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7c0103120 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.069+0000 7fa7bd7fa640 1 -- 172.21.15.37:0/460265024 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa7b8037020 con 0x7fa7c010c170 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.070+0000 7fa7bd7fa640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa7ac0a32d0 0x7fa7ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:40:59.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.070+0000 7fa7bf7fe640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa7ac0a32d0 0x7fa7ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:40:59.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.070+0000 7fa7bd7fa640 1 -- 172.21.15.37:0/460265024 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa7b80f0b80 con 0x7fa7c010c170 2024-09-19T06:40:59.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.070+0000 7fa7bf7fe640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa7ac0a32d0 0x7fa7ac0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009890 tx=0x7fa7b0008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:40:59.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.074+0000 7fa7bd7fa640 1 -- 172.21.15.37:0/460265024 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa7b80bd7d0 con 0x7fa7c010c170 2024-09-19T06:40:59.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.289+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fa7c01a5220 con 0x7fa7c010c170 2024-09-19T06:40:59.291 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:59 smithi037 ceph-mon[31211]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:59.291 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:40:59 smithi037 ceph-mon[31211]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:40:59.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.289+0000 7fa7bd7fa640 1 -- 172.21.15.37:0/460265024 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+180 (secure 0 0 0) 0x7fa7b80bd0d0 con 0x7fa7c010c170 2024-09-19T06:40:59.291 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:40:59.292 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false}},"mutes":[]} 2024-09-19T06:40:59.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa7ac0a32d0 msgr2=0x7fa7ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:59.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa7ac0a32d0 0x7fa7ac0a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa7b0009890 tx=0x7fa7b0008040 comp rx=0 tx=0).stop 2024-09-19T06:40:59.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 msgr2=0x7fa7c019cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:40:59.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c019cd40 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8006010 tx=0x7fa7b802f710 comp rx=0 tx=0).stop 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 shutdown_connections 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa7ac0a32d0 0x7fa7ac0a5790 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa7c010c170 0x7fa7c019cd40 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa7c0103660 0x7fa7c01989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 --2- 172.21.15.37:0/460265024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa7c0102d40 0x7fa7c0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.291+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 >> 172.21.15.37:0/460265024 conn(0x7fa7c00fc7a0 msgr2=0x7fa7c00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.292+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 shutdown_connections 2024-09-19T06:40:59.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:40:59.292+0000 7fa7c5f0f640 1 -- 172.21.15.37:0/460265024 wait complete. 2024-09-19T06:40:59.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:59 smithi096 ceph-mon[35004]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:59.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:40:59 smithi096 ceph-mon[35004]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:40:59.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:59 smithi151 ceph-mon[35236]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:40:59.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:40:59 smithi151 ceph-mon[35236]: osdmap e83: 12 total, 12 up, 12 in 2024-09-19T06:41:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:00 smithi096 ceph-mon[35004]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-19T06:41:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:00 smithi096 ceph-mon[35004]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T06:41:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:00 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/460265024' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:00 smithi037 ceph-mon[31211]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-19T06:41:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:00 smithi037 ceph-mon[31211]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T06:41:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:00 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/460265024' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:00.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:00 smithi151 ceph-mon[35236]: pgmap v314: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s 2024-09-19T06:41:00.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:00 smithi151 ceph-mon[35236]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-19T06:41:00.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:00 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/460265024' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:02.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:02 smithi096 ceph-mon[35004]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-09-19T06:41:02.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:02 smithi037 ceph-mon[31211]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-09-19T06:41:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:02 smithi151 ceph-mon[35236]: pgmap v315: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s; 72 B/s, 0 objects/s recovering 2024-09-19T06:41:02.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T06:41:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:03.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 -- 172.21.15.37:0/4273823925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c34102d30 msgr2=0x7f7c34103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 --2- 172.21.15.37:0/4273823925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c34102d30 0x7f7c34103110 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7c24009cd0 tx=0x7f7c2402a910 comp rx=0 tx=0).stop 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 -- 172.21.15.37:0/4273823925 shutdown_connections 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 --2- 172.21.15.37:0/4273823925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c341082a0 0x7f7c3410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 --2- 172.21.15.37:0/4273823925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c341036e0 0x7f7c34107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 --2- 172.21.15.37:0/4273823925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c34102d30 0x7f7c34103110 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 -- 172.21.15.37:0/4273823925 >> 172.21.15.37:0/4273823925 conn(0x7f7c340fc7d0 msgr2=0x7f7c340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 -- 172.21.15.37:0/4273823925 shutdown_connections 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 -- 172.21.15.37:0/4273823925 wait complete. 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.660+0000 7f7c39f79640 1 Processor -- start 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c39f79640 1 -- start start 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c39f79640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c34102d30 0x7f7c34198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c39f79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c341036e0 0x7f7c34198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:03.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c39f79640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c341082a0 0x7f7c3419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c39f79640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c341120e0 con 0x7f7c341036e0 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c39f79640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34111f60 con 0x7f7c341082a0 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c39f79640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c34112260 con 0x7f7c34102d30 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c341036e0 0x7f7c34198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c341036e0 0x7f7c34198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58590/0 (socket says 172.21.15.37:58590) 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 -- 172.21.15.37:0/3361304265 learned_addr learned my addr 172.21.15.37:0/3361304265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c337fe640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c34102d30 0x7f7c34198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c33fff640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c341082a0 0x7f7c3419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 -- 172.21.15.37:0/3361304265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c34102d30 msgr2=0x7f7c34198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c34102d30 0x7f7c34198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 -- 172.21.15.37:0/3361304265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c341082a0 msgr2=0x7f7c3419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c341082a0 0x7f7c3419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 -- 172.21.15.37:0/3361304265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c3419d5d0 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c33fff640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c341082a0 0x7f7c3419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.661+0000 7f7c32ffd640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c341036e0 0x7f7c34198b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c28006fd0 tx=0x7f7c2800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.662+0000 7f7c30ff9640 1 -- 172.21.15.37:0/3361304265 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c28011830 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.662+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c3419d8c0 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.662+0000 7f7c30ff9640 1 -- 172.21.15.37:0/3361304265 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c28016070 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.662+0000 7f7c30ff9640 1 -- 172.21.15.37:0/3361304265 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800f2e0 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.662+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c341a51a0 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.663+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c3406a850 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.664+0000 7f7c30ff9640 1 -- 172.21.15.37:0/3361304265 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7c2801b460 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.665+0000 7f7c30ff9640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7c1c0a32d0 0x7f7c1c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.665+0000 7f7c30ff9640 1 -- 172.21.15.37:0/3361304265 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7c280cf700 con 0x7f7c341036e0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.665+0000 7f7c337fe640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7c1c0a32d0 0x7f7c1c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:03.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.665+0000 7f7c337fe640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7c1c0a32d0 0x7f7c1c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c240098d0 tx=0x7f7c24009900 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:03.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.668+0000 7f7c30ff9640 1 -- 172.21.15.37:0/3361304265 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c2809c3b0 con 0x7f7c341036e0 2024-09-19T06:41:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.884+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f7c34103110 con 0x7f7c341036e0 2024-09-19T06:41:03.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.884+0000 7f7c30ff9640 1 -- 172.21.15.37:0/3361304265 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+180 (secure 0 0 0) 0x7f7c34103110 con 0x7f7c341036e0 2024-09-19T06:41:03.886 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:03.886 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_WARN","checks":{"PG_AVAILABILITY":{"severity":"HEALTH_WARN","summary":{"message":"Reduced data availability: 1 pg peering","count":1},"muted":false}},"mutes":[]} 2024-09-19T06:41:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7c1c0a32d0 msgr2=0x7f7c1c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7c1c0a32d0 0x7f7c1c0a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c240098d0 tx=0x7f7c24009900 comp rx=0 tx=0).stop 2024-09-19T06:41:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c341036e0 msgr2=0x7f7c34198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:03.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c341036e0 0x7f7c34198b60 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7c28006fd0 tx=0x7f7c2800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 shutdown_connections 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7c1c0a32d0 0x7f7c1c0a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c341082a0 0x7f7c3419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c341036e0 0x7f7c34198b60 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 --2- 172.21.15.37:0/3361304265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c34102d30 0x7f7c34198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 >> 172.21.15.37:0/3361304265 conn(0x7f7c340fc7d0 msgr2=0x7f7c34105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 shutdown_connections 2024-09-19T06:41:03.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:03.886+0000 7f7c39f79640 1 -- 172.21.15.37:0/3361304265 wait complete. 2024-09-19T06:41:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:04 smithi037 ceph-mon[31211]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:04 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3361304265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:04 smithi096 ceph-mon[35004]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:04 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3361304265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:04 smithi151 ceph-mon[35236]: pgmap v316: 97 pgs: 1 peering, 96 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:04.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:04 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3361304265' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:05.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:05 smithi096 ceph-mon[35004]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:05.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:05 smithi096 ceph-mon[35004]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T06:41:05.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:05 smithi096 ceph-mon[35004]: Cluster is now healthy 2024-09-19T06:41:05.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:05 smithi037 ceph-mon[31211]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:05.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:05 smithi037 ceph-mon[31211]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T06:41:05.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:05 smithi037 ceph-mon[31211]: Cluster is now healthy 2024-09-19T06:41:05.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:05 smithi151 ceph-mon[35236]: pgmap v317: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 5 op/s; 75 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:05.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:05 smithi151 ceph-mon[35236]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-19T06:41:05.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:05 smithi151 ceph-mon[35236]: Cluster is now healthy 2024-09-19T06:41:07.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:07 smithi096 ceph-mon[35004]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 66 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:07.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:07 smithi037 ceph-mon[31211]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 66 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:07.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:07 smithi151 ceph-mon[35236]: pgmap v318: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 6 op/s; 66 B/s, 1 keys/s, 0 objects/s recovering 2024-09-19T06:41:07.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-19T06:41:07.894 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:08.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 -- 172.21.15.37:0/1243001461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28102380 msgr2=0x7f2d2810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:08.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1243001461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28102380 0x7f2d2810e9c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2d20009cd0 tx=0x7f2d2002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 -- 172.21.15.37:0/1243001461 shutdown_connections 2024-09-19T06:41:08.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1243001461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28102380 0x7f2d2810e9c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1243001461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281019e0 0x7f2d28101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1243001461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d28107af0 0x7f2d28107ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 -- 172.21.15.37:0/1243001461 >> 172.21.15.37:0/1243001461 conn(0x7f2d280fd700 msgr2=0x7f2d280ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 -- 172.21.15.37:0/1243001461 shutdown_connections 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.352+0000 7f2d2759e640 1 -- 172.21.15.37:0/1243001461 wait complete. 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 Processor -- start 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 -- start start 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281019e0 0x7f2d281983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d28102380 0x7f2d28198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28107af0 0x7f2d2819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28111f60 con 0x7f2d28107af0 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28111de0 con 0x7f2d281019e0 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2759e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2d281120e0 con 0x7f2d28102380 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28107af0 0x7f2d2819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:08.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28107af0 0x7f2d2819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58604/0 (socket says 172.21.15.37:58604) 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 -- 172.21.15.37:0/1823916961 learned_addr learned my addr 172.21.15.37:0/1823916961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d2659c640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281019e0 0x7f2d281983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d25d9b640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d28102380 0x7f2d28198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 -- 172.21.15.37:0/1823916961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d28102380 msgr2=0x7f2d28198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d28102380 0x7f2d28198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 -- 172.21.15.37:0/1823916961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281019e0 msgr2=0x7f2d281983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281019e0 0x7f2d281983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.353+0000 7f2d26d9d640 1 -- 172.21.15.37:0/1823916961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d2819d390 con 0x7f2d28107af0 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.354+0000 7f2d26d9d640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28107af0 0x7f2d2819ccb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d20033040 tx=0x7f2d20003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.354+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1823916961 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20004040 con 0x7f2d28107af0 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.354+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d2819d620 con 0x7f2d28107af0 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.354+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1823916961 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2d200046c0 con 0x7f2d28107af0 2024-09-19T06:41:08.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.354+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1823916961 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d20030eb0 con 0x7f2d28107af0 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.354+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281a4f60 con 0x7f2d28107af0 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.355+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d28101e40 con 0x7f2d28107af0 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.356+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1823916961 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f2d200041e0 con 0x7f2d28107af0 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.357+0000 7f2d1f7fe640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2d140a32d0 0x7f2d140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.357+0000 7f2d2659c640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2d140a32d0 0x7f2d140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.357+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1823916961 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f2d2002e030 con 0x7f2d28107af0 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.358+0000 7f2d2659c640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2d140a32d0 0x7f2d140a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009990 tx=0x7f2d18009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:08.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.360+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1823916961 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2d200bdc70 con 0x7f2d28107af0 2024-09-19T06:41:08.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.574+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2d28199740 con 0x7f2d28107af0 2024-09-19T06:41:08.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.574+0000 7f2d1f7fe640 1 -- 172.21.15.37:0/1823916961 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2d28199740 con 0x7f2d28107af0 2024-09-19T06:41:08.576 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:08.576 INFO:teuthology.orchestra.run.smithi037.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-19T06:41:08.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.576+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2d140a32d0 msgr2=0x7f2d140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.576+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2d140a32d0 0x7f2d140a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009990 tx=0x7f2d18009110 comp rx=0 tx=0).stop 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.576+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28107af0 msgr2=0x7f2d2819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.576+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28107af0 0x7f2d2819ccb0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2d20033040 tx=0x7f2d20003970 comp rx=0 tx=0).stop 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 shutdown_connections 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f2d140a32d0 0x7f2d140a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2d28107af0 0x7f2d2819ccb0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2d28102380 0x7f2d28198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 --2- 172.21.15.37:0/1823916961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281019e0 0x7f2d281983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 >> 172.21.15.37:0/1823916961 conn(0x7f2d280fd700 msgr2=0x7f2d280fdd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 shutdown_connections 2024-09-19T06:41:08.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:08.577+0000 7f2d2759e640 1 -- 172.21.15.37:0/1823916961 wait complete. 2024-09-19T06:41:09.215 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-19T06:41:09.216 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:41:09.361 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:09 smithi037 ceph-mon[31211]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 59 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:09.362 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:09 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1823916961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:09.476 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:09.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:09 smithi096 ceph-mon[35004]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 59 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:09.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:09 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1823916961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:09.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:09 smithi151 ceph-mon[35236]: pgmap v319: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s; 59 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:09.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:09 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1823916961' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 -- 172.21.15.37:0/1102306239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b81008d0 msgr2=0x7fb4b8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1102306239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b81008d0 0x7fb4b8100cb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009900 tx=0x7fb4b402a410 comp rx=0 tx=0).stop 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 -- 172.21.15.37:0/1102306239 shutdown_connections 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1102306239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810e480 0x7fb4b8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1102306239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b8101280 0x7fb4b810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1102306239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b81008d0 0x7fb4b8100cb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 -- 172.21.15.37:0/1102306239 >> 172.21.15.37:0/1102306239 conn(0x7fb4b80fc7a0 msgr2=0x7fb4b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 -- 172.21.15.37:0/1102306239 shutdown_connections 2024-09-19T06:41:09.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.920+0000 7fb4c0398640 1 -- 172.21.15.37:0/1102306239 wait complete. 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 Processor -- start 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 -- start start 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81008d0 0x7fb4b819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b8101280 0x7fb4b819abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810e480 0x7fb4b819ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8114190 con 0x7fb4b8101280 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8114010 con 0x7fb4b81008d0 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4c0398640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb4b8114310 con 0x7fb4b810e480 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4bd933640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b8101280 0x7fb4b819abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4bd933640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b8101280 0x7fb4b819abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53312/0 (socket says 172.21.15.37:53312) 2024-09-19T06:41:09.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4bd933640 1 -- 172.21.15.37:0/1262619557 learned_addr learned my addr 172.21.15.37:0/1262619557 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4be134640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81008d0 0x7fb4b819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4be935640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810e480 0x7fb4b819ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4bd933640 1 -- 172.21.15.37:0/1262619557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810e480 msgr2=0x7fb4b819ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4bd933640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810e480 0x7fb4b819ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4bd933640 1 -- 172.21.15.37:0/1262619557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81008d0 msgr2=0x7fb4b819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.921+0000 7fb4bd933640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81008d0 0x7fb4b819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.922+0000 7fb4bd933640 1 -- 172.21.15.37:0/1262619557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4b819f620 con 0x7fb4b8101280 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.922+0000 7fb4bd933640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b8101280 0x7fb4b819abb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c890 tx=0x7fb4a800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.922+0000 7fb4af7fe640 1 -- 172.21.15.37:0/1262619557 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8004220 con 0x7fb4b8101280 2024-09-19T06:41:09.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.922+0000 7fb4af7fe640 1 -- 172.21.15.37:0/1262619557 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb4a80043c0 con 0x7fb4b8101280 2024-09-19T06:41:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.922+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4b819f910 con 0x7fb4b8101280 2024-09-19T06:41:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.922+0000 7fb4af7fe640 1 -- 172.21.15.37:0/1262619557 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a80049e0 con 0x7fb4b8101280 2024-09-19T06:41:09.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.922+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4b81a71f0 con 0x7fb4b8101280 2024-09-19T06:41:09.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.923+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4b8100cb0 con 0x7fb4b8101280 2024-09-19T06:41:09.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.924+0000 7fb4af7fe640 1 -- 172.21.15.37:0/1262619557 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fb4a80079b0 con 0x7fb4b8101280 2024-09-19T06:41:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.925+0000 7fb4af7fe640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb4a40a3350 0x7fb4a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.925+0000 7fb4be134640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb4a40a3350 0x7fb4a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.925+0000 7fb4af7fe640 1 -- 172.21.15.37:0/1262619557 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb4a80cfa20 con 0x7fb4b8101280 2024-09-19T06:41:09.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.925+0000 7fb4be134640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb4a40a3350 0x7fb4a40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009230 tx=0x7fb4b40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:09.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:09.928+0000 7fb4af7fe640 1 -- 172.21.15.37:0/1262619557 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb4a8014030 con 0x7fb4b8101280 2024-09-19T06:41:10.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.140+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4b819b9d0 con 0x7fb4b8101280 2024-09-19T06:41:10.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.141+0000 7fb4af7fe640 1 -- 172.21.15.37:0/1262619557 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11903 (secure 0 0 0) 0x7fb4b819b9d0 con 0x7fb4b8101280 2024-09-19T06:41:10.143 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:10.143 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"btime":"2024-09-19T06:40:04:443772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:40:04.075579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:41:10.143 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb4a40a3350 msgr2=0x7fb4a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb4a40a3350 0x7fb4a40a5810 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb4b4009230 tx=0x7fb4b40058d0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b8101280 msgr2=0x7fb4b819abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b8101280 0x7fb4b819abb0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c890 tx=0x7fb4a800cd50 comp rx=0 tx=0).stop 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 shutdown_connections 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb4a40a3350 0x7fb4a40a5810 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb4b810e480 0x7fb4b819ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb4b8101280 0x7fb4b819abb0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 --2- 172.21.15.37:0/1262619557 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4b81008d0 0x7fb4b819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:10.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 >> 172.21.15.37:0/1262619557 conn(0x7fb4b80fc7a0 msgr2=0x7fb4b80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:10.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 shutdown_connections 2024-09-19T06:41:10.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:10.143+0000 7fb4c0398640 1 -- 172.21.15.37:0/1262619557 wait complete. 2024-09-19T06:41:10.512 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:10 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1262619557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:10 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1262619557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:10 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1262619557' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:10.860 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:41:11.118 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:11.493 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:11 smithi037 ceph-mon[31211]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 -- 172.21.15.37:0/1388677138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 msgr2=0x7fcf381029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 --2- 172.21.15.37:0/1388677138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381029b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009900 tx=0x7fcf2802a480 comp rx=0 tx=0).stop 2024-09-19T06:41:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 -- 172.21.15.37:0/1388677138 shutdown_connections 2024-09-19T06:41:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 --2- 172.21.15.37:0/1388677138 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf38102ef0 0x7fcf3810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 --2- 172.21.15.37:0/1388677138 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381029b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 --2- 172.21.15.37:0/1388677138 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf38100e20 0x7fcf38101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 -- 172.21.15.37:0/1388677138 >> 172.21.15.37:0/1388677138 conn(0x7fcf380fc7d0 msgr2=0x7fcf380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 -- 172.21.15.37:0/1388677138 shutdown_connections 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 -- 172.21.15.37:0/1388677138 wait complete. 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 Processor -- start 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.574+0000 7fcf40105640 1 -- start start 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf40105640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf38100e20 0x7fcf38199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf40105640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf40105640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf38102ef0 0x7fcf381a43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf40105640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38110fa0 con 0x7fcf38102570 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf40105640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38110e20 con 0x7fcf38102ef0 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf40105640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf38111120 con 0x7fcf38100e20 2024-09-19T06:41:11.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53338/0 (socket says 172.21.15.37:53338) 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 -- 172.21.15.37:0/1167637530 learned_addr learned my addr 172.21.15.37:0/1167637530 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3e6a2640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf38102ef0 0x7fcf381a43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3dea1640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf38100e20 0x7fcf38199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 -- 172.21.15.37:0/1167637530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf38100e20 msgr2=0x7fcf38199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf38100e20 0x7fcf38199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 -- 172.21.15.37:0/1167637530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf38102ef0 msgr2=0x7fcf381a43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf38102ef0 0x7fcf381a43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 -- 172.21.15.37:0/1167637530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf381a4930 con 0x7fcf38102570 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf3d6a0640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381998b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006010 tx=0x7fcf28003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf2effd640 1 -- 172.21.15.37:0/1167637530 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf2803f070 con 0x7fcf38102570 2024-09-19T06:41:11.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf381a4b00 con 0x7fcf38102570 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf2effd640 1 -- 172.21.15.37:0/1167637530 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf28031e30 con 0x7fcf38102570 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.575+0000 7fcf2effd640 1 -- 172.21.15.37:0/1167637530 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28035030 con 0x7fcf38102570 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.576+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf381a4f60 con 0x7fcf38102570 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.576+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf38101200 con 0x7fcf38102570 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.578+0000 7fcf2effd640 1 -- 172.21.15.37:0/1167637530 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcf28042070 con 0x7fcf38102570 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.578+0000 7fcf2effd640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf240a32d0 0x7fcf240a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.579+0000 7fcf2effd640 1 -- 172.21.15.37:0/1167637530 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcf280f0a00 con 0x7fcf38102570 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.579+0000 7fcf3dea1640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf240a32d0 0x7fcf240a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.579+0000 7fcf3dea1640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf240a32d0 0x7fcf240a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009890 tx=0x7fcf34008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:11.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.582+0000 7fcf2effd640 1 -- 172.21.15.37:0/1167637530 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcf280bd630 con 0x7fcf38102570 2024-09-19T06:41:11.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:11 smithi151 ceph-mon[35236]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:11.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:11 smithi096 ceph-mon[35004]: pgmap v320: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 50 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:11.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.797+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf381a5190 con 0x7fcf38102570 2024-09-19T06:41:11.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.798+0000 7fcf2effd640 1 -- 172.21.15.37:0/1167637530 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11903 (secure 0 0 0) 0x7fcf280bcf30 con 0x7fcf38102570 2024-09-19T06:41:11.799 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:11.799 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"btime":"2024-09-19T06:40:04:443772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:40:04.075579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:41:11.799 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-19T06:41:11.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf240a32d0 msgr2=0x7fcf240a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:11.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf240a32d0 0x7fcf240a5790 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009890 tx=0x7fcf34008040 comp rx=0 tx=0).stop 2024-09-19T06:41:11.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 msgr2=0x7fcf381998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381998b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcf28006010 tx=0x7fcf28003c80 comp rx=0 tx=0).stop 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 shutdown_connections 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcf240a32d0 0x7fcf240a5790 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf38102ef0 0x7fcf381a43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf38102570 0x7fcf381998b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 --2- 172.21.15.37:0/1167637530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf38100e20 0x7fcf38199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.800+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 >> 172.21.15.37:0/1167637530 conn(0x7fcf380fc7d0 msgr2=0x7fcf381068a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.801+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 shutdown_connections 2024-09-19T06:41:11.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:11.801+0000 7fcf40105640 1 -- 172.21.15.37:0/1167637530 wait complete. 2024-09-19T06:41:12.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:12 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1167637530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:12.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:41:12.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:12 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1167637530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:12 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1167637530' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:12.789 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 -- 172.21.15.37:0/942516035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 msgr2=0x7f88a81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 --2- 172.21.15.37:0/942516035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a81021c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8898009900 tx=0x7f889802a410 comp rx=0 tx=0).stop 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 -- 172.21.15.37:0/942516035 shutdown_connections 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 --2- 172.21.15.37:0/942516035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8106f10 0x7f88a810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 --2- 172.21.15.37:0/942516035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88a8102790 0x7f88a81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 --2- 172.21.15.37:0/942516035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a81021c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 -- 172.21.15.37:0/942516035 >> 172.21.15.37:0/942516035 conn(0x7f88a80fd8c0 msgr2=0x7f88a80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 -- 172.21.15.37:0/942516035 shutdown_connections 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.208+0000 7f88adbb7640 1 -- 172.21.15.37:0/942516035 wait complete. 2024-09-19T06:41:13.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.209+0000 7f88adbb7640 1 Processor -- start 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.209+0000 7f88adbb7640 1 -- start start 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.209+0000 7f88adbb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.209+0000 7f88adbb7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8102790 0x7f88a8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.209+0000 7f88adbb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88a8106f10 0x7f88a8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88adbb7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8111f90 con 0x7f88a8101de0 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88adbb7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8111e10 con 0x7f88a8106f10 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88adbb7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88a8112110 con 0x7f88a8102790 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53364/0 (socket says 172.21.15.37:53364) 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 -- 172.21.15.37:0/3793590609 learned_addr learned my addr 172.21.15.37:0/3793590609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a7fff640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88a8106f10 0x7f88a8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a6ffd640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8102790 0x7f88a8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:13.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 -- 172.21.15.37:0/3793590609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8102790 msgr2=0x7f88a8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8102790 0x7f88a8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 -- 172.21.15.37:0/3793590609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88a8106f10 msgr2=0x7f88a8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88a8106f10 0x7f88a8074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 -- 172.21.15.37:0/3793590609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a81a6950 con 0x7f88a8101de0 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a6ffd640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8102790 0x7f88a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a77fe640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a80744f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8898006010 tx=0x7f88980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a4ff9640 1 -- 172.21.15.37:0/3793590609 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f889803f030 con 0x7f88a8101de0 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a81a6be0 con 0x7f88a8101de0 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a4ff9640 1 -- 172.21.15.37:0/3793590609 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8898003ce0 con 0x7f88a8101de0 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.210+0000 7f88a4ff9640 1 -- 172.21.15.37:0/3793590609 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8898035070 con 0x7f88a8101de0 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.211+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a81a70c0 con 0x7f88a8101de0 2024-09-19T06:41:13.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.211+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a81021c0 con 0x7f88a8101de0 2024-09-19T06:41:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.213+0000 7f88a4ff9640 1 -- 172.21.15.37:0/3793590609 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f8898042030 con 0x7f88a8101de0 2024-09-19T06:41:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.213+0000 7f88a4ff9640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f88900a32d0 0x7f88900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.213+0000 7f88a6ffd640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f88900a32d0 0x7f88900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.213+0000 7f88a4ff9640 1 -- 172.21.15.37:0/3793590609 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f88980effe0 con 0x7f88a8101de0 2024-09-19T06:41:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.214+0000 7f88a6ffd640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f88900a32d0 0x7f88900a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f88a8075f80 tx=0x7f889c005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:13.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.217+0000 7f88a4ff9640 1 -- 172.21.15.37:0/3793590609 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f88980bcce0 con 0x7f88a8101de0 2024-09-19T06:41:13.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.397+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88a8109b60 con 0x7f88a8101de0 2024-09-19T06:41:13.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.399+0000 7f88a4ff9640 1 -- 172.21.15.37:0/3793590609 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22795 (secure 0 0 0) 0x7f88980bc5e0 con 0x7f88a8101de0 2024-09-19T06:41:13.400 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:13.400 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:40:58.113982+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:41:13.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f88900a32d0 msgr2=0x7f88900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:13.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f88900a32d0 0x7f88900a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f88a8075f80 tx=0x7f889c005e50 comp rx=0 tx=0).stop 2024-09-19T06:41:13.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 msgr2=0x7f88a80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a80744f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8898006010 tx=0x7f88980042f0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 shutdown_connections 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f88900a32d0 0x7f88900a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88a8106f10 0x7f88a8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f88a8102790 0x7f88a8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 --2- 172.21.15.37:0/3793590609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f88a8101de0 0x7f88a80744f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 >> 172.21.15.37:0/3793590609 conn(0x7f88a80fd8c0 msgr2=0x7f88a80fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 shutdown_connections 2024-09-19T06:41:13.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:13.401+0000 7f88adbb7640 1 -- 172.21.15.37:0/3793590609 wait complete. 2024-09-19T06:41:13.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:13 smithi037 ceph-mon[31211]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:13.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:13 smithi151 ceph-mon[35236]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:13.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:13 smithi096 ceph-mon[35004]: pgmap v321: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-09-19T06:41:14.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:41:14.287 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:14 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3793590609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:14.327 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:14.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:14 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3793590609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:14.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 -- 172.21.15.37:0/3670922803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c810c170 msgr2=0x7f29c810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:14.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 --2- 172.21.15.37:0/3670922803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c810c170 0x7f29c810e610 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f29c0009ca0 tx=0x7f29c002a950 comp rx=0 tx=0).stop 2024-09-19T06:41:14.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 -- 172.21.15.37:0/3670922803 shutdown_connections 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 --2- 172.21.15.37:0/3670922803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c810c170 0x7f29c810e610 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 --2- 172.21.15.37:0/3670922803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29c8103660 0x7f29c810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 --2- 172.21.15.37:0/3670922803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29c8102d40 0x7f29c8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 -- 172.21.15.37:0/3670922803 >> 172.21.15.37:0/3670922803 conn(0x7f29c80fc7a0 msgr2=0x7f29c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 -- 172.21.15.37:0/3670922803 shutdown_connections 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.733+0000 7f29cd04a640 1 -- 172.21.15.37:0/3670922803 wait complete. 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.734+0000 7f29cd04a640 1 Processor -- start 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.734+0000 7f29cd04a640 1 -- start start 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29cd04a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c8102d40 0x7f29c8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29cd04a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29c8103660 0x7f29c8198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29cd04a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29c810c170 0x7f29c819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:14.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29cd04a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112080 con 0x7f29c8102d40 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29cd04a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8111f00 con 0x7f29c8103660 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29cd04a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f29c8112200 con 0x7f29c810c170 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c8102d40 0x7f29c8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c8102d40 0x7f29c8198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53376/0 (socket says 172.21.15.37:53376) 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 -- 172.21.15.37:0/2248616218 learned_addr learned my addr 172.21.15.37:0/2248616218 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c759e640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29c810c170 0x7f29c819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c659c640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29c8103660 0x7f29c8198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 -- 172.21.15.37:0/2248616218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29c810c170 msgr2=0x7f29c819ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29c810c170 0x7f29c819ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 -- 172.21.15.37:0/2248616218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29c8103660 msgr2=0x7f29c8198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29c8103660 0x7f29c8198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 -- 172.21.15.37:0/2248616218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29c819d510 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c759e640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29c810c170 0x7f29c819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29c6d9d640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c8102d40 0x7f29c8198560 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29b800b4a0 tx=0x7f29b800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.735+0000 7f29affff640 1 -- 172.21.15.37:0/2248616218 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8014070 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.736+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29c819d800 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.736+0000 7f29affff640 1 -- 172.21.15.37:0/2248616218 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f29b8010af0 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.736+0000 7f29affff640 1 -- 172.21.15.37:0/2248616218 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29b8017d10 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.736+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29c81a50e0 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.737+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29c81031a0 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.738+0000 7f29affff640 1 -- 172.21.15.37:0/2248616218 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f29b8017070 con 0x7f29c8102d40 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.738+0000 7f29affff640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f29b00a32d0 0x7f29b00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:14.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.739+0000 7f29c659c640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f29b00a32d0 0x7f29b00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:14.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.739+0000 7f29affff640 1 -- 172.21.15.37:0/2248616218 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f29b80ceea0 con 0x7f29c8102d40 2024-09-19T06:41:14.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.739+0000 7f29c659c640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f29b00a32d0 0x7f29b00a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f29c80fed30 tx=0x7f29bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:14.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.743+0000 7f29affff640 1 -- 172.21.15.37:0/2248616218 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f29b809bb50 con 0x7f29c8102d40 2024-09-19T06:41:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:14 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3793590609' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.955+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29c8107240 con 0x7f29c8102d40 2024-09-19T06:41:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.956+0000 7f29affff640 1 -- 172.21.15.37:0/2248616218 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11903 (secure 0 0 0) 0x7f29b809b450 con 0x7f29c8102d40 2024-09-19T06:41:14.957 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:14.958 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"btime":"2024-09-19T06:40:04:443772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:40:04.075579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:41:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-19T06:41:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f29b00a32d0 msgr2=0x7f29b00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f29b00a32d0 0x7f29b00a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f29c80fed30 tx=0x7f29bc008040 comp rx=0 tx=0).stop 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c8102d40 msgr2=0x7f29c8198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c8102d40 0x7f29c8198560 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f29b800b4a0 tx=0x7f29b800b960 comp rx=0 tx=0).stop 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 shutdown_connections 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f29b00a32d0 0x7f29b00a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f29c810c170 0x7f29c819ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29c8103660 0x7f29c8198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 --2- 172.21.15.37:0/2248616218 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f29c8102d40 0x7f29c8198560 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.958+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 >> 172.21.15.37:0/2248616218 conn(0x7f29c80fc7a0 msgr2=0x7f29c80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.959+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 shutdown_connections 2024-09-19T06:41:14.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:14.959+0000 7f29cd04a640 1 -- 172.21.15.37:0/2248616218 wait complete. 2024-09-19T06:41:15.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:15 smithi151 ceph-mon[35236]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:41:15.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:15 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2248616218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:15.617 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:15 smithi037 ceph-mon[31211]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:41:15.617 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:15 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2248616218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:15.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:39:53.022194+0000', 'modified': '2024-09-19T06:40:04.075579+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24470, 'mds_2': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/672218237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 672218237}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 672218237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3741792814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3741792814}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3741792814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6839/3546164673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 3546164673}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 3546164673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.96:6835/946199433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 946199433}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 946199433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.96:6837/3655112770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 3655112770}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 3655112770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6839/1066552330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 1066552330}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 1066552330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24449, 24470, 24497]} 2024-09-19T06:41:15.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T06:41:15.647 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-19T06:41:15.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:15 smithi096 ceph-mon[35004]: pgmap v322: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:41:15.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:15 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2248616218' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:15.904 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:16.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.378+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/842660389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4101de0 msgr2=0x7fbfd41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.378+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/842660389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4101de0 0x7fbfd41021c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4009900 tx=0x7fbfc402a410 comp rx=0 tx=0).stop 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/842660389 shutdown_connections 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/842660389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfd4106f10 0x7fbfd410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/842660389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd4102790 0x7fbfd41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/842660389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4101de0 0x7fbfd41021c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/842660389 >> 172.21.15.37:0/842660389 conn(0x7fbfd40fd880 msgr2=0x7fbfd40ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/842660389 shutdown_connections 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/842660389 wait complete. 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 Processor -- start 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.379+0000 7fbfdaa5f640 1 -- start start 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfdaa5f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd4101de0 0x7fbfd4198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfdaa5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4102790 0x7fbfd4198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfdaa5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfd4106f10 0x7fbfd419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfdaa5f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4111fc0 con 0x7fbfd4102790 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfdaa5f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4111e40 con 0x7fbfd4101de0 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfdaa5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd4112140 con 0x7fbfd4106f10 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4102790 0x7fbfd4198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4102790 0x7fbfd4198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53396/0 (socket says 172.21.15.37:53396) 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 -- 172.21.15.37:0/472717822 learned_addr learned my addr 172.21.15.37:0/472717822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfd4106f10 0x7fbfd419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd4101de0 0x7fbfd4198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 -- 172.21.15.37:0/472717822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfd4106f10 msgr2=0x7fbfd419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfd4106f10 0x7fbfd419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 -- 172.21.15.37:0/472717822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd4101de0 msgr2=0x7fbfd4198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd4101de0 0x7fbfd4198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 -- 172.21.15.37:0/472717822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd419d4b0 con 0x7fbfd4102790 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4102790 0x7fbfd4198a40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800e9f0 tx=0x7fbfc800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd17fa640 1 -- 172.21.15.37:0/472717822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc800cd80 con 0x7fbfd4102790 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd17fa640 1 -- 172.21.15.37:0/472717822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfc8010910 con 0x7fbfd4102790 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd419d7a0 con 0x7fbfd4102790 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.380+0000 7fbfd17fa640 1 -- 172.21.15.37:0/472717822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc8018990 con 0x7fbfd4102790 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.381+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd41a5080 con 0x7fbfd4102790 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfd17fa640 1 -- 172.21.15.37:0/472717822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbfc8010ac0 con 0x7fbfd4102790 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfdce38740 1 Processor -- start 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfdce38740 1 -- start start 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdf81db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdf8c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf8efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfdce38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633bdee8090 con 0x5633bdf819d0 2024-09-19T06:41:16.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.383+0000 7fbfdce38740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5633bde8fa70 con 0x5633bdf8cbb0 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfdce38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5633bdee4050 con 0x5633bdf822f0 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdf81db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdf81db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53408/0 (socket says 172.21.15.37:53408) 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/3468279986 learned_addr learned my addr 172.21.15.37:0/3468279986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf8efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdf8c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/3468279986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 msgr2=0x5633bdf8c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdf8c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/3468279986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 msgr2=0x5633bdf8efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf8efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd17fa640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbfbc0a32d0 0x7fbfbc0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/3468279986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bde9c130 con 0x5633bdf819d0 2024-09-19T06:41:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdf8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbfbc0a32d0 0x7fbfbc0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd17fa640 1 -- 172.21.15.37:0/472717822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbfc80cff50 con 0x7fbfd4102790 2024-09-19T06:41:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdf81db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009e60 tx=0x7fbfcc010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18eba479c97d69c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfba7fc640 1 -- 172.21.15.37:0/3468279986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc016070 con 0x5633bdf819d0 2024-09-19T06:41:16.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfba7fc640 1 -- 172.21.15.37:0/3468279986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc004d20 con 0x5633bdf819d0 2024-09-19T06:41:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.384+0000 7fbfba7fc640 1 -- 172.21.15.37:0/3468279986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0127d0 con 0x5633bdf819d0 2024-09-19T06:41:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 -- 172.21.15.37:0/3468279986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 msgr2=0x5633bdf81db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbfbc0a32d0 0x7fbfbc0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4000c00 tx=0x7fbfc4005b90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:16.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdf81db0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc009e60 tx=0x7fbfcc010740 comp rx=0 tx=0).stop 2024-09-19T06:41:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 -- 172.21.15.37:0/3468279986 shutdown_connections 2024-09-19T06:41:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf8efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdf8c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3468279986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdf81db0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 -- 172.21.15.37:0/3468279986 >> 172.21.15.37:0/3468279986 conn(0x5633bdf7df60 msgr2=0x5633bdf7e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 -- 172.21.15.37:0/3468279986 shutdown_connections 2024-09-19T06:41:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.385+0000 7fbfdce38740 1 -- 172.21.15.37:0/3468279986 wait complete. 2024-09-19T06:41:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 -- start start 2024-09-19T06:41:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 Processor -- start 2024-09-19T06:41:16.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf8c490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf897e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf89d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf8f780 con 0x5633bdf8cbb0 2024-09-19T06:41:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5633bdea2f20 con 0x5633bdf822f0 2024-09-19T06:41:16.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf8f900 con 0x5633bdf819d0 2024-09-19T06:41:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfdce38740 10 client.0 _ll_register_callbacks cb 0x5633bde6c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfd8ffc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf89d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfd8ffc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf89d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53420/0 (socket says 172.21.15.37:53420) 2024-09-19T06:41:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/2040185702 learned_addr learned my addr 172.21.15.37:0/2040185702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:16.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf897e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.386+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf8c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/2040185702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 msgr2=0x5633bdf8c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf8c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 msgr2=0x5633bdf897e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf897e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/2040185702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bdc592b0 con 0x5633bdf8cbb0 2024-09-19T06:41:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf89d20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0059a0 tx=0x7fbfcc009a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af38721b08b78dba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfb37fe640 1 -- 172.21.15.37:0/2040185702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc029060 con 0x5633bdf8cbb0 2024-09-19T06:41:16.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfb37fe640 1 -- 172.21.15.37:0/2040185702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbfcc0050e0 con 0x5633bdf8cbb0 2024-09-19T06:41:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfcc004d20 con 0x5633bdf8cbb0 2024-09-19T06:41:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfb37fe640 1 -- 172.21.15.37:0/2040185702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbfcc016050 con 0x5633bdf8cbb0 2024-09-19T06:41:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfdce38740 10 client.14976 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5633bdf9b0c0 con 0x5633bdf8cbb0 2024-09-19T06:41:16.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.387+0000 7fbfb37fe640 1 -- 172.21.15.37:0/2040185702 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc004d20 con 0x5633bdf8cbb0 2024-09-19T06:41:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfb37fe640 1 -- 172.21.15.37:0/2040185702 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbfcc00f040 con 0x5633bdf8cbb0 2024-09-19T06:41:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfdce38740 10 client.14976 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfdce38740 10 client.14976 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:41:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfdce38740 20 client.14976 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:16.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfdce38740 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633bdf9d340 0x5633bdf9e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfdce38740 4 client.14976 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633bdef5600 con 0x5633bdf9d340 2024-09-19T06:41:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.388+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633bdf9d340 0x5633bdf9e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.389+0000 7fbfb37fe640 10 client.14976 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.389+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633bdf9d340 0x5633bdf9e7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8136d4d3e8c40fb8 server_cookie=f20f97e3f33f5e56 in_seq=0 out_seq=0 2024-09-19T06:41:16.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.389+0000 7fbfb37fe640 1 -- 172.21.15.37:0/2040185702 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5633bdef5600 con 0x5633bdf9d340 2024-09-19T06:41:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.389+0000 7fbfb37fe640 10 client.14976 handle_command_reply: tid=0 2024-09-19T06:41:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.389+0000 7fbfdce38740 1 client.14976 shutdown 2024-09-19T06:41:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 msgr2=0x5633bdf89d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf89d20 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0059a0 tx=0x7fbfcc009a60 comp rx=0 tx=0).stop 2024-09-19T06:41:16.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 shutdown_connections 2024-09-19T06:41:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633bdf9d340 0x5633bdf9e7b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf89d20 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf897e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfb37fe640 0 client.14976 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:16.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 --2- 172.21.15.37:0/2040185702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf8c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 >> 172.21.15.37:0/2040185702 conn(0x5633bdf7df60 msgr2=0x5633bdf83c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:16.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 shutdown_connections 2024-09-19T06:41:16.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 1 -- 172.21.15.37:0/2040185702 wait complete. 2024-09-19T06:41:16.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.390+0000 7fbfdce38740 20 client.14976 trim_cache size 0 max 16384 2024-09-19T06:41:16.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 Processor -- start 2024-09-19T06:41:16.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 -- start start 2024-09-19T06:41:16.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf9e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf89d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf83b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf9caf0 con 0x5633bdf8cbb0 2024-09-19T06:41:16.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf9d1c0 con 0x5633bdf822f0 2024-09-19T06:41:16.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfdce38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf9d890 con 0x5633bdf819d0 2024-09-19T06:41:16.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf83b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf83b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53434/0 (socket says 172.21.15.37:53434) 2024-09-19T06:41:16.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/876544397 learned_addr learned my addr 172.21.15.37:0/876544397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf89d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf9e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/876544397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 msgr2=0x5633bdf9e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf9e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/876544397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 msgr2=0x5633bdf89d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf89d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd8ffc640 1 -- 172.21.15.37:0/876544397 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633bdf8a290 con 0x5633bdf8cbb0 2024-09-19T06:41:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.408+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf89d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf83b30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc010f90 tx=0x7fbfcc005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c5f49511610553a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfb9ffb640 1 -- 172.21.15.37:0/876544397 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc004cf0 con 0x5633bdf8cbb0 2024-09-19T06:41:16.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfb9ffb640 1 -- 172.21.15.37:0/876544397 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc024e00 con 0x5633bdf8cbb0 2024-09-19T06:41:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfb9ffb640 1 -- 172.21.15.37:0/876544397 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfcc0051d0 con 0x5633bdf8cbb0 2024-09-19T06:41:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfdce38740 1 -- 172.21.15.37:0/876544397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 msgr2=0x5633bdf83b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfdce38740 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf83b30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc010f90 tx=0x7fbfcc005b00 comp rx=0 tx=0).stop 2024-09-19T06:41:16.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfdce38740 1 -- 172.21.15.37:0/876544397 shutdown_connections 2024-09-19T06:41:16.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfdce38740 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf8cbb0 0x5633bdf83b30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfdce38740 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf822f0 0x5633bdf89d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfdce38740 1 --2- 172.21.15.37:0/876544397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf819d0 0x5633bdf9e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.409+0000 7fbfdce38740 1 -- 172.21.15.37:0/876544397 >> 172.21.15.37:0/876544397 conn(0x5633bdf7df60 msgr2=0x5633bdf83420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:16.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 -- 172.21.15.37:0/876544397 shutdown_connections 2024-09-19T06:41:16.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 -- 172.21.15.37:0/876544397 wait complete. 2024-09-19T06:41:16.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 -- start start 2024-09-19T06:41:16.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 Processor -- start 2024-09-19T06:41:16.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdfa6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdfa7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf863b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf9e630 con 0x5633bdf819d0 2024-09-19T06:41:16.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf9df60 con 0x5633bdf8cbb0 2024-09-19T06:41:16.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5633bdf9ed00 con 0x5633bdf822f0 2024-09-19T06:41:16.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.410+0000 7fbfdce38740 10 client.0 _ll_register_callbacks cb 0x7fbfcc009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:16.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdfa6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdfa6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53442/0 (socket says 172.21.15.37:53442) 2024-09-19T06:41:16.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 -- 172.21.15.37:0/3726763538 learned_addr learned my addr 172.21.15.37:0/3726763538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:16.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdfa7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf863b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 -- 172.21.15.37:0/3726763538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 msgr2=0x5633bdfa7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdfa7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 -- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 msgr2=0x5633bdf863b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf863b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 -- 172.21.15.37:0/3726763538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633bdf86be0 con 0x5633bdf819d0 2024-09-19T06:41:16.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd8ffc640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfd3fff640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdfa6e10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40059d0 tx=0x7fbfc402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7edc97b98d9de2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfb37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfb37fe640 1 -- 172.21.15.37:0/3726763538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc4035050 con 0x5633bdf819d0 2024-09-19T06:41:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfb37fe640 1 -- 172.21.15.37:0/3726763538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbfc4005560 con 0x5633bdf819d0 2024-09-19T06:41:16.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfcc024e00 con 0x5633bdf819d0 2024-09-19T06:41:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfb37fe640 1 -- 172.21.15.37:0/3726763538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fbfc405e070 con 0x5633bdf819d0 2024-09-19T06:41:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfdce38740 10 client.14982 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.411+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5633bdfa5790 con 0x5633bdf819d0 2024-09-19T06:41:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfb37fe640 1 -- 172.21.15.37:0/3726763538 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbfcc024e00 con 0x5633bdf819d0 2024-09-19T06:41:16.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfb37fe640 1 -- 172.21.15.37:0/3726763538 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fbfc4032900 con 0x5633bdf819d0 2024-09-19T06:41:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfdce38740 10 client.14982 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfdce38740 10 client.14982 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:41:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfdce38740 20 client.14982 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633be00c310 0x5633be00b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:16.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfdce38740 4 client.14982 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:41:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "status"}) -- 0x5633bdeefb10 con 0x5633be00c310 2024-09-19T06:41:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.412+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633be00c310 0x5633be00b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfd37fe640 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633be00c310 0x5633be00b020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e0623e2b2249d9a server_cookie=881a32043f2cd84e in_seq=0 out_seq=0 2024-09-19T06:41:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfb37fe640 10 client.14982 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:16.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfb37fe640 1 -- 172.21.15.37:0/3726763538 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5633bdeefb10 con 0x5633be00c310 2024-09-19T06:41:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfb37fe640 10 client.14982 handle_command_reply: tid=0 2024-09-19T06:41:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 client.14982 shutdown 2024-09-19T06:41:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 msgr2=0x5633bdfa6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdfa6e10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fbfc40059d0 tx=0x7fbfc402a410 comp rx=0 tx=0).stop 2024-09-19T06:41:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 shutdown_connections 2024-09-19T06:41:16.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x5633be00c310 0x5633be00b020 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5633bdf8cbb0 0x5633bdf863b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5633bdf822f0 0x5633bdfa7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 --2- 172.21.15.37:0/3726763538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5633bdf819d0 0x5633bdfa6e10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfb37fe640 0 client.14982 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 >> 172.21.15.37:0/3726763538 conn(0x5633bdf7df60 msgr2=0x5633bdf83c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 shutdown_connections 2024-09-19T06:41:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 1 -- 172.21.15.37:0/3726763538 wait complete. 2024-09-19T06:41:16.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.413+0000 7fbfdce38740 20 client.14982 trim_cache size 0 max 16384 2024-09-19T06:41:16.444 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:41:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-19T06:41:16.444 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24449, 2024-09-19T06:41:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:41:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:41:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:41:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 74.974327627999998, 2024-09-19T06:41:16.445 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 24, 2024-09-19T06:41:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 83, 2024-09-19T06:41:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T06:41:16.446 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 113.386074804 2024-09-19T06:41:16.446 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:16.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbfbc0a32d0 msgr2=0x7fbfbc0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbfbc0a32d0 0x7fbfbc0a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fbfc4000c00 tx=0x7fbfc4005b90 comp rx=0 tx=0).stop 2024-09-19T06:41:16.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4102790 msgr2=0x7fbfd4198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:16.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4102790 0x7fbfd4198a40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800e9f0 tx=0x7fbfc800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 shutdown_connections 2024-09-19T06:41:16.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbfbc0a32d0 0x7fbfbc0a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbfd4106f10 0x7fbfd419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbfd4102790 0x7fbfd4198a40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 --2- 172.21.15.37:0/472717822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbfd4101de0 0x7fbfd4198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:16.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 >> 172.21.15.37:0/472717822 conn(0x7fbfd40fd880 msgr2=0x7fbfd40fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:16.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 shutdown_connections 2024-09-19T06:41:16.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:16.414+0000 7fbfdaa5f640 1 -- 172.21.15.37:0/472717822 wait complete. 2024-09-19T06:41:17.180 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:41:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:17.557 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:17 smithi037 ceph-mon[31211]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:17.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:17 smithi151 ceph-mon[35236]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:17 smithi096 ceph-mon[35004]: pgmap v323: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:17.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/1041841433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44106f10 msgr2=0x7fcd4410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/1041841433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44106f10 0x7fcd4410ea90 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fcd3c009ca0 tx=0x7fcd3c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/1041841433 shutdown_connections 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/1041841433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44106f10 0x7fcd4410ea90 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/1041841433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd44102790 0x7fcd441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/1041841433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd44101de0 0x7fcd441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/1041841433 >> 172.21.15.37:0/1041841433 conn(0x7fcd440fd8c0 msgr2=0x7fcd440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/1041841433 shutdown_connections 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/1041841433 wait complete. 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 Processor -- start 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.905+0000 7fcd4b1aa640 1 -- start start 2024-09-19T06:41:17.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd4b1aa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd44101de0 0x7fcd441983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd4b1aa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44102790 0x7fcd441988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd4b1aa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd44106f10 0x7fcd4419cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd4b1aa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44111f90 con 0x7fcd44102790 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd4b1aa640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44111e10 con 0x7fcd44106f10 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd4b1aa640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd44112110 con 0x7fcd44101de0 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44102790 0x7fcd441988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44102790 0x7fcd441988e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53460/0 (socket says 172.21.15.37:53460) 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 -- 172.21.15.37:0/695828287 learned_addr learned my addr 172.21.15.37:0/695828287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd49747640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd44106f10 0x7fcd4419cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd48f46640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd44101de0 0x7fcd441983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 -- 172.21.15.37:0/695828287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd44101de0 msgr2=0x7fcd441983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd44101de0 0x7fcd441983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 -- 172.21.15.37:0/695828287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd44106f10 msgr2=0x7fcd4419cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd44106f10 0x7fcd4419cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 -- 172.21.15.37:0/695828287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd4419d350 con 0x7fcd44102790 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.906+0000 7fcd43fff640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44102790 0x7fcd441988e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b4a0 tx=0x7fcd2c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.907+0000 7fcd41ffb640 1 -- 172.21.15.37:0/695828287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c015020 con 0x7fcd44102790 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.907+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd4419d640 con 0x7fcd44102790 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.907+0000 7fcd41ffb640 1 -- 172.21.15.37:0/695828287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2c011d10 con 0x7fcd44102790 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.907+0000 7fcd41ffb640 1 -- 172.21.15.37:0/695828287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c00f2e0 con 0x7fcd44102790 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.907+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd441a4f20 con 0x7fcd44102790 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.909+0000 7fcd41ffb640 1 -- 172.21.15.37:0/695828287 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fcd2c018020 con 0x7fcd44102790 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.909+0000 7fcd4d583740 1 Processor -- start 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.909+0000 7fcd4d583740 1 -- start start 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd41ffb640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcd300a32d0 0x7fcd300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d56c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d571b0 0x55da48d65a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 0x55da48d683b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd4d583740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da48cbd090 con 0x55da48d56890 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd4d583740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da48c64a70 con 0x55da48d571b0 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd4d583740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da48cb9050 con 0x55da48d65fc0 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d56c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d56c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53474/0 (socket says 172.21.15.37:53474) 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 -- 172.21.15.37:0/507501780 learned_addr learned my addr 172.21.15.37:0/507501780 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd49747640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcd300a32d0 0x7fcd300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd41ffb640 1 -- 172.21.15.37:0/695828287 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcd2c0cf2b0 con 0x7fcd44102790 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd49747640 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 0x55da48d683b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd43fff640 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d571b0 0x55da48d65a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 -- 172.21.15.37:0/507501780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 msgr2=0x55da48d683b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 0x55da48d683b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 -- 172.21.15.37:0/507501780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d571b0 msgr2=0x55da48d65a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d571b0 0x55da48d65a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 -- 172.21.15.37:0/507501780 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da48c71130 con 0x55da48d56890 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd48f46640 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d56c70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400ad00 tx=0x7fcd3400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7554a3913a2c7d28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd49747640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcd300a32d0 0x7fcd300a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55da48d54050 tx=0x7fcd3c0093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.910+0000 7fcd3affd640 1 -- 172.21.15.37:0/507501780 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3401d070 con 0x55da48d56890 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd3affd640 1 -- 172.21.15.37:0/507501780 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd34018b20 con 0x55da48d56890 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd3affd640 1 -- 172.21.15.37:0/507501780 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd34010be0 con 0x55da48d56890 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 -- 172.21.15.37:0/507501780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 msgr2=0x55da48d56c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d56c70 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcd3400ad00 tx=0x7fcd3400f790 comp rx=0 tx=0).stop 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 -- 172.21.15.37:0/507501780 shutdown_connections 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 0x55da48d683b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d571b0 0x55da48d65a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 --2- 172.21.15.37:0/507501780 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d56c70 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 -- 172.21.15.37:0/507501780 >> 172.21.15.37:0/507501780 conn(0x55da48d52f60 msgr2=0x55da48d53340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 -- 172.21.15.37:0/507501780 shutdown_connections 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.911+0000 7fcd4d583740 1 -- 172.21.15.37:0/507501780 wait complete. 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 -- start start 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 Processor -- start 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d658a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d571b0 0x55da48d62bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da48d68b90 con 0x55da48d56890 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da48c77f20 con 0x55da48d65fc0 2024-09-19T06:41:17.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da48d68d10 con 0x55da48d571b0 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd4d583740 10 client.0 _ll_register_callbacks cb 0x55da48c41030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd48f46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd48f46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53476/0 (socket says 172.21.15.37:53476) 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd48f46640 1 -- 172.21.15.37:0/983162986 learned_addr learned my addr 172.21.15.37:0/983162986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd43fff640 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d571b0 0x55da48d62bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.912+0000 7fcd49747640 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd48f46640 1 -- 172.21.15.37:0/983162986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d571b0 msgr2=0x55da48d62bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd48f46640 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d571b0 0x55da48d62bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd48f46640 1 -- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 msgr2=0x55da48d63130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd48f46640 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d63130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd48f46640 1 -- 172.21.15.37:0/983162986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da48a2e2b0 con 0x55da48d56890 2024-09-19T06:41:17.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd48f46640 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d658a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd3401b040 tx=0x7fcd34010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec5514f9da84200 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd23fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd23fff640 1 -- 172.21.15.37:0/983162986 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd3401d040 con 0x55da48d56890 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd34018b20 con 0x55da48d56890 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd23fff640 1 -- 172.21.15.37:0/983162986 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcd34018440 con 0x55da48d56890 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd23fff640 1 -- 172.21.15.37:0/983162986 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcd34018640 con 0x55da48d56890 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd4d583740 10 client.14994 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55da48d74540 con 0x55da48d56890 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.913+0000 7fcd23fff640 1 -- 172.21.15.37:0/983162986 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd34018b20 con 0x55da48d56890 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd23fff640 1 -- 172.21.15.37:0/983162986 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fcd34020030 con 0x55da48d56890 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd4d583740 10 client.14994 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd4d583740 10 client.14994 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd4d583740 20 client.14994 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:17.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd4d583740 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48d76770 0x55da48d77be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd4d583740 4 client.14994 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da48cca600 con 0x55da48d76770 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd43fff640 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48d76770 0x55da48d77be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd23fff640 10 client.14994 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.914+0000 7fcd43fff640 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48d76770 0x55da48d77be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=347ebea308313d43 server_cookie=65b69cd3d90b4036 in_seq=0 out_seq=0 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd23fff640 1 -- 172.21.15.37:0/983162986 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55da48cca600 con 0x55da48d76770 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd23fff640 10 client.14994 handle_command_reply: tid=0 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 client.14994 shutdown 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 msgr2=0x55da48d658a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d658a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcd3401b040 tx=0x7fcd34010bd0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 shutdown_connections 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48d76770 0x55da48d77be0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d63130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d571b0 0x55da48d62bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 --2- 172.21.15.37:0/983162986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d56890 0x55da48d658a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 >> 172.21.15.37:0/983162986 conn(0x55da48d52f60 msgr2=0x55da48d58cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.915+0000 7fcd23fff640 0 client.14994 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:17.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.916+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 shutdown_connections 2024-09-19T06:41:17.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.916+0000 7fcd4d583740 1 -- 172.21.15.37:0/983162986 wait complete. 2024-09-19T06:41:17.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.916+0000 7fcd4d583740 20 client.14994 trim_cache size 0 max 16384 2024-09-19T06:41:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.933+0000 7fcd4d583740 1 Processor -- start 2024-09-19T06:41:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd4d583740 1 -- start start 2024-09-19T06:41:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d56890 0x55da48d76770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d5fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 0x55da48d60390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd4d583740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da48d75f20 con 0x55da48d571b0 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd4d583740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da48d765f0 con 0x55da48d56890 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd4d583740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da48d76cc0 con 0x55da48d65fc0 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d5fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d5fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53478/0 (socket says 172.21.15.37:53478) 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 -- 172.21.15.37:0/1063224309 learned_addr learned my addr 172.21.15.37:0/1063224309 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd48f46640 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d56890 0x55da48d76770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 -- 172.21.15.37:0/1063224309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 msgr2=0x55da48d60390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 0x55da48d60390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 -- 172.21.15.37:0/1063224309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d56890 msgr2=0x55da48d76770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d56890 0x55da48d76770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 -- 172.21.15.37:0/1063224309 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da48d686f0 con 0x55da48d571b0 2024-09-19T06:41:17.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd43fff640 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d5fe50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c09b500 tx=0x7fcd2c010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b32177c5eb8fc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd3a7fc640 1 -- 172.21.15.37:0/1063224309 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c09b800 con 0x55da48d571b0 2024-09-19T06:41:17.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.934+0000 7fcd3a7fc640 1 -- 172.21.15.37:0/1063224309 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2c09fbb0 con 0x55da48d571b0 2024-09-19T06:41:17.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd3a7fc640 1 -- 172.21.15.37:0/1063224309 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c097e70 con 0x55da48d571b0 2024-09-19T06:41:17.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 -- 172.21.15.37:0/1063224309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 msgr2=0x55da48d5fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d5fe50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c09b500 tx=0x7fcd2c010f60 comp rx=0 tx=0).stop 2024-09-19T06:41:17.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 -- 172.21.15.37:0/1063224309 shutdown_connections 2024-09-19T06:41:17.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d65fc0 0x55da48d60390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d5fe50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 --2- 172.21.15.37:0/1063224309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d56890 0x55da48d76770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 -- 172.21.15.37:0/1063224309 >> 172.21.15.37:0/1063224309 conn(0x55da48d52f60 msgr2=0x55da48d5e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 -- 172.21.15.37:0/1063224309 shutdown_connections 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 -- 172.21.15.37:0/1063224309 wait complete. 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.935+0000 7fcd4d583740 1 -- start start 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 1 Processor -- start 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d56890 0x55da48d80330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d79660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d79ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55da48d77a60 con 0x55da48d571b0 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55da48d77390 con 0x55da48d65fc0 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55da48d78130 con 0x55da48d56890 2024-09-19T06:41:17.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd4d583740 10 client.0 _ll_register_callbacks cb 0x7fcd3401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d79660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d79660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53486/0 (socket says 172.21.15.37:53486) 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 -- 172.21.15.37:0/2220293307 learned_addr learned my addr 172.21.15.37:0/2220293307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd48f46640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d56890 0x55da48d80330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 -- 172.21.15.37:0/2220293307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d56890 msgr2=0x55da48d80330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd49747640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d79ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d56890 0x55da48d80330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 -- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 msgr2=0x55da48d79ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d79ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd43fff640 1 -- 172.21.15.37:0/2220293307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da48d7a0e0 con 0x55da48d571b0 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd48f46640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d56890 0x55da48d80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.936+0000 7fcd49747640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d79ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:17.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd43fff640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d79660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0981d0 tx=0x7fcd2c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d780a46f1f5ad67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd23fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd23fff640 1 -- 172.21.15.37:0/2220293307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c099b00 con 0x55da48d571b0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd23fff640 1 -- 172.21.15.37:0/2220293307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fcd2c0cce30 con 0x55da48d571b0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd2c09fbb0 con 0x55da48d571b0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd23fff640 1 -- 172.21.15.37:0/2220293307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fcd2c09b800 con 0x55da48d571b0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 10 client.15000 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55da48d7ebe0 con 0x55da48d571b0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd23fff640 1 -- 172.21.15.37:0/2220293307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd2c09fbb0 con 0x55da48d571b0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd23fff640 1 -- 172.21.15.37:0/2220293307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fcd34019ce0 con 0x55da48d571b0 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 10 client.15000 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 10 client.15000 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 20 client.15000 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:17.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48de51d0 0x55da48de3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 4 client.15000 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.937+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "status"}) -- 0x55da48cc4b10 con 0x55da48de51d0 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd48f46640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48de51d0 0x55da48de3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd48f46640 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48de51d0 0x55da48de3ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bb181ce56bc497f1 server_cookie=d2ffa36e4e8a325d in_seq=0 out_seq=0 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd23fff640 10 client.15000 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd23fff640 1 -- 172.21.15.37:0/2220293307 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55da48cc4b10 con 0x55da48de51d0 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd23fff640 10 client.15000 handle_command_reply: tid=0 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd4d583740 1 client.15000 shutdown 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 msgr2=0x55da48d79660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd4d583740 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d79660 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c0981d0 tx=0x7fcd2c00bed0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 shutdown_connections 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd4d583740 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55da48de51d0 0x55da48de3ee0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.938+0000 7fcd4d583740 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55da48d65fc0 0x55da48d79ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4d583740 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55da48d571b0 0x55da48d79660 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4d583740 1 --2- 172.21.15.37:0/2220293307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55da48d56890 0x55da48d80330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 >> 172.21.15.37:0/2220293307 conn(0x55da48d52f60 msgr2=0x55da48d517e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd23fff640 0 client.15000 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 shutdown_connections 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4d583740 1 -- 172.21.15.37:0/2220293307 wait complete. 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4d583740 20 client.15000 trim_cache size 0 max 16384 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:41:17.945 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24470, 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 77.509960164000006, 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 24, 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 83, 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 107.180950001 2024-09-19T06:41:17.946 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcd300a32d0 msgr2=0x7fcd300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcd300a32d0 0x7fcd300a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55da48d54050 tx=0x7fcd3c0093f0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44102790 msgr2=0x7fcd441988e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44102790 0x7fcd441988e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00b4a0 tx=0x7fcd2c00b8a0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 shutdown_connections 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fcd300a32d0 0x7fcd300a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd44106f10 0x7fcd4419cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd44102790 0x7fcd441988e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 --2- 172.21.15.37:0/695828287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd44101de0 0x7fcd441983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 >> 172.21.15.37:0/695828287 conn(0x7fcd440fd8c0 msgr2=0x7fcd440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 shutdown_connections 2024-09-19T06:41:17.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:17.939+0000 7fcd4b1aa640 1 -- 172.21.15.37:0/695828287 wait complete. 2024-09-19T06:41:18.684 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:41:18.940 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:19.366 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:19 smithi037 ceph-mon[31211]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:41:19.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.365+0000 7f720aa59640 1 -- 172.21.15.37:0/3033760290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204102790 msgr2=0x7f72041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.365+0000 7f720aa59640 1 --2- 172.21.15.37:0/3033760290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204102790 0x7f72041067e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f71f8009900 tx=0x7f71f802a410 comp rx=0 tx=0).stop 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 -- 172.21.15.37:0/3033760290 shutdown_connections 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 --2- 172.21.15.37:0/3033760290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204106f10 0x7f720410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 --2- 172.21.15.37:0/3033760290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204102790 0x7f72041067e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 --2- 172.21.15.37:0/3033760290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204101de0 0x7f72041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 -- 172.21.15.37:0/3033760290 >> 172.21.15.37:0/3033760290 conn(0x7f72040fd8c0 msgr2=0x7f72040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 -- 172.21.15.37:0/3033760290 shutdown_connections 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 -- 172.21.15.37:0/3033760290 wait complete. 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 Processor -- start 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.366+0000 7f720aa59640 1 -- start start 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f720aa59640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204101de0 0x7f72041984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f720aa59640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204102790 0x7f72041989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f720aa59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204106f10 0x7f720419cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f720aa59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7204111f70 con 0x7f7204106f10 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f720aa59640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7204111df0 con 0x7f7204102790 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f720aa59640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f72041120f0 con 0x7f7204101de0 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204106f10 0x7f720419cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204106f10 0x7f720419cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53506/0 (socket says 172.21.15.37:53506) 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 -- 172.21.15.37:0/163829162 learned_addr learned my addr 172.21.15.37:0/163829162 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7203fff640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204101de0 0x7f72041984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f72037fe640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204102790 0x7f72041989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 -- 172.21.15.37:0/163829162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204101de0 msgr2=0x7f72041984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204101de0 0x7f72041984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 -- 172.21.15.37:0/163829162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204102790 msgr2=0x7f72041989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204102790 0x7f72041989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f7208ff6640 1 -- 172.21.15.37:0/163829162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f720419d460 con 0x7f7204106f10 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.367+0000 7f72037fe640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204102790 0x7f72041989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.368+0000 7f7208ff6640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204106f10 0x7f720419cd80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00eaa0 tx=0x7f71fc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.368+0000 7f72017fa640 1 -- 172.21.15.37:0/163829162 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc00cdb0 con 0x7f7204106f10 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.368+0000 7f72017fa640 1 -- 172.21.15.37:0/163829162 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71fc005020 con 0x7f7204106f10 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.368+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f720419d750 con 0x7f7204106f10 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.368+0000 7f72017fa640 1 -- 172.21.15.37:0/163829162 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc0116b0 con 0x7f7204106f10 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.368+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72041a5030 con 0x7f7204106f10 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.370+0000 7f72017fa640 1 -- 172.21.15.37:0/163829162 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f71fc005560 con 0x7f7204106f10 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.370+0000 7f720ce32740 1 Processor -- start 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.370+0000 7f720ce32740 1 -- start start 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.370+0000 7f720ce32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a55c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f720ce32740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a64a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f720ce32740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f720ce32740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5624799bc090 con 0x562479a64fc0 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f720ce32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562479963a70 con 0x562479a558b0 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f720ce32740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5624799b8050 con 0x562479a561d0 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a673b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53522/0 (socket says 172.21.15.37:53522) 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 -- 172.21.15.37:0/4222782005 learned_addr learned my addr 172.21.15.37:0/4222782005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a55c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f7203fff640 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a64a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72017fa640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f71ec0a32d0 0x7f71ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 -- 172.21.15.37:0/4222782005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 msgr2=0x562479a64a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a64a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 -- 172.21.15.37:0/4222782005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 msgr2=0x562479a55c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a55c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 -- 172.21.15.37:0/4222782005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562479970130 con 0x562479a64fc0 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f7208ff6640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f71ec0a32d0 0x7f71ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72017fa640 1 -- 172.21.15.37:0/163829162 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f71fc0cf950 con 0x7f7204106f10 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.371+0000 7f72037fe640 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a673b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71f8032ed0 tx=0x7f71f8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87299a355c18c17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f71ea7fc640 1 -- 172.21.15.37:0/4222782005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8035040 con 0x562479a64fc0 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f71ea7fc640 1 -- 172.21.15.37:0/4222782005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f8045070 con 0x562479a64fc0 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f71ea7fc640 1 -- 172.21.15.37:0/4222782005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8003b50 con 0x562479a64fc0 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f7208ff6640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f71ec0a32d0 0x7f71ec0a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71fc09bd80 tx=0x7f71fc0054d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 -- 172.21.15.37:0/4222782005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 msgr2=0x562479a673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a673b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f71f8032ed0 tx=0x7f71f8000bc0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 -- 172.21.15.37:0/4222782005 shutdown_connections 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a673b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a64a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 --2- 172.21.15.37:0/4222782005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a55c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 -- 172.21.15.37:0/4222782005 >> 172.21.15.37:0/4222782005 conn(0x562479a51f60 msgr2=0x562479a52340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.372+0000 7f720ce32740 1 -- 172.21.15.37:0/4222782005 shutdown_connections 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 -- 172.21.15.37:0/4222782005 wait complete. 2024-09-19T06:41:19.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 -- start start 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 Processor -- start 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a64880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a61bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a62110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562479a67b90 con 0x562479a64fc0 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562479976f20 con 0x562479a558b0 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562479a67d10 con 0x562479a561d0 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.373+0000 7f720ce32740 10 client.0 _ll_register_callbacks cb 0x562479940030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a62110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a62110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53536/0 (socket says 172.21.15.37:53536) 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 -- 172.21.15.37:0/816364942 learned_addr learned my addr 172.21.15.37:0/816364942 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:19.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f7203fff640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a61bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a64880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 -- 172.21.15.37:0/816364942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 msgr2=0x562479a61bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a61bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 -- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 msgr2=0x562479a64880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a64880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 -- 172.21.15.37:0/816364942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56247972d2b0 con 0x562479a64fc0 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f7203fff640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a61bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f72037fe640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a62110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71f802ac00 tx=0x7f71f8003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c625e6adac02d264 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f71e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.374+0000 7f71e37fe640 1 -- 172.21.15.37:0/816364942 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8004200 con 0x562479a64fc0 2024-09-19T06:41:19.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f71e37fe640 1 -- 172.21.15.37:0/816364942 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f71f80043a0 con 0x562479a64fc0 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f8045070 con 0x562479a64fc0 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f71e37fe640 1 -- 172.21.15.37:0/816364942 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f71f8066050 con 0x562479a64fc0 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 10 client.15012 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562479a58610 con 0x562479a64fc0 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f71e37fe640 1 -- 172.21.15.37:0/816364942 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f8045070 con 0x562479a64fc0 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f71e37fe640 1 -- 172.21.15.37:0/816364942 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f71f805d070 con 0x562479a64fc0 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 10 client.15012 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 10 client.15012 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 20 client.15012 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479a5a820 0x562479a59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 4 client.15012 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:19.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.375+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5624799c9600 con 0x562479a5a820 2024-09-19T06:41:19.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.376+0000 7f7203fff640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479a5a820 0x562479a59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.376+0000 7f7203fff640 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479a5a820 0x562479a59a80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30990485d98c95e2 server_cookie=b76721e34a29ff94 in_seq=0 out_seq=0 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.376+0000 7f71e37fe640 10 client.15012 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f71e37fe640 1 -- 172.21.15.37:0/816364942 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5624799c9600 con 0x562479a5a820 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f71e37fe640 10 client.15012 handle_command_reply: tid=0 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 client.15012 shutdown 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 msgr2=0x562479a62110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a62110 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f71f802ac00 tx=0x7f71f8003790 comp rx=0 tx=0).stop 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 shutdown_connections 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479a5a820 0x562479a59a80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a62110 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a61bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 --2- 172.21.15.37:0/816364942 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a64880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f71e37fe640 0 client.15012 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.377+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 >> 172.21.15.37:0/816364942 conn(0x562479a51f60 msgr2=0x562479a57cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.378+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 shutdown_connections 2024-09-19T06:41:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.378+0000 7f720ce32740 1 -- 172.21.15.37:0/816364942 wait complete. 2024-09-19T06:41:19.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.378+0000 7f720ce32740 20 client.15012 trim_cache size 0 max 16384 2024-09-19T06:41:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.395+0000 7f720ce32740 1 Processor -- start 2024-09-19T06:41:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f720ce32740 1 -- start start 2024-09-19T06:41:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f720ce32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f720ce32740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a561d0 0x562479a5d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f720ce32740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a64fc0 0x562479a4cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f720ce32740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562479a59fd0 con 0x562479a561d0 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f720ce32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562479a5a6a0 con 0x562479a558b0 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f720ce32740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562479a5ad70 con 0x562479a64fc0 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a561d0 0x562479a5d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a561d0 0x562479a5d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53552/0 (socket says 172.21.15.37:53552) 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 -- 172.21.15.37:0/2637521624 learned_addr learned my addr 172.21.15.37:0/2637521624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f72037fe640 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a64fc0 0x562479a4cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f72037fe640 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 -- 172.21.15.37:0/2637521624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a64fc0 msgr2=0x562479a4cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a64fc0 0x562479a4cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 -- 172.21.15.37:0/2637521624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 msgr2=0x562479a59a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a59a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 -- 172.21.15.37:0/2637521624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562479a64bf0 con 0x562479a561d0 2024-09-19T06:41:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.396+0000 7f7203fff640 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a561d0 0x562479a5d240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71f4004700 tx=0x7f71f400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3492ad0c43adea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f71e9ffb640 1 -- 172.21.15.37:0/2637521624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4022070 con 0x562479a561d0 2024-09-19T06:41:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f71e9ffb640 1 -- 172.21.15.37:0/2637521624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f4003e40 con 0x562479a561d0 2024-09-19T06:41:19.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f71e9ffb640 1 -- 172.21.15.37:0/2637521624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4015070 con 0x562479a561d0 2024-09-19T06:41:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 -- 172.21.15.37:0/2637521624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a561d0 msgr2=0x562479a5d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a561d0 0x562479a5d240 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f71f4004700 tx=0x7f71f400be10 comp rx=0 tx=0).stop 2024-09-19T06:41:19.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 -- 172.21.15.37:0/2637521624 shutdown_connections 2024-09-19T06:41:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a64fc0 0x562479a4cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a561d0 0x562479a5d240 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 --2- 172.21.15.37:0/2637521624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 -- 172.21.15.37:0/2637521624 >> 172.21.15.37:0/2637521624 conn(0x562479a51f60 msgr2=0x562479a52340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 -- 172.21.15.37:0/2637521624 shutdown_connections 2024-09-19T06:41:19.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.397+0000 7f720ce32740 1 -- 172.21.15.37:0/2637521624 wait complete. 2024-09-19T06:41:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 -- start start 2024-09-19T06:41:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 Processor -- start 2024-09-19T06:41:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a738c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a73e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a74340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562479a5bb10 con 0x562479a64fc0 2024-09-19T06:41:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562479a5b440 con 0x562479a558b0 2024-09-19T06:41:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562479a5c1e0 con 0x562479a561d0 2024-09-19T06:41:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f720ce32740 10 client.0 _ll_register_callbacks cb 0x7f71f8032d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f72037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a74340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f72037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a74340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53554/0 (socket says 172.21.15.37:53554) 2024-09-19T06:41:19.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.398+0000 7f72037fe640 1 -- 172.21.15.37:0/1023202935 learned_addr learned my addr 172.21.15.37:0/1023202935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f72037fe640 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a738c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f7203fff640 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a73e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f72037fe640 1 -- 172.21.15.37:0/1023202935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 msgr2=0x562479a73e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f72037fe640 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a73e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f72037fe640 1 -- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 msgr2=0x562479a738c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:19.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f72037fe640 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a738c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f72037fe640 1 -- 172.21.15.37:0/1023202935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562479ad9ae0 con 0x562479a64fc0 2024-09-19T06:41:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f72037fe640 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a74340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71f803d450 tx=0x7f71f8041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f14f730168d4cc98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f71e37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f71e37fe640 1 -- 172.21.15.37:0/1023202935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8035070 con 0x562479a64fc0 2024-09-19T06:41:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f4003e40 con 0x562479a64fc0 2024-09-19T06:41:19.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f71e37fe640 1 -- 172.21.15.37:0/1023202935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f71f803dc90 con 0x562479a64fc0 2024-09-19T06:41:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f71e37fe640 1 -- 172.21.15.37:0/1023202935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f71f8031940 con 0x562479a64fc0 2024-09-19T06:41:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f720ce32740 10 client.15018 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.399+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562479adadc0 con 0x562479a64fc0 2024-09-19T06:41:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f71e37fe640 1 -- 172.21.15.37:0/1023202935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f4003e40 con 0x562479a64fc0 2024-09-19T06:41:19.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f71e37fe640 1 -- 172.21.15.37:0/1023202935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f71f805d040 con 0x562479a64fc0 2024-09-19T06:41:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f720ce32740 10 client.15018 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f720ce32740 10 client.15018 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:41:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f720ce32740 20 client.15018 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f720ce32740 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479adf8e0 0x562479ade5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f720ce32740 4 client.15018 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:41:19.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "status"}) -- 0x5624799c3b10 con 0x562479adf8e0 2024-09-19T06:41:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f7203fff640 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479adf8e0 0x562479ade5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f7203fff640 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479adf8e0 0x562479ade5f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97091bcb51dd91cb server_cookie=d8444dbf778ae62e in_seq=0 out_seq=0 2024-09-19T06:41:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.400+0000 7f71e37fe640 10 client.15018 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f71e37fe640 1 -- 172.21.15.37:0/1023202935 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5624799c3b10 con 0x562479adf8e0 2024-09-19T06:41:19.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f71e37fe640 10 client.15018 handle_command_reply: tid=0 2024-09-19T06:41:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 client.15018 shutdown 2024-09-19T06:41:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 msgr2=0x562479a74340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a74340 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f71f803d450 tx=0x7f71f8041040 comp rx=0 tx=0).stop 2024-09-19T06:41:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 shutdown_connections 2024-09-19T06:41:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562479adf8e0 0x562479ade5f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562479a64fc0 0x562479a74340 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f71e37fe640 0 client.15018 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562479a561d0 0x562479a73e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 --2- 172.21.15.37:0/1023202935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562479a558b0 0x562479a738c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 >> 172.21.15.37:0/1023202935 conn(0x562479a51f60 msgr2=0x562479a5d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 shutdown_connections 2024-09-19T06:41:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 1 -- 172.21.15.37:0/1023202935 wait complete. 2024-09-19T06:41:19.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.401+0000 7f720ce32740 20 client.15018 trim_cache size 0 max 16384 2024-09-19T06:41:19.415 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:19.415 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:41:19.416 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-19T06:41:19.416 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24497, 2024-09-19T06:41:19.416 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:41:19.416 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:41:19.416 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:41:19.416 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 85.993945585999995, 2024-09-19T06:41:19.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 24, 2024-09-19T06:41:19.417 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 83, 2024-09-19T06:41:19.417 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T06:41:19.417 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 97.798391738000007 2024-09-19T06:41:19.417 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f71ec0a32d0 msgr2=0x7f71ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f71ec0a32d0 0x7f71ec0a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f71fc09bd80 tx=0x7f71fc0054d0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204106f10 msgr2=0x7f720419cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204106f10 0x7f720419cd80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f71fc00eaa0 tx=0x7f71fc00ef60 comp rx=0 tx=0).stop 2024-09-19T06:41:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 shutdown_connections 2024-09-19T06:41:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f71ec0a32d0 0x7f71ec0a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7204106f10 0x7f720419cd80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7204102790 0x7f72041989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 --2- 172.21.15.37:0/163829162 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7204101de0 0x7f72041984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 >> 172.21.15.37:0/163829162 conn(0x7f72040fd8c0 msgr2=0x7f72040fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 shutdown_connections 2024-09-19T06:41:19.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:19.402+0000 7f720aa59640 1 -- 172.21.15.37:0/163829162 wait complete. 2024-09-19T06:41:19.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:19 smithi151 ceph-mon[35236]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:41:19.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:19 smithi096 ceph-mon[35004]: pgmap v324: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:41:20.062 INFO:tasks.cephadm:Setup complete, yielding 2024-09-19T06:41:20.063 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:41:20.074 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:41:20.075 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-19T06:41:20.335 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 -- 172.21.15.37:0/2543360951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 msgr2=0x7fa61c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 --2- 172.21.15.37:0/2543360951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c103280 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa604009900 tx=0x7fa60402a410 comp rx=0 tx=0).stop 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 -- 172.21.15.37:0/2543360951 shutdown_connections 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 --2- 172.21.15.37:0/2543360951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa61c1037c0 0x7fa61c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 --2- 172.21.15.37:0/2543360951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c103280 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 --2- 172.21.15.37:0/2543360951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa61c101c00 0x7fa61c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 -- 172.21.15.37:0/2543360951 >> 172.21.15.37:0/2543360951 conn(0x7fa61c0fd390 msgr2=0x7fa61c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:20.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 -- 172.21.15.37:0/2543360951 shutdown_connections 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 -- 172.21.15.37:0/2543360951 wait complete. 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.815+0000 7fa621113640 1 Processor -- start 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa621113640 1 -- start start 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa621113640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa61c101c00 0x7fa61c075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa621113640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c0745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa621113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa61c1037c0 0x7fa61c074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa621113640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10d900 con 0x7fa61c102e00 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa621113640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10d780 con 0x7fa61c101c00 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa621113640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa61c10da80 con 0x7fa61c1037c0 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c0745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c0745e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38894/0 (socket says 172.21.15.37:38894) 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 -- 172.21.15.37:0/4030583774 learned_addr learned my addr 172.21.15.37:0/4030583774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:20.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61b59e640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa61c1037c0 0x7fa61c074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61ad9d640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa61c101c00 0x7fa61c075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 -- 172.21.15.37:0/4030583774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa61c1037c0 msgr2=0x7fa61c074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa61c1037c0 0x7fa61c074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 -- 172.21.15.37:0/4030583774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa61c101c00 msgr2=0x7fa61c075f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa61c101c00 0x7fa61c075f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61a59c640 1 -- 172.21.15.37:0/4030583774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61c0753e0 con 0x7fa61c102e00 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.816+0000 7fa61ad9d640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa61c101c00 0x7fa61c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.817+0000 7fa61a59c640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c0745e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa604009230 tx=0x7fa604004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.817+0000 7fa613fff640 1 -- 172.21.15.37:0/4030583774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60403f030 con 0x7fa61c102e00 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.817+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61c1a2050 con 0x7fa61c102e00 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.817+0000 7fa613fff640 1 -- 172.21.15.37:0/4030583774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa604003ce0 con 0x7fa61c102e00 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.817+0000 7fa613fff640 1 -- 172.21.15.37:0/4030583774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604035070 con 0x7fa61c102e00 2024-09-19T06:41:20.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.817+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61c1a25d0 con 0x7fa61c102e00 2024-09-19T06:41:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.818+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa61c108fb0 con 0x7fa61c102e00 2024-09-19T06:41:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.819+0000 7fa613fff640 1 -- 172.21.15.37:0/4030583774 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa604042030 con 0x7fa61c102e00 2024-09-19T06:41:20.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.820+0000 7fa613fff640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6080a32d0 0x7fa6080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.820+0000 7fa61ad9d640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6080a32d0 0x7fa6080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.821+0000 7fa613fff640 1 -- 172.21.15.37:0/4030583774 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa6040f0630 con 0x7fa61c102e00 2024-09-19T06:41:20.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.821+0000 7fa61ad9d640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6080a32d0 0x7fa6080a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa61c102c60 tx=0x7fa60c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:20.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:20.824+0000 7fa613fff640 1 -- 172.21.15.37:0/4030583774 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa6040bd170 con 0x7fa61c102e00 2024-09-19T06:41:21.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.014+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fa61c062630 con 0x7fa6080a32d0 2024-09-19T06:41:21.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.016+0000 7fa613fff640 1 -- 172.21.15.37:0/4030583774 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fa61c062630 con 0x7fa6080a32d0 2024-09-19T06:41:21.016 INFO:teuthology.orchestra.run.smithi037.stdout:Backend: cephadm 2024-09-19T06:41:21.016 INFO:teuthology.orchestra.run.smithi037.stdout:Available: Yes 2024-09-19T06:41:21.017 INFO:teuthology.orchestra.run.smithi037.stdout:Paused: No 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6080a32d0 msgr2=0x7fa6080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6080a32d0 0x7fa6080a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa61c102c60 tx=0x7fa60c008040 comp rx=0 tx=0).stop 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 msgr2=0x7fa61c0745e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c0745e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa604009230 tx=0x7fa604004430 comp rx=0 tx=0).stop 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 shutdown_connections 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa6080a32d0 0x7fa6080a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa61c1037c0 0x7fa61c074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa61c102e00 0x7fa61c0745e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 --2- 172.21.15.37:0/4030583774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa61c101c00 0x7fa61c075f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:21.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 >> 172.21.15.37:0/4030583774 conn(0x7fa61c0fd390 msgr2=0x7fa61c106180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:21.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 shutdown_connections 2024-09-19T06:41:21.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:21.018+0000 7fa621113640 1 -- 172.21.15.37:0/4030583774 wait complete. 2024-09-19T06:41:21.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:21 smithi151 ceph-mon[35236]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:41:21.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:21 smithi151 ceph-mon[35236]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:21.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:21 smithi037 ceph-mon[31211]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:41:21.639 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:21 smithi037 ceph-mon[31211]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:21.671 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-19T06:41:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:21 smithi096 ceph-mon[35004]: pgmap v325: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-19T06:41:21.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:21 smithi096 ceph-mon[35004]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:21.928 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:22.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.383+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/509023621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 msgr2=0x7fa8c4102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:22.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.383+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/509023621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4102150 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009900 tx=0x7fa8b802a410 comp rx=0 tx=0).stop 2024-09-19T06:41:22.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/509023621 shutdown_connections 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/509023621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8c4102690 0x7fa8c4108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/509023621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4102150 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/509023621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8c4100ad0 0x7fa8c4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/509023621 >> 172.21.15.37:0/509023621 conn(0x7fa8c40fc280 msgr2=0x7fa8c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/509023621 shutdown_connections 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/509023621 wait complete. 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 Processor -- start 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 -- start start 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8c4100ad0 0x7fa8c4196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.384+0000 7fa8ca9e4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8c4102690 0x7fa8c419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:22.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8ca9e4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410c750 con 0x7fa8c4101cd0 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8ca9e4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410c5d0 con 0x7fa8c4100ad0 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8ca9e4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa8c410c8d0 con 0x7fa8c4102690 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38904/0 (socket says 172.21.15.37:38904) 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 -- 172.21.15.37:0/848217742 learned_addr learned my addr 172.21.15.37:0/848217742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c8f81640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8c4102690 0x7fa8c419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c3fff640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8c4100ad0 0x7fa8c4196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 -- 172.21.15.37:0/848217742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8c4102690 msgr2=0x7fa8c419e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8c4102690 0x7fa8c419e3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 -- 172.21.15.37:0/848217742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8c4100ad0 msgr2=0x7fa8c4196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8c4100ad0 0x7fa8c4196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 -- 172.21.15.37:0/848217742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8c419ea10 con 0x7fa8c4101cd0 2024-09-19T06:41:22.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c3fff640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8c4100ad0 0x7fa8c4196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c37fe640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4197320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009230 tx=0x7fa8b8004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c17fa640 1 -- 172.21.15.37:0/848217742 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b803f030 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c17fa640 1 -- 172.21.15.37:0/848217742 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8b8003ce0 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8c17fa640 1 -- 172.21.15.37:0/848217742 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8035070 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8c419eca0 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.385+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8c419f180 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.386+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8c4107f20 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.387+0000 7fa8c17fa640 1 -- 172.21.15.37:0/848217742 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa8b8042030 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.388+0000 7fa8c17fa640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa8ac0a3300 0x7fa8ac0a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.388+0000 7fa8c3fff640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa8ac0a3300 0x7fa8ac0a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.388+0000 7fa8c17fa640 1 -- 172.21.15.37:0/848217742 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fa8b80f00f0 con 0x7fa8c4101cd0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.389+0000 7fa8c3fff640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa8ac0a3300 0x7fa8ac0a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4101b30 tx=0x7fa8b4005e70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:22.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.392+0000 7fa8c17fa640 1 -- 172.21.15.37:0/848217742 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa8b80bcc90 con 0x7fa8c4101cd0 2024-09-19T06:41:22.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.577+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fa8c4062630 con 0x7fa8ac0a3300 2024-09-19T06:41:22.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.590+0000 7fa8c17fa640 1 -- 172.21.15.37:0/848217742 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4154 (secure 0 0 0) 0x7fa8c4062630 con 0x7fa8ac0a3300 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:mds.a smithi037 running (103s) 26s ago 103s 13.3M - 19.1.1-361-g2f599a46 d2baa855c067 df1b81e71043 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:mds.b smithi096 running (101s) 34s ago 100s 26.4M - 19.1.1-361-g2f599a46 d2baa855c067 a47f4a9cf11a 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:mds.c smithi151 running (114s) 34s ago 114s 18.4M - 19.1.1-361-g2f599a46 d2baa855c067 90aecbb0c824 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:mds.d smithi037 running (2m) 26s ago 2m 13.4M - 19.1.1-361-g2f599a46 d2baa855c067 9059d2ccd358 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:mds.e smithi096 running (119s) 34s ago 119s 17.9M - 19.1.1-361-g2f599a46 d2baa855c067 1c294244b3ee 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:mds.f smithi151 running (117s) 34s ago 116s 14.4M - 19.1.1-361-g2f599a46 d2baa855c067 f12ce8dc4629 2024-09-19T06:41:22.591 INFO:teuthology.orchestra.run.smithi037.stdout:mds.g smithi037 running (107s) 26s ago 106s 15.8M - 19.1.1-361-g2f599a46 d2baa855c067 5bf2c8397de1 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mds.h smithi096 running (112s) 34s ago 111s 20.2M - 19.1.1-361-g2f599a46 d2baa855c067 1cfd244352b2 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mds.i smithi151 running (109s) 34s ago 109s 17.7M - 19.1.1-361-g2f599a46 d2baa855c067 b5479bcc4846 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mds.j smithi037 running (2m) 26s ago 2m 14.2M - 19.1.1-361-g2f599a46 d2baa855c067 6512a1479098 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mds.k smithi096 running (2m) 34s ago 2m 17.0M - 19.1.1-361-g2f599a46 d2baa855c067 89ff57701125 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mds.l smithi151 running (2m) 34s ago 2m 13.5M - 19.1.1-361-g2f599a46 d2baa855c067 1e9f1562a3db 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.x smithi037 *:9283,8765 running (9m) 26s ago 9m 508M - 19.1.1-361-g2f599a46 d2baa855c067 0d11568288e0 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.y smithi096 *:8443,8765 running (7m) 34s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 3b0d4123ccc8 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mgr.z smithi151 *:8443,8765 running (7m) 34s ago 7m 442M - 19.1.1-361-g2f599a46 d2baa855c067 ef213e9b1b08 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mon.a smithi037 running (9m) 26s ago 9m 53.0M 2048M 19.1.1-361-g2f599a46 d2baa855c067 98ade29977a1 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mon.b smithi096 running (7m) 34s ago 7m 38.9M 2048M 19.1.1-361-g2f599a46 d2baa855c067 fbdc43f0f542 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:mon.c smithi151 running (7m) 34s ago 7m 38.5M 2048M 19.1.1-361-g2f599a46 d2baa855c067 50dfa29c6126 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 smithi037 running (7m) 26s ago 7m 43.9M 4273M 19.1.1-361-g2f599a46 d2baa855c067 538a9aac3e85 2024-09-19T06:41:22.592 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 smithi096 running (6m) 34s ago 6m 45.1M 4272M 19.1.1-361-g2f599a46 d2baa855c067 213bf026dadc 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 smithi151 running (6m) 34s ago 6m 57.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 13bf3e81dc71 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 smithi037 running (5m) 26s ago 5m 54.4M 4273M 19.1.1-361-g2f599a46 d2baa855c067 8fd8a5b539ed 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 smithi096 running (5m) 34s ago 5m 41.2M 4272M 19.1.1-361-g2f599a46 d2baa855c067 1fa96ec4c71d 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 smithi151 running (5m) 34s ago 5m 43.5M 4272M 19.1.1-361-g2f599a46 d2baa855c067 22f12d51ec75 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 smithi037 running (4m) 26s ago 4m 52.6M 4273M 19.1.1-361-g2f599a46 d2baa855c067 5dfff3f4ce32 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 smithi096 running (4m) 34s ago 4m 42.9M 4272M 19.1.1-361-g2f599a46 d2baa855c067 50a23e586100 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 smithi151 running (3m) 34s ago 3m 53.3M 4272M 19.1.1-361-g2f599a46 d2baa855c067 95a3611889c8 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 smithi037 running (3m) 26s ago 3m 50.8M 4273M 19.1.1-361-g2f599a46 d2baa855c067 369466b1a889 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 smithi096 running (2m) 34s ago 2m 43.8M 4272M 19.1.1-361-g2f599a46 d2baa855c067 d81e1b69ffc5 2024-09-19T06:41:22.593 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 smithi151 running (2m) 34s ago 2m 51.7M 4272M 19.1.1-361-g2f599a46 d2baa855c067 39f747294b5d 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa8ac0a3300 msgr2=0x7fa8ac0a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa8ac0a3300 0x7fa8ac0a57c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa8c4101b30 tx=0x7fa8b4005e70 comp rx=0 tx=0).stop 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 msgr2=0x7fa8c4197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4197320 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa8b8009230 tx=0x7fa8b8004360 comp rx=0 tx=0).stop 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 shutdown_connections 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fa8ac0a3300 0x7fa8ac0a57c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8c4102690 0x7fa8c419e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa8c4101cd0 0x7fa8c4197320 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 --2- 172.21.15.37:0/848217742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8c4100ad0 0x7fa8c4196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 >> 172.21.15.37:0/848217742 conn(0x7fa8c40fc280 msgr2=0x7fa8c41050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 shutdown_connections 2024-09-19T06:41:22.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:22.592+0000 7fa8ca9e4640 1 -- 172.21.15.37:0/848217742 wait complete. 2024-09-19T06:41:23.221 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-19T06:41:23.480 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:23.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:23 smithi037 ceph-mon[31211]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:23.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:23 smithi037 ceph-mon[31211]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:23.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:23 smithi151 ceph-mon[35236]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:23.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:23 smithi151 ceph-mon[35236]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:23.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:23 smithi096 ceph-mon[35004]: pgmap v326: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:23.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:23 smithi096 ceph-mon[35004]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.950+0000 7f7a0257b640 1 -- 172.21.15.37:0/1768074704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc1026d0 msgr2=0x7f79fc108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.950+0000 7f7a0257b640 1 --2- 172.21.15.37:0/1768074704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc1026d0 0x7f79fc108f60 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f79f4009ca0 tx=0x7f79f402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.950+0000 7f7a0257b640 1 -- 172.21.15.37:0/1768074704 shutdown_connections 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.950+0000 7f7a0257b640 1 --2- 172.21.15.37:0/1768074704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc1026d0 0x7f79fc108f60 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.950+0000 7f7a0257b640 1 --2- 172.21.15.37:0/1768074704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79fc101d10 0x7f79fc102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.950+0000 7f7a0257b640 1 --2- 172.21.15.37:0/1768074704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc100b10 0x7f79fc100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.950+0000 7f7a0257b640 1 -- 172.21.15.37:0/1768074704 >> 172.21.15.37:0/1768074704 conn(0x7f79fc0fc2a0 msgr2=0x7f79fc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.951+0000 7f7a0257b640 1 -- 172.21.15.37:0/1768074704 shutdown_connections 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.951+0000 7f7a0257b640 1 -- 172.21.15.37:0/1768074704 wait complete. 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.951+0000 7f7a0257b640 1 Processor -- start 2024-09-19T06:41:23.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.951+0000 7f7a0257b640 1 -- start start 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f7a0257b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc100b10 0x7f79fc196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f7a0257b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc101d10 0x7f79fc1971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f7a0257b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79fc1026d0 0x7f79fc19e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f7a0257b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc10c770 con 0x7f79fc100b10 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f7a0257b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc10c5f0 con 0x7f79fc101d10 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f7a0257b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79fc10c8f0 con 0x7f79fc1026d0 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc100b10 0x7f79fc196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc100b10 0x7f79fc196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38916/0 (socket says 172.21.15.37:38916) 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 -- 172.21.15.37:0/2295980983 learned_addr learned my addr 172.21.15.37:0/2295980983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fb7fe640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc101d10 0x7f79fc1971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 -- 172.21.15.37:0/2295980983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79fc1026d0 msgr2=0x7f79fc19e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f7a00b18640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79fc1026d0 0x7f79fc19e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:23.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79fc1026d0 0x7f79fc19e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 -- 172.21.15.37:0/2295980983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc101d10 msgr2=0x7f79fc1971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc101d10 0x7f79fc1971a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 -- 172.21.15.37:0/2295980983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79fc19e880 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79fbfff640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc100b10 0x7f79fc196c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c970 tx=0x7f79ec00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.952+0000 7f79f97fa640 1 -- 172.21.15.37:0/2295980983 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec007c10 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.953+0000 7f79f97fa640 1 -- 172.21.15.37:0/2295980983 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79ec007db0 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.953+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79fc19eb70 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.953+0000 7f79f97fa640 1 -- 172.21.15.37:0/2295980983 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00f940 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.953+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79fc19f0b0 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.954+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79fc107de0 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.955+0000 7f79f97fa640 1 -- 172.21.15.37:0/2295980983 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f79ec00fae0 con 0x7f79fc100b10 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.956+0000 7f79f97fa640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f79e40a32d0 0x7f79e40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.956+0000 7f79fb7fe640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f79e40a32d0 0x7f79e40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:23.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.956+0000 7f79f97fa640 1 -- 172.21.15.37:0/2295980983 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f79ec0d0280 con 0x7f79fc100b10 2024-09-19T06:41:23.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.956+0000 7f79fb7fe640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f79e40a32d0 0x7f79e40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f79f000ba40 tx=0x7f79f0005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:23.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:23.959+0000 7f79f97fa640 1 -- 172.21.15.37:0/2295980983 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f79ec09cee0 con 0x7f79fc100b10 2024-09-19T06:41:24.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.142+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f79fc062630 con 0x7f79e40a32d0 2024-09-19T06:41:24.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.148+0000 7f79f97fa640 1 -- 172.21.15.37:0/2295980983 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f79fc062630 con 0x7f79e40a32d0 2024-09-19T06:41:24.148 INFO:teuthology.orchestra.run.smithi037.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-19T06:41:24.149 INFO:teuthology.orchestra.run.smithi037.stdout:mds.all 12/12 36s ago 2m smithi037=a;smithi037=d;smithi037=g;smithi037=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi151=c;smithi151=f;smithi151=i;smithi151=l;count:12 2024-09-19T06:41:24.149 INFO:teuthology.orchestra.run.smithi037.stdout:mgr 3/3 36s ago 7m smithi037=x;smithi096=y;smithi151=z;count:3 2024-09-19T06:41:24.149 INFO:teuthology.orchestra.run.smithi037.stdout:mon 3/3 36s ago 7m smithi037:172.21.15.37=a;smithi096:172.21.15.96=b;smithi151:172.21.15.151=c;count:3 2024-09-19T06:41:24.149 INFO:teuthology.orchestra.run.smithi037.stdout:osd 12 36s ago - 2024-09-19T06:41:24.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f79e40a32d0 msgr2=0x7f79e40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:24.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f79e40a32d0 0x7f79e40a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f79f000ba40 tx=0x7f79f0005cc0 comp rx=0 tx=0).stop 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc100b10 msgr2=0x7f79fc196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc100b10 0x7f79fc196c60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f79ec00c970 tx=0x7f79ec00ce30 comp rx=0 tx=0).stop 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 shutdown_connections 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f79e40a32d0 0x7f79e40a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79fc1026d0 0x7f79fc19e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79fc101d10 0x7f79fc1971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 --2- 172.21.15.37:0/2295980983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79fc100b10 0x7f79fc196c60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 >> 172.21.15.37:0/2295980983 conn(0x7f79fc0fc2a0 msgr2=0x7f79fc104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 shutdown_connections 2024-09-19T06:41:24.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:24.150+0000 7f7a0257b640 1 -- 172.21.15.37:0/2295980983 wait complete. 2024-09-19T06:41:24.523 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:24 smithi037 ceph-mon[31211]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:24 smithi151 ceph-mon[35236]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:24.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:24 smithi096 ceph-mon[35004]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:24.959 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-19T06:41:25.220 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:25.571 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:25 smithi037 ceph-mon[31211]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:41:25.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:25 smithi151 ceph-mon[35236]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 -- 172.21.15.37:0/2441201836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 msgr2=0x7fc28c102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 --2- 172.21.15.37:0/2441201836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c102000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009900 tx=0x7fc27c02a410 comp rx=0 tx=0).stop 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 -- 172.21.15.37:0/2441201836 shutdown_connections 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 --2- 172.21.15.37:0/2441201836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc28c1037c0 0x7fc28c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 --2- 172.21.15.37:0/2441201836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c102e00 0x7fc28c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 --2- 172.21.15.37:0/2441201836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c102000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 -- 172.21.15.37:0/2441201836 >> 172.21.15.37:0/2441201836 conn(0x7fc28c0fd390 msgr2=0x7fc28c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 -- 172.21.15.37:0/2441201836 shutdown_connections 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 -- 172.21.15.37:0/2441201836 wait complete. 2024-09-19T06:41:25.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 Processor -- start 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.654+0000 7fc292895640 1 -- start start 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc292895640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc292895640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc28c102e00 0x7fc28c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc292895640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c1037c0 0x7fc28c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc292895640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c10d960 con 0x7fc28c101c00 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc292895640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c10d7e0 con 0x7fc28c102e00 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc292895640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc28c10dae0 con 0x7fc28c1037c0 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c1980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38930/0 (socket says 172.21.15.37:38930) 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 -- 172.21.15.37:0/1169625712 learned_addr learned my addr 172.21.15.37:0/1169625712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:25.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28b7fe640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc28c102e00 0x7fc28c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 -- 172.21.15.37:0/1169625712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c1037c0 msgr2=0x7fc28c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc290e32640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c1037c0 0x7fc28c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c1037c0 0x7fc28c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 -- 172.21.15.37:0/1169625712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc28c102e00 msgr2=0x7fc28c1985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc28c102e00 0x7fc28c1985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 -- 172.21.15.37:0/1169625712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc28c19fcd0 con 0x7fc28c101c00 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc290e32640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c1037c0 0x7fc28c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28b7fe640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc28c102e00 0x7fc28c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.655+0000 7fc28bfff640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c1980a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc27c031610 tx=0x7fc27c0043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.656+0000 7fc2897fa640 1 -- 172.21.15.37:0/1169625712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c03f030 con 0x7fc28c101c00 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.656+0000 7fc2897fa640 1 -- 172.21.15.37:0/1169625712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc27c003ce0 con 0x7fc28c101c00 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.656+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc28c19ff60 con 0x7fc28c101c00 2024-09-19T06:41:25.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.656+0000 7fc2897fa640 1 -- 172.21.15.37:0/1169625712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc27c035070 con 0x7fc28c101c00 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.656+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc28c1a0430 con 0x7fc28c101c00 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.657+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc28c1090a0 con 0x7fc28c101c00 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.658+0000 7fc2897fa640 1 -- 172.21.15.37:0/1169625712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc27c042030 con 0x7fc28c101c00 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.658+0000 7fc2897fa640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc2740a3380 0x7fc2740a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.658+0000 7fc28b7fe640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc2740a3380 0x7fc2740a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.659+0000 7fc2897fa640 1 -- 172.21.15.37:0/1169625712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fc27c0f0180 con 0x7fc28c101c00 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.659+0000 7fc28b7fe640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc2740a3380 0x7fc2740a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc280006fd0 tx=0x7fc280008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:25.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.663+0000 7fc2897fa640 1 -- 172.21.15.37:0/1169625712 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc27c0bcd20 con 0x7fc28c101c00 2024-09-19T06:41:25.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:25 smithi096 ceph-mon[35004]: pgmap v327: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2024-09-19T06:41:25.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.847+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fc28c062630 con 0x7fc2740a3380 2024-09-19T06:41:25.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.849+0000 7fc2897fa640 1 -- 172.21.15.37:0/1169625712 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fc28c062630 con 0x7fc2740a3380 2024-09-19T06:41:25.849 INFO:teuthology.orchestra.run.smithi037.stdout:HOST ADDR LABELS STATUS 2024-09-19T06:41:25.849 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 172.21.15.37 2024-09-19T06:41:25.850 INFO:teuthology.orchestra.run.smithi037.stdout:smithi096 172.21.15.96 2024-09-19T06:41:25.850 INFO:teuthology.orchestra.run.smithi037.stdout:smithi151 172.21.15.151 2024-09-19T06:41:25.850 INFO:teuthology.orchestra.run.smithi037.stdout:3 hosts in cluster 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc2740a3380 msgr2=0x7fc2740a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc2740a3380 0x7fc2740a5840 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc280006fd0 tx=0x7fc280008040 comp rx=0 tx=0).stop 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 msgr2=0x7fc28c1980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c1980a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc27c031610 tx=0x7fc27c0043b0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 shutdown_connections 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fc2740a3380 0x7fc2740a5840 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc28c1037c0 0x7fc28c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc28c102e00 0x7fc28c1985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 --2- 172.21.15.37:0/1169625712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc28c101c00 0x7fc28c1980a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 >> 172.21.15.37:0/1169625712 conn(0x7fc28c0fd390 msgr2=0x7fc28c106270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 shutdown_connections 2024-09-19T06:41:25.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:25.851+0000 7fc292895640 1 -- 172.21.15.37:0/1169625712 wait complete. 2024-09-19T06:41:26.586 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:26 smithi037 ceph-mon[31211]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:26.611 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-19T06:41:26.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:26 smithi151 ceph-mon[35236]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:26.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:26 smithi096 ceph-mon[35004]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:26.864 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:27.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 -- 172.21.15.37:0/658392894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f201037c0 msgr2=0x7f3f2010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 --2- 172.21.15.37:0/658392894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f201037c0 0x7f3f2010a050 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3f18009ca0 tx=0x7f3f1802a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 -- 172.21.15.37:0/658392894 shutdown_connections 2024-09-19T06:41:27.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 --2- 172.21.15.37:0/658392894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f201037c0 0x7f3f2010a050 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 --2- 172.21.15.37:0/658392894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f20102e00 0x7f3f20103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 --2- 172.21.15.37:0/658392894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f20101c00 0x7f3f20102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 -- 172.21.15.37:0/658392894 >> 172.21.15.37:0/658392894 conn(0x7f3f200fd390 msgr2=0x7f3f200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:27.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 -- 172.21.15.37:0/658392894 shutdown_connections 2024-09-19T06:41:27.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.338+0000 7f3f27237640 1 -- 172.21.15.37:0/658392894 wait complete. 2024-09-19T06:41:27.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.339+0000 7f3f27237640 1 Processor -- start 2024-09-19T06:41:27.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.339+0000 7f3f27237640 1 -- start start 2024-09-19T06:41:27.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f27237640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f20101c00 0x7f3f2019a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f27237640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f20102e00 0x7f3f2019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f27237640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f201037c0 0x7f3f201a15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f27237640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2010d880 con 0x7f3f20101c00 2024-09-19T06:41:27.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f27237640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2010d700 con 0x7f3f20102e00 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f27237640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f2010da00 con 0x7f3f201037c0 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f20101c00 0x7f3f2019a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f20101c00 0x7f3f2019a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38956/0 (socket says 172.21.15.37:38956) 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 -- 172.21.15.37:0/2102217220 learned_addr learned my addr 172.21.15.37:0/2102217220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f1ffff640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f20102e00 0x7f3f2019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 -- 172.21.15.37:0/2102217220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f201037c0 msgr2=0x7f3f201a15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f201037c0 0x7f3f201a15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 -- 172.21.15.37:0/2102217220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f20102e00 msgr2=0x7f3f2019a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f20102e00 0x7f3f2019a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 -- 172.21.15.37:0/2102217220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f201a1c50 con 0x7f3f20101c00 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.340+0000 7f3f24fd3640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f20101c00 0x7f3f2019a020 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d950 tx=0x7f3f1400dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:27.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.341+0000 7f3f1dffb640 1 -- 172.21.15.37:0/2102217220 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14017070 con 0x7f3f20101c00 2024-09-19T06:41:27.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.341+0000 7f3f1dffb640 1 -- 172.21.15.37:0/2102217220 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f1400be10 con 0x7f3f20101c00 2024-09-19T06:41:27.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.341+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f201a1f40 con 0x7f3f20101c00 2024-09-19T06:41:27.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.341+0000 7f3f1dffb640 1 -- 172.21.15.37:0/2102217220 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400f620 con 0x7f3f20101c00 2024-09-19T06:41:27.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.341+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f201a2390 con 0x7f3f20101c00 2024-09-19T06:41:27.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.342+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f201a25f0 con 0x7f3f20101c00 2024-09-19T06:41:27.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.343+0000 7f3f1dffb640 1 -- 172.21.15.37:0/2102217220 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3f1401a070 con 0x7f3f20101c00 2024-09-19T06:41:27.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.344+0000 7f3f1dffb640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3f100a3270 0x7f3f100a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:27.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.344+0000 7f3f1dffb640 1 -- 172.21.15.37:0/2102217220 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3f140cf3b0 con 0x7f3f20101c00 2024-09-19T06:41:27.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.344+0000 7f3f1ffff640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3f100a3270 0x7f3f100a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:27.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.345+0000 7f3f1ffff640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3f100a3270 0x7f3f100a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f2019b540 tx=0x7f3f0c00a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:27.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.347+0000 7f3f1dffb640 1 -- 172.21.15.37:0/2102217220 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f1409c0b0 con 0x7f3f20101c00 2024-09-19T06:41:27.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:27 smithi037 ceph-mon[31211]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:27.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:27 smithi037 ceph-mon[31211]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:41:27.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.532+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f3f20075b40 con 0x7f3f100a3270 2024-09-19T06:41:27.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.535+0000 7f3f1dffb640 1 -- 172.21.15.37:0/2102217220 <== mgr.14150 v2:172.21.15.37:6800/871019432 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f3f20075b40 con 0x7f3f100a3270 2024-09-19T06:41:27.536 INFO:teuthology.orchestra.run.smithi037.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-19T06:41:27.536 INFO:teuthology.orchestra.run.smithi037.stdout:smithi037 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5331005T400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-19T06:41:27.536 INFO:teuthology.orchestra.run.smithi037.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T06:41:27.536 INFO:teuthology.orchestra.run.smithi037.stdout:smithi151 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JJ400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-19T06:41:27.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3f100a3270 msgr2=0x7f3f100a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3f100a3270 0x7f3f100a5730 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f2019b540 tx=0x7f3f0c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f20101c00 msgr2=0x7f3f2019a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f20101c00 0x7f3f2019a020 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d950 tx=0x7f3f1400dd50 comp rx=0 tx=0).stop 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 shutdown_connections 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3f100a3270 0x7f3f100a5730 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f201037c0 0x7f3f201a15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f20102e00 0x7f3f2019a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 --2- 172.21.15.37:0/2102217220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f20101c00 0x7f3f2019a020 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 >> 172.21.15.37:0/2102217220 conn(0x7f3f200fd390 msgr2=0x7f3f20074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 shutdown_connections 2024-09-19T06:41:27.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:27.539+0000 7f3f27237640 1 -- 172.21.15.37:0/2102217220 wait complete. 2024-09-19T06:41:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:27 smithi151 ceph-mon[35236]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:27 smithi151 ceph-mon[35236]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:41:27.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:27 smithi096 ceph-mon[35004]: pgmap v328: 97 pgs: 97 active+clean; 638 KiB data, 352 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-19T06:41:27.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:27 smithi096 ceph-mon[35004]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-09-19T06:41:28.223 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-19T06:41:28.232 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:41:28.233 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-19T06:41:28.504 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:28.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:28 smithi037 ceph-mon[31211]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:28.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:28 smithi151 ceph-mon[35236]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:28.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:28 smithi096 ceph-mon[35004]: from='client.15048 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:28.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 -- 172.21.15.37:0/1171535512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a20101c00 msgr2=0x7f3a20102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:28.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 --2- 172.21.15.37:0/1171535512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a20101c00 0x7f3a20102000 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009900 tx=0x7f3a1c02a410 comp rx=0 tx=0).stop 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 -- 172.21.15.37:0/1171535512 shutdown_connections 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 --2- 172.21.15.37:0/1171535512 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a201037c0 0x7f3a2010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 --2- 172.21.15.37:0/1171535512 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a20102e00 0x7f3a20103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 --2- 172.21.15.37:0/1171535512 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a20101c00 0x7f3a20102000 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 -- 172.21.15.37:0/1171535512 >> 172.21.15.37:0/1171535512 conn(0x7f3a200fd390 msgr2=0x7f3a200ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 -- 172.21.15.37:0/1171535512 shutdown_connections 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.957+0000 7f3a27629640 1 -- 172.21.15.37:0/1171535512 wait complete. 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 Processor -- start 2024-09-19T06:41:28.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 -- start start 2024-09-19T06:41:28.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a20101c00 0x7f3a20197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:28.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a20102e00 0x7f3a20198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:28.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a201037c0 0x7f3a2019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:28.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2010d8e0 con 0x7f3a201037c0 2024-09-19T06:41:28.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2010d760 con 0x7f3a20101c00 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.958+0000 7f3a27629640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3a2010da60 con 0x7f3a20102e00 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a201037c0 0x7f3a2019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a201037c0 0x7f3a2019f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:38970/0 (socket says 172.21.15.37:38970) 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 -- 172.21.15.37:0/1066166741 learned_addr learned my addr 172.21.15.37:0/1066166741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a253c5640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a20101c00 0x7f3a20197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a24bc4640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a20102e00 0x7f3a20198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 -- 172.21.15.37:0/1066166741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a20102e00 msgr2=0x7f3a20198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a20102e00 0x7f3a20198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 -- 172.21.15.37:0/1066166741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a20101c00 msgr2=0x7f3a20197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a20101c00 0x7f3a20197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 -- 172.21.15.37:0/1066166741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a2019fb10 con 0x7f3a201037c0 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a253c5640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a20101c00 0x7f3a20197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a25bc6640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a201037c0 0x7f3a2019f4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800b970 tx=0x7f3a1800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:28.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a167fc640 1 -- 172.21.15.37:0/1066166741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a18019070 con 0x7f3a201037c0 2024-09-19T06:41:28.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a167fc640 1 -- 172.21.15.37:0/1066166741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3a18014440 con 0x7f3a201037c0 2024-09-19T06:41:28.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a2019fe00 con 0x7f3a201037c0 2024-09-19T06:41:28.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a167fc640 1 -- 172.21.15.37:0/1066166741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a180075d0 con 0x7f3a201037c0 2024-09-19T06:41:28.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.959+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a201a0330 con 0x7f3a201037c0 2024-09-19T06:41:28.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.960+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a20109000 con 0x7f3a201037c0 2024-09-19T06:41:28.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.962+0000 7f3a167fc640 1 -- 172.21.15.37:0/1066166741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3a1801c070 con 0x7f3a201037c0 2024-09-19T06:41:28.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.962+0000 7f3a167fc640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3a0c0a32f0 0x7f3a0c0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:28.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.963+0000 7f3a253c5640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3a0c0a32f0 0x7f3a0c0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:28.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.963+0000 7f3a167fc640 1 -- 172.21.15.37:0/1066166741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f3a180cf290 con 0x7f3a201037c0 2024-09-19T06:41:28.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.963+0000 7f3a253c5640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3a0c0a32f0 0x7f3a0c0a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005ab0 tx=0x7f3a1c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:28.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:28.966+0000 7f3a167fc640 1 -- 172.21.15.37:0/1066166741 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3a1809b8e0 con 0x7f3a201037c0 2024-09-19T06:41:29.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.180+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f3a2002da10 con 0x7f3a201037c0 2024-09-19T06:41:29.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.180+0000 7f3a167fc640 1 -- 172.21.15.37:0/1066166741 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3081 (secure 0 0 0) 0x7f3a1809fbe0 con 0x7f3a201037c0 2024-09-19T06:41:29.181 INFO:teuthology.orchestra.run.smithi037.stdout:e24 2024-09-19T06:41:29.181 INFO:teuthology.orchestra.run.smithi037.stdout:btime 2024-09-19T06:40:04:443772+0000 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:legacy client fscid: 1 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:Filesystem 'cephfs' (1) 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:fs_name cephfs 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 24 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-09-19T06:39:53.022194+0000 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-09-19T06:40:04.075579+0000 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:tableserver 0 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:root 0 2024-09-19T06:41:29.182 INFO:teuthology.orchestra.run.smithi037.stdout:session_timeout 300 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:session_autoclose 300 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:max_file_size 1099511627776 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:max_xattr_size 65536 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:required_client_features {} 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure 0 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:last_failure_osd_epoch 0 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:max_mds 3 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:in 0,1,2 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:up {0=24497,1=24470,2=24449} 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:failed 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:damaged 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:stopped 2024-09-19T06:41:29.183 INFO:teuthology.orchestra.run.smithi037.stdout:data_pools [3] 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:metadata_pool 2 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:inline_data disabled 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:balancer 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:bal_rank_mask -1 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:standby_count_wanted 0 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:qdb_cluster leader: 24497 members: 24449,24470,24497 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.b{0:24497} state up:active seq 4 addr [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.i{0:24445} state up:standby-replay seq 1 addr [v2:172.21.15.151:6838/3546164673,v1:172.21.15.151:6839/3546164673] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.h{1:24470} state up:active seq 9 addr [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.c{1:24430} state up:standby-replay seq 1 addr [v2:172.21.15.151:6836/3741792814,v1:172.21.15.151:6837/3741792814] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.e{2:24449} state up:active seq 11 addr [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.k{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.96:6832/672218237,v1:172.21.15.96:6833/672218237] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.184 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout:Standby daemons: 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.37:6834/146682084,v1:172.21.15.37:6835/146682084] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.37:6836/773501481,v1:172.21.15.37:6837/773501481] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.g{-1:14598} state up:standby seq 1 addr [v2:172.21.15.37:6838/2512161212,v1:172.21.15.37:6839/2512161212] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.a{-1:14610} state up:standby seq 1 addr [v2:172.21.15.37:6840/2995019412,v1:172.21.15.37:6841/2995019412] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.l{-1:24400} state up:standby seq 1 addr [v2:172.21.15.151:6832/3007825106,v1:172.21.15.151:6833/3007825106] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.185 INFO:teuthology.orchestra.run.smithi037.stdout:[mds.f{-1:24424} state up:standby seq 1 addr [v2:172.21.15.151:6834/2848315285,v1:172.21.15.151:6835/2848315285] compat {c=[1],r=[1],i=[1fff]}] 2024-09-19T06:41:29.186 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-19T06:41:29.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3a0c0a32f0 msgr2=0x7f3a0c0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:29.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3a0c0a32f0 0x7f3a0c0a57b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c005ab0 tx=0x7f3a1c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a201037c0 msgr2=0x7f3a2019f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a201037c0 0x7f3a2019f4a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800b970 tx=0x7f3a1800bd70 comp rx=0 tx=0).stop 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 shutdown_connections 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f3a0c0a32f0 0x7f3a0c0a57b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3a201037c0 0x7f3a2019f4a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a20102e00 0x7f3a20198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 --2- 172.21.15.37:0/1066166741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3a20101c00 0x7f3a20197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.182+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 >> 172.21.15.37:0/1066166741 conn(0x7f3a200fd390 msgr2=0x7f3a201061d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.183+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 shutdown_connections 2024-09-19T06:41:29.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:29.183+0000 7f3a27629640 1 -- 172.21.15.37:0/1066166741 wait complete. 2024-09-19T06:41:29.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:29 smithi037 ceph-mon[31211]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:41:29.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:29 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1066166741' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:41:29.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:29 smithi151 ceph-mon[35236]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:41:29.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:29 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1066166741' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:41:29.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:29 smithi096 ceph-mon[35004]: pgmap v329: 97 pgs: 97 active+clean; 638 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-19T06:41:29.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:29 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1066166741' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-19T06:41:29.905 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-19T06:41:30.167 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:30.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.623+0000 7fbaf4c43640 1 -- 172.21.15.37:0/1636281590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0101c00 msgr2=0x7fbaf0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.623+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/1636281590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0101c00 0x7fbaf0102000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009900 tx=0x7fbae002a410 comp rx=0 tx=0).stop 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.623+0000 7fbaf4c43640 1 -- 172.21.15.37:0/1636281590 shutdown_connections 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.623+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/1636281590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaf01037c0 0x7fbaf010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.623+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/1636281590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaf0102e00 0x7fbaf0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.623+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/1636281590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0101c00 0x7fbaf0102000 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.623+0000 7fbaf4c43640 1 -- 172.21.15.37:0/1636281590 >> 172.21.15.37:0/1636281590 conn(0x7fbaf00fd390 msgr2=0x7fbaf00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.624+0000 7fbaf4c43640 1 -- 172.21.15.37:0/1636281590 shutdown_connections 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.624+0000 7fbaf4c43640 1 -- 172.21.15.37:0/1636281590 wait complete. 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.624+0000 7fbaf4c43640 1 Processor -- start 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.624+0000 7fbaf4c43640 1 -- start start 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.624+0000 7fbaf4c43640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaf0101c00 0x7fbaf0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:30.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaf4c43640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0102e00 0x7fbaf0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaf4c43640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaf01037c0 0x7fbaf019f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaf4c43640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf010d8e0 con 0x7fbaf0102e00 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaf4c43640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf010d760 con 0x7fbaf0101c00 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaf4c43640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf010da60 con 0x7fbaf01037c0 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0102e00 0x7fbaf0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0102e00 0x7fbaf0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49082/0 (socket says 172.21.15.37:49082) 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 -- 172.21.15.37:0/2653698477 learned_addr learned my addr 172.21.15.37:0/2653698477 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaee59c640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaf0101c00 0x7fbaf0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 -- 172.21.15.37:0/2653698477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaf01037c0 msgr2=0x7fbaf019f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaf01037c0 0x7fbaf019f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 -- 172.21.15.37:0/2653698477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaf0101c00 msgr2=0x7fbaf0197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaf0101c00 0x7fbaf0197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 -- 172.21.15.37:0/2653698477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf019fb50 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaedd9b640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0102e00 0x7fbaf0198460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbad800b4a0 tx=0x7fbad800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbae77fe640 1 -- 172.21.15.37:0/2653698477 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8014070 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf019fe40 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbae77fe640 1 -- 172.21.15.37:0/2653698477 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad8010af0 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbae77fe640 1 -- 172.21.15.37:0/2653698477 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8017d10 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.625+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf01a03c0 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.626+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf0108fa0 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.628+0000 7fbae77fe640 1 -- 172.21.15.37:0/2653698477 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbad8017070 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.628+0000 7fbae77fe640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbadc0a3270 0x7fbadc0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.629+0000 7fbaee59c640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbadc0a3270 0x7fbadc0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.629+0000 7fbae77fe640 1 -- 172.21.15.37:0/2653698477 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fbad80cf140 con 0x7fbaf0102e00 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.629+0000 7fbaee59c640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbadc0a3270 0x7fbadc0a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbae0006010 tx=0x7fbae0000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:30.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.632+0000 7fbae77fe640 1 -- 172.21.15.37:0/2653698477 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbad809bde0 con 0x7fbaf0102e00 2024-09-19T06:41:30.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.808+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fbaf002d9c0 con 0x7fbaf0102e00 2024-09-19T06:41:30.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.809+0000 7fbae77fe640 1 -- 172.21.15.37:0/2653698477 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v83) ==== 56+0+5328 (secure 0 0 0) 0x7fbad809b6e0 con 0x7fbaf0102e00 2024-09-19T06:41:30.810 INFO:teuthology.orchestra.run.smithi037.stdout:epoch 83 2024-09-19T06:41:30.810 INFO:teuthology.orchestra.run.smithi037.stdout:fsid b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:created 2024-09-19T06:32:04.919737+0000 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:modified 2024-09-19T06:40:58.113982+0000 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:crush_version 26 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:full_ratio 0.95 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:backfillfull_ratio 0.9 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:nearfull_ratio 0.85 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:require_min_compat_client luminous 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:min_compat_client luminous 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:require_osd_release squid 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:stretch_mode_enabled false 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-19T06:41:30.811 INFO:teuthology.orchestra.run.smithi037.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 81 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:max_osd 12 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.0 up in weight 1 up_from 8 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6802/2764918113,v1:172.21.15.37:6803/2764918113] [v2:172.21.15.37:6804/2764918113,v1:172.21.15.37:6805/2764918113] exists,up 49e46443-fcf6-403e-95af-871ca077aee4 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/515532313,v1:172.21.15.96:6801/515532313] [v2:172.21.15.96:6802/515532313,v1:172.21.15.96:6803/515532313] exists,up 971dd8c5-2680-4d3a-812a-a7e391ff5560 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.2 up in weight 1 up_from 18 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6800/1198314257,v1:172.21.15.151:6801/1198314257] [v2:172.21.15.151:6802/1198314257,v1:172.21.15.151:6803/1198314257] exists,up fd59ee43-9b0c-4440-a57c-7f944560b77c 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.3 up in weight 1 up_from 25 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6810/1041463019,v1:172.21.15.37:6811/1041463019] [v2:172.21.15.37:6812/1041463019,v1:172.21.15.37:6813/1041463019] exists,up 77fbc8c9-cfb7-49d4-a395-c747b5061988 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.4 up in weight 1 up_from 32 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/3832301595,v1:172.21.15.96:6809/3832301595] [v2:172.21.15.96:6810/3832301595,v1:172.21.15.96:6811/3832301595] exists,up c58779a9-0c41-444b-b1c1-93e50dbca0eb 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.5 up in weight 1 up_from 37 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6808/4035837098,v1:172.21.15.151:6809/4035837098] [v2:172.21.15.151:6810/4035837098,v1:172.21.15.151:6811/4035837098] exists,up 9318d75e-c472-4c7e-b758-53c7f9cb12e9 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.6 up in weight 1 up_from 44 up_thru 79 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6818/3128436148,v1:172.21.15.37:6819/3128436148] [v2:172.21.15.37:6820/3128436148,v1:172.21.15.37:6821/3128436148] exists,up b76fc6b5-0f24-447c-9541-f19b2dcf0066 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.7 up in weight 1 up_from 49 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/4105644959,v1:172.21.15.96:6817/4105644959] [v2:172.21.15.96:6818/4105644959,v1:172.21.15.96:6819/4105644959] exists,up 57af0c79-adbc-4e9c-a848-4586e582c9b6 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.8 up in weight 1 up_from 56 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6816/2450296865,v1:172.21.15.151:6817/2450296865] [v2:172.21.15.151:6818/2450296865,v1:172.21.15.151:6819/2450296865] exists,up f560456d-ef7e-4b12-8bcf-be9bab3c3def 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.9 up in weight 1 up_from 63 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.37:6826/3305545849,v1:172.21.15.37:6827/3305545849] [v2:172.21.15.37:6828/3305545849,v1:172.21.15.37:6829/3305545849] exists,up ab6dfa58-1497-4cee-9145-64b049b10e93 2024-09-19T06:41:30.812 INFO:teuthology.orchestra.run.smithi037.stdout:osd.10 up in weight 1 up_from 68 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/2313306979,v1:172.21.15.96:6825/2313306979] [v2:172.21.15.96:6826/2313306979,v1:172.21.15.96:6827/2313306979] exists,up 8817f0ca-2ce3-483e-a70c-68d96cacdbd6 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:osd.11 up in weight 1 up_from 75 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.151:6824/3357779771,v1:172.21.15.151:6825/3357779771] [v2:172.21.15.151:6826/3357779771,v1:172.21.15.151:6827/3357779771] exists,up d075a370-835b-443b-a582-a19ac7a79167 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.5 [8,11] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.c [2,4] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.12 [5,11] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1a [8,1] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 2.1d [2,4] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.0 [5,9] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.11 [4,8] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.13 [4,9] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.14 [4,6] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:pg_upmap_items 3.27 [3,9] 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/2601148313 expires 2024-09-20T06:32:56.109611+0000 2024-09-19T06:41:30.813 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3941836172 expires 2024-09-20T06:32:56.109611+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6801/2963797247 expires 2024-09-20T06:32:32.108497+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/2963797247 expires 2024-09-20T06:32:32.108497+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/1459260343 expires 2024-09-20T06:32:56.109611+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:6800/2601148313 expires 2024-09-20T06:32:56.109611+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2641687433 expires 2024-09-20T06:32:56.109611+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/180594249 expires 2024-09-20T06:32:32.108497+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/3313349719 expires 2024-09-20T06:32:32.108497+0000 2024-09-19T06:41:30.814 INFO:teuthology.orchestra.run.smithi037.stdout:blocklist 172.21.15.37:0/2591230747 expires 2024-09-20T06:32:32.108497+0000 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbadc0a3270 msgr2=0x7fbadc0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbadc0a3270 0x7fbadc0a5730 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbae0006010 tx=0x7fbae0000bc0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0102e00 msgr2=0x7fbaf0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0102e00 0x7fbaf0198460 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbad800b4a0 tx=0x7fbad800b960 comp rx=0 tx=0).stop 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 shutdown_connections 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fbadc0a3270 0x7fbadc0a5730 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbaf01037c0 0x7fbaf019f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbaf0102e00 0x7fbaf0198460 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 --2- 172.21.15.37:0/2653698477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbaf0101c00 0x7fbaf0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:30.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 >> 172.21.15.37:0/2653698477 conn(0x7fbaf00fd390 msgr2=0x7fbaf0106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:30.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 shutdown_connections 2024-09-19T06:41:30.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:30.812+0000 7fbaf4c43640 1 -- 172.21.15.37:0/2653698477 wait complete. 2024-09-19T06:41:31.464 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:31 smithi037 ceph-mon[31211]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-19T06:41:31.465 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:31 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2653698477' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:41:31.495 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-19T06:41:31.506 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:41:31.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:31 smithi151 ceph-mon[35236]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-19T06:41:31.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:31 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2653698477' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:41:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:31.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:31 smithi096 ceph-mon[35004]: pgmap v330: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-19T06:41:31.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:31 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2653698477' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-19T06:41:32.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 -- 172.21.15.37:0/3187924590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 msgr2=0x7f7b741029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 --2- 172.21.15.37:0/3187924590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741029b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009900 tx=0x7f7b6402a480 comp rx=0 tx=0).stop 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 -- 172.21.15.37:0/3187924590 shutdown_connections 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 --2- 172.21.15.37:0/3187924590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74102ef0 0x7f7b7410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 --2- 172.21.15.37:0/3187924590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741029b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 --2- 172.21.15.37:0/3187924590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b74100e20 0x7f7b74101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 -- 172.21.15.37:0/3187924590 >> 172.21.15.37:0/3187924590 conn(0x7f7b740fc7d0 msgr2=0x7f7b740febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 -- 172.21.15.37:0/3187924590 shutdown_connections 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.226+0000 7f7b7b555640 1 -- 172.21.15.37:0/3187924590 wait complete. 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 Processor -- start 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 -- start start 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74100e20 0x7f7b74199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b74102ef0 0x7f7b741a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74110ff0 con 0x7f7b74102570 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74110e70 con 0x7f7b74100e20 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b7b555640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b74111170 con 0x7f7b74102ef0 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49100/0 (socket says 172.21.15.37:49100) 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 -- 172.21.15.37:0/1869887134 learned_addr learned my addr 172.21.15.37:0/1869887134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b79af2640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b74102ef0 0x7f7b741a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b792f1640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74100e20 0x7f7b74199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 -- 172.21.15.37:0/1869887134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b74102ef0 msgr2=0x7f7b741a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b74102ef0 0x7f7b741a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 -- 172.21.15.37:0/1869887134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74100e20 msgr2=0x7f7b74199360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74100e20 0x7f7b74199360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b78af0640 1 -- 172.21.15.37:0/1869887134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b741a4920 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b79af2640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b74102ef0 0x7f7b741a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.227+0000 7f7b792f1640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74100e20 0x7f7b74199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.228+0000 7f7b78af0640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741998a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402aa70 tx=0x7f7b64003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.228+0000 7f7b6a7fc640 1 -- 172.21.15.37:0/1869887134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b6403f070 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.228+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b741a4af0 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.228+0000 7f7b6a7fc640 1 -- 172.21.15.37:0/1869887134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b64031c20 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.228+0000 7f7b6a7fc640 1 -- 172.21.15.37:0/1869887134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b64035030 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.228+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b741a4fa0 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.229+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b74101200 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.231+0000 7f7b6a7fc640 1 -- 172.21.15.37:0/1869887134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7b64042070 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.231+0000 7f7b6a7fc640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7b600a3240 0x7f7b600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.232+0000 7f7b792f1640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7b600a3240 0x7f7b600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.232+0000 7f7b6a7fc640 1 -- 172.21.15.37:0/1869887134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7b640f0700 con 0x7f7b74102570 2024-09-19T06:41:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.232+0000 7f7b792f1640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7b600a3240 0x7f7b600a5700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009930 tx=0x7f7b70008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.235+0000 7f7b6a7fc640 1 -- 172.21.15.37:0/1869887134 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7b640bd3a0 con 0x7f7b74102570 2024-09-19T06:41:32.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.447+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7b74109090 con 0x7f7b74102570 2024-09-19T06:41:32.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.448+0000 7f7b6a7fc640 1 -- 172.21.15.37:0/1869887134 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11903 (secure 0 0 0) 0x7f7b640bcca0 con 0x7f7b74102570 2024-09-19T06:41:32.449 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:32.449 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"btime":"2024-09-19T06:40:04:443772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:40:04.075579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:41:32.449 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-19T06:41:32.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.450+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7b600a3240 msgr2=0x7f7b600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:32.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.450+0000 7f7b7b555640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7b600a3240 0x7f7b600a5700 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7b70009930 tx=0x7f7b70008040 comp rx=0 tx=0).stop 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.450+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 msgr2=0x7f7b741998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.450+0000 7f7b7b555640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741998a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7b6402aa70 tx=0x7f7b64003b60 comp rx=0 tx=0).stop 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 shutdown_connections 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7b600a3240 0x7f7b600a5700 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b74102ef0 0x7f7b741a43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b74102570 0x7f7b741998a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 --2- 172.21.15.37:0/1869887134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b74100e20 0x7f7b74199360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 >> 172.21.15.37:0/1869887134 conn(0x7f7b740fc7d0 msgr2=0x7f7b74105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 shutdown_connections 2024-09-19T06:41:32.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:32.451+0000 7f7b7b555640 1 -- 172.21.15.37:0/1869887134 wait complete. 2024-09-19T06:41:33.132 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:41:33.392 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:33.513 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:33 smithi037 ceph-mon[31211]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:33.514 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:33 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1869887134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:33.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:33 smithi151 ceph-mon[35236]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:33.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:33 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1869887134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:33.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:33 smithi096 ceph-mon[35004]: pgmap v331: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:33.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:33 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1869887134' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 -- 172.21.15.37:0/484084391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc1008d0 msgr2=0x7f97bc100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 --2- 172.21.15.37:0/484084391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc1008d0 0x7f97bc100cb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f97ac009900 tx=0x7f97ac02a410 comp rx=0 tx=0).stop 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 -- 172.21.15.37:0/484084391 shutdown_connections 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 --2- 172.21.15.37:0/484084391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97bc10e480 0x7f97bc110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 --2- 172.21.15.37:0/484084391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc101280 0x7f97bc10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 --2- 172.21.15.37:0/484084391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc1008d0 0x7f97bc100cb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 -- 172.21.15.37:0/484084391 >> 172.21.15.37:0/484084391 conn(0x7f97bc0fc7a0 msgr2=0x7f97bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:33.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 -- 172.21.15.37:0/484084391 shutdown_connections 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.841+0000 7f97c148a640 1 -- 172.21.15.37:0/484084391 wait complete. 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.842+0000 7f97c148a640 1 Processor -- start 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.842+0000 7f97c148a640 1 -- start start 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97c148a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1008d0 0x7f97bc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97c148a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97bc101280 0x7f97bc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97c148a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc10e480 0x7f97bc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97c148a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc1141b0 con 0x7f97bc10e480 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97c148a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc114030 con 0x7f97bc1008d0 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97c148a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97bc114330 con 0x7f97bc101280 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc10e480 0x7f97bc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:33.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc10e480 0x7f97bc19ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49114/0 (socket says 172.21.15.37:49114) 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 -- 172.21.15.37:0/1546039628 learned_addr learned my addr 172.21.15.37:0/1546039628 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97baffd640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1008d0 0x7f97bc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 -- 172.21.15.37:0/1546039628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97bc101280 msgr2=0x7f97bc19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97bc101280 0x7f97bc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 -- 172.21.15.37:0/1546039628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1008d0 msgr2=0x7f97bc19a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1008d0 0x7f97bc19a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 -- 172.21.15.37:0/1546039628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97bc19f610 con 0x7f97bc10e480 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97baffd640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1008d0 0x7f97bc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97bb7fe640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc10e480 0x7f97bc19ef30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009fd0 tx=0x7f97b400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97a3fff640 1 -- 172.21.15.37:0/1546039628 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b4004d20 con 0x7f97bc10e480 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97bc19f900 con 0x7f97bc10e480 2024-09-19T06:41:33.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97a3fff640 1 -- 172.21.15.37:0/1546039628 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97b4005020 con 0x7f97bc10e480 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.843+0000 7f97a3fff640 1 -- 172.21.15.37:0/1546039628 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97b40116d0 con 0x7f97bc10e480 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.844+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97bc1a71e0 con 0x7f97bc10e480 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.845+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97bc100cb0 con 0x7f97bc10e480 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.846+0000 7f97a3fff640 1 -- 172.21.15.37:0/1546039628 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f97b40051d0 con 0x7f97bc10e480 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.846+0000 7f97a3fff640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f97a40a3240 0x7f97a40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.847+0000 7f97baffd640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f97a40a3240 0x7f97a40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.847+0000 7f97a3fff640 1 -- 172.21.15.37:0/1546039628 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f97b40d0080 con 0x7f97bc10e480 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.847+0000 7f97baffd640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f97a40a3240 0x7f97a40a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005ab0 tx=0x7f97ac005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:33.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:33.850+0000 7f97a3fff640 1 -- 172.21.15.37:0/1546039628 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f97b4016030 con 0x7f97bc10e480 2024-09-19T06:41:34.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.061+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97bc02d7c0 con 0x7f97bc10e480 2024-09-19T06:41:34.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.062+0000 7f97a3fff640 1 -- 172.21.15.37:0/1546039628 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11903 (secure 0 0 0) 0x7f97b409cd20 con 0x7f97bc10e480 2024-09-19T06:41:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:34.063 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":24,"btime":"2024-09-19T06:40:04:443772+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:40:04.075579+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:41:34.063 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 24 2024-09-19T06:41:34.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f97a40a3240 msgr2=0x7f97a40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:34.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f97a40a3240 0x7f97a40a5700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f97ac005ab0 tx=0x7f97ac005a20 comp rx=0 tx=0).stop 2024-09-19T06:41:34.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc10e480 msgr2=0x7f97bc19ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:34.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc10e480 0x7f97bc19ef30 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f97b4009fd0 tx=0x7f97b400ee60 comp rx=0 tx=0).stop 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 shutdown_connections 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f97a40a3240 0x7f97a40a5700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97bc10e480 0x7f97bc19ef30 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97bc101280 0x7f97bc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 --2- 172.21.15.37:0/1546039628 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97bc1008d0 0x7f97bc19a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 >> 172.21.15.37:0/1546039628 conn(0x7f97bc0fc7a0 msgr2=0x7f97bc0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.064+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 shutdown_connections 2024-09-19T06:41:34.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:34.065+0000 7f97c148a640 1 -- 172.21.15.37:0/1546039628 wait complete. 2024-09-19T06:41:34.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:34 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1546039628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:34.666 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:34 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1546039628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:34.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:41:34.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:34 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1546039628' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:34.959 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 -- 172.21.15.37:0/4198106632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330103780 msgr2=0x7ff330107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 --2- 172.21.15.37:0/4198106632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330103780 0x7ff330107bc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff318009930 tx=0x7ff31802a420 comp rx=0 tx=0).stop 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 -- 172.21.15.37:0/4198106632 shutdown_connections 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 --2- 172.21.15.37:0/4198106632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff330108300 0x7ff33010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 --2- 172.21.15.37:0/4198106632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330103780 0x7ff330107bc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 --2- 172.21.15.37:0/4198106632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff330102dd0 0x7ff3301031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 -- 172.21.15.37:0/4198106632 >> 172.21.15.37:0/4198106632 conn(0x7ff3300fc7d0 msgr2=0x7ff3300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 -- 172.21.15.37:0/4198106632 shutdown_connections 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 -- 172.21.15.37:0/4198106632 wait complete. 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.535+0000 7ff335981640 1 Processor -- start 2024-09-19T06:41:35.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff335981640 1 -- start start 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff335981640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330102dd0 0x7ff3301985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff335981640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff330103780 0x7ff330198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff335981640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff330108300 0x7ff33019cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff335981640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff330112070 con 0x7ff330102dd0 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff335981640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff330111ef0 con 0x7ff330108300 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff335981640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3301121f0 con 0x7ff330103780 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff32effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330102dd0 0x7ff3301985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff32effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330102dd0 0x7ff3301985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49138/0 (socket says 172.21.15.37:49138) 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff32effd640 1 -- 172.21.15.37:0/459995139 learned_addr learned my addr 172.21.15.37:0/459995139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff32f7fe640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff330108300 0x7ff33019cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.536+0000 7ff32e7fc640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff330103780 0x7ff330198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff32effd640 1 -- 172.21.15.37:0/459995139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff330103780 msgr2=0x7ff330198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff32effd640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff330103780 0x7ff330198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff32effd640 1 -- 172.21.15.37:0/459995139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff330108300 msgr2=0x7ff33019cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff32effd640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff330108300 0x7ff33019cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff32effd640 1 -- 172.21.15.37:0/459995139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff33019d5a0 con 0x7ff330102dd0 2024-09-19T06:41:35.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff32effd640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330102dd0 0x7ff3301985f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff32800db80 tx=0x7ff32800b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:35.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff33497f640 1 -- 172.21.15.37:0/459995139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff328017070 con 0x7ff330102dd0 2024-09-19T06:41:35.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff33019d890 con 0x7ff330102dd0 2024-09-19T06:41:35.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff33497f640 1 -- 172.21.15.37:0/459995139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff328003b70 con 0x7ff330102dd0 2024-09-19T06:41:35.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff33497f640 1 -- 172.21.15.37:0/459995139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3280106c0 con 0x7ff330102dd0 2024-09-19T06:41:35.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.537+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3301a5170 con 0x7ff330102dd0 2024-09-19T06:41:35.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.538+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3301031b0 con 0x7ff330102dd0 2024-09-19T06:41:35.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.540+0000 7ff33497f640 1 -- 172.21.15.37:0/459995139 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ff328004020 con 0x7ff330102dd0 2024-09-19T06:41:35.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.541+0000 7ff33497f640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff31c0a3240 0x7ff31c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:35.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.541+0000 7ff32e7fc640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff31c0a3240 0x7ff31c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:35.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.541+0000 7ff33497f640 1 -- 172.21.15.37:0/459995139 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7ff3280cfa40 con 0x7ff330102dd0 2024-09-19T06:41:35.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.542+0000 7ff32e7fc640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff31c0a3240 0x7ff31c0a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff318005940 tx=0x7ff3180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:35.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.544+0000 7ff33497f640 1 -- 172.21.15.37:0/459995139 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff328020070 con 0x7ff330102dd0 2024-09-19T06:41:35.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:35 smithi151 ceph-mon[35236]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:35.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.720+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff33010a290 con 0x7ff330102dd0 2024-09-19T06:41:35.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:35 smithi037 ceph-mon[31211]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:35.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.722+0000 7ff33497f640 1 -- 172.21.15.37:0/459995139 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) ==== 74+0+22795 (secure 0 0 0) 0x7ff32809c6e0 con 0x7ff330102dd0 2024-09-19T06:41:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:35.723 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":83,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:40:58.113982+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:41:35.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff31c0a3240 msgr2=0x7ff31c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff31c0a3240 0x7ff31c0a5700 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff318005940 tx=0x7ff3180058d0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330102dd0 msgr2=0x7ff3301985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:35.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330102dd0 0x7ff3301985f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff32800db80 tx=0x7ff32800b630 comp rx=0 tx=0).stop 2024-09-19T06:41:35.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 shutdown_connections 2024-09-19T06:41:35.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7ff31c0a3240 0x7ff31c0a5700 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff330108300 0x7ff33019cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff330103780 0x7ff330198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 --2- 172.21.15.37:0/459995139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff330102dd0 0x7ff3301985f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:35.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 >> 172.21.15.37:0/459995139 conn(0x7ff3300fc7d0 msgr2=0x7ff330105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:35.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.724+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 shutdown_connections 2024-09-19T06:41:35.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:35.725+0000 7ff335981640 1 -- 172.21.15.37:0/459995139 wait complete. 2024-09-19T06:41:35.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:35 smithi096 ceph-mon[35004]: pgmap v332: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:36.415 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:39:53.022194+0000', 'modified': '2024-09-19T06:40:04.075579+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24470, 'mds_2': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/672218237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 672218237}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 672218237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3741792814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3741792814}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3741792814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6839/3546164673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 3546164673}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 3546164673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.96:6835/946199433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 946199433}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 946199433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.96:6837/3655112770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 3655112770}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 3655112770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6839/1066552330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 1066552330}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 1066552330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24449, 24470, 24497]} 2024-09-19T06:41:36.415 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T06:41:36.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-19T06:41:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:36 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/459995139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:36.676 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:36.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:36 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/459995139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:36.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:36 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/459995139' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:37.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 -- 172.21.15.37:0/928860299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 msgr2=0x7f7fa0103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/928860299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa0103120 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009900 tx=0x7f7f9c02a480 comp rx=0 tx=0).stop 2024-09-19T06:41:37.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 -- 172.21.15.37:0/928860299 shutdown_connections 2024-09-19T06:41:37.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/928860299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa010c170 0x7f7fa010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/928860299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103660 0x7f7fa010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/928860299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa0103120 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 -- 172.21.15.37:0/928860299 >> 172.21.15.37:0/928860299 conn(0x7f7fa00fc7a0 msgr2=0x7f7fa00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 -- 172.21.15.37:0/928860299 shutdown_connections 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.119+0000 7f7fa72ab640 1 -- 172.21.15.37:0/928860299 wait complete. 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 Processor -- start 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 -- start start 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa01009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103660 0x7f7fa0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa010c170 0x7f7fa0101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0111f70 con 0x7f7fa0102d40 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa0111df0 con 0x7f7fa0103660 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa72ab640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7fa01120f0 con 0x7f7fa010c170 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa5047640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa01009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa5047640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa01009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49148/0 (socket says 172.21.15.37:49148) 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa5047640 1 -- 172.21.15.37:0/4095904700 learned_addr learned my addr 172.21.15.37:0/4095904700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa4846640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103660 0x7f7fa0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.120+0000 7f7fa5848640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa010c170 0x7f7fa0101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa5047640 1 -- 172.21.15.37:0/4095904700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa010c170 msgr2=0x7f7fa0101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa5047640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa010c170 0x7f7fa0101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa5047640 1 -- 172.21.15.37:0/4095904700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103660 msgr2=0x7f7fa0100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa5047640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103660 0x7f7fa0100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa5047640 1 -- 172.21.15.37:0/4095904700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fa01a68d0 con 0x7f7fa0102d40 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa5047640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa01009a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c006010 tx=0x7f7f9c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7f967fc640 1 -- 172.21.15.37:0/4095904700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c03f030 con 0x7f7fa0102d40 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7f967fc640 1 -- 172.21.15.37:0/4095904700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9c003ce0 con 0x7f7fa0102d40 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fa01a6b60 con 0x7f7fa0102d40 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7f967fc640 1 -- 172.21.15.37:0/4095904700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c035070 con 0x7f7fa0102d40 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.121+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fa01a7040 con 0x7f7fa0102d40 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.123+0000 7f7f967fc640 1 -- 172.21.15.37:0/4095904700 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f7f9c042030 con 0x7f7fa0102d40 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 Processor -- start 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 -- start start 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315e9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c315f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559c31550090 con 0x559c315e99d0 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c314f7a70 con 0x559c315f4bb0 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa9684740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c3154c050 con 0x559c315ea2f0 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315e9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315e9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49162/0 (socket says 172.21.15.37:49162) 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 -- 172.21.15.37:0/628392857 learned_addr learned my addr 172.21.15.37:0/628392857 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5047640 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c315f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7f967fc640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7f8c0a3240 0x7f7f8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa4846640 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 -- 172.21.15.37:0/628392857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 msgr2=0x559c315f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c315f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 -- 172.21.15.37:0/628392857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 msgr2=0x559c315f6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f6fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.124+0000 7f7fa5848640 1 -- 172.21.15.37:0/628392857 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c31504130 con 0x559c315e99d0 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.125+0000 7f7fa5848640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7f8c0a3240 0x7f7f8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.125+0000 7f7f967fc640 1 -- 172.21.15.37:0/4095904700 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7f9c0efe20 con 0x7f7fa0102d40 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.125+0000 7f7fa5848640 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315e9db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f98010440 tx=0x7f7f98010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d458ac7641e5e80f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.125+0000 7f7f8b7fe640 1 -- 172.21.15.37:0/628392857 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9801d400 con 0x559c315e99d0 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.125+0000 7f7f8b7fe640 1 -- 172.21.15.37:0/628392857 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9801da80 con 0x559c315e99d0 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.125+0000 7f7f8b7fe640 1 -- 172.21.15.37:0/628392857 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9801c940 con 0x559c315e99d0 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 -- 172.21.15.37:0/628392857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 msgr2=0x559c315e9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315e9db0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f7f98010440 tx=0x7f7f98010900 comp rx=0 tx=0).stop 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa5848640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7f8c0a3240 0x7f7f8c0a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f98021000 tx=0x7f7f9801e000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:37.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 -- 172.21.15.37:0/628392857 shutdown_connections 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c315f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 --2- 172.21.15.37:0/628392857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315e9db0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 -- 172.21.15.37:0/628392857 >> 172.21.15.37:0/628392857 conn(0x559c315e5f60 msgr2=0x559c315e6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 -- 172.21.15.37:0/628392857 shutdown_connections 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 -- 172.21.15.37:0/628392857 wait complete. 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 -- start start 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.126+0000 7f7fa9684740 1 Processor -- start 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315e99d0 0x559c315f4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315ea2f0 0x559c315f17a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559c315f7780 con 0x559c315ea2f0 2024-09-19T06:41:37.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c3150af20 con 0x559c315f4bb0 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c315f7900 con 0x559c315e99d0 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 10 client.0 _ll_register_callbacks cb 0x559c314d4030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315ea2f0 0x559c315f17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315ea2f0 0x559c315f17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49176/0 (socket says 172.21.15.37:49176) 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 -- 172.21.15.37:0/1433084134 learned_addr learned my addr 172.21.15.37:0/1433084134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa5848640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa5047640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315e99d0 0x559c315f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 -- 172.21.15.37:0/1433084134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315e99d0 msgr2=0x559c315f4450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315e99d0 0x559c315f4450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 -- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 msgr2=0x559c315f1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 -- 172.21.15.37:0/1433084134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c312c12b0 con 0x559c315ea2f0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa5848640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa5047640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315e99d0 0x559c315f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa4846640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315ea2f0 0x559c315f17a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000b9e0 tx=0x7f7f9000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d48dbe47f9df13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7f88ff9640 1 -- 172.21.15.37:0/1433084134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9001b440 con 0x559c315ea2f0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7f88ff9640 1 -- 172.21.15.37:0/1433084134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7f9001b5e0 con 0x559c315ea2f0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9801da80 con 0x559c315ea2f0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.127+0000 7f7f88ff9640 1 -- 172.21.15.37:0/1433084134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f9001b7e0 con 0x559c315ea2f0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 10 client.15090 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559c31603130 con 0x559c315ea2f0 2024-09-19T06:41:37.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7f88ff9640 1 -- 172.21.15.37:0/1433084134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9801da80 con 0x559c315ea2f0 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7f88ff9640 1 -- 172.21.15.37:0/1433084134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f90019020 con 0x559c315ea2f0 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 10 client.15090 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 20 client.15090 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c316057c0 0x559c31606c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 4 client.15090 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c3155d600 con 0x559c316057c0 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.128+0000 7f7fa5047640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c316057c0 0x559c31606c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.129+0000 7f7fa5047640 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c316057c0 0x559c31606c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30d226da26a17e64 server_cookie=2e6c8b5876af4400 in_seq=0 out_seq=0 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.129+0000 7f7f88ff9640 10 client.15090 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.129+0000 7f7f88ff9640 1 -- 172.21.15.37:0/1433084134 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559c3155d600 con 0x559c316057c0 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.129+0000 7f7f88ff9640 10 client.15090 handle_command_reply: tid=0 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.129+0000 7f7fa9684740 1 client.15090 shutdown 2024-09-19T06:41:37.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315ea2f0 msgr2=0x559c315f17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315ea2f0 0x559c315f17a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7f9000b9e0 tx=0x7f7f9000bea0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 shutdown_connections 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c316057c0 0x559c31606c30 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c315f1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315ea2f0 0x559c315f17a0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7f88ff9640 0 client.15090 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 --2- 172.21.15.37:0/1433084134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315e99d0 0x559c315f4450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 >> 172.21.15.37:0/1433084134 conn(0x559c315e5f60 msgr2=0x559c315ebc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 shutdown_connections 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 1 -- 172.21.15.37:0/1433084134 wait complete. 2024-09-19T06:41:37.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.130+0000 7f7fa9684740 20 client.15090 trim_cache size 0 max 16384 2024-09-19T06:41:37.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 Processor -- start 2024-09-19T06:41:37.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 -- start start 2024-09-19T06:41:37.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315eea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315ea2f0 0x559c315eef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315f4bb0 0x559c3160ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559c31604f70 con 0x559c315e99d0 2024-09-19T06:41:37.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c31605640 con 0x559c315ea2f0 2024-09-19T06:41:37.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa9684740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c31605d10 con 0x559c315f4bb0 2024-09-19T06:41:37.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa5047640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315eea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa5047640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315eea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49178/0 (socket says 172.21.15.37:49178) 2024-09-19T06:41:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.148+0000 7f7fa5047640 1 -- 172.21.15.37:0/483582534 learned_addr learned my addr 172.21.15.37:0/483582534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa4846640 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315ea2f0 0x559c315eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5848640 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315f4bb0 0x559c3160ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5047640 1 -- 172.21.15.37:0/483582534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315f4bb0 msgr2=0x559c3160ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5047640 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315f4bb0 0x559c3160ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5047640 1 -- 172.21.15.37:0/483582534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315ea2f0 msgr2=0x559c315eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5047640 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315ea2f0 0x559c315eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5047640 1 -- 172.21.15.37:0/483582534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c315eaee0 con 0x559c315e99d0 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5848640 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315f4bb0 0x559c3160ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa4846640 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315ea2f0 0x559c315eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7fa5047640 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315eea00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c000c00 tx=0x7f7f9c0ed770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ebce283eab6e25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7f8affd640 1 -- 172.21.15.37:0/483582534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c049040 con 0x559c315e99d0 2024-09-19T06:41:37.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7f8affd640 1 -- 172.21.15.37:0/483582534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f9c03b580 con 0x559c315e99d0 2024-09-19T06:41:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.149+0000 7f7f8affd640 1 -- 172.21.15.37:0/483582534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0b8ba0 con 0x559c315e99d0 2024-09-19T06:41:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 -- 172.21.15.37:0/483582534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 msgr2=0x559c315eea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315eea00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c000c00 tx=0x7f7f9c0ed770 comp rx=0 tx=0).stop 2024-09-19T06:41:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 -- 172.21.15.37:0/483582534 shutdown_connections 2024-09-19T06:41:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315f4bb0 0x559c3160ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315ea2f0 0x559c315eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 --2- 172.21.15.37:0/483582534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c315eea00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 -- 172.21.15.37:0/483582534 >> 172.21.15.37:0/483582534 conn(0x559c315e5f60 msgr2=0x559c315f0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 -- 172.21.15.37:0/483582534 shutdown_connections 2024-09-19T06:41:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 -- 172.21.15.37:0/483582534 wait complete. 2024-09-19T06:41:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 -- start start 2024-09-19T06:41:37.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.150+0000 7f7fa9684740 1 Processor -- start 2024-09-19T06:41:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c31609f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c3160a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa9684740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c3166dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa9684740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559c31606ab0 con 0x559c315e99d0 2024-09-19T06:41:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa9684740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559c316063e0 con 0x559c315f4bb0 2024-09-19T06:41:37.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa9684740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559c31607180 con 0x559c315ea2f0 2024-09-19T06:41:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa9684740 10 client.0 _ll_register_callbacks cb 0x7f7f90004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c31609f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c31609f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49190/0 (socket says 172.21.15.37:49190) 2024-09-19T06:41:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 -- 172.21.15.37:0/799473850 learned_addr learned my addr 172.21.15.37:0/799473850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa4846640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c3160a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5848640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c3166dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 -- 172.21.15.37:0/799473850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 msgr2=0x559c3160a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c3160a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 -- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 msgr2=0x559c3166dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c3166dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 -- 172.21.15.37:0/799473850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3160ad20 con 0x559c315e99d0 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5848640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c3166dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7fa5047640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c31609f50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0c4bc0 tx=0x7f7f9c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d7bcb417f07343 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.151+0000 7f7f88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7f88ff9640 1 -- 172.21.15.37:0/799473850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c049050 con 0x559c315e99d0 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7f88ff9640 1 -- 172.21.15.37:0/799473850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f7f9c073c30 con 0x559c315e99d0 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559c31557b10 con 0x559c315e99d0 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7f88ff9640 1 -- 172.21.15.37:0/799473850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f7f9c03bcb0 con 0x559c315e99d0 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 10 client.15096 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:37.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x559c3166f440 con 0x559c315e99d0 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7f88ff9640 1 -- 172.21.15.37:0/799473850 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x559c31557b10 con 0x559c315e99d0 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7f88ff9640 1 -- 172.21.15.37:0/799473850 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7f7f9c071bd0 con 0x559c315e99d0 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 10 client.15096 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 10 client.15096 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 20 client.15096 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c31673d90 0x559c31672aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 4 client.15096 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.152+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "status"}) -- 0x559c314d40e0 con 0x559c31673d90 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa4846640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c31673d90 0x559c31672aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:37.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa4846640 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c31673d90 0x559c31672aa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9df6043f8d4c6813 server_cookie=d7b0c739cf0fb280 in_seq=0 out_seq=0 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7f88ff9640 10 client.15096 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7f88ff9640 1 -- 172.21.15.37:0/799473850 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559c314d40e0 con 0x559c31673d90 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7f88ff9640 10 client.15096 handle_command_reply: tid=0 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 client.15096 shutdown 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 msgr2=0x559c31609f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c31609f50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0c4bc0 tx=0x7f7f9c02a480 comp rx=0 tx=0).stop 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 shutdown_connections 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559c31673d90 0x559c31672aa0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559c315f4bb0 0x559c3166dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559c315ea2f0 0x559c3160a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 --2- 172.21.15.37:0/799473850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559c315e99d0 0x559c31609f50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 >> 172.21.15.37:0/799473850 conn(0x559c315e5f60 msgr2=0x559c315f0460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:37.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.153+0000 7f7f88ff9640 0 client.15096 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 shutdown_connections 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa9684740 1 -- 172.21.15.37:0/799473850 wait complete. 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa9684740 20 client.15096 trim_cache size 0 max 16384 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24449, 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:41:37.169 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:41:37.170 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 95.715782684000004, 2024-09-19T06:41:37.170 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 24, 2024-09-19T06:41:37.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 83, 2024-09-19T06:41:37.170 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T06:41:37.170 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 134.127527805 2024-09-19T06:41:37.170 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:37.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7f8c0a3240 msgr2=0x7f7f8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7f8c0a3240 0x7f7f8c0a5700 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7f98021000 tx=0x7f7f9801e000 comp rx=0 tx=0).stop 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 msgr2=0x7f7fa01009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa01009a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c006010 tx=0x7f7f9c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 shutdown_connections 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f7f8c0a3240 0x7f7f8c0a5700 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7fa010c170 0x7f7fa0101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7fa0103660 0x7f7fa0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 --2- 172.21.15.37:0/4095904700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7fa0102d40 0x7f7fa01009a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 >> 172.21.15.37:0/4095904700 conn(0x7f7fa00fc7a0 msgr2=0x7f7fa00fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 shutdown_connections 2024-09-19T06:41:37.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:37.154+0000 7f7fa72ab640 1 -- 172.21.15.37:0/4095904700 wait complete. 2024-09-19T06:41:37.450 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:37 smithi037 ceph-mon[31211]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:37.675 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:37 smithi096 ceph-mon[35004]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:37.764 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:37 smithi151 ceph-mon[35236]: pgmap v333: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:37.933 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:41:38.189 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:38.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 -- 172.21.15.37:0/2271322321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4100e20 msgr2=0x7fb5a4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 --2- 172.21.15.37:0/2271322321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4100e20 0x7fb5a4101200 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009900 tx=0x7fb5a002a480 comp rx=0 tx=0).stop 2024-09-19T06:41:38.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 -- 172.21.15.37:0/2271322321 shutdown_connections 2024-09-19T06:41:38.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 --2- 172.21.15.37:0/2271322321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5a4102ef0 0x7fb5a410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 --2- 172.21.15.37:0/2271322321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5a4102570 0x7fb5a41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 --2- 172.21.15.37:0/2271322321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4100e20 0x7fb5a4101200 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 -- 172.21.15.37:0/2271322321 >> 172.21.15.37:0/2271322321 conn(0x7fb5a40fc7d0 msgr2=0x7fb5a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 -- 172.21.15.37:0/2271322321 shutdown_connections 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 -- 172.21.15.37:0/2271322321 wait complete. 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 Processor -- start 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.638+0000 7fb5abede640 1 -- start start 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5abede640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5a4100e20 0x7fb5a41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5abede640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4102570 0x7fb5a4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5abede640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5a4102ef0 0x7fb5a419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5abede640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41120a0 con 0x7fb5a4102570 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5abede640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4111f20 con 0x7fb5a4102ef0 2024-09-19T06:41:38.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5abede640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4112220 con 0x7fb5a4100e20 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4102570 0x7fb5a4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4102570 0x7fb5a4198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49214/0 (socket says 172.21.15.37:49214) 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 -- 172.21.15.37:0/1941526776 learned_addr learned my addr 172.21.15.37:0/1941526776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5a4102ef0 0x7fb5a419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5a4100e20 0x7fb5a41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 -- 172.21.15.37:0/1941526776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5a4100e20 msgr2=0x7fb5a41984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5a4100e20 0x7fb5a41984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 -- 172.21.15.37:0/1941526776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5a4102ef0 msgr2=0x7fb5a419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5a4102ef0 0x7fb5a419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 -- 172.21.15.37:0/1941526776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a419d470 con 0x7fb5a4102570 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4102570 0x7fb5a4198a00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c920 tx=0x7fb59400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb59affd640 1 -- 172.21.15.37:0/1941526776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594007c70 con 0x7fb5a4102570 2024-09-19T06:41:38.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb59affd640 1 -- 172.21.15.37:0/1941526776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb594007e10 con 0x7fb5a4102570 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.639+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a419d760 con 0x7fb5a4102570 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.640+0000 7fb59affd640 1 -- 172.21.15.37:0/1941526776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594018890 con 0x7fb5a4102570 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.640+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41a5040 con 0x7fb5a4102570 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.642+0000 7fb59affd640 1 -- 172.21.15.37:0/1941526776 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb594018a70 con 0x7fb5a4102570 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.642+0000 7fb5ae2b7740 1 Processor -- start 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.642+0000 7fb5ae2b7740 1 -- start start 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 0x55e019a02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a032f0 0x55e019a0d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e019969090 con 0x55e019a032f0 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e019910a70 con 0x55e019a0dbb0 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e019965050 con 0x55e019a029d0 2024-09-19T06:41:38.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a032f0 0x55e019a0d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a032f0 0x55e019a0d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49224/0 (socket says 172.21.15.37:49224) 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/3132742264 learned_addr learned my addr 172.21.15.37:0/3132742264 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb59affd640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb5900a3240 0x7fb5900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 0x55e019a02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9479640 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb5900a3240 0x7fb5900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/3132742264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 msgr2=0x55e019a02db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 0x55e019a02db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/3132742264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 msgr2=0x55e019a0ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb59affd640 1 -- 172.21.15.37:0/1941526776 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb5940cfb30 con 0x7fb5a4102570 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/3132742264 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e01991d130 con 0x55e019a032f0 2024-09-19T06:41:38.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.643+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a032f0 0x55e019a0d670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00314e0 tx=0x7fb5a0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec162b0996af19e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb5900a3240 0x7fb5900a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0070b0 tx=0x7fb59c00c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb58ffff640 1 -- 172.21.15.37:0/3132742264 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0035030 con 0x55e019a032f0 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb58ffff640 1 -- 172.21.15.37:0/3132742264 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a0003ad0 con 0x55e019a032f0 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb58ffff640 1 -- 172.21.15.37:0/3132742264 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0031c60 con 0x55e019a032f0 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/3132742264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a032f0 msgr2=0x55e019a0d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a032f0 0x55e019a0d670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb5a00314e0 tx=0x7fb5a0004170 comp rx=0 tx=0).stop 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/3132742264 shutdown_connections 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a032f0 0x55e019a0d670 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/3132742264 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 0x55e019a02db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/3132742264 >> 172.21.15.37:0/3132742264 conn(0x55e0199fef60 msgr2=0x55e0199ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/3132742264 shutdown_connections 2024-09-19T06:41:38.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.644+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/3132742264 wait complete. 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 -- start start 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 Processor -- start 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a0d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a032f0 0x55e019a0a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e019a10780 con 0x55e019a029d0 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e019923f20 con 0x55e019a0dbb0 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e019a10900 con 0x55e019a032f0 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5ae2b7740 10 client.0 _ll_register_callbacks cb 0x55e0198ed030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5a9c7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a0d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5a9c7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a0d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49228/0 (socket says 172.21.15.37:49228) 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/1406998851 learned_addr learned my addr 172.21.15.37:0/1406998851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:38.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.645+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a032f0 0x55e019a0a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/1406998851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a032f0 msgr2=0x55e019a0a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a032f0 0x55e019a0a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 msgr2=0x55e019a0acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/1406998851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0196da2b0 con 0x55e019a029d0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a032f0 0x55e019a0a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a0d440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0031770 tx=0x7fb5a00317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333379ab2424643f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb58d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb58d7fa640 1 -- 172.21.15.37:0/1406998851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0035070 con 0x55e019a029d0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb58d7fa640 1 -- 172.21.15.37:0/1406998851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb5a00035f0 con 0x55e019a029d0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a0003ad0 con 0x55e019a029d0 2024-09-19T06:41:38.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb58d7fa640 1 -- 172.21.15.37:0/1406998851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5a0054070 con 0x55e019a029d0 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5ae2b7740 10 client.15108 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e019a1c090 con 0x55e019a029d0 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.646+0000 7fb58d7fa640 1 -- 172.21.15.37:0/1406998851 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb5a0003ad0 con 0x55e019a029d0 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb58d7fa640 1 -- 172.21.15.37:0/1406998851 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5a003c8f0 con 0x55e019a029d0 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5ae2b7740 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5ae2b7740 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5ae2b7740 20 client.15108 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a1e280 0x55e019a1f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5ae2b7740 4 client.15108 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e019976600 con 0x55e019a1e280 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a1e280 0x55e019a1f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a1e280 0x55e019a1f6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=74bae70762415ea6 server_cookie=238227e75e0b657e in_seq=0 out_seq=0 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.647+0000 7fb58d7fa640 10 client.15108 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:38.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb58d7fa640 1 -- 172.21.15.37:0/1406998851 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e019976600 con 0x55e019a1e280 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb58d7fa640 10 client.15108 handle_command_reply: tid=0 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 client.15108 shutdown 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 msgr2=0x55e019a0d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a0d440 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0031770 tx=0x7fb5a00317a0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 shutdown_connections 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a1e280 0x55e019a1f6f0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a0dbb0 0x55e019a0acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb58d7fa640 0 client.15108 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a032f0 0x55e019a0a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1406998851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a0d440 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 >> 172.21.15.37:0/1406998851 conn(0x55e0199fef60 msgr2=0x55e019a04c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 shutdown_connections 2024-09-19T06:41:38.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.648+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1406998851 wait complete. 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.649+0000 7fb5ae2b7740 20 client.15108 trim_cache size 0 max 16384 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.666+0000 7fb5ae2b7740 1 Processor -- start 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.666+0000 7fb5ae2b7740 1 -- start start 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.666+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 0x55e019a1e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a07a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a0dbb0 0x55e019a07f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e019a1da30 con 0x55e019a0dbb0 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e019a1e100 con 0x55e019a032f0 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e019a1e7d0 con 0x55e019a029d0 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a0dbb0 0x55e019a07f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a0dbb0 0x55e019a07f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49242/0 (socket says 172.21.15.37:49242) 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 -- 172.21.15.37:0/1297648394 learned_addr learned my addr 172.21.15.37:0/1297648394 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:38.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5a9479640 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a07a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 -- 172.21.15.37:0/1297648394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 msgr2=0x55e019a1e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 0x55e019a1e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24470, 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:41:38.710 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 98.243540878999994, 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 24, 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 83, 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 127.91453015800001 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 -- 172.21.15.37:0/1297648394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 msgr2=0x55e019a07a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a07a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 -- 172.21.15.37:0/1297648394 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e019a0b550 con 0x55e019a0dbb0 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a0dbb0 0x55e019a07f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb59c01a710 tx=0x7fb59c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=802b0e7f94dfbe5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:38.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb58f7fe640 1 -- 172.21.15.37:0/1297648394 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c013040 con 0x55e019a0dbb0 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb58f7fe640 1 -- 172.21.15.37:0/1297648394 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59c028070 con 0x55e019a0dbb0 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.667+0000 7fb58f7fe640 1 -- 172.21.15.37:0/1297648394 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0189a0 con 0x55e019a0dbb0 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1297648394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a0dbb0 msgr2=0x55e019a07f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a0dbb0 0x55e019a07f50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fb59c01a710 tx=0x7fb59c004a40 comp rx=0 tx=0).stop 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1297648394 shutdown_connections 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a0dbb0 0x55e019a07f50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a07a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/1297648394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a029d0 0x55e019a1e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1297648394 >> 172.21.15.37:0/1297648394 conn(0x55e0199fef60 msgr2=0x55e019a09060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1297648394 shutdown_connections 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/1297648394 wait complete. 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 -- start start 2024-09-19T06:41:38.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.668+0000 7fb5ae2b7740 1 Processor -- start 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a27f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a232a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5ae2b7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a0dbb0 0x55e019a237e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e019a1f570 con 0x55e019a029d0 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e019a1eea0 con 0x55e019a032f0 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5ae2b7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e019a1fc40 con 0x55e019a0dbb0 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5ae2b7740 10 client.0 _ll_register_callbacks cb 0x7fb5a0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a27f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a27f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49250/0 (socket says 172.21.15.37:49250) 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/2718205159 learned_addr learned my addr 172.21.15.37:0/2718205159 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9479640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a232a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/2718205159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a0dbb0 msgr2=0x55e019a237e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a0dbb0 0x55e019a237e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a0dbb0 0x55e019a237e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 msgr2=0x55e019a232a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a232a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 -- 172.21.15.37:0/2718205159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e019a23e50 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5aa47b640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a0dbb0 0x55e019a237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb5a9c7a640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a27f70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009900 tx=0x7fb5a00318a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1186e6ab1b4c52 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.669+0000 7fb58d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb58d7fa640 1 -- 172.21.15.37:0/2718205159 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a005e060 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb58d7fa640 1 -- 172.21.15.37:0/2718205159 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb5a0004400 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb59c028070 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb58d7fa640 1 -- 172.21.15.37:0/2718205159 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fb5a0035040 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 10 client.15114 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55e019a88860 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb58d7fa640 1 -- 172.21.15.37:0/2718205159 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb59c028070 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb58d7fa640 1 -- 172.21.15.37:0/2718205159 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fb5a003c440 con 0x55e019a029d0 2024-09-19T06:41:38.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 10 client.15114 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 10 client.15114 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 20 client.15114 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a8d400 0x55e019a8c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 4 client.15114 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "status"}) -- 0x55e019970b10 con 0x55e019a8d400 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.670+0000 7fb5a9479640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a8d400 0x55e019a8c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb58d7fa640 10 client.15114 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5a9479640 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a8d400 0x55e019a8c110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=368e26a082dc0727 server_cookie=2dd0459cd1d428ad in_seq=0 out_seq=0 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb58d7fa640 1 -- 172.21.15.37:0/2718205159 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e019970b10 con 0x55e019a8d400 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb58d7fa640 10 client.15114 handle_command_reply: tid=0 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 client.15114 shutdown 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 msgr2=0x55e019a27f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a27f70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0009900 tx=0x7fb5a00318a0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 shutdown_connections 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e019a8d400 0x55e019a8c110 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e019a0dbb0 0x55e019a237e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e019a032f0 0x55e019a232a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb58d7fa640 0 client.15114 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 --2- 172.21.15.37:0/2718205159 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e019a029d0 0x55e019a27f70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 >> 172.21.15.37:0/2718205159 conn(0x55e0199fef60 msgr2=0x55e019a18b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 shutdown_connections 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 1 -- 172.21.15.37:0/2718205159 wait complete. 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.671+0000 7fb5ae2b7740 20 client.15114 trim_cache size 0 max 16384 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb5900a3240 msgr2=0x7fb5900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb5900a3240 0x7fb5900a5700 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb59c0070b0 tx=0x7fb59c00c040 comp rx=0 tx=0).stop 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4102570 msgr2=0x7fb5a4198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4102570 0x7fb5a4198a00 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fb59400c920 tx=0x7fb59400cde0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 shutdown_connections 2024-09-19T06:41:38.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb5900a3240 0x7fb5900a5700 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5a4102ef0 0x7fb5a419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb5a4102570 0x7fb5a4198a00 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 --2- 172.21.15.37:0/1941526776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb5a4100e20 0x7fb5a41984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 >> 172.21.15.37:0/1941526776 conn(0x7fb5a40fc7d0 msgr2=0x7fb5a4106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 shutdown_connections 2024-09-19T06:41:38.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:38.672+0000 7fb5abede640 1 -- 172.21.15.37:0/1941526776 wait complete. 2024-09-19T06:41:39.499 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:41:39.761 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:39.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:39 smithi096 ceph-mon[35004]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:39.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:39 smithi037 ceph-mon[31211]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:39.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:39 smithi151 ceph-mon[35236]: pgmap v334: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.189+0000 7fde0a608640 1 -- 172.21.15.37:0/3887227106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04102790 msgr2=0x7fde041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.189+0000 7fde0a608640 1 --2- 172.21.15.37:0/3887227106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04102790 0x7fde041067e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009900 tx=0x7fddf802a410 comp rx=0 tx=0).stop 2024-09-19T06:41:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.189+0000 7fde0a608640 1 -- 172.21.15.37:0/3887227106 shutdown_connections 2024-09-19T06:41:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.189+0000 7fde0a608640 1 --2- 172.21.15.37:0/3887227106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde04106f10 0x7fde0410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.189+0000 7fde0a608640 1 --2- 172.21.15.37:0/3887227106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04102790 0x7fde041067e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.189+0000 7fde0a608640 1 --2- 172.21.15.37:0/3887227106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde04101de0 0x7fde041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.189+0000 7fde0a608640 1 -- 172.21.15.37:0/3887227106 >> 172.21.15.37:0/3887227106 conn(0x7fde040fd8c0 msgr2=0x7fde040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.190+0000 7fde0a608640 1 -- 172.21.15.37:0/3887227106 shutdown_connections 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.190+0000 7fde0a608640 1 -- 172.21.15.37:0/3887227106 wait complete. 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 Processor -- start 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 -- start start 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde04101de0 0x7fde041985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde04102790 0x7fde04198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04106f10 0x7fde0419ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde04112010 con 0x7fde04106f10 2024-09-19T06:41:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde04111e90 con 0x7fde04102790 2024-09-19T06:41:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde0a608640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde04112190 con 0x7fde04101de0 2024-09-19T06:41:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde08ba5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04106f10 0x7fde0419ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde08ba5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04106f10 0x7fde0419ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50120/0 (socket says 172.21.15.37:50120) 2024-09-19T06:41:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde08ba5640 1 -- 172.21.15.37:0/3307805035 learned_addr learned my addr 172.21.15.37:0/3307805035 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde037fe640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde04102790 0x7fde04198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.191+0000 7fde03fff640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde04101de0 0x7fde041985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde08ba5640 1 -- 172.21.15.37:0/3307805035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde04101de0 msgr2=0x7fde041985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde08ba5640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde04101de0 0x7fde041985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde08ba5640 1 -- 172.21.15.37:0/3307805035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde04102790 msgr2=0x7fde04198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde08ba5640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde04102790 0x7fde04198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde08ba5640 1 -- 172.21.15.37:0/3307805035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0419d560 con 0x7fde04106f10 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde08ba5640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04106f10 0x7fde0419ce80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b970 tx=0x7fddfc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde017fa640 1 -- 172.21.15.37:0/3307805035 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc019070 con 0x7fde04106f10 2024-09-19T06:41:40.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0419d850 con 0x7fde04106f10 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde017fa640 1 -- 172.21.15.37:0/3307805035 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddfc00cb40 con 0x7fde04106f10 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde017fa640 1 -- 172.21.15.37:0/3307805035 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc007650 con 0x7fde04106f10 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.192+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde041a5130 con 0x7fde04106f10 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.194+0000 7fde017fa640 1 -- 172.21.15.37:0/3307805035 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fddfc01c070 con 0x7fde04106f10 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 Processor -- start 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 -- start start 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e91db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68e9c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e9cbb0 0x55ef68e9efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ef68df8090 con 0x55ef68e922f0 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef68d9fa70 con 0x55ef68e9cbb0 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef68df4050 con 0x55ef68e919d0 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68e9c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde017fa640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fddec0a3240 0x7fddec0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68e9c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50124/0 (socket says 172.21.15.37:50124) 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 -- 172.21.15.37:0/3961481328 learned_addr learned my addr 172.21.15.37:0/3961481328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde037fe640 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e9cbb0 0x55ef68e9efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde037fe640 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e91db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde08ba5640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fddec0a3240 0x7fddec0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde017fa640 1 -- 172.21.15.37:0/3307805035 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fddfc0cefd0 con 0x7fde04106f10 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 -- 172.21.15.37:0/3961481328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 msgr2=0x55ef68e91db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e91db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 -- 172.21.15.37:0/3961481328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e9cbb0 msgr2=0x55ef68e9efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e9cbb0 0x55ef68e9efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.195+0000 7fde03fff640 1 -- 172.21.15.37:0/3961481328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef68dac130 con 0x55ef68e922f0 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.196+0000 7fde08ba5640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fddec0a3240 0x7fddec0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fddfc09b2d0 tx=0x7fddfc08d040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.196+0000 7fde03fff640 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68e9c670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fddf400b820 tx=0x7fddf400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cf9df119d64d8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.196+0000 7fddea7fc640 1 -- 172.21.15.37:0/3961481328 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf400d8d0 con 0x55ef68e922f0 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.196+0000 7fddea7fc640 1 -- 172.21.15.37:0/3961481328 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf4019070 con 0x55ef68e922f0 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.196+0000 7fddea7fc640 1 -- 172.21.15.37:0/3961481328 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf40123e0 con 0x55ef68e922f0 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 -- 172.21.15.37:0/3961481328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 msgr2=0x55ef68e9c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68e9c670 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fddf400b820 tx=0x7fddf400bce0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 -- 172.21.15.37:0/3961481328 shutdown_connections 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e9cbb0 0x55ef68e9efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68e9c670 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/3961481328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e91db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 -- 172.21.15.37:0/3961481328 >> 172.21.15.37:0/3961481328 conn(0x55ef68e8df60 msgr2=0x55ef68e8e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 -- 172.21.15.37:0/3961481328 shutdown_connections 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 -- 172.21.15.37:0/3961481328 wait complete. 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 -- start start 2024-09-19T06:41:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.197+0000 7fde0c9e1740 1 Processor -- start 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e9c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e996b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e99bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ef68e9f780 con 0x55ef68e9cbb0 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef68db2f20 con 0x55ef68e922f0 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef68e9f900 con 0x55ef68e919d0 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 10 client.0 _ll_register_callbacks cb 0x55ef68d7c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e99bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e99bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50138/0 (socket says 172.21.15.37:50138) 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 -- 172.21.15.37:0/1235831714 learned_addr learned my addr 172.21.15.37:0/1235831714 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde03fff640 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e9c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 -- 172.21.15.37:0/1235831714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 msgr2=0x55ef68e9c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e9c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 -- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 msgr2=0x55ef68e996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 -- 172.21.15.37:0/1235831714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef68b692b0 con 0x55ef68e9cbb0 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde037fe640 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e99bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009900 tx=0x7fddf8040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9975b9419a106ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fdde37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fdde37fe640 1 -- 172.21.15.37:0/1235831714 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8035040 con 0x55ef68e9cbb0 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fdde37fe640 1 -- 172.21.15.37:0/1235831714 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fddf8004200 con 0x55ef68e9cbb0 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.198+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddf4019070 con 0x55ef68e9cbb0 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fdde37fe640 1 -- 172.21.15.37:0/1235831714 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fddf8051070 con 0x55ef68e9cbb0 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 10 client.15126 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ef68eaaf90 con 0x55ef68e9cbb0 2024-09-19T06:41:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fdde37fe640 1 -- 172.21.15.37:0/1235831714 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf4019070 con 0x55ef68e9cbb0 2024-09-19T06:41:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fdde37fe640 1 -- 172.21.15.37:0/1235831714 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fddf803a040 con 0x55ef68e9cbb0 2024-09-19T06:41:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 10 client.15126 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:41:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 20 client.15126 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68ead4f0 0x55ef68eae960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 4 client.15126 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef68e05600 con 0x55ef68ead4f0 2024-09-19T06:41:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.199+0000 7fde03fff640 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68ead4f0 0x55ef68eae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.200+0000 7fdde37fe640 10 client.15126 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.200+0000 7fde03fff640 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68ead4f0 0x55ef68eae960 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ffd45f87a061e73 server_cookie=740ba2f57fc27abe in_seq=0 out_seq=0 2024-09-19T06:41:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.200+0000 7fdde37fe640 1 -- 172.21.15.37:0/1235831714 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef68e05600 con 0x55ef68ead4f0 2024-09-19T06:41:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.200+0000 7fdde37fe640 10 client.15126 handle_command_reply: tid=0 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 client.15126 shutdown 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 msgr2=0x55ef68e99bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e99bf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fddf8009900 tx=0x7fddf8040040 comp rx=0 tx=0).stop 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 shutdown_connections 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68ead4f0 0x55ef68eae960 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e99bf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fdde37fe640 0 client.15126 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1235831714 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68e9c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 >> 172.21.15.37:0/1235831714 conn(0x55ef68e8df60 msgr2=0x55ef68e93c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 shutdown_connections 2024-09-19T06:41:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1235831714 wait complete. 2024-09-19T06:41:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.201+0000 7fde0c9e1740 20 client.15126 trim_cache size 0 max 16384 2024-09-19T06:41:40.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 Processor -- start 2024-09-19T06:41:40.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 -- start start 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68eae290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e98bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e97f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ef68eacca0 con 0x55ef68e9cbb0 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef68ead370 con 0x55ef68e922f0 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef68eada40 con 0x55ef68e919d0 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e97f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.219+0000 7fde037fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e97f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50146/0 (socket says 172.21.15.37:50146) 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 -- 172.21.15.37:0/1889263635 learned_addr learned my addr 172.21.15.37:0/1889263635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68eae290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde03fff640 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e98bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 -- 172.21.15.37:0/1889263635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 msgr2=0x55ef68eae290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68eae290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 -- 172.21.15.37:0/1889263635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 msgr2=0x55ef68e98bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e98bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 -- 172.21.15.37:0/1889263635 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef68e9f890 con 0x55ef68e9cbb0 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde03fff640 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e98bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fde037fe640 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e97f80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fddf802ab40 tx=0x7fddf8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fc0a9d484b7dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fdde9ffb640 1 -- 172.21.15.37:0/1889263635 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8035070 con 0x55ef68e9cbb0 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fdde9ffb640 1 -- 172.21.15.37:0/1889263635 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf803d480 con 0x55ef68e9cbb0 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.220+0000 7fdde9ffb640 1 -- 172.21.15.37:0/1889263635 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf8004460 con 0x55ef68e9cbb0 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1889263635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 msgr2=0x55ef68e97f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e97f80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fddf802ab40 tx=0x7fddf8001e50 comp rx=0 tx=0).stop 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1889263635 shutdown_connections 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e9cbb0 0x55ef68e97f80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e922f0 0x55ef68e98bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/1889263635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e919d0 0x55ef68eae290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1889263635 >> 172.21.15.37:0/1889263635 conn(0x55ef68e8df60 msgr2=0x55ef68e99670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1889263635 shutdown_connections 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 -- 172.21.15.37:0/1889263635 wait complete. 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.221+0000 7fde0c9e1740 1 -- start start 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 1 Processor -- start 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e919d0 0x55ef68eb70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68eb0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e9cbb0 0x55ef68eb0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ef68eae7e0 con 0x55ef68e922f0 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ef68eae110 con 0x55ef68e919d0 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ef68eaeeb0 con 0x55ef68e9cbb0 2024-09-19T06:41:40.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde0c9e1740 10 client.0 _ll_register_callbacks cb 0x7fddf80314d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:40.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde03fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68eb0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde03fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68eb0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50152/0 (socket says 172.21.15.37:50152) 2024-09-19T06:41:40.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde03fff640 1 -- 172.21.15.37:0/743668001 learned_addr learned my addr 172.21.15.37:0/743668001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:40.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde037fe640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e9cbb0 0x55ef68eb0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.222+0000 7fde037fe640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e919d0 0x55ef68eb70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde03fff640 1 -- 172.21.15.37:0/743668001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e9cbb0 msgr2=0x55ef68eb0960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde03fff640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e9cbb0 0x55ef68eb0960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde03fff640 1 -- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e919d0 msgr2=0x55ef68eb70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde03fff640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e919d0 0x55ef68eb70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde03fff640 1 -- 172.21.15.37:0/743668001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef68eb0f50 con 0x55ef68e922f0 2024-09-19T06:41:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde037fe640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e919d0 0x55ef68eb70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde03fff640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68eb0420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fddf4014a90 tx=0x7fddf4014ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebeefcf77eade72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fdde37fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fdde37fe640 1 -- 172.21.15.37:0/743668001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddf402b070 con 0x55ef68e922f0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fdde37fe640 1 -- 172.21.15.37:0/743668001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fddf400dad0 con 0x55ef68e922f0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ef68dffb10 con 0x55ef68e922f0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fdde37fe640 1 -- 172.21.15.37:0/743668001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fddf4037050 con 0x55ef68e922f0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde0c9e1740 10 client.15132 fetch_fsmap learned FSMap version 24 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55ef68eb59e0 con 0x55ef68e922f0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.223+0000 7fdde37fe640 1 -- 172.21.15.37:0/743668001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddf4019040 con 0x55ef68e922f0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fdde37fe640 1 -- 172.21.15.37:0/743668001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 24) ==== 8346+0+0 (secure 0 0 0) 0x7fddf4013070 con 0x55ef68e922f0 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde0c9e1740 10 client.15132 fetch_fsmap finished waiting for FSMap version 24 2024-09-19T06:41:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde0c9e1740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde0c9e1740 20 client.15132 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68f1c2a0 0x55ef68f1afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde0c9e1740 4 client.15132 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "status"}) -- 0x55ef68dffb10 con 0x55ef68f1c2a0 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde037fe640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68f1c2a0 0x55ef68f1afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fdde37fe640 10 client.15132 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.224+0000 7fde037fe640 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68f1c2a0 0x55ef68f1afb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cbb7af7261e3b00d server_cookie=c65303c3ceef7cee in_seq=0 out_seq=0 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fdde37fe640 1 -- 172.21.15.37:0/743668001 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55ef68dffb10 con 0x55ef68f1c2a0 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fdde37fe640 10 client.15132 handle_command_reply: tid=0 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 client.15132 shutdown 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 msgr2=0x55ef68eb0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68eb0420 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fddf4014a90 tx=0x7fddf4014ac0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 shutdown_connections 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ef68f1c2a0 0x55ef68f1afb0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ef68e9cbb0 0x55ef68eb0960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ef68e922f0 0x55ef68eb0420 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 --2- 172.21.15.37:0/743668001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ef68e919d0 0x55ef68eb70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 >> 172.21.15.37:0/743668001 conn(0x55ef68e8df60 msgr2=0x55ef68e97f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fdde37fe640 0 client.15132 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 shutdown_connections 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 1 -- 172.21.15.37:0/743668001 wait complete. 2024-09-19T06:41:40.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.225+0000 7fde0c9e1740 20 client.15132 trim_cache size 0 max 16384 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24497, 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 106.818916941, 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 24, 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 83, 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 81, 2024-09-19T06:41:40.246 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 118.623361687 2024-09-19T06:41:40.247 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fddec0a3240 msgr2=0x7fddec0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fddec0a3240 0x7fddec0a5700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fddfc09b2d0 tx=0x7fddfc08d040 comp rx=0 tx=0).stop 2024-09-19T06:41:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04106f10 msgr2=0x7fde0419ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04106f10 0x7fde0419ce80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fddfc00b970 tx=0x7fddfc00be30 comp rx=0 tx=0).stop 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 shutdown_connections 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fddec0a3240 0x7fddec0a5700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde04106f10 0x7fde0419ce80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde04102790 0x7fde04198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 --2- 172.21.15.37:0/3307805035 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde04101de0 0x7fde041985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 >> 172.21.15.37:0/3307805035 conn(0x7fde040fd8c0 msgr2=0x7fde040fed30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 shutdown_connections 2024-09-19T06:41:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.226+0000 7fde0a608640 1 -- 172.21.15.37:0/3307805035 wait complete. 2024-09-19T06:41:40.855 INFO:teuthology.run_tasks:Running task sequential... 2024-09-19T06:41:40.866 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-19T06:41:40.866 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-19T06:41:40.867 INFO:teuthology.task.print:Enabling mgr modules 2024-09-19T06:41:40.867 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-09-19T06:41:40.867 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:41:40.868 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:41:40.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-09-19T06:41:40.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 -- 172.21.15.37:0/4280090637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c100a40 msgr2=0x7f1b6c100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 --2- 172.21.15.37:0/4280090637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c100a40 0x7f1b6c100e40 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c009900 tx=0x7f1b5c02a410 comp rx=0 tx=0).stop 2024-09-19T06:41:40.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 -- 172.21.15.37:0/4280090637 shutdown_connections 2024-09-19T06:41:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 --2- 172.21.15.37:0/4280090637 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c1025f0 0x7f1b6c108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 --2- 172.21.15.37:0/4280090637 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b6c101c30 0x7f1b6c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 --2- 172.21.15.37:0/4280090637 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c100a40 0x7f1b6c100e40 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 -- 172.21.15.37:0/4280090637 >> 172.21.15.37:0/4280090637 conn(0x7f1b6c0fc1e0 msgr2=0x7f1b6c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 -- 172.21.15.37:0/4280090637 shutdown_connections 2024-09-19T06:41:40.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.974+0000 7f1b7252c640 1 -- 172.21.15.37:0/4280090637 wait complete. 2024-09-19T06:41:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 Processor -- start 2024-09-19T06:41:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 -- start start 2024-09-19T06:41:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c100a40 0x7f1b6c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c101c30 0x7f1b6c19cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b6c1025f0 0x7f1b6c19f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c10c710 con 0x7f1b6c101c30 2024-09-19T06:41:40.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c10c590 con 0x7f1b6c1025f0 2024-09-19T06:41:40.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b7252c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1b6c10c890 con 0x7f1b6c100a40 2024-09-19T06:41:40.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c101c30 0x7f1b6c19cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c101c30 0x7f1b6c19cb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50170/0 (socket says 172.21.15.37:50170) 2024-09-19T06:41:40.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.975+0000 7f1b6b7fe640 1 -- 172.21.15.37:0/1609512005 learned_addr learned my addr 172.21.15.37:0/1609512005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:40.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b70ac9640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b6c1025f0 0x7f1b6c19f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6bfff640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c100a40 0x7f1b6c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6b7fe640 1 -- 172.21.15.37:0/1609512005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c100a40 msgr2=0x7f1b6c06a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6b7fe640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c100a40 0x7f1b6c06a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6b7fe640 1 -- 172.21.15.37:0/1609512005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b6c1025f0 msgr2=0x7f1b6c19f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:40.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6b7fe640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b6c1025f0 0x7f1b6c19f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:40.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6b7fe640 1 -- 172.21.15.37:0/1609512005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b6c19fa50 con 0x7f1b6c101c30 2024-09-19T06:41:40.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6bfff640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c100a40 0x7f1b6c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:40.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b6b7fe640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c101c30 0x7f1b6c19cb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c890 tx=0x7f1b6000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b697fa640 1 -- 172.21.15.37:0/1609512005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60004220 con 0x7f1b6c101c30 2024-09-19T06:41:40.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b697fa640 1 -- 172.21.15.37:0/1609512005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1b600043c0 con 0x7f1b6c101c30 2024-09-19T06:41:40.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b6c19fce0 con 0x7f1b6c101c30 2024-09-19T06:41:40.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b697fa640 1 -- 172.21.15.37:0/1609512005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b600049e0 con 0x7f1b6c101c30 2024-09-19T06:41:40.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.976+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b6c1a01d0 con 0x7f1b6c101c30 2024-09-19T06:41:40.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.977+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b6c107d60 con 0x7f1b6c101c30 2024-09-19T06:41:40.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.978+0000 7f1b697fa640 1 -- 172.21.15.37:0/1609512005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f1b600079b0 con 0x7f1b6c101c30 2024-09-19T06:41:40.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.979+0000 7f1b697fa640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1b540a32f0 0x7f1b540a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:40.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.979+0000 7f1b6bfff640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1b540a32f0 0x7f1b540a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:40.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.979+0000 7f1b697fa640 1 -- 172.21.15.37:0/1609512005 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f1b600cf9f0 con 0x7f1b6c101c30 2024-09-19T06:41:40.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.980+0000 7f1b6bfff640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1b540a32f0 0x7f1b540a57b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c02aa00 tx=0x7f1b5c000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:40.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:40.983+0000 7f1b697fa640 1 -- 172.21.15.37:0/1609512005 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1b60014030 con 0x7f1b6c101c30 2024-09-19T06:41:41.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.207+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f1b6c062e90 con 0x7f1b6c101c30 2024-09-19T06:41:41.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.374+0000 7f1b697fa640 1 -- 172.21.15.37:0/1609512005 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f1b6009c690 con 0x7f1b6c101c30 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1b540a32f0 msgr2=0x7f1b540a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1b540a32f0 0x7f1b540a57b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c02aa00 tx=0x7f1b5c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c101c30 msgr2=0x7f1b6c19cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c101c30 0x7f1b6c19cb20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000c890 tx=0x7f1b6000cd50 comp rx=0 tx=0).stop 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 shutdown_connections 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f1b540a32f0 0x7f1b540a57b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1b6c1025f0 0x7f1b6c19f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1b6c101c30 0x7f1b6c19cb20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 --2- 172.21.15.37:0/1609512005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1b6c100a40 0x7f1b6c06a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 >> 172.21.15.37:0/1609512005 conn(0x7f1b6c0fc1e0 msgr2=0x7f1b6c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 shutdown_connections 2024-09-19T06:41:41.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:41.377+0000 7f1b7252c640 1 -- 172.21.15.37:0/1609512005 wait complete. 2024-09-19T06:41:41.396 INFO:teuthology.run_tasks:Running task kclient... 2024-09-19T06:41:41.422 INFO:tasks.kclient:Mounting kernel clients... 2024-09-19T06:41:41.422 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-19T06:41:41.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:41.422 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-19T06:41:41.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:41.441 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-19T06:41:41.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:41:41.513 INFO:teuthology.orchestra.run.smithi037.stderr:Cannot find device "ceph-brx" 2024-09-19T06:41:41.513 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-19T06:41:41.513 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-19T06:41:41.513 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:41:41.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-19T06:41:41.731 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-mon[35004]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:41.731 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1609512005' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:41:41.732 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: ignoring --setuser ceph since I am not root 2024-09-19T06:41:41.732 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: ignoring --setgroup ceph since I am not root 2024-09-19T06:41:41.732 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-mgr[36368]: -- 172.21.15.96:0/413758613 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55edc86210e0 con 0x55edc85ff400 2024-09-19T06:41:41.732 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:41.527+0000 7f7bd55a1040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:41:41.732 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:41.584+0000 7f7bd55a1040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:41.732 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:41.729+0000 7f7bd55a1040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:41:41.732 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-mon[35236]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:41.733 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1609512005' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:41:41.733 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: ignoring --setuser ceph since I am not root 2024-09-19T06:41:41.733 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: ignoring --setgroup ceph since I am not root 2024-09-19T06:41:41.733 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-mgr[37049]: -- 172.21.15.151:0/3356515330 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d90bbbb0e0 con 0x55d90bb98800 2024-09-19T06:41:41.733 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:41.526+0000 7f2b28546040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:41:41.733 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:41.582+0000 7f2b28546040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:41.733 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:41.730+0000 7f2b28546040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:41:41.763 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-mon[31211]: pgmap v335: 97 pgs: 97 active+clean; 639 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-19T06:41:41.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1609512005' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-09-19T06:41:41.764 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setuser ceph since I am not root 2024-09-19T06:41:41.764 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ignoring --setgroup ceph since I am not root 2024-09-19T06:41:41.764 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:41.535+0000 7f15e69ee040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-19T06:41:41.764 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:41.600+0000 7f15e69ee040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:41.764 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:41.761+0000 7f15e69ee040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-19T06:41:41.821 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:42.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:41.796+0000 7f7bd55a1040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:41:42.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:41.940+0000 7f7bd55a1040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:42.025 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:41 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:41.842+0000 7f15e69ee040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:41:42.025 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:42 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:41.999+0000 7f15e69ee040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:42.084 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:41.796+0000 7f2b28546040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-19T06:41:42.084 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:41.942+0000 7f2b28546040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:42.084 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:42.082+0000 7f2b28546040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 -- 172.21.15.37:0/2544269549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc101df0 msgr2=0x7f25fc1021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 --2- 172.21.15.37:0/2544269549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc101df0 0x7f25fc1021d0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f25f0009900 tx=0x7f25f002a410 comp rx=0 tx=0).stop 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 -- 172.21.15.37:0/2544269549 shutdown_connections 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 --2- 172.21.15.37:0/2544269549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc106f20 0x7f25fc10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 --2- 172.21.15.37:0/2544269549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25fc1027a0 0x7f25fc1067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 --2- 172.21.15.37:0/2544269549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc101df0 0x7f25fc1021d0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 -- 172.21.15.37:0/2544269549 >> 172.21.15.37:0/2544269549 conn(0x7f25fc0fd8f0 msgr2=0x7f25fc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:42.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 -- 172.21.15.37:0/2544269549 shutdown_connections 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.276+0000 7f2603658640 1 -- 172.21.15.37:0/2544269549 wait complete. 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 Processor -- start 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 -- start start 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25fc101df0 0x7f25fc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc1027a0 0x7f25fc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc106f20 0x7f25fc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc111f90 con 0x7f25fc106f20 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc111e10 con 0x7f25fc1027a0 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2603658640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc112110 con 0x7f25fc101df0 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2600bf3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc1027a0 0x7f25fc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2601bf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc106f20 0x7f25fc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2600bf3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc1027a0 0x7f25fc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34162/0 (socket says 172.21.15.37:34162) 2024-09-19T06:41:42.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2600bf3640 1 -- 172.21.15.37:0/3746676863 learned_addr learned my addr 172.21.15.37:0/3746676863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f2601bf5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc106f20 0x7f25fc074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50222/0 (socket says 172.21.15.37:50222) 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.277+0000 7f26013f4640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25fc101df0 0x7f25fc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2600bf3640 1 -- 172.21.15.37:0/3746676863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25fc101df0 msgr2=0x7f25fc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2600bf3640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25fc101df0 0x7f25fc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2600bf3640 1 -- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc106f20 msgr2=0x7f25fc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2600bf3640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc106f20 0x7f25fc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2600bf3640 1 -- 172.21.15.37:0/3746676863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc1a68f0 con 0x7f25fc1027a0 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2601bf5640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc106f20 0x7f25fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2600bf3640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc1027a0 0x7f25fc074a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25f800d950 tx=0x7f25f800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f25ee7fc640 1 -- 172.21.15.37:0/3746676863 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f8017030 con 0x7f25fc1027a0 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f25ee7fc640 1 -- 172.21.15.37:0/3746676863 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f8004020 con 0x7f25fc1027a0 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc1a6be0 con 0x7f25fc1027a0 2024-09-19T06:41:42.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f25ee7fc640 1 -- 172.21.15.37:0/3746676863 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f80047a0 con 0x7f25fc1027a0 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.278+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc1a7120 con 0x7f25fc1027a0 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.279+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25fc1021d0 con 0x7f25fc1027a0 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.281+0000 7f25ee7fc640 1 -- 172.21.15.37:0/3746676863 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f25f80041d0 con 0x7f25fc1027a0 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.281+0000 7f25ee7fc640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 0x7f25e80a5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.281+0000 7f26013f4640 1 -- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 msgr2=0x7f25e80a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/871019432 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.281+0000 7f25ee7fc640 1 -- 172.21.15.37:0/3746676863 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f25f80cf2c0 con 0x7f25fc1027a0 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.281+0000 7f26013f4640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 0x7f25e80a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:42.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.285+0000 7f25ee7fc640 1 -- 172.21.15.37:0/3746676863 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f25f809c040 con 0x7f25fc1027a0 2024-09-19T06:41:42.377 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:42.079+0000 7f7bd55a1040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:41:42.378 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:42 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:42.151+0000 7f15e69ee040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-19T06:41:42.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.482+0000 7f26013f4640 1 -- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 msgr2=0x7f25e80a5750 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/871019432 2024-09-19T06:41:42.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.482+0000 7f26013f4640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 0x7f25e80a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:41:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.502+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f25fc075a20 con 0x7f25fc1027a0 2024-09-19T06:41:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.502+0000 7f25ee7fc640 1 -- 172.21.15.37:0/3746676863 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7f25fc075a20 con 0x7f25fc1027a0 2024-09-19T06:41:42.504 INFO:teuthology.orchestra.run.smithi037.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-19T06:41:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 msgr2=0x7f25e80a5750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:41:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 0x7f25e80a5750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc1027a0 msgr2=0x7f25fc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc1027a0 0x7f25fc074a30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f25f800d950 tx=0x7f25f800de10 comp rx=0 tx=0).stop 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 shutdown_connections 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f25e80a3290 0x7f25e80a5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f25fc106f20 0x7f25fc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f25fc1027a0 0x7f25fc074a30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 --2- 172.21.15.37:0/3746676863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f25fc101df0 0x7f25fc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 >> 172.21.15.37:0/3746676863 conn(0x7f25fc0fd8f0 msgr2=0x7f25fc0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 shutdown_connections 2024-09-19T06:41:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:42.505+0000 7f2603658640 1 -- 172.21.15.37:0/3746676863 wait complete. 2024-09-19T06:41:42.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:42 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1609512005' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:41:42.635 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:42 smithi096 ceph-mon[35004]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T06:41:42.636 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:42.390+0000 7f7bd55a1040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:41:42.636 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:42.450+0000 7f7bd55a1040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:41:42.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:42 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1609512005' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:41:42.642 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:42 smithi151 ceph-mon[35236]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T06:41:42.643 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:42.395+0000 7f2b28546040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:41:42.643 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:42.456+0000 7f2b28546040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:41:42.643 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:42.641+0000 7f2b28546040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:41:42.754 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:42 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1609512005' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-19T06:41:42.754 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:42 smithi037 ceph-mon[31211]: mgrmap e19: x(active, since 8m), standbys: z, y 2024-09-19T06:41:42.754 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:42 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:42.485+0000 7f15e69ee040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-19T06:41:42.755 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:42 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:42.556+0000 7f15e69ee040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-19T06:41:42.755 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:42 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:42.752+0000 7f15e69ee040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:41:43.024 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:42.634+0000 7f7bd55a1040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-19T06:41:43.206 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi037.front.sepia.ceph.com 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-19T06:41:43.207 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:41:43.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:43.207 DEBUG:teuthology.orchestra.run.smithi037:> ip addr 2024-09-19T06:41:43.225 INFO:teuthology.orchestra.run.smithi037.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-19T06:41:43.225 INFO:teuthology.orchestra.run.smithi037.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-19T06:41:43.225 INFO:teuthology.orchestra.run.smithi037.stdout: inet 127.0.0.1/8 scope host lo 2024-09-19T06:41:43.225 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:41:43.225 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 ::1/128 scope host 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:18 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f0 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:6b:fd:19 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp7s0f1 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:94 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f0 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: inet 172.21.15.37/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft 42175sec preferred_lft 42175sec 2024-09-19T06:41:43.226 INFO:teuthology.orchestra.run.smithi037.stdout: inet6 fe80::ec4:7aff:fe88:7794/64 scope link 2024-09-19T06:41:43.227 INFO:teuthology.orchestra.run.smithi037.stdout: valid_lft forever preferred_lft forever 2024-09-19T06:41:43.227 INFO:teuthology.orchestra.run.smithi037.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-19T06:41:43.227 INFO:teuthology.orchestra.run.smithi037.stdout: link/ether 0c:c4:7a:88:77:95 brd ff:ff:ff:ff:ff:ff 2024-09-19T06:41:43.227 INFO:teuthology.orchestra.run.smithi037.stdout: altname enp1s0f1 2024-09-19T06:41:43.227 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-19T06:41:43.227 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:41:43.227 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-19T06:41:43.227 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add name ceph-brx type bridge 2024-09-19T06:41:43.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr flush dev ceph-brx 2024-09-19T06:41:43.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set ceph-brx up 2024-09-19T06:41:43.228 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-19T06:41:43.228 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-19T06:41:43.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:41:43.428 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:43 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3746676863' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:41:43.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:43.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:43.452 DEBUG:teuthology.orchestra.run.smithi037:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-19T06:41:43.477 INFO:teuthology.orchestra.run.smithi037.stdout:1 2024-09-19T06:41:43.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:43.479 DEBUG:teuthology.orchestra.run.smithi037:> route 2024-09-19T06:41:43.554 INFO:teuthology.orchestra.run.smithi037.stdout:Kernel IP routing table 2024-09-19T06:41:43.554 INFO:teuthology.orchestra.run.smithi037.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-19T06:41:43.554 INFO:teuthology.orchestra.run.smithi037.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-19T06:41:43.554 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-19T06:41:43.554 INFO:teuthology.orchestra.run.smithi037.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-19T06:41:43.555 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:41:43.555 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-19T06:41:43.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-19T06:41:43.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-19T06:41:43.555 DEBUG:teuthology.orchestra.run.smithi037:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-19T06:41:43.555 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-19T06:41:43.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:41:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:43 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3746676863' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:41:43.774 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:43.511+0000 7f7bd55a1040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:41:43.774 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:43.578+0000 7f7bd55a1040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:41:43.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:43.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:43.842 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-19T06:41:43.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:43.859 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list-id 2024-09-19T06:41:43.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:43 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3746676863' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-19T06:41:43.862 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:43.520+0000 7f2b28546040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:41:43.862 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:43.587+0000 7f2b28546040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:41:43.915 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:41:43.915 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-19T06:41:43.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:41:43.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-19T06:41:43.915 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-19T06:41:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:41:44.013 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:43 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:43.707+0000 7f15e69ee040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-19T06:41:44.013 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:43 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:43.781+0000 7f15e69ee040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-19T06:41:44.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:44.037 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-19T06:41:44.037 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:41:44.037 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-19T06:41:44.037 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-19T06:41:44.037 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-19T06:41:44.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-19T06:41:44.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-19T06:41:44.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-19T06:41:44.038 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-19T06:41:44.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:41:44.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:44.195 DEBUG:teuthology.orchestra.run.smithi037:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-19T06:41:44.195 DEBUG:teuthology.orchestra.run.smithi037:> set -e 2024-09-19T06:41:44.195 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set brx.0 up 2024-09-19T06:41:44.195 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link set dev brx.0 master ceph-brx 2024-09-19T06:41:44.195 DEBUG:teuthology.orchestra.run.smithi037:> ') 2024-09-19T06:41:44.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:41:44.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:44.275 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:41:44.275 DEBUG:teuthology.orchestra.run.smithi037:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-19T06:41:44.295 INFO:teuthology.orchestra.run.smithi037.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-19T06:41:44.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-19T06:41:44.295 DEBUG:teuthology.orchestra.run.smithi037:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:41:44.352 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-19T06:41:44.352 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-19T06:41:44.352 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4 2024-09-19T06:41:44.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:44.352 DEBUG:teuthology.orchestra.run.smithi037:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T06:41:44.448 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T06:41:44.448 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T06:41:44.538 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:44.278+0000 7f7bd55a1040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:41:44.538 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:44.392+0000 7f7bd55a1040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:41:44.558 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:44.296+0000 7f2b28546040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:41:44.558 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:44.413+0000 7f2b28546040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:41:44.593 INFO:teuthology.orchestra.run.smithi037.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-19T06:41:44.593 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-19T06:41:44.593 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: resolved to: "172.21.15.37:3300,172.21.15.96:3300,172.21.15.151:3300" 2024-09-19T06:41:44.593 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: trying mount with new device syntax: 0@b715dee2-7650-11ef-bceb-c7b262605968.cephfs=/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4 2024-09-19T06:41:44.593 INFO:teuthology.orchestra.run.smithi037.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.37:3300/172.21.15.96:3300/172.21.15.151:3300" will pass to kernel 2024-09-19T06:41:44.596 INFO:teuthology.orchestra.run.smithi037.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-19T06:41:44.597 INFO:teuthology.orchestra.run.smithi037.stdout: You just mounted a file system that supports labels which does not 2024-09-19T06:41:44.597 INFO:teuthology.orchestra.run.smithi037.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-19T06:41:44.597 INFO:teuthology.orchestra.run.smithi037.stdout: applications will generate AVC messages and not be allowed access to 2024-09-19T06:41:44.597 INFO:teuthology.orchestra.run.smithi037.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-19T06:41:44.597 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-19T06:41:44.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:41:44.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-19T06:41:44.624 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T06:41:44.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T06:41:44.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:44.724 INFO:teuthology.orchestra.run.smithi037.stdout:client24845 2024-09-19T06:41:44.724 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:41:44.724 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:41:44.742 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T06:41:44.816 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:44.536+0000 7f7bd55a1040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:44.816 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:44.598+0000 7f7bd55a1040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:41:44.816 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:44.748+0000 7f7bd55a1040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:41:44.817 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:44.814+0000 7f7bd55a1040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:41:44.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T06:41:44.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:44.836 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:44.556+0000 7f2b28546040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:44.836 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:44.619+0000 7f2b28546040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:41:44.837 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:44.767+0000 7f2b28546040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:41:44.840 INFO:teuthology.orchestra.run.smithi037.stdout:client24845 2024-09-19T06:41:44.840 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-19T06:41:44.848 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:44 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:44.567+0000 7f15e69ee040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-19T06:41:44.848 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:44 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:44.693+0000 7f15e69ee040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-19T06:41:44.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-19T06:41:44.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:44.899 INFO:teuthology.orchestra.run.smithi037.stdout:client24845 2024-09-19T06:41:44.900 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/b715dee2-7650-11ef-bceb-c7b262605968.client24845/status') 2024-09-19T06:41:44.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-19T06:41:45.000 INFO:teuthology.orchestra.run.smithi037.stdout:instance: client.24845 (3)172.21.15.37:0/3621062609 2024-09-19T06:41:45.000 INFO:teuthology.orchestra.run.smithi037.stdout:blocklisted: false 2024-09-19T06:41:45.001 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records in 2024-09-19T06:41:45.001 INFO:teuthology.orchestra.run.smithi037.stderr:0+1 records out 2024-09-19T06:41:45.001 INFO:teuthology.orchestra.run.smithi037.stderr:71 bytes copied, 2.6692e-05 s, 2.7 MB/s 2024-09-19T06:41:45.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-19T06:41:45.009 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T06:41:45.017 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:41:45.018 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:41:45.018 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-19T06:41:45.097 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:44.874+0000 7f7bd55a1040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:41:45.097 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:45.031+0000 7f7bd55a1040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:41:45.097 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:45.095+0000 7f7bd55a1040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:41:45.112 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:44.835+0000 7f2b28546040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:41:45.112 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:44.894+0000 7f2b28546040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:41:45.112 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:45.052+0000 7f2b28546040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:41:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 -- 172.21.15.37:0/981987060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9184100a40 msgr2=0x7f9184100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 --2- 172.21.15.37:0/981987060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9184100a40 0x7f9184100e40 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9174009900 tx=0x7f917402a410 comp rx=0 tx=0).stop 2024-09-19T06:41:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 -- 172.21.15.37:0/981987060 shutdown_connections 2024-09-19T06:41:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 --2- 172.21.15.37:0/981987060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91841025f0 0x7f9184108e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 --2- 172.21.15.37:0/981987060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 0x7f91841020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 --2- 172.21.15.37:0/981987060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9184100a40 0x7f9184100e40 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 -- 172.21.15.37:0/981987060 >> 172.21.15.37:0/981987060 conn(0x7f91840fc1e0 msgr2=0x7f91840fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 -- 172.21.15.37:0/981987060 shutdown_connections 2024-09-19T06:41:45.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.122+0000 7f918a3a6640 1 -- 172.21.15.37:0/981987060 wait complete. 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 Processor -- start 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 -- start start 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9184100a40 0x7f918406a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 0x7f918419cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91841025f0 0x7f918419f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f918410c6a0 con 0x7f91841025f0 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f918410c520 con 0x7f9184101c30 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f918a3a6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f918410c820 con 0x7f9184100a40 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f9188943640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91841025f0 0x7f918419f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f91837fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 0x7f918419cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f9183fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9184100a40 0x7f918406a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:45.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f9188943640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91841025f0 0x7f918419f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50228/0 (socket says 172.21.15.37:50228) 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f9188943640 1 -- 172.21.15.37:0/706568288 learned_addr learned my addr 172.21.15.37:0/706568288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.123+0000 7f91837fe640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 0x7f918419cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34182/0 (socket says 172.21.15.37:34182) 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91837fe640 1 -- 172.21.15.37:0/706568288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9184100a40 msgr2=0x7f918406a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91837fe640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9184100a40 0x7f918406a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91837fe640 1 -- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91841025f0 msgr2=0x7f918419f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91837fe640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91841025f0 0x7f918419f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91837fe640 1 -- 172.21.15.37:0/706568288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918419fa40 con 0x7f9184101c30 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f9183fff640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9184100a40 0x7f918406a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91837fe640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 0x7f918419cb10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f917800e9f0 tx=0x7f917800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91817fa640 1 -- 172.21.15.37:0/706568288 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f917800ccc0 con 0x7f9184101c30 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91817fa640 1 -- 172.21.15.37:0/706568288 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9178010910 con 0x7f9184101c30 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918419fcd0 con 0x7f9184101c30 2024-09-19T06:41:45.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f91817fa640 1 -- 172.21.15.37:0/706568288 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9178018960 con 0x7f9184101c30 2024-09-19T06:41:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.124+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91841a01d0 con 0x7f9184101c30 2024-09-19T06:41:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.125+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9184107d60 con 0x7f9184101c30 2024-09-19T06:41:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.126+0000 7f91817fa640 1 -- 172.21.15.37:0/706568288 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7f9178010ac0 con 0x7f9184101c30 2024-09-19T06:41:45.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.127+0000 7f91817fa640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 0x7f916c0a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.127+0000 7f9183fff640 1 -- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 msgr2=0x7f916c0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/871019432 2024-09-19T06:41:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.127+0000 7f91817fa640 1 -- 172.21.15.37:0/706568288 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7f91780cfe80 con 0x7f9184101c30 2024-09-19T06:41:45.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.127+0000 7f9183fff640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 0x7f916c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:45.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.130+0000 7f91817fa640 1 -- 172.21.15.37:0/706568288 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f917809c530 con 0x7f9184101c30 2024-09-19T06:41:45.149 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:44 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:44.846+0000 7f15e69ee040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-19T06:41:45.149 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:44 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:44.912+0000 7f15e69ee040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-19T06:41:45.150 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.073+0000 7f15e69ee040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-19T06:41:45.150 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.147+0000 7f15e69ee040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-19T06:41:45.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.328+0000 7f9183fff640 1 -- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 msgr2=0x7f916c0a5800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/871019432 2024-09-19T06:41:45.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.328+0000 7f9183fff640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 0x7f916c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:41:45.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.348+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f918402d7c0 con 0x7f9184101c30 2024-09-19T06:41:45.361 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:45.162+0000 7f7bd55a1040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:41:45.361 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:45.243+0000 7f7bd55a1040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:45.361 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:45.301+0000 7f7bd55a1040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:41:45.361 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:45.359+0000 7f7bd55a1040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:41:45.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:45.116+0000 7f2b28546040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:41:45.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:45.183+0000 7f2b28546040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:41:45.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:45.265+0000 7f2b28546040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:45.384 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:45.324+0000 7f2b28546040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:41:45.385 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:45.382+0000 7f2b28546040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:41:45.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.403+0000 7f91817fa640 1 -- 172.21.15.37:0/706568288 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f91780a0830 con 0x7f9184101c30 2024-09-19T06:41:45.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.405+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 msgr2=0x7f916c0a5800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 0x7f916c0a5800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 msgr2=0x7f918419cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 0x7f918419cb10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f917800e9f0 tx=0x7f917800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 shutdown_connections 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7f916c0a3340 0x7f916c0a5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f91841025f0 0x7f918419f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9184101c30 0x7f918419cb10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 --2- 172.21.15.37:0/706568288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9184100a40 0x7f918406a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 >> 172.21.15.37:0/706568288 conn(0x7f91840fc1e0 msgr2=0x7f9184104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 shutdown_connections 2024-09-19T06:41:45.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.406+0000 7f918a3a6640 1 -- 172.21.15.37:0/706568288 wait complete. 2024-09-19T06:41:45.409 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.211+0000 7f15e69ee040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-19T06:41:45.409 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.379+0000 7f15e69ee040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-19T06:41:45.419 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 -- 172.21.15.37:0/1890805406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c1025f0 msgr2=0x7fb26c108e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 --2- 172.21.15.37:0/1890805406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c1025f0 0x7fb26c108e80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb264009ca0 tx=0x7fb26402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 -- 172.21.15.37:0/1890805406 shutdown_connections 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 --2- 172.21.15.37:0/1890805406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c1025f0 0x7fb26c108e80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 --2- 172.21.15.37:0/1890805406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb26c101c30 0x7fb26c1020b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 --2- 172.21.15.37:0/1890805406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb26c100a40 0x7fb26c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 -- 172.21.15.37:0/1890805406 >> 172.21.15.37:0/1890805406 conn(0x7fb26c0fc1e0 msgr2=0x7fb26c0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:45.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 -- 172.21.15.37:0/1890805406 shutdown_connections 2024-09-19T06:41:45.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.526+0000 7fb2747b9640 1 -- 172.21.15.37:0/1890805406 wait complete. 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 Processor -- start 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 -- start start 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c100a40 0x7fb26c06a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb26c101c30 0x7fb26c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb26c1025f0 0x7fb26c1a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10c710 con 0x7fb26c100a40 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10c590 con 0x7fb26c1025f0 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb2747b9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb26c10c890 con 0x7fb26c101c30 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c100a40 0x7fb26c06a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c100a40 0x7fb26c06a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50240/0 (socket says 172.21.15.37:50240) 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 -- 172.21.15.37:0/3215921128 learned_addr learned my addr 172.21.15.37:0/3215921128 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:45.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb271d54640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb26c101c30 0x7fb26c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272d56640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb26c1025f0 0x7fb26c1a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb26c101c30 msgr2=0x7fb26c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb26c101c30 0x7fb26c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb26c1025f0 msgr2=0x7fb26c1a1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb26c1025f0 0x7fb26c1a1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.527+0000 7fb272555640 1 -- 172.21.15.37:0/3215921128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb26c1a1c00 con 0x7fb26c100a40 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.528+0000 7fb272555640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c100a40 0x7fb26c06a410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9c0 tx=0x7fb26800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.528+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26800cde0 con 0x7fb26c100a40 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.528+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb26c1a1e90 con 0x7fb26c100a40 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.528+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb268010910 con 0x7fb26c100a40 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.528+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb268018a10 con 0x7fb26c100a40 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.528+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb26c1a2350 con 0x7fb26c100a40 2024-09-19T06:41:45.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.529+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb26c107d60 con 0x7fb26c100a40 2024-09-19T06:41:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.530+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 19) ==== 138372+0+0 (secure 0 0 0) 0x7fb26801f020 con 0x7fb26c100a40 2024-09-19T06:41:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.531+0000 7fb2637fe640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 0x7fb2580a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.531+0000 7fb271d54640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 msgr2=0x7fb2580a5830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/871019432 2024-09-19T06:41:45.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.531+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8269+0+0 (secure 0 0 0) 0x7fb2680cfb60 con 0x7fb26c100a40 2024-09-19T06:41:45.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.531+0000 7fb271d54640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 0x7fb2580a5830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-19T06:41:45.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.535+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb26809c860 con 0x7fb26c100a40 2024-09-19T06:41:45.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/706568288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:41:45.678 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-mon[31211]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:41:45.678 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.452+0000 7f15e69ee040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-19T06:41:45.678 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.523+0000 7f15e69ee040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-19T06:41:45.678 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.613+0000 7f15e69ee040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-19T06:41:45.678 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.676+0000 7f15e69ee040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-19T06:41:45.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.732+0000 7fb271d54640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 msgr2=0x7fb2580a5830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/871019432 2024-09-19T06:41:45.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.732+0000 7fb271d54640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 0x7fb2580a5830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-19T06:41:45.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:45.739+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 --> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fb26c062790 con 0x7fb2580a3370 2024-09-19T06:41:45.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/706568288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:41:45.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:45 smithi096 ceph-mon[35004]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:41:45.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/706568288' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:41:45.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:45 smithi151 ceph-mon[35236]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-19T06:41:46.025 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:45 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:45.739+0000 7f15e69ee040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-19T06:41:46.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:46.133+0000 7fb271d54640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 msgr2=0x7fb2580a5830 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.37:6800/871019432 2024-09-19T06:41:46.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:46.133+0000 7fb271d54640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 0x7fb2580a5830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-19T06:41:46.410 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y[36364]: 2024-09-19T06:41:46.150+0000 7f7bd55a1040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:41:46.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:46.425+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 20) ==== 138372+0+0 (secure 0 0 0) 0x7fb268010430 con 0x7fb26c100a40 2024-09-19T06:41:46.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:46.593+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 8 ==== mgrmap(e 21) ==== 138040+0+0 (secure 0 0 0) 0x7fb268045ca0 con 0x7fb26c100a40 2024-09-19T06:41:46.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:46.593+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 msgr2=0x7fb2580a5830 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-19T06:41:46.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:46.593+0000 7fb2637fe640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/871019432,v1:172.21.15.37:6801/871019432] conn(0x7fb2580a3370 0x7fb2580a5830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:46.612 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z[37045]: 2024-09-19T06:41:46.174+0000 7f2b28546040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:41:46.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: Standby manager daemon y restarted 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: Standby manager daemon y started 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: Standby manager daemon z restarted 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: Standby manager daemon z started 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:46.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:41:46.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:46 smithi151 ceph-mon[35236]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T06:41:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:41:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: Standby manager daemon y restarted 2024-09-19T06:41:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: Standby manager daemon y started 2024-09-19T06:41:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:41:46.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:46.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:41:46.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:46.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: Standby manager daemon z restarted 2024-09-19T06:41:46.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: Standby manager daemon z started 2024-09-19T06:41:46.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:41:46.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:46.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:41:46.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:46 smithi096 ceph-mon[35004]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:46.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-19T06:41:46.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:41:46.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: Standby manager daemon y restarted 2024-09-19T06:41:46.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: Standby manager daemon y started 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.96:0/3287328983' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: Standby manager daemon z restarted 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: Standby manager daemon z started 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-mon[31211]: from='mgr.? 172.21.15.151:0/1811587884' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-19T06:41:46.777 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.578+0000 7f15e69ee040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-19T06:41:47.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.828+0000 7f154e240640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.828+0000 7f154e240640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.828+0000 7f154e240640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.275 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.828+0000 7f154e240640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.828+0000 7f154e240640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.830+0000 7f1550244640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.830+0000 7f1550244640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.830+0000 7f1550244640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.830+0000 7f1550244640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.276 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:46.830+0000 7f1550244640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:47.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.597+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mon.0 v2:172.21.15.37:3300/0 9 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7fb268077cf0 con 0x7fb26c100a40 2024-09-19T06:41:47.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.598+0000 7fb2637fe640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb258031cc0 0x7fb2580320a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:47.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.598+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7fb26c062790 con 0x7fb258031cc0 2024-09-19T06:41:47.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.600+0000 7fb271d54640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb258031cc0 0x7fb2580320a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:47.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.604+0000 7fb271d54640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb258031cc0 0x7fb2580320a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb26c06a180 tx=0x7fb25c005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:47.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.620+0000 7fb2637fe640 1 -- 172.21.15.37:0/3215921128 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7fb26c062790 con 0x7fb258031cc0 2024-09-19T06:41:47.622 INFO:teuthology.orchestra.run.smithi037.stdout:{} 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb258031cc0 msgr2=0x7fb2580320a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb258031cc0 0x7fb2580320a0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb26c06a180 tx=0x7fb25c005f90 comp rx=0 tx=0).stop 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c100a40 msgr2=0x7fb26c06a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 --2- 172.21.15.37:0/3215921128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb26c100a40 0x7fb26c06a410 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb26800e9c0 tx=0x7fb26800ee80 comp rx=0 tx=0).stop 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb272d56640 1 -- 172.21.15.37:0/3215921128 reap_dead start 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 shutdown_connections 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 >> 172.21.15.37:0/3215921128 conn(0x7fb26c0fc1e0 msgr2=0x7fb26c104f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 shutdown_connections 2024-09-19T06:41:47.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:47.622+0000 7fb2747b9640 1 -- 172.21.15.37:0/3215921128 wait complete. 2024-09-19T06:41:47.635 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-19T06:41:47.646 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-19T06:41:48.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T06:41:48.036 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:47.611+0000 7f154b1fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:48.036 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:47.611+0000 7f154b1fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:48.036 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:47.611+0000 7f154b1fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:48.036 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:47.611+0000 7f154b1fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:48.036 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:47.611+0000 7f154b1fa640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:41:48.036 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T06:41:48.036 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: Active manager daemon x restarted 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: Activating manager daemon x 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: mgrmap e21: x(active, starting, since 0.0126839s), standbys: z, y 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:41:48.037 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:41:48.038 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: Manager daemon x is now available 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:41:48.039 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:47 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: Active manager daemon x restarted 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: Activating manager daemon x 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: mgrmap e21: x(active, starting, since 0.0126839s), standbys: z, y 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:41:48.040 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:41:48.041 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: Manager daemon x is now available 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:48.042 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:47 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: mgrmap e20: x(active, since 8m), standbys: z, y 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: Active manager daemon x restarted 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: Activating manager daemon x 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: osdmap e84: 12 total, 12 up, 12 in 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: mgrmap e21: x(active, starting, since 0.0126839s), standbys: z, y 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-19T06:41:48.043 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:41:48.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-19T06:41:48.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-19T06:41:48.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-19T06:41:48.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-19T06:41:48.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-19T06:41:48.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: Manager daemon x is now available 2024-09-19T06:41:48.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:48.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-19T06:41:48.046 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:47 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-19T06:41:48.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 -- 172.21.15.37:0/1609363851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 msgr2=0x7f220c104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 --2- 172.21.15.37:0/1609363851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c104980 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2200009900 tx=0x7f220002a410 comp rx=0 tx=0).stop 2024-09-19T06:41:48.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 -- 172.21.15.37:0/1609363851 shutdown_connections 2024-09-19T06:41:48.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 --2- 172.21.15.37:0/1609363851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f220c1077f0 0x7f220c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 --2- 172.21.15.37:0/1609363851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f220c104ec0 0x7f220c1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 --2- 172.21.15.37:0/1609363851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c104980 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 -- 172.21.15.37:0/1609363851 >> 172.21.15.37:0/1609363851 conn(0x7f220c0fc170 msgr2=0x7f220c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:48.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 -- 172.21.15.37:0/1609363851 shutdown_connections 2024-09-19T06:41:48.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.149+0000 7f2212f2a640 1 -- 172.21.15.37:0/1609363851 wait complete. 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 Processor -- start 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 -- start start 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f220c104ec0 0x7f220c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f220c1077f0 0x7f220c19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10d7d0 con 0x7f220c102590 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10d650 con 0x7f220c104ec0 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.150+0000 7f2212f2a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f220c10d950 con 0x7f220c1077f0 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50394/0 (socket says 172.21.15.37:50394) 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 -- 172.21.15.37:0/3549111443 learned_addr learned my addr 172.21.15.37:0/3549111443 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:48.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f220bfff640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f220c104ec0 0x7f220c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f22114c7640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f220c1077f0 0x7f220c19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 -- 172.21.15.37:0/3549111443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f220c1077f0 msgr2=0x7f220c19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f220c1077f0 0x7f220c19f440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 -- 172.21.15.37:0/3549111443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f220c104ec0 msgr2=0x7f220c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f220c104ec0 0x7f220c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 -- 172.21.15.37:0/3549111443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c19fab0 con 0x7f220c102590 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2210cc6640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c197e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2200009230 tx=0x7f2200031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2209ffb640 1 -- 172.21.15.37:0/3549111443 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f220003f030 con 0x7f220c102590 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2209ffb640 1 -- 172.21.15.37:0/3549111443 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2200003ad0 con 0x7f220c102590 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c19fd40 con 0x7f220c102590 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.151+0000 7f2209ffb640 1 -- 172.21.15.37:0/3549111443 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2200035070 con 0x7f220c102590 2024-09-19T06:41:48.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.152+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a0180 con 0x7f220c102590 2024-09-19T06:41:48.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.153+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220c0fdc60 con 0x7f220c102590 2024-09-19T06:41:48.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.154+0000 7f2209ffb640 1 -- 172.21.15.37:0/3549111443 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f2200042030 con 0x7f220c102590 2024-09-19T06:41:48.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.155+0000 7f2209ffb640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21fc0a3050 0x7f21fc0a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.155+0000 7f2209ffb640 1 -- 172.21.15.37:0/3549111443 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f22000f0010 con 0x7f220c102590 2024-09-19T06:41:48.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.155+0000 7f220bfff640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21fc0a3050 0x7f21fc0a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.156+0000 7f220bfff640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21fc0a3050 0x7f21fc0a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f220c1993a0 tx=0x7f21f800a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:48.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.162+0000 7f2209ffb640 1 -- 172.21.15.37:0/3549111443 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f22000bcf00 con 0x7f220c102590 2024-09-19T06:41:48.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.342+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f220c062df0 con 0x7f220c102590 2024-09-19T06:41:48.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.344+0000 7f2209ffb640 1 -- 172.21.15.37:0/3549111443 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23170 (secure 0 0 0) 0x7f22000bc800 con 0x7f220c102590 2024-09-19T06:41:48.346 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:48.346 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:41:46.581293+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3406854118":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/1436144832":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6801/871019432":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1644258429":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/711456420":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6800/871019432":"2024-09-20T06:41:46.581265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:41:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21fc0a3050 msgr2=0x7f21fc0a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21fc0a3050 0x7f21fc0a5510 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f220c1993a0 tx=0x7f21f800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 msgr2=0x7f220c197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c197e90 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2200009230 tx=0x7f2200031b60 comp rx=0 tx=0).stop 2024-09-19T06:41:48.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 shutdown_connections 2024-09-19T06:41:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f220c1077f0 0x7f220c19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f220c104ec0 0x7f220c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21fc0a3050 0x7f21fc0a5510 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 --2- 172.21.15.37:0/3549111443 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f220c102590 0x7f220c197e90 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.346+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 >> 172.21.15.37:0/3549111443 conn(0x7f220c0fc170 msgr2=0x7f220c102fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.347+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 shutdown_connections 2024-09-19T06:41:48.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.347+0000 7f2212f2a640 1 -- 172.21.15.37:0/3549111443 wait complete. 2024-09-19T06:41:48.360 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T06:35:14.622171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T06:39:49.247361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T06:39:51.369694+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:41:48.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T06:41:48.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.473+0000 7f62592f1640 1 -- 172.21.15.37:0/2594760785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 msgr2=0x7f6254108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.473+0000 7f62592f1640 1 --2- 172.21.15.37:0/2594760785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 0x7f6254108ef0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f624c009ca0 tx=0x7f624c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 -- 172.21.15.37:0/2594760785 shutdown_connections 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 --2- 172.21.15.37:0/2594760785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 0x7f6254108ef0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 --2- 172.21.15.37:0/2594760785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6254100c90 0x7f6254101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 --2- 172.21.15.37:0/2594760785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6254101ff0 0x7f62541023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 -- 172.21.15.37:0/2594760785 >> 172.21.15.37:0/2594760785 conn(0x7f62540fb910 msgr2=0x7f62540fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 -- 172.21.15.37:0/2594760785 shutdown_connections 2024-09-19T06:41:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 -- 172.21.15.37:0/2594760785 wait complete. 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 Processor -- start 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.474+0000 7f62592f1640 1 -- start start 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62592f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6254100c90 0x7f62541006f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62592f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 0x7f625419cb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62592f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6254101ff0 0x7f625419f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62592f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f625410c6a0 con 0x7f6254101650 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62592f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f625410c520 con 0x7f6254101ff0 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62592f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f625410c820 con 0x7f6254100c90 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 0x7f625419cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 0x7f625419cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50420/0 (socket says 172.21.15.37:50420) 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62527fc640 1 -- 172.21.15.37:0/886715580 learned_addr learned my addr 172.21.15.37:0/886715580 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62537fe640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6254101ff0 0x7f625419f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f6252ffd640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6254100c90 0x7f62541006f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62537fe640 1 -- 172.21.15.37:0/886715580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6254100c90 msgr2=0x7f62541006f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62537fe640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6254100c90 0x7f62541006f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62537fe640 1 -- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 msgr2=0x7f625419cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62537fe640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 0x7f625419cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62537fe640 1 -- 172.21.15.37:0/886715580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6254196950 con 0x7f6254101ff0 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.475+0000 7f62537fe640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6254101ff0 0x7f625419f1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f624c031ae0 tx=0x7f624c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.476+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c003b70 con 0x7f6254101ff0 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.476+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6254196b80 con 0x7f6254101ff0 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.476+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f624c003d10 con 0x7f6254101ff0 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.476+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f624c02fc00 con 0x7f6254101ff0 2024-09-19T06:41:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.476+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62541970a0 con 0x7f6254101ff0 2024-09-19T06:41:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.477+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f625402d7c0 con 0x7f6254101ff0 2024-09-19T06:41:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.478+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 22) ==== 138167+0+0 (secure 0 0 0) 0x7f624c004050 con 0x7f6254101ff0 2024-09-19T06:41:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.478+0000 7f623bfff640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f623c0a3050 0x7f623c0a5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.479+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f624c0f0610 con 0x7f6254101ff0 2024-09-19T06:41:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.479+0000 7f6252ffd640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f623c0a3050 0x7f623c0a5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.479+0000 7f6252ffd640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f623c0a3050 0x7f623c0a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6254069140 tx=0x7f6244005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:48.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.484+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f624c0bd500 con 0x7f6254101ff0 2024-09-19T06:41:48.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.605+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f624c0b9470 con 0x7f6254101ff0 2024-09-19T06:41:48.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.706+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6254107900 con 0x7f6254101ff0 2024-09-19T06:41:48.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.707+0000 7f623bfff640 1 -- 172.21.15.37:0/886715580 <== mon.2 v2:172.21.15.96:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f624c0c11b0 con 0x7f6254101ff0 2024-09-19T06:41:48.708 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f623c0a3050 msgr2=0x7f623c0a5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f623c0a3050 0x7f623c0a5510 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6254069140 tx=0x7f6244005f90 comp rx=0 tx=0).stop 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6254101ff0 msgr2=0x7f625419f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6254101ff0 0x7f625419f1a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f624c031ae0 tx=0x7f624c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 shutdown_connections 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6254101ff0 0x7f625419f1a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f623c0a3050 0x7f623c0a5510 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6254101650 0x7f625419cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 --2- 172.21.15.37:0/886715580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6254100c90 0x7f62541006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 >> 172.21.15.37:0/886715580 conn(0x7f62540fb910 msgr2=0x7f6254105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:48.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 shutdown_connections 2024-09-19T06:41:48.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.709+0000 7f62592f1640 1 -- 172.21.15.37:0/886715580 wait complete. 2024-09-19T06:41:48.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-mon[31211]: mgrmap e22: x(active, since 1.01855s), standbys: z, y 2024-09-19T06:41:48.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-mon[31211]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:48.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-mon[31211]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:48.721 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3549111443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:48.722 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T06:41:48.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 -- 172.21.15.37:0/2237762207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100d40 msgr2=0x7faae41011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 --2- 172.21.15.37:0/2237762207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100d40 0x7faae41011a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7faad0009900 tx=0x7faad002a410 comp rx=0 tx=0).stop 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 -- 172.21.15.37:0/2237762207 shutdown_connections 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 --2- 172.21.15.37:0/2237762207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae41016e0 0x7faae4101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 --2- 172.21.15.37:0/2237762207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100d40 0x7faae41011a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 --2- 172.21.15.37:0/2237762207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae4100400 0x7faae4100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 -- 172.21.15.37:0/2237762207 >> 172.21.15.37:0/2237762207 conn(0x7faae40fc1b0 msgr2=0x7faae40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 -- 172.21.15.37:0/2237762207 shutdown_connections 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 -- 172.21.15.37:0/2237762207 wait complete. 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 Processor -- start 2024-09-19T06:41:48.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.834+0000 7faaeb080640 1 -- start start 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faaeb080640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100400 0x7faae419a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faaeb080640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae4100d40 0x7faae419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faaeb080640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae41016e0 0x7faae41a1640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faaeb080640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faae410f920 con 0x7faae4100400 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faaeb080640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faae410f7a0 con 0x7faae41016e0 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faaeb080640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faae410faa0 con 0x7faae4100d40 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100400 0x7faae419a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100400 0x7faae419a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50454/0 (socket says 172.21.15.37:50454) 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 -- 172.21.15.37:0/3994427860 learned_addr learned my addr 172.21.15.37:0/3994427860 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae3fff640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae4100d40 0x7faae419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae961d640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae41016e0 0x7faae41a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 -- 172.21.15.37:0/3994427860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae4100d40 msgr2=0x7faae419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae4100d40 0x7faae419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 -- 172.21.15.37:0/3994427860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae41016e0 msgr2=0x7faae41a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae41016e0 0x7faae41a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 -- 172.21.15.37:0/3994427860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae41a1cb0 con 0x7faae4100400 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae8e1c640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100400 0x7faae419a090 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faad800e9f0 tx=0x7faad800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae1ffb640 1 -- 172.21.15.37:0/3994427860 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad800cd80 con 0x7faae4100400 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae1ffb640 1 -- 172.21.15.37:0/3994427860 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faad8010910 con 0x7faae4100400 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae41a1fa0 con 0x7faae4100400 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.835+0000 7faae1ffb640 1 -- 172.21.15.37:0/3994427860 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad8018990 con 0x7faae4100400 2024-09-19T06:41:48.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.836+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae41a2400 con 0x7faae4100400 2024-09-19T06:41:48.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.837+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae410b320 con 0x7faae4100400 2024-09-19T06:41:48.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.838+0000 7faae1ffb640 1 -- 172.21.15.37:0/3994427860 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7faad8010ac0 con 0x7faae4100400 2024-09-19T06:41:48.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.839+0000 7faae1ffb640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faad40a3450 0x7faad40a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:48.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.839+0000 7faae3fff640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faad40a3450 0x7faad40a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:48.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.839+0000 7faae1ffb640 1 -- 172.21.15.37:0/3994427860 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7faad80d05f0 con 0x7faae4100400 2024-09-19T06:41:48.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.840+0000 7faae3fff640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faad40a3450 0x7faad40a5910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faad0000c00 tx=0x7faad00315b0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:48.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:48.842+0000 7faae1ffb640 1 -- 172.21.15.37:0/3994427860 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7faad809c7b0 con 0x7faae4100400 2024-09-19T06:41:48.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:48 smithi151 ceph-mon[35236]: mgrmap e22: x(active, since 1.01855s), standbys: z, y 2024-09-19T06:41:48.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:48 smithi151 ceph-mon[35236]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:48.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:48 smithi151 ceph-mon[35236]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:48.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:48 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3549111443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:49.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:48 smithi096 ceph-mon[35004]: mgrmap e22: x(active, since 1.01855s), standbys: z, y 2024-09-19T06:41:49.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:48 smithi096 ceph-mon[35004]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:49.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:48 smithi096 ceph-mon[35004]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:49.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:48 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3549111443' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:49.025 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:48.741+0000 7f1530205640 -1 log_channel(cephadm) log [ERR] : [19/Sep/2024:06:41:48] ENGINE Error in HTTPServer.serve 2024-09-19T06:41:49.025 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: Traceback (most recent call last): 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._connections.run(self.expiration_interval) 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._run(expiration_interval) 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: s = self.context.wrap_socket( 2024-09-19T06:41:49.026 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:41:49.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: return self.sslsocket_class._create( 2024-09-19T06:41:49.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:41:49.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self.do_handshake() 2024-09-19T06:41:49.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:41:49.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: self._sslobj.do_handshake() 2024-09-19T06:41:49.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:41:49.027 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:41:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:41:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.026+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7faae402da50 con 0x7faae4100400 2024-09-19T06:41:49.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.027+0000 7faae1ffb640 1 -- 172.21.15.37:0/3994427860 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7faad80a0ab0 con 0x7faae4100400 2024-09-19T06:41:49.029 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-09-19T06:41:49.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faad40a3450 msgr2=0x7faad40a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faad40a3450 0x7faad40a5910 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7faad0000c00 tx=0x7faad00315b0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100400 msgr2=0x7faae419a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100400 0x7faae419a090 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7faad800e9f0 tx=0x7faad800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 shutdown_connections 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faae41016e0 0x7faae41a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faae4100d40 0x7faae419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faad40a3450 0x7faad40a5910 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 --2- 172.21.15.37:0/3994427860 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faae4100400 0x7faae419a090 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 >> 172.21.15.37:0/3994427860 conn(0x7faae40fc1b0 msgr2=0x7faae40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 shutdown_connections 2024-09-19T06:41:49.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.030+0000 7faaeb080640 1 -- 172.21.15.37:0/3994427860 wait complete. 2024-09-19T06:41:49.046 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T06:41:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 -- 172.21.15.37:0/2881303326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f241025b0 msgr2=0x7f4f24108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 --2- 172.21.15.37:0/2881303326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f241025b0 0x7f4f24108e40 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009ca0 tx=0x7f4f1c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 -- 172.21.15.37:0/2881303326 shutdown_connections 2024-09-19T06:41:49.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 --2- 172.21.15.37:0/2881303326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f241025b0 0x7f4f24108e40 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 --2- 172.21.15.37:0/2881303326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f24101bf0 0x7f4f24102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 --2- 172.21.15.37:0/2881303326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f24100a00 0x7f4f24100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 -- 172.21.15.37:0/2881303326 >> 172.21.15.37:0/2881303326 conn(0x7f4f240fc140 msgr2=0x7f4f240fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 -- 172.21.15.37:0/2881303326 shutdown_connections 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.156+0000 7f4f2b552640 1 -- 172.21.15.37:0/2881303326 wait complete. 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 Processor -- start 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 -- start start 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f24100a00 0x7f4f24197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f24101bf0 0x7f4f24198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f241025b0 0x7f4f2419f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:49.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2410d7d0 con 0x7f4f24100a00 2024-09-19T06:41:49.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2410d650 con 0x7f4f241025b0 2024-09-19T06:41:49.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f2b552640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f2410d950 con 0x7f4f24101bf0 2024-09-19T06:41:49.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f24100a00 0x7f4f24197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:49.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f24100a00 0x7f4f24197df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50468/0 (socket says 172.21.15.37:50468) 2024-09-19T06:41:49.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 -- 172.21.15.37:0/966837789 learned_addr learned my addr 172.21.15.37:0/966837789 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:49.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f29aef640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f241025b0 0x7f4f2419f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:49.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f28aed640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f24101bf0 0x7f4f24198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 -- 172.21.15.37:0/966837789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f24101bf0 msgr2=0x7f4f24198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f24101bf0 0x7f4f24198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 -- 172.21.15.37:0/966837789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f241025b0 msgr2=0x7f4f2419f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f241025b0 0x7f4f2419f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f292ee640 1 -- 172.21.15.37:0/966837789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f2419fa10 con 0x7f4f24100a00 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.157+0000 7f4f29aef640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f241025b0 0x7f4f2419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.158+0000 7f4f292ee640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f24100a00 0x7f4f24197df0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000e9c0 tx=0x7f4f2000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.158+0000 7f4f1a7fc640 1 -- 172.21.15.37:0/966837789 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f2000cd80 con 0x7f4f24100a00 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.158+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f2419fd00 con 0x7f4f24100a00 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.158+0000 7f4f1a7fc640 1 -- 172.21.15.37:0/966837789 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f200035f0 con 0x7f4f24100a00 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.158+0000 7f4f1a7fc640 1 -- 172.21.15.37:0/966837789 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f20010690 con 0x7f4f24100a00 2024-09-19T06:41:49.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.158+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f241a01e0 con 0x7f4f24100a00 2024-09-19T06:41:49.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.159+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f24107e10 con 0x7f4f24100a00 2024-09-19T06:41:49.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.160+0000 7f4f1a7fc640 1 -- 172.21.15.37:0/966837789 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f4f200037a0 con 0x7f4f24100a00 2024-09-19T06:41:49.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.161+0000 7f4f1a7fc640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4f100a3450 0x7f4f100a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:49.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.161+0000 7f4f1a7fc640 1 -- 172.21.15.37:0/966837789 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f4f200d02e0 con 0x7f4f24100a00 2024-09-19T06:41:49.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.161+0000 7f4f28aed640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4f100a3450 0x7f4f100a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:49.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.162+0000 7f4f28aed640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4f100a3450 0x7f4f100a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f140095f0 tx=0x7f4f14005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:49.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.169+0000 7f4f1a7fc640 1 -- 172.21.15.37:0/966837789 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f4f2009cbe0 con 0x7f4f24100a00 2024-09-19T06:41:49.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.350+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4f2402d7c0 con 0x7f4f24100a00 2024-09-19T06:41:49.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.350+0000 7f4f1a7fc640 1 -- 172.21.15.37:0/966837789 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f4f2009c4e0 con 0x7f4f24100a00 2024-09-19T06:41:49.352 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-09-19T06:41:49.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4f100a3450 msgr2=0x7f4f100a5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:49.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4f100a3450 0x7f4f100a5910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4f140095f0 tx=0x7f4f14005f90 comp rx=0 tx=0).stop 2024-09-19T06:41:49.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f24100a00 msgr2=0x7f4f24197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:49.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f24100a00 0x7f4f24197df0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4f2000e9c0 tx=0x7f4f2000ee80 comp rx=0 tx=0).stop 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 shutdown_connections 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f241025b0 0x7f4f2419f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f24101bf0 0x7f4f24198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4f100a3450 0x7f4f100a5910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 --2- 172.21.15.37:0/966837789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f24100a00 0x7f4f24197df0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.352+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 >> 172.21.15.37:0/966837789 conn(0x7f4f240fc140 msgr2=0x7f4f24104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.353+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 shutdown_connections 2024-09-19T06:41:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:49.353+0000 7f4f2b552640 1 -- 172.21.15.37:0/966837789 wait complete. 2024-09-19T06:41:49.366 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-19T06:41:49.366 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:41:49.642 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:49.761 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: [19/Sep/2024:06:41:48] ENGINE Bus STARTING 2024-09-19T06:41:49.761 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:49.761 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T06:41:49.761 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.761 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.761 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: [19/Sep/2024:06:41:48] ENGINE Serving on http://172.21.15.37:8765 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/886715580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: [19/Sep/2024:06:41:48] ENGINE Serving on https://172.21.15.37:7150 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: [19/Sep/2024:06:41:48] ENGINE Bus STARTED 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: [19/Sep/2024:06:41:48] ENGINE Error in HTTPServer.serve 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: Traceback (most recent call last): 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._run(expiration_interval) 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:41:49.762 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: s = self.context.wrap_socket( 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: return self.sslsocket_class._create( 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: self.do_handshake() 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: self._sslobj.do_handshake() 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.763 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3994427860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: Unable to set osd_memory_target on smithi151 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.764 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: Unable to set osd_memory_target on smithi096 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/966837789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:49.765 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:49 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: [19/Sep/2024:06:41:48] ENGINE Bus STARTING 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: [19/Sep/2024:06:41:48] ENGINE Serving on http://172.21.15.37:8765 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/886715580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:41:49.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: [19/Sep/2024:06:41:48] ENGINE Serving on https://172.21.15.37:7150 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: [19/Sep/2024:06:41:48] ENGINE Bus STARTED 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: [19/Sep/2024:06:41:48] ENGINE Error in HTTPServer.serve 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: Traceback (most recent call last): 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: self._run(expiration_interval) 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:41:49.864 INFO:journalctl@ceph.mon.c.smithi151.stdout: s = self.context.wrap_socket( 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout: return self.sslsocket_class._create( 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout: self.do_handshake() 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout: self._sslobj.do_handshake() 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3994427860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: Unable to set osd_memory_target on smithi151 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: Unable to set osd_memory_target on smithi096 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-19T06:41:49.866 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/966837789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:49.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:49 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:50.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: [19/Sep/2024:06:41:48] ENGINE Bus STARTING 2024-09-19T06:41:50.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:50.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-09-19T06:41:50.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: [19/Sep/2024:06:41:48] ENGINE Serving on http://172.21.15.37:8765 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/886715580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: [19/Sep/2024:06:41:48] ENGINE Serving on https://172.21.15.37:7150 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: [19/Sep/2024:06:41:48] ENGINE Bus STARTED 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: [19/Sep/2024:06:41:48] ENGINE Error in HTTPServer.serve 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout: Traceback (most recent call last): 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._connections.run(self.expiration_interval) 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-19T06:41:50.025 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._run(expiration_interval) 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: s = self.context.wrap_socket( 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: return self.sslsocket_class._create( 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: self.do_handshake() 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: self._sslobj.do_handshake() 2024-09-19T06:41:50.026 INFO:journalctl@ceph.mon.b.smithi096.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3994427860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: Unable to set osd_memory_target on smithi151 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.027 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: Unable to set osd_memory_target on smithi096 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/966837789' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.028 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:41:50.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:50.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:49 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 -- 172.21.15.37:0/2041008663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 msgr2=0x7f70d01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 --2- 172.21.15.37:0/2041008663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d01067e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009900 tx=0x7f70c402a410 comp rx=0 tx=0).stop 2024-09-19T06:41:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 -- 172.21.15.37:0/2041008663 shutdown_connections 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 --2- 172.21.15.37:0/2041008663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d0106f10 0x7f70d010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 --2- 172.21.15.37:0/2041008663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d01067e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 --2- 172.21.15.37:0/2041008663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70d0101de0 0x7f70d01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 -- 172.21.15.37:0/2041008663 >> 172.21.15.37:0/2041008663 conn(0x7f70d00fd8e0 msgr2=0x7f70d00ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 -- 172.21.15.37:0/2041008663 shutdown_connections 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.144+0000 7f70d689b640 1 -- 172.21.15.37:0/2041008663 wait complete. 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.145+0000 7f70d689b640 1 Processor -- start 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.145+0000 7f70d689b640 1 -- start start 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70d689b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d0101de0 0x7f70d0198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70d689b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d0198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70d689b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70d0106f10 0x7f70d019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70d689b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0111f70 con 0x7f70d0102790 2024-09-19T06:41:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70d689b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f70d0111df0 con 0x7f70d0106f10 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70d689b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f70d01120f0 con 0x7f70d0101de0 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d0198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d0198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54804/0 (socket says 172.21.15.37:54804) 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 -- 172.21.15.37:0/1836701121 learned_addr learned my addr 172.21.15.37:0/1836701121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cffff640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d0101de0 0x7f70d0198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70d4e38640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70d0106f10 0x7f70d019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 -- 172.21.15.37:0/1836701121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d0101de0 msgr2=0x7f70d0198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d0101de0 0x7f70d0198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 -- 172.21.15.37:0/1836701121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70d0106f10 msgr2=0x7f70d019cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70d0106f10 0x7f70d019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 -- 172.21.15.37:0/1836701121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70d019d3e0 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cffff640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d0101de0 0x7f70d0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.146+0000 7f70cf7fe640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d0198970 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f70c4006010 tx=0x7f70c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.147+0000 7f70cd7fa640 1 -- 172.21.15.37:0/1836701121 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c403f030 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.147+0000 7f70cd7fa640 1 -- 172.21.15.37:0/1836701121 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c4003ce0 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.147+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70d019d670 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.147+0000 7f70cd7fa640 1 -- 172.21.15.37:0/1836701121 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70c4035070 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.147+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70d0074420 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.148+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70d01032c0 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.149+0000 7f70cd7fa640 1 -- 172.21.15.37:0/1836701121 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 23) ==== 138526+0+0 (secure 0 0 0) 0x7f70c4042030 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.150+0000 7f70cd7fa640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70b80a3440 0x7f70b80a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.150+0000 7f70cffff640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70b80a3440 0x7f70b80a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.150+0000 7f70cd7fa640 1 -- 172.21.15.37:0/1836701121 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f70c40f0570 con 0x7f70d0102790 2024-09-19T06:41:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.151+0000 7f70cffff640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70b80a3440 0x7f70b80a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f70c0005d00 tx=0x7f70c000b410 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.154+0000 7f70cd7fa640 1 -- 172.21.15.37:0/1836701121 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f70c40bcea0 con 0x7f70d0102790 2024-09-19T06:41:50.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.399+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f70d010a0f0 con 0x7f70d0102790 2024-09-19T06:41:50.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.400+0000 7f70cd7fa640 1 -- 172.21.15.37:0/1836701121 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7f70c40bc7a0 con 0x7f70d0102790 2024-09-19T06:41:50.402 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:50.403 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:41:50.403 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:41:50.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70b80a3440 msgr2=0x7f70b80a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:50.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70b80a3440 0x7f70b80a5900 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f70c0005d00 tx=0x7f70c000b410 comp rx=0 tx=0).stop 2024-09-19T06:41:50.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 msgr2=0x7f70d0198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:50.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d0198970 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f70c4006010 tx=0x7f70c40042f0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 shutdown_connections 2024-09-19T06:41:50.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f70d0106f10 0x7f70d019cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70b80a3440 0x7f70b80a5900 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f70d0102790 0x7f70d0198970 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 --2- 172.21.15.37:0/1836701121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f70d0101de0 0x7f70d0198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:50.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 >> 172.21.15.37:0/1836701121 conn(0x7f70d00fd8e0 msgr2=0x7f70d00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:50.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 shutdown_connections 2024-09-19T06:41:50.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:50.404+0000 7f70d689b640 1 -- 172.21.15.37:0/1836701121 wait complete. 2024-09-19T06:41:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi037 to 177.0M 2024-09-19T06:41:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Unable to set osd_memory_target on smithi037 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T06:41:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi037:/etc/ceph/ceph.conf 2024-09-19T06:41:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1836701121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:50.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:50.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:50 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi037 to 177.0M 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Unable to set osd_memory_target on smithi037 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi037:/etc/ceph/ceph.conf 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:50.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1836701121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.864 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:50.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:50.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:50.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:50 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:51.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi037 to 177.0M 2024-09-19T06:41:51.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Unable to set osd_memory_target on smithi037 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T06:41:51.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi037:/etc/ceph/ceph.conf 2024-09-19T06:41:51.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi096:/etc/ceph/ceph.conf 2024-09-19T06:41:51.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi151:/etc/ceph/ceph.conf 2024-09-19T06:41:51.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi037:/etc/ceph/ceph.client.admin.keyring 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: Updating smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1836701121' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.025 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:51.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:51.026 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:50 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:51.389 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-19T06:41:51.389 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-19T06:41:51.389 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:41:51.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 -- 172.21.15.37:0/82679523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 msgr2=0x7f3f9810baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 --2- 172.21.15.37:0/82679523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f9810baf0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009900 tx=0x7f3f8802a480 comp rx=0 tx=0).stop 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 -- 172.21.15.37:0/82679523 shutdown_connections 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 --2- 172.21.15.37:0/82679523 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f9810c0c0 0x7f3f9810e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 --2- 172.21.15.37:0/82679523 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f9810baf0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 --2- 172.21.15.37:0/82679523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f98102cf0 0x7f3f981030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 -- 172.21.15.37:0/82679523 >> 172.21.15.37:0/82679523 conn(0x7f3f98077a10 msgr2=0x7f3f980fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 -- 172.21.15.37:0/82679523 shutdown_connections 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.507+0000 7f3f9f477640 1 -- 172.21.15.37:0/82679523 wait complete. 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 Processor -- start 2024-09-19T06:41:51.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 -- start start 2024-09-19T06:41:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f98102cf0 0x7f3f98100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f98100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f9810c0c0 0x7f3f981013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98111ee0 con 0x7f3f98103610 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98111d60 con 0x7f3f9810c0c0 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9f477640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3f98112060 con 0x7f3f98102cf0 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9ca12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f98100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9ca12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f98100e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54812/0 (socket says 172.21.15.37:54812) 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9ca12640 1 -- 172.21.15.37:0/3945614767 learned_addr learned my addr 172.21.15.37:0/3945614767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9da14640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f9810c0c0 0x7f3f981013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.508+0000 7f3f9d213640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f98102cf0 0x7f3f98100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9ca12640 1 -- 172.21.15.37:0/3945614767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f98102cf0 msgr2=0x7f3f98100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9ca12640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f98102cf0 0x7f3f98100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9ca12640 1 -- 172.21.15.37:0/3945614767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f9810c0c0 msgr2=0x7f3f981013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9ca12640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f9810c0c0 0x7f3f981013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9ca12640 1 -- 172.21.15.37:0/3945614767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f981a68d0 con 0x7f3f98103610 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9ca12640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f98100e80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f88004770 tx=0x7f3f88003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f8e7fc640 1 -- 172.21.15.37:0/3945614767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f8803f070 con 0x7f3f98103610 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f8e7fc640 1 -- 172.21.15.37:0/3945614767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f88031ca0 con 0x7f3f98103610 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f981a6b60 con 0x7f3f98103610 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f8e7fc640 1 -- 172.21.15.37:0/3945614767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f88035030 con 0x7f3f98103610 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.509+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f981a7050 con 0x7f3f98103610 2024-09-19T06:41:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.510+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f981030d0 con 0x7f3f98103610 2024-09-19T06:41:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.511+0000 7f3f8e7fc640 1 -- 172.21.15.37:0/3945614767 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f3f88042070 con 0x7f3f98103610 2024-09-19T06:41:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.512+0000 7f3f8e7fc640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3f840a34e0 0x7f3f840a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.512+0000 7f3f9d213640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3f840a34e0 0x7f3f840a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.512+0000 7f3f8e7fc640 1 -- 172.21.15.37:0/3945614767 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f3f880f0680 con 0x7f3f98103610 2024-09-19T06:41:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.513+0000 7f3f9d213640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3f840a34e0 0x7f3f840a59a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400bbe0 tx=0x7f3f9400b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.516+0000 7f3f8e7fc640 1 -- 172.21.15.37:0/3945614767 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f3f880bc830 con 0x7f3f98103610 2024-09-19T06:41:51.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:51 smithi037 ceph-mon[31211]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:51.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:51 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.608 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:51 smithi037 ceph-mon[31211]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T06:41:51.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.690+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f3f98062630 con 0x7f3f840a34e0 2024-09-19T06:41:51.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.697+0000 7f3f8e7fc640 1 -- 172.21.15.37:0/3945614767 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258000 (secure 0 0 0) 0x7f3f98062630 con 0x7f3f840a34e0 2024-09-19T06:41:51.701 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:51.702 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:41:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.699+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3f840a34e0 msgr2=0x7f3f840a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3f840a34e0 0x7f3f840a59a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400bbe0 tx=0x7f3f9400b390 comp rx=0 tx=0).stop 2024-09-19T06:41:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 msgr2=0x7f3f98100e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f98100e80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3f88004770 tx=0x7f3f88003c10 comp rx=0 tx=0).stop 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 shutdown_connections 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3f9810c0c0 0x7f3f981013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3f840a34e0 0x7f3f840a59a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3f98103610 0x7f3f98100e80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 --2- 172.21.15.37:0/3945614767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3f98102cf0 0x7f3f98100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 >> 172.21.15.37:0/3945614767 conn(0x7f3f98077a10 msgr2=0x7f3f980fd150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 shutdown_connections 2024-09-19T06:41:51.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.700+0000 7f3f9f477640 1 -- 172.21.15.37:0/3945614767 wait complete. 2024-09-19T06:41:51.713 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:41:50.600810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186976,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":887,"num_read_kb":595,"num_write":475,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364700,"kb_used_data":11264,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577124,"statfs":{"total":1151940427776,"available":1151566974976,"internally_reserved":0,"allocated":11534336,"data_stored":8420198,"data_compressed":256268,"data_compressed_allocated":339968,"data_compressed_original":3129344,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990992"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934024+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:41:46.934024+0000","last_peered":"2024-09-19T06:41:46.934024+0000","last_clean":"2024-09-19T06:41:46.934024+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:41:46.934024+0000","last_undegraded":"2024-09-19T06:41:46.934024+0000","last_fullsized":"2024-09-19T06:41:46.934024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866799+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:41:46.866799+0000","last_peered":"2024-09-19T06:41:46.866799+0000","last_clean":"2024-09-19T06:41:46.866799+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:41:46.866799+0000","last_undegraded":"2024-09-19T06:41:46.866799+0000","last_fullsized":"2024-09-19T06:41:46.866799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866503+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:41:46.866503+0000","last_peered":"2024-09-19T06:41:46.866503+0000","last_clean":"2024-09-19T06:41:46.866503+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:41:46.866503+0000","last_undegraded":"2024-09-19T06:41:46.866503+0000","last_fullsized":"2024-09-19T06:41:46.866503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607414+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:41:46.607414+0000","last_peered":"2024-09-19T06:41:46.607414+0000","last_clean":"2024-09-19T06:41:46.607414+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:41:46.607414+0000","last_undegraded":"2024-09-19T06:41:46.607414+0000","last_fullsized":"2024-09-19T06:41:46.607414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679815+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:41:46.679815+0000","last_peered":"2024-09-19T06:41:46.679815+0000","last_clean":"2024-09-19T06:41:46.679815+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:41:46.679815+0000","last_undegraded":"2024-09-19T06:41:46.679815+0000","last_fullsized":"2024-09-19T06:41:46.679815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590191+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:41:46.590191+0000","last_peered":"2024-09-19T06:41:46.590191+0000","last_clean":"2024-09-19T06:41:46.590191+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:41:46.590191+0000","last_undegraded":"2024-09-19T06:41:46.590191+0000","last_fullsized":"2024-09-19T06:41:46.590191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860437+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:41:46.860437+0000","last_peered":"2024-09-19T06:41:46.860437+0000","last_clean":"2024-09-19T06:41:46.860437+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:41:46.860437+0000","last_undegraded":"2024-09-19T06:41:46.860437+0000","last_fullsized":"2024-09-19T06:41:46.860437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607193+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:41:46.607193+0000","last_peered":"2024-09-19T06:41:46.607193+0000","last_clean":"2024-09-19T06:41:46.607193+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:41:46.607193+0000","last_undegraded":"2024-09-19T06:41:46.607193+0000","last_fullsized":"2024-09-19T06:41:46.607193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609566+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:41:46.609566+0000","last_peered":"2024-09-19T06:41:46.609566+0000","last_clean":"2024-09-19T06:41:46.609566+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:41:46.609566+0000","last_undegraded":"2024-09-19T06:41:46.609566+0000","last_fullsized":"2024-09-19T06:41:46.609566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590543+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:41:46.590543+0000","last_peered":"2024-09-19T06:41:46.590543+0000","last_clean":"2024-09-19T06:41:46.590543+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:41:46.590543+0000","last_undegraded":"2024-09-19T06:41:46.590543+0000","last_fullsized":"2024-09-19T06:41:46.590543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860924+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:41:46.860924+0000","last_peered":"2024-09-19T06:41:46.860924+0000","last_clean":"2024-09-19T06:41:46.860924+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:41:46.860924+0000","last_undegraded":"2024-09-19T06:41:46.860924+0000","last_fullsized":"2024-09-19T06:41:46.860924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607557+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:41:46.607557+0000","last_peered":"2024-09-19T06:41:46.607557+0000","last_clean":"2024-09-19T06:41:46.607557+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:41:46.607557+0000","last_undegraded":"2024-09-19T06:41:46.607557+0000","last_fullsized":"2024-09-19T06:41:46.607557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587632+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:41:46.587632+0000","last_peered":"2024-09-19T06:41:46.587632+0000","last_clean":"2024-09-19T06:41:46.587632+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:41:46.587632+0000","last_undegraded":"2024-09-19T06:41:46.587632+0000","last_fullsized":"2024-09-19T06:41:46.587632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866079+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:41:46.866079+0000","last_peered":"2024-09-19T06:41:46.866079+0000","last_clean":"2024-09-19T06:41:46.866079+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:41:46.866079+0000","last_undegraded":"2024-09-19T06:41:46.866079+0000","last_fullsized":"2024-09-19T06:41:46.866079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.865755+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:41:46.865755+0000","last_peered":"2024-09-19T06:41:46.865755+0000","last_clean":"2024-09-19T06:41:46.865755+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:41:46.865755+0000","last_undegraded":"2024-09-19T06:41:46.865755+0000","last_fullsized":"2024-09-19T06:41:46.865755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609034+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:41:46.609034+0000","last_peered":"2024-09-19T06:41:46.609034+0000","last_clean":"2024-09-19T06:41:46.609034+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:41:46.609034+0000","last_undegraded":"2024-09-19T06:41:46.609034+0000","last_fullsized":"2024-09-19T06:41:46.609034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934022+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:41:46.934022+0000","last_peered":"2024-09-19T06:41:46.934022+0000","last_clean":"2024-09-19T06:41:46.934022+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:41:46.934022+0000","last_undegraded":"2024-09-19T06:41:46.934022+0000","last_fullsized":"2024-09-19T06:41:46.934022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677589+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:41:46.677589+0000","last_peered":"2024-09-19T06:41:46.677589+0000","last_clean":"2024-09-19T06:41:46.677589+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:41:46.677589+0000","last_undegraded":"2024-09-19T06:41:46.677589+0000","last_fullsized":"2024-09-19T06:41:46.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609334+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:41:46.609334+0000","last_peered":"2024-09-19T06:41:46.609334+0000","last_clean":"2024-09-19T06:41:46.609334+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:41:46.609334+0000","last_undegraded":"2024-09-19T06:41:46.609334+0000","last_fullsized":"2024-09-19T06:41:46.609334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933853+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:41:46.933853+0000","last_peered":"2024-09-19T06:41:46.933853+0000","last_clean":"2024-09-19T06:41:46.933853+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:41:46.933853+0000","last_undegraded":"2024-09-19T06:41:46.933853+0000","last_fullsized":"2024-09-19T06:41:46.933853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587643+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:41:46.587643+0000","last_peered":"2024-09-19T06:41:46.587643+0000","last_clean":"2024-09-19T06:41:46.587643+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:41:46.587643+0000","last_undegraded":"2024-09-19T06:41:46.587643+0000","last_fullsized":"2024-09-19T06:41:46.587643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609138+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:41:46.609138+0000","last_peered":"2024-09-19T06:41:46.609138+0000","last_clean":"2024-09-19T06:41:46.609138+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:41:46.609138+0000","last_undegraded":"2024-09-19T06:41:46.609138+0000","last_fullsized":"2024-09-19T06:41:46.609138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607201+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:41:46.607201+0000","last_peered":"2024-09-19T06:41:46.607201+0000","last_clean":"2024-09-19T06:41:46.607201+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:41:46.607201+0000","last_undegraded":"2024-09-19T06:41:46.607201+0000","last_fullsized":"2024-09-19T06:41:46.607201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588420+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:41:46.588420+0000","last_peered":"2024-09-19T06:41:46.588420+0000","last_clean":"2024-09-19T06:41:46.588420+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:41:46.588420+0000","last_undegraded":"2024-09-19T06:41:46.588420+0000","last_fullsized":"2024-09-19T06:41:46.588420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677514+0000","last_change":"2024-09-19T06:40:58.134245+0000","last_active":"2024-09-19T06:41:46.677514+0000","last_peered":"2024-09-19T06:41:46.677514+0000","last_clean":"2024-09-19T06:41:46.677514+0000","last_became_active":"2024-09-19T06:40:58.133514+0000","last_became_peered":"2024-09-19T06:40:58.133514+0000","last_unstale":"2024-09-19T06:41:46.677514+0000","last_undegraded":"2024-09-19T06:41:46.677514+0000","last_fullsized":"2024-09-19T06:41:46.677514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:23.267466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609405+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:41:46.609405+0000","last_peered":"2024-09-19T06:41:46.609405+0000","last_clean":"2024-09-19T06:41:46.609405+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:41:46.609405+0000","last_undegraded":"2024-09-19T06:41:46.609405+0000","last_fullsized":"2024-09-19T06:41:46.609405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933946+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:41:46.933946+0000","last_peered":"2024-09-19T06:41:46.933946+0000","last_clean":"2024-09-19T06:41:46.933946+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:41:46.933946+0000","last_undegraded":"2024-09-19T06:41:46.933946+0000","last_fullsized":"2024-09-19T06:41:46.933946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587547+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:41:46.587547+0000","last_peered":"2024-09-19T06:41:46.587547+0000","last_clean":"2024-09-19T06:41:46.587547+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:41:46.587547+0000","last_undegraded":"2024-09-19T06:41:46.587547+0000","last_fullsized":"2024-09-19T06:41:46.587547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866245+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:41:46.866245+0000","last_peered":"2024-09-19T06:41:46.866245+0000","last_clean":"2024-09-19T06:41:46.866245+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:41:46.866245+0000","last_undegraded":"2024-09-19T06:41:46.866245+0000","last_fullsized":"2024-09-19T06:41:46.866245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609008+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:41:46.609008+0000","last_peered":"2024-09-19T06:41:46.609008+0000","last_clean":"2024-09-19T06:41:46.609008+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:41:46.609008+0000","last_undegraded":"2024-09-19T06:41:46.609008+0000","last_fullsized":"2024-09-19T06:41:46.609008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609041+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:41:46.609041+0000","last_peered":"2024-09-19T06:41:46.609041+0000","last_clean":"2024-09-19T06:41:46.609041+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:41:46.609041+0000","last_undegraded":"2024-09-19T06:41:46.609041+0000","last_fullsized":"2024-09-19T06:41:46.609041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933663+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:41:46.933663+0000","last_peered":"2024-09-19T06:41:46.933663+0000","last_clean":"2024-09-19T06:41:46.933663+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:41:46.933663+0000","last_undegraded":"2024-09-19T06:41:46.933663+0000","last_fullsized":"2024-09-19T06:41:46.933663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679766+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:41:46.679766+0000","last_peered":"2024-09-19T06:41:46.679766+0000","last_clean":"2024-09-19T06:41:46.679766+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:41:46.679766+0000","last_undegraded":"2024-09-19T06:41:46.679766+0000","last_fullsized":"2024-09-19T06:41:46.679766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860151+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:41:46.860151+0000","last_peered":"2024-09-19T06:41:46.860151+0000","last_clean":"2024-09-19T06:41:46.860151+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:41:46.860151+0000","last_undegraded":"2024-09-19T06:41:46.860151+0000","last_fullsized":"2024-09-19T06:41:46.860151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588409+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:41:46.588409+0000","last_peered":"2024-09-19T06:41:46.588409+0000","last_clean":"2024-09-19T06:41:46.588409+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:41:46.588409+0000","last_undegraded":"2024-09-19T06:41:46.588409+0000","last_fullsized":"2024-09-19T06:41:46.588409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590296+0000","last_change":"2024-09-19T06:40:58.136626+0000","last_active":"2024-09-19T06:41:46.590296+0000","last_peered":"2024-09-19T06:41:46.590296+0000","last_clean":"2024-09-19T06:41:46.590296+0000","last_became_active":"2024-09-19T06:40:58.132752+0000","last_became_peered":"2024-09-19T06:40:58.132752+0000","last_unstale":"2024-09-19T06:41:46.590296+0000","last_undegraded":"2024-09-19T06:41:46.590296+0000","last_fullsized":"2024-09-19T06:41:46.590296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587271+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:41:46.587271+0000","last_peered":"2024-09-19T06:41:46.587271+0000","last_clean":"2024-09-19T06:41:46.587271+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:41:46.587271+0000","last_undegraded":"2024-09-19T06:41:46.587271+0000","last_fullsized":"2024-09-19T06:41:46.587271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607281+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:41:46.607281+0000","last_peered":"2024-09-19T06:41:46.607281+0000","last_clean":"2024-09-19T06:41:46.607281+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:41:46.607281+0000","last_undegraded":"2024-09-19T06:41:46.607281+0000","last_fullsized":"2024-09-19T06:41:46.607281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588546+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:41:46.588546+0000","last_peered":"2024-09-19T06:41:46.588546+0000","last_clean":"2024-09-19T06:41:46.588546+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:41:46.588546+0000","last_undegraded":"2024-09-19T06:41:46.588546+0000","last_fullsized":"2024-09-19T06:41:46.588546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677238+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:41:46.677238+0000","last_peered":"2024-09-19T06:41:46.677238+0000","last_clean":"2024-09-19T06:41:46.677238+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:41:46.677238+0000","last_undegraded":"2024-09-19T06:41:46.677238+0000","last_fullsized":"2024-09-19T06:41:46.677238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590558+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:41:46.590558+0000","last_peered":"2024-09-19T06:41:46.590558+0000","last_clean":"2024-09-19T06:41:46.590558+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:41:46.590558+0000","last_undegraded":"2024-09-19T06:41:46.590558+0000","last_fullsized":"2024-09-19T06:41:46.590558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609442+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:41:46.609442+0000","last_peered":"2024-09-19T06:41:46.609442+0000","last_clean":"2024-09-19T06:41:46.609442+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:41:46.609442+0000","last_undegraded":"2024-09-19T06:41:46.609442+0000","last_fullsized":"2024-09-19T06:41:46.609442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860979+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:41:46.860979+0000","last_peered":"2024-09-19T06:41:46.860979+0000","last_clean":"2024-09-19T06:41:46.860979+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:41:46.860979+0000","last_undegraded":"2024-09-19T06:41:46.860979+0000","last_fullsized":"2024-09-19T06:41:46.860979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867221+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:41:46.867221+0000","last_peered":"2024-09-19T06:41:46.867221+0000","last_clean":"2024-09-19T06:41:46.867221+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:41:46.867221+0000","last_undegraded":"2024-09-19T06:41:46.867221+0000","last_fullsized":"2024-09-19T06:41:46.867221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588728+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:41:46.588728+0000","last_peered":"2024-09-19T06:41:46.588728+0000","last_clean":"2024-09-19T06:41:46.588728+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:41:46.588728+0000","last_undegraded":"2024-09-19T06:41:46.588728+0000","last_fullsized":"2024-09-19T06:41:46.588728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609314+0000","last_change":"2024-09-19T06:40:58.135595+0000","last_active":"2024-09-19T06:41:46.609314+0000","last_peered":"2024-09-19T06:41:46.609314+0000","last_clean":"2024-09-19T06:41:46.609314+0000","last_became_active":"2024-09-19T06:40:58.135031+0000","last_became_peered":"2024-09-19T06:40:58.135031+0000","last_unstale":"2024-09-19T06:41:46.609314+0000","last_undegraded":"2024-09-19T06:41:46.609314+0000","last_fullsized":"2024-09-19T06:41:46.609314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:53.542630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'34","reported_seq":278,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860101+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:41:46.860101+0000","last_peered":"2024-09-19T06:41:46.860101+0000","last_clean":"2024-09-19T06:41:46.860101+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:41:46.860101+0000","last_undegraded":"2024-09-19T06:41:46.860101+0000","last_fullsized":"2024-09-19T06:41:46.860101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":168,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680048+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:41:46.680048+0000","last_peered":"2024-09-19T06:41:46.680048+0000","last_clean":"2024-09-19T06:41:46.680048+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:41:46.680048+0000","last_undegraded":"2024-09-19T06:41:46.680048+0000","last_fullsized":"2024-09-19T06:41:46.680048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609208+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:41:46.609208+0000","last_peered":"2024-09-19T06:41:46.609208+0000","last_clean":"2024-09-19T06:41:46.609208+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:41:46.609208+0000","last_undegraded":"2024-09-19T06:41:46.609208+0000","last_fullsized":"2024-09-19T06:41:46.609208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587384+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:41:46.587384+0000","last_peered":"2024-09-19T06:41:46.587384+0000","last_clean":"2024-09-19T06:41:46.587384+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:41:46.587384+0000","last_undegraded":"2024-09-19T06:41:46.587384+0000","last_fullsized":"2024-09-19T06:41:46.587384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933722+0000","last_change":"2024-09-19T06:40:58.143264+0000","last_active":"2024-09-19T06:41:46.933722+0000","last_peered":"2024-09-19T06:41:46.933722+0000","last_clean":"2024-09-19T06:41:46.933722+0000","last_became_active":"2024-09-19T06:40:58.135400+0000","last_became_peered":"2024-09-19T06:40:58.135400+0000","last_unstale":"2024-09-19T06:41:46.933722+0000","last_undegraded":"2024-09-19T06:41:46.933722+0000","last_fullsized":"2024-09-19T06:41:46.933722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:30.500765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860840+0000","last_change":"2024-09-19T06:40:58.134136+0000","last_active":"2024-09-19T06:41:46.860840+0000","last_peered":"2024-09-19T06:41:46.860840+0000","last_clean":"2024-09-19T06:41:46.860840+0000","last_became_active":"2024-09-19T06:40:58.133734+0000","last_became_peered":"2024-09-19T06:40:58.133734+0000","last_unstale":"2024-09-19T06:41:46.860840+0000","last_undegraded":"2024-09-19T06:41:46.860840+0000","last_fullsized":"2024-09-19T06:41:46.860840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:50.822670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607548+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:41:46.607548+0000","last_peered":"2024-09-19T06:41:46.607548+0000","last_clean":"2024-09-19T06:41:46.607548+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:41:46.607548+0000","last_undegraded":"2024-09-19T06:41:46.607548+0000","last_fullsized":"2024-09-19T06:41:46.607548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860373+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:41:46.860373+0000","last_peered":"2024-09-19T06:41:46.860373+0000","last_clean":"2024-09-19T06:41:46.860373+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:41:46.860373+0000","last_undegraded":"2024-09-19T06:41:46.860373+0000","last_fullsized":"2024-09-19T06:41:46.860373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679751+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:41:46.679751+0000","last_peered":"2024-09-19T06:41:46.679751+0000","last_clean":"2024-09-19T06:41:46.679751+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:41:46.679751+0000","last_undegraded":"2024-09-19T06:41:46.679751+0000","last_fullsized":"2024-09-19T06:41:46.679751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588476+0000","last_change":"2024-09-19T06:40:58.135978+0000","last_active":"2024-09-19T06:41:46.588476+0000","last_peered":"2024-09-19T06:41:46.588476+0000","last_clean":"2024-09-19T06:41:46.588476+0000","last_became_active":"2024-09-19T06:40:58.135581+0000","last_became_peered":"2024-09-19T06:40:58.135581+0000","last_unstale":"2024-09-19T06:41:46.588476+0000","last_undegraded":"2024-09-19T06:41:46.588476+0000","last_fullsized":"2024-09-19T06:41:46.588476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:54.532003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680159+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:41:46.680159+0000","last_peered":"2024-09-19T06:41:46.680159+0000","last_clean":"2024-09-19T06:41:46.680159+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:41:46.680159+0000","last_undegraded":"2024-09-19T06:41:46.680159+0000","last_fullsized":"2024-09-19T06:41:46.680159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933860+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:41:46.933860+0000","last_peered":"2024-09-19T06:41:46.933860+0000","last_clean":"2024-09-19T06:41:46.933860+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:41:46.933860+0000","last_undegraded":"2024-09-19T06:41:46.933860+0000","last_fullsized":"2024-09-19T06:41:46.933860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587529+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:41:46.587529+0000","last_peered":"2024-09-19T06:41:46.587529+0000","last_clean":"2024-09-19T06:41:46.587529+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:41:46.587529+0000","last_undegraded":"2024-09-19T06:41:46.587529+0000","last_fullsized":"2024-09-19T06:41:46.587529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.797821+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:41:46.797821+0000","last_peered":"2024-09-19T06:41:46.797821+0000","last_clean":"2024-09-19T06:41:46.797821+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:41:46.797821+0000","last_undegraded":"2024-09-19T06:41:46.797821+0000","last_fullsized":"2024-09-19T06:41:46.797821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860872+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:41:46.860872+0000","last_peered":"2024-09-19T06:41:46.860872+0000","last_clean":"2024-09-19T06:41:46.860872+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:41:46.860872+0000","last_undegraded":"2024-09-19T06:41:46.860872+0000","last_fullsized":"2024-09-19T06:41:46.860872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607210+0000","last_change":"2024-09-19T06:40:58.138283+0000","last_active":"2024-09-19T06:41:46.607210+0000","last_peered":"2024-09-19T06:41:46.607210+0000","last_clean":"2024-09-19T06:41:46.607210+0000","last_became_active":"2024-09-19T06:40:58.132079+0000","last_became_peered":"2024-09-19T06:40:58.132079+0000","last_unstale":"2024-09-19T06:41:46.607210+0000","last_undegraded":"2024-09-19T06:41:46.607210+0000","last_fullsized":"2024-09-19T06:41:46.607210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:12.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860532+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:41:46.860532+0000","last_peered":"2024-09-19T06:41:46.860532+0000","last_clean":"2024-09-19T06:41:46.860532+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:41:46.860532+0000","last_undegraded":"2024-09-19T06:41:46.860532+0000","last_fullsized":"2024-09-19T06:41:46.860532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607259+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:41:46.607259+0000","last_peered":"2024-09-19T06:41:46.607259+0000","last_clean":"2024-09-19T06:41:46.607259+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:41:46.607259+0000","last_undegraded":"2024-09-19T06:41:46.607259+0000","last_fullsized":"2024-09-19T06:41:46.607259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607408+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:41:46.607408+0000","last_peered":"2024-09-19T06:41:46.607408+0000","last_clean":"2024-09-19T06:41:46.607408+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:41:46.607408+0000","last_undegraded":"2024-09-19T06:41:46.607408+0000","last_fullsized":"2024-09-19T06:41:46.607408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860541+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:41:46.860541+0000","last_peered":"2024-09-19T06:41:46.860541+0000","last_clean":"2024-09-19T06:41:46.860541+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:41:46.860541+0000","last_undegraded":"2024-09-19T06:41:46.860541+0000","last_fullsized":"2024-09-19T06:41:46.860541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587269+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:41:46.587269+0000","last_peered":"2024-09-19T06:41:46.587269+0000","last_clean":"2024-09-19T06:41:46.587269+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:41:46.587269+0000","last_undegraded":"2024-09-19T06:41:46.587269+0000","last_fullsized":"2024-09-19T06:41:46.587269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":135,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.605518+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:41:46.605518+0000","last_peered":"2024-09-19T06:41:46.605518+0000","last_clean":"2024-09-19T06:41:46.605518+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:41:46.605518+0000","last_undegraded":"2024-09-19T06:41:46.605518+0000","last_fullsized":"2024-09-19T06:41:46.605518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677201+0000","last_change":"2024-09-19T06:40:58.132715+0000","last_active":"2024-09-19T06:41:46.677201+0000","last_peered":"2024-09-19T06:41:46.677201+0000","last_clean":"2024-09-19T06:41:46.677201+0000","last_became_active":"2024-09-19T06:40:58.132333+0000","last_became_peered":"2024-09-19T06:40:58.132333+0000","last_unstale":"2024-09-19T06:41:46.677201+0000","last_undegraded":"2024-09-19T06:41:46.677201+0000","last_fullsized":"2024-09-19T06:41:46.677201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:03.256998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860643+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:41:46.860643+0000","last_peered":"2024-09-19T06:41:46.860643+0000","last_clean":"2024-09-19T06:41:46.860643+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:41:46.860643+0000","last_undegraded":"2024-09-19T06:41:46.860643+0000","last_fullsized":"2024-09-19T06:41:46.860643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590285+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:41:46.590285+0000","last_peered":"2024-09-19T06:41:46.590285+0000","last_clean":"2024-09-19T06:41:46.590285+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:41:46.590285+0000","last_undegraded":"2024-09-19T06:41:46.590285+0000","last_fullsized":"2024-09-19T06:41:46.590285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677533+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:41:46.677533+0000","last_peered":"2024-09-19T06:41:46.677533+0000","last_clean":"2024-09-19T06:41:46.677533+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:41:46.677533+0000","last_undegraded":"2024-09-19T06:41:46.677533+0000","last_fullsized":"2024-09-19T06:41:46.677533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590332+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:41:46.590332+0000","last_peered":"2024-09-19T06:41:46.590332+0000","last_clean":"2024-09-19T06:41:46.590332+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:41:46.590332+0000","last_undegraded":"2024-09-19T06:41:46.590332+0000","last_fullsized":"2024-09-19T06:41:46.590332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933826+0000","last_change":"2024-09-19T06:40:58.140944+0000","last_active":"2024-09-19T06:41:46.933826+0000","last_peered":"2024-09-19T06:41:46.933826+0000","last_clean":"2024-09-19T06:41:46.933826+0000","last_became_active":"2024-09-19T06:40:58.132427+0000","last_became_peered":"2024-09-19T06:40:58.132427+0000","last_unstale":"2024-09-19T06:41:46.933826+0000","last_undegraded":"2024-09-19T06:41:46.933826+0000","last_fullsized":"2024-09-19T06:41:46.933826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:32.472242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587689+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:41:46.587689+0000","last_peered":"2024-09-19T06:41:46.587689+0000","last_clean":"2024-09-19T06:41:46.587689+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:41:46.587689+0000","last_undegraded":"2024-09-19T06:41:46.587689+0000","last_fullsized":"2024-09-19T06:41:46.587689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'47","reported_seq":94,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.800662+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:41:46.800662+0000","last_peered":"2024-09-19T06:41:46.800662+0000","last_clean":"2024-09-19T06:41:46.800662+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:41:46.800662+0000","last_undegraded":"2024-09-19T06:41:46.800662+0000","last_fullsized":"2024-09-19T06:41:46.800662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607761+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:41:46.607761+0000","last_peered":"2024-09-19T06:41:46.607761+0000","last_clean":"2024-09-19T06:41:46.607761+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:41:46.607761+0000","last_undegraded":"2024-09-19T06:41:46.607761+0000","last_fullsized":"2024-09-19T06:41:46.607761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607492+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:41:46.607492+0000","last_peered":"2024-09-19T06:41:46.607492+0000","last_clean":"2024-09-19T06:41:46.607492+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:41:46.607492+0000","last_undegraded":"2024-09-19T06:41:46.607492+0000","last_fullsized":"2024-09-19T06:41:46.607492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866926+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:41:46.866926+0000","last_peered":"2024-09-19T06:41:46.866926+0000","last_clean":"2024-09-19T06:41:46.866926+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:41:46.866926+0000","last_undegraded":"2024-09-19T06:41:46.866926+0000","last_fullsized":"2024-09-19T06:41:46.866926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609037+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:41:46.609037+0000","last_peered":"2024-09-19T06:41:46.609037+0000","last_clean":"2024-09-19T06:41:46.609037+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:41:46.609037+0000","last_undegraded":"2024-09-19T06:41:46.609037+0000","last_fullsized":"2024-09-19T06:41:46.609037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679748+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:41:46.679748+0000","last_peered":"2024-09-19T06:41:46.679748+0000","last_clean":"2024-09-19T06:41:46.679748+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:41:46.679748+0000","last_undegraded":"2024-09-19T06:41:46.679748+0000","last_fullsized":"2024-09-19T06:41:46.679748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607361+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:41:46.607361+0000","last_peered":"2024-09-19T06:41:46.607361+0000","last_clean":"2024-09-19T06:41:46.607361+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:41:46.607361+0000","last_undegraded":"2024-09-19T06:41:46.607361+0000","last_fullsized":"2024-09-19T06:41:46.607361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933699+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:41:46.933699+0000","last_peered":"2024-09-19T06:41:46.933699+0000","last_clean":"2024-09-19T06:41:46.933699+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:41:46.933699+0000","last_undegraded":"2024-09-19T06:41:46.933699+0000","last_fullsized":"2024-09-19T06:41:46.933699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609110+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:41:46.609110+0000","last_peered":"2024-09-19T06:41:46.609110+0000","last_clean":"2024-09-19T06:41:46.609110+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:41:46.609110+0000","last_undegraded":"2024-09-19T06:41:46.609110+0000","last_fullsized":"2024-09-19T06:41:46.609110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607487+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:41:46.607487+0000","last_peered":"2024-09-19T06:41:46.607487+0000","last_clean":"2024-09-19T06:41:46.607487+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:41:46.607487+0000","last_undegraded":"2024-09-19T06:41:46.607487+0000","last_fullsized":"2024-09-19T06:41:46.607487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679819+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:41:46.679819+0000","last_peered":"2024-09-19T06:41:46.679819+0000","last_clean":"2024-09-19T06:41:46.679819+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:41:46.679819+0000","last_undegraded":"2024-09-19T06:41:46.679819+0000","last_fullsized":"2024-09-19T06:41:46.679819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860513+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:41:46.860513+0000","last_peered":"2024-09-19T06:41:46.860513+0000","last_clean":"2024-09-19T06:41:46.860513+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:41:46.860513+0000","last_undegraded":"2024-09-19T06:41:46.860513+0000","last_fullsized":"2024-09-19T06:41:46.860513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588680+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:41:46.588680+0000","last_peered":"2024-09-19T06:41:46.588680+0000","last_clean":"2024-09-19T06:41:46.588680+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:41:46.588680+0000","last_undegraded":"2024-09-19T06:41:46.588680+0000","last_fullsized":"2024-09-19T06:41:46.588680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680018+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:41:46.680018+0000","last_peered":"2024-09-19T06:41:46.680018+0000","last_clean":"2024-09-19T06:41:46.680018+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:41:46.680018+0000","last_undegraded":"2024-09-19T06:41:46.680018+0000","last_fullsized":"2024-09-19T06:41:46.680018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609092+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:41:46.609092+0000","last_peered":"2024-09-19T06:41:46.609092+0000","last_clean":"2024-09-19T06:41:46.609092+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:41:46.609092+0000","last_undegraded":"2024-09-19T06:41:46.609092+0000","last_fullsized":"2024-09-19T06:41:46.609092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680073+0000","last_change":"2024-09-19T06:40:58.139755+0000","last_active":"2024-09-19T06:41:46.680073+0000","last_peered":"2024-09-19T06:41:46.680073+0000","last_clean":"2024-09-19T06:41:46.680073+0000","last_became_active":"2024-09-19T06:40:58.135411+0000","last_became_peered":"2024-09-19T06:40:58.135411+0000","last_unstale":"2024-09-19T06:41:46.680073+0000","last_undegraded":"2024-09-19T06:41:46.680073+0000","last_fullsized":"2024-09-19T06:41:46.680073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:38.951366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867192+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:41:46.867192+0000","last_peered":"2024-09-19T06:41:46.867192+0000","last_clean":"2024-09-19T06:41:46.867192+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:41:46.867192+0000","last_undegraded":"2024-09-19T06:41:46.867192+0000","last_fullsized":"2024-09-19T06:41:46.867192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607877+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:41:46.607877+0000","last_peered":"2024-09-19T06:41:46.607877+0000","last_clean":"2024-09-19T06:41:46.607877+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:41:46.607877+0000","last_undegraded":"2024-09-19T06:41:46.607877+0000","last_fullsized":"2024-09-19T06:41:46.607877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607731+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:41:46.607731+0000","last_peered":"2024-09-19T06:41:46.607731+0000","last_clean":"2024-09-19T06:41:46.607731+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:41:46.607731+0000","last_undegraded":"2024-09-19T06:41:46.607731+0000","last_fullsized":"2024-09-19T06:41:46.607731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677656+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:41:46.677656+0000","last_peered":"2024-09-19T06:41:46.677656+0000","last_clean":"2024-09-19T06:41:46.677656+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:41:46.677656+0000","last_undegraded":"2024-09-19T06:41:46.677656+0000","last_fullsized":"2024-09-19T06:41:46.677656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596374,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":751,"num_read_kb":476,"num_write":292,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1789122,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":437263,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":1236161,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":774142,"data_compressed":24564,"data_compressed_allocated":32768,"data_compressed_original":352256,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":446360,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1029778,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445851,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":1170299,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":447797,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":970978,"data_compressed":33961,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":511861,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":511571,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438137,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134170,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200032,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:41:51.718 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:41:51.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.855+0000 7f6838f4b640 1 -- 172.21.15.37:0/2538163050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 msgr2=0x7f683410e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.855+0000 7f6838f4b640 1 --2- 172.21.15.37:0/2538163050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f683410e980 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f6824009cd0 tx=0x7f682402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.855+0000 7f6838f4b640 1 -- 172.21.15.37:0/2538163050 shutdown_connections 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.855+0000 7f6838f4b640 1 --2- 172.21.15.37:0/2538163050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f683410e980 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 --2- 172.21.15.37:0/2538163050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6834101ab0 0x7f6834101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 --2- 172.21.15.37:0/2538163050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6834107ac0 0x7f6834107ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 -- 172.21.15.37:0/2538163050 >> 172.21.15.37:0/2538163050 conn(0x7f68340fd7d0 msgr2=0x7f68340ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 -- 172.21.15.37:0/2538163050 shutdown_connections 2024-09-19T06:41:51.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 -- 172.21.15.37:0/2538163050 wait complete. 2024-09-19T06:41:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 Processor -- start 2024-09-19T06:41:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 -- start start 2024-09-19T06:41:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6834101ab0 0x7f68341982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f6834198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6834107ac0 0x7f683419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.856+0000 7f6838f4b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6834111f00 con 0x7f6834102450 2024-09-19T06:41:51.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6838f4b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6834111d80 con 0x7f6834101ab0 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6838f4b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6834112080 con 0x7f6834107ac0 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f6834198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f6834198830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54820/0 (socket says 172.21.15.37:54820) 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 -- 172.21.15.37:0/3436626071 learned_addr learned my addr 172.21.15.37:0/3436626071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6832d9d640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6834107ac0 0x7f683419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f683259c640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6834101ab0 0x7f68341982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 -- 172.21.15.37:0/3436626071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6834107ac0 msgr2=0x7f683419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6834107ac0 0x7f683419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 -- 172.21.15.37:0/3436626071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6834101ab0 msgr2=0x7f68341982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6834101ab0 0x7f68341982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 -- 172.21.15.37:0/3436626071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f683419d2e0 con 0x7f6834102450 2024-09-19T06:41:51.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6831d9b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f6834198830 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c970 tx=0x7f681c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f682b7fe640 1 -- 172.21.15.37:0/3436626071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c007bb0 con 0x7f6834102450 2024-09-19T06:41:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f682b7fe640 1 -- 172.21.15.37:0/3436626071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f681c007d50 con 0x7f6834102450 2024-09-19T06:41:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f683419d5d0 con 0x7f6834102450 2024-09-19T06:41:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f682b7fe640 1 -- 172.21.15.37:0/3436626071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f681c00f450 con 0x7f6834102450 2024-09-19T06:41:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.857+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68341a4e60 con 0x7f6834102450 2024-09-19T06:41:51.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.858+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6834101f10 con 0x7f6834102450 2024-09-19T06:41:51.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:51 smithi151 ceph-mon[35236]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:51.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:51 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:51.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:51 smithi151 ceph-mon[35236]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T06:41:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.859+0000 7f682b7fe640 1 -- 172.21.15.37:0/3436626071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f681c004f40 con 0x7f6834102450 2024-09-19T06:41:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.860+0000 7f682b7fe640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68200a3560 0x7f68200a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.861+0000 7f683259c640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68200a3560 0x7f68200a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:51.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.861+0000 7f682b7fe640 1 -- 172.21.15.37:0/3436626071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f681c0cfbf0 con 0x7f6834102450 2024-09-19T06:41:51.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.861+0000 7f683259c640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68200a3560 0x7f68200a5a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f682c00ab60 tx=0x7f682c00a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:51.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:51.865+0000 7f682b7fe640 1 -- 172.21.15.37:0/3436626071 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f681c014030 con 0x7f6834102450 2024-09-19T06:41:52.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:51 smithi096 ceph-mon[35004]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:41:52.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:51 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:52.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:51 smithi096 ceph-mon[35004]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-09-19T06:41:52.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.040+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f68340fe520 con 0x7f68200a3560 2024-09-19T06:41:52.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.047+0000 7f682b7fe640 1 -- 172.21.15.37:0/3436626071 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258000 (secure 0 0 0) 0x7f68340fe520 con 0x7f68200a3560 2024-09-19T06:41:52.052 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:52.053 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:41:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68200a3560 msgr2=0x7f68200a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68200a3560 0x7f68200a5a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f682c00ab60 tx=0x7f682c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 msgr2=0x7f6834198830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:52.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f6834198830 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f681c00c970 tx=0x7f681c00cd70 comp rx=0 tx=0).stop 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 shutdown_connections 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6834107ac0 0x7f683419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68200a3560 0x7f68200a5a20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6834102450 0x7f6834198830 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 --2- 172.21.15.37:0/3436626071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6834101ab0 0x7f68341982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 >> 172.21.15.37:0/3436626071 conn(0x7f68340fd7d0 msgr2=0x7f68340fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 shutdown_connections 2024-09-19T06:41:52.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.050+0000 7f6838f4b640 1 -- 172.21.15.37:0/3436626071 wait complete. 2024-09-19T06:41:52.063 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:41:50.600810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186976,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":887,"num_read_kb":595,"num_write":475,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364700,"kb_used_data":11264,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577124,"statfs":{"total":1151940427776,"available":1151566974976,"internally_reserved":0,"allocated":11534336,"data_stored":8420198,"data_compressed":256268,"data_compressed_allocated":339968,"data_compressed_original":3129344,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990992"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934024+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:41:46.934024+0000","last_peered":"2024-09-19T06:41:46.934024+0000","last_clean":"2024-09-19T06:41:46.934024+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:41:46.934024+0000","last_undegraded":"2024-09-19T06:41:46.934024+0000","last_fullsized":"2024-09-19T06:41:46.934024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866799+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:41:46.866799+0000","last_peered":"2024-09-19T06:41:46.866799+0000","last_clean":"2024-09-19T06:41:46.866799+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:41:46.866799+0000","last_undegraded":"2024-09-19T06:41:46.866799+0000","last_fullsized":"2024-09-19T06:41:46.866799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866503+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:41:46.866503+0000","last_peered":"2024-09-19T06:41:46.866503+0000","last_clean":"2024-09-19T06:41:46.866503+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:41:46.866503+0000","last_undegraded":"2024-09-19T06:41:46.866503+0000","last_fullsized":"2024-09-19T06:41:46.866503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607414+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:41:46.607414+0000","last_peered":"2024-09-19T06:41:46.607414+0000","last_clean":"2024-09-19T06:41:46.607414+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:41:46.607414+0000","last_undegraded":"2024-09-19T06:41:46.607414+0000","last_fullsized":"2024-09-19T06:41:46.607414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679815+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:41:46.679815+0000","last_peered":"2024-09-19T06:41:46.679815+0000","last_clean":"2024-09-19T06:41:46.679815+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:41:46.679815+0000","last_undegraded":"2024-09-19T06:41:46.679815+0000","last_fullsized":"2024-09-19T06:41:46.679815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590191+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:41:46.590191+0000","last_peered":"2024-09-19T06:41:46.590191+0000","last_clean":"2024-09-19T06:41:46.590191+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:41:46.590191+0000","last_undegraded":"2024-09-19T06:41:46.590191+0000","last_fullsized":"2024-09-19T06:41:46.590191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860437+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:41:46.860437+0000","last_peered":"2024-09-19T06:41:46.860437+0000","last_clean":"2024-09-19T06:41:46.860437+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:41:46.860437+0000","last_undegraded":"2024-09-19T06:41:46.860437+0000","last_fullsized":"2024-09-19T06:41:46.860437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607193+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:41:46.607193+0000","last_peered":"2024-09-19T06:41:46.607193+0000","last_clean":"2024-09-19T06:41:46.607193+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:41:46.607193+0000","last_undegraded":"2024-09-19T06:41:46.607193+0000","last_fullsized":"2024-09-19T06:41:46.607193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609566+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:41:46.609566+0000","last_peered":"2024-09-19T06:41:46.609566+0000","last_clean":"2024-09-19T06:41:46.609566+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:41:46.609566+0000","last_undegraded":"2024-09-19T06:41:46.609566+0000","last_fullsized":"2024-09-19T06:41:46.609566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590543+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:41:46.590543+0000","last_peered":"2024-09-19T06:41:46.590543+0000","last_clean":"2024-09-19T06:41:46.590543+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:41:46.590543+0000","last_undegraded":"2024-09-19T06:41:46.590543+0000","last_fullsized":"2024-09-19T06:41:46.590543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860924+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:41:46.860924+0000","last_peered":"2024-09-19T06:41:46.860924+0000","last_clean":"2024-09-19T06:41:46.860924+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:41:46.860924+0000","last_undegraded":"2024-09-19T06:41:46.860924+0000","last_fullsized":"2024-09-19T06:41:46.860924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607557+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:41:46.607557+0000","last_peered":"2024-09-19T06:41:46.607557+0000","last_clean":"2024-09-19T06:41:46.607557+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:41:46.607557+0000","last_undegraded":"2024-09-19T06:41:46.607557+0000","last_fullsized":"2024-09-19T06:41:46.607557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587632+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:41:46.587632+0000","last_peered":"2024-09-19T06:41:46.587632+0000","last_clean":"2024-09-19T06:41:46.587632+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:41:46.587632+0000","last_undegraded":"2024-09-19T06:41:46.587632+0000","last_fullsized":"2024-09-19T06:41:46.587632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866079+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:41:46.866079+0000","last_peered":"2024-09-19T06:41:46.866079+0000","last_clean":"2024-09-19T06:41:46.866079+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:41:46.866079+0000","last_undegraded":"2024-09-19T06:41:46.866079+0000","last_fullsized":"2024-09-19T06:41:46.866079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.865755+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:41:46.865755+0000","last_peered":"2024-09-19T06:41:46.865755+0000","last_clean":"2024-09-19T06:41:46.865755+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:41:46.865755+0000","last_undegraded":"2024-09-19T06:41:46.865755+0000","last_fullsized":"2024-09-19T06:41:46.865755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609034+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:41:46.609034+0000","last_peered":"2024-09-19T06:41:46.609034+0000","last_clean":"2024-09-19T06:41:46.609034+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:41:46.609034+0000","last_undegraded":"2024-09-19T06:41:46.609034+0000","last_fullsized":"2024-09-19T06:41:46.609034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934022+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:41:46.934022+0000","last_peered":"2024-09-19T06:41:46.934022+0000","last_clean":"2024-09-19T06:41:46.934022+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:41:46.934022+0000","last_undegraded":"2024-09-19T06:41:46.934022+0000","last_fullsized":"2024-09-19T06:41:46.934022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677589+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:41:46.677589+0000","last_peered":"2024-09-19T06:41:46.677589+0000","last_clean":"2024-09-19T06:41:46.677589+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:41:46.677589+0000","last_undegraded":"2024-09-19T06:41:46.677589+0000","last_fullsized":"2024-09-19T06:41:46.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609334+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:41:46.609334+0000","last_peered":"2024-09-19T06:41:46.609334+0000","last_clean":"2024-09-19T06:41:46.609334+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:41:46.609334+0000","last_undegraded":"2024-09-19T06:41:46.609334+0000","last_fullsized":"2024-09-19T06:41:46.609334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933853+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:41:46.933853+0000","last_peered":"2024-09-19T06:41:46.933853+0000","last_clean":"2024-09-19T06:41:46.933853+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:41:46.933853+0000","last_undegraded":"2024-09-19T06:41:46.933853+0000","last_fullsized":"2024-09-19T06:41:46.933853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587643+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:41:46.587643+0000","last_peered":"2024-09-19T06:41:46.587643+0000","last_clean":"2024-09-19T06:41:46.587643+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:41:46.587643+0000","last_undegraded":"2024-09-19T06:41:46.587643+0000","last_fullsized":"2024-09-19T06:41:46.587643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609138+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:41:46.609138+0000","last_peered":"2024-09-19T06:41:46.609138+0000","last_clean":"2024-09-19T06:41:46.609138+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:41:46.609138+0000","last_undegraded":"2024-09-19T06:41:46.609138+0000","last_fullsized":"2024-09-19T06:41:46.609138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607201+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:41:46.607201+0000","last_peered":"2024-09-19T06:41:46.607201+0000","last_clean":"2024-09-19T06:41:46.607201+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:41:46.607201+0000","last_undegraded":"2024-09-19T06:41:46.607201+0000","last_fullsized":"2024-09-19T06:41:46.607201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588420+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:41:46.588420+0000","last_peered":"2024-09-19T06:41:46.588420+0000","last_clean":"2024-09-19T06:41:46.588420+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:41:46.588420+0000","last_undegraded":"2024-09-19T06:41:46.588420+0000","last_fullsized":"2024-09-19T06:41:46.588420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677514+0000","last_change":"2024-09-19T06:40:58.134245+0000","last_active":"2024-09-19T06:41:46.677514+0000","last_peered":"2024-09-19T06:41:46.677514+0000","last_clean":"2024-09-19T06:41:46.677514+0000","last_became_active":"2024-09-19T06:40:58.133514+0000","last_became_peered":"2024-09-19T06:40:58.133514+0000","last_unstale":"2024-09-19T06:41:46.677514+0000","last_undegraded":"2024-09-19T06:41:46.677514+0000","last_fullsized":"2024-09-19T06:41:46.677514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:23.267466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609405+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:41:46.609405+0000","last_peered":"2024-09-19T06:41:46.609405+0000","last_clean":"2024-09-19T06:41:46.609405+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:41:46.609405+0000","last_undegraded":"2024-09-19T06:41:46.609405+0000","last_fullsized":"2024-09-19T06:41:46.609405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933946+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:41:46.933946+0000","last_peered":"2024-09-19T06:41:46.933946+0000","last_clean":"2024-09-19T06:41:46.933946+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:41:46.933946+0000","last_undegraded":"2024-09-19T06:41:46.933946+0000","last_fullsized":"2024-09-19T06:41:46.933946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587547+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:41:46.587547+0000","last_peered":"2024-09-19T06:41:46.587547+0000","last_clean":"2024-09-19T06:41:46.587547+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:41:46.587547+0000","last_undegraded":"2024-09-19T06:41:46.587547+0000","last_fullsized":"2024-09-19T06:41:46.587547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866245+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:41:46.866245+0000","last_peered":"2024-09-19T06:41:46.866245+0000","last_clean":"2024-09-19T06:41:46.866245+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:41:46.866245+0000","last_undegraded":"2024-09-19T06:41:46.866245+0000","last_fullsized":"2024-09-19T06:41:46.866245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609008+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:41:46.609008+0000","last_peered":"2024-09-19T06:41:46.609008+0000","last_clean":"2024-09-19T06:41:46.609008+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:41:46.609008+0000","last_undegraded":"2024-09-19T06:41:46.609008+0000","last_fullsized":"2024-09-19T06:41:46.609008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609041+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:41:46.609041+0000","last_peered":"2024-09-19T06:41:46.609041+0000","last_clean":"2024-09-19T06:41:46.609041+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:41:46.609041+0000","last_undegraded":"2024-09-19T06:41:46.609041+0000","last_fullsized":"2024-09-19T06:41:46.609041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933663+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:41:46.933663+0000","last_peered":"2024-09-19T06:41:46.933663+0000","last_clean":"2024-09-19T06:41:46.933663+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:41:46.933663+0000","last_undegraded":"2024-09-19T06:41:46.933663+0000","last_fullsized":"2024-09-19T06:41:46.933663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679766+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:41:46.679766+0000","last_peered":"2024-09-19T06:41:46.679766+0000","last_clean":"2024-09-19T06:41:46.679766+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:41:46.679766+0000","last_undegraded":"2024-09-19T06:41:46.679766+0000","last_fullsized":"2024-09-19T06:41:46.679766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860151+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:41:46.860151+0000","last_peered":"2024-09-19T06:41:46.860151+0000","last_clean":"2024-09-19T06:41:46.860151+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:41:46.860151+0000","last_undegraded":"2024-09-19T06:41:46.860151+0000","last_fullsized":"2024-09-19T06:41:46.860151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588409+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:41:46.588409+0000","last_peered":"2024-09-19T06:41:46.588409+0000","last_clean":"2024-09-19T06:41:46.588409+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:41:46.588409+0000","last_undegraded":"2024-09-19T06:41:46.588409+0000","last_fullsized":"2024-09-19T06:41:46.588409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590296+0000","last_change":"2024-09-19T06:40:58.136626+0000","last_active":"2024-09-19T06:41:46.590296+0000","last_peered":"2024-09-19T06:41:46.590296+0000","last_clean":"2024-09-19T06:41:46.590296+0000","last_became_active":"2024-09-19T06:40:58.132752+0000","last_became_peered":"2024-09-19T06:40:58.132752+0000","last_unstale":"2024-09-19T06:41:46.590296+0000","last_undegraded":"2024-09-19T06:41:46.590296+0000","last_fullsized":"2024-09-19T06:41:46.590296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587271+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:41:46.587271+0000","last_peered":"2024-09-19T06:41:46.587271+0000","last_clean":"2024-09-19T06:41:46.587271+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:41:46.587271+0000","last_undegraded":"2024-09-19T06:41:46.587271+0000","last_fullsized":"2024-09-19T06:41:46.587271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607281+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:41:46.607281+0000","last_peered":"2024-09-19T06:41:46.607281+0000","last_clean":"2024-09-19T06:41:46.607281+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:41:46.607281+0000","last_undegraded":"2024-09-19T06:41:46.607281+0000","last_fullsized":"2024-09-19T06:41:46.607281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588546+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:41:46.588546+0000","last_peered":"2024-09-19T06:41:46.588546+0000","last_clean":"2024-09-19T06:41:46.588546+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:41:46.588546+0000","last_undegraded":"2024-09-19T06:41:46.588546+0000","last_fullsized":"2024-09-19T06:41:46.588546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677238+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:41:46.677238+0000","last_peered":"2024-09-19T06:41:46.677238+0000","last_clean":"2024-09-19T06:41:46.677238+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:41:46.677238+0000","last_undegraded":"2024-09-19T06:41:46.677238+0000","last_fullsized":"2024-09-19T06:41:46.677238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590558+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:41:46.590558+0000","last_peered":"2024-09-19T06:41:46.590558+0000","last_clean":"2024-09-19T06:41:46.590558+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:41:46.590558+0000","last_undegraded":"2024-09-19T06:41:46.590558+0000","last_fullsized":"2024-09-19T06:41:46.590558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609442+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:41:46.609442+0000","last_peered":"2024-09-19T06:41:46.609442+0000","last_clean":"2024-09-19T06:41:46.609442+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:41:46.609442+0000","last_undegraded":"2024-09-19T06:41:46.609442+0000","last_fullsized":"2024-09-19T06:41:46.609442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860979+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:41:46.860979+0000","last_peered":"2024-09-19T06:41:46.860979+0000","last_clean":"2024-09-19T06:41:46.860979+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:41:46.860979+0000","last_undegraded":"2024-09-19T06:41:46.860979+0000","last_fullsized":"2024-09-19T06:41:46.860979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867221+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:41:46.867221+0000","last_peered":"2024-09-19T06:41:46.867221+0000","last_clean":"2024-09-19T06:41:46.867221+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:41:46.867221+0000","last_undegraded":"2024-09-19T06:41:46.867221+0000","last_fullsized":"2024-09-19T06:41:46.867221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588728+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:41:46.588728+0000","last_peered":"2024-09-19T06:41:46.588728+0000","last_clean":"2024-09-19T06:41:46.588728+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:41:46.588728+0000","last_undegraded":"2024-09-19T06:41:46.588728+0000","last_fullsized":"2024-09-19T06:41:46.588728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609314+0000","last_change":"2024-09-19T06:40:58.135595+0000","last_active":"2024-09-19T06:41:46.609314+0000","last_peered":"2024-09-19T06:41:46.609314+0000","last_clean":"2024-09-19T06:41:46.609314+0000","last_became_active":"2024-09-19T06:40:58.135031+0000","last_became_peered":"2024-09-19T06:40:58.135031+0000","last_unstale":"2024-09-19T06:41:46.609314+0000","last_undegraded":"2024-09-19T06:41:46.609314+0000","last_fullsized":"2024-09-19T06:41:46.609314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:53.542630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'34","reported_seq":278,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860101+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:41:46.860101+0000","last_peered":"2024-09-19T06:41:46.860101+0000","last_clean":"2024-09-19T06:41:46.860101+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:41:46.860101+0000","last_undegraded":"2024-09-19T06:41:46.860101+0000","last_fullsized":"2024-09-19T06:41:46.860101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":168,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680048+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:41:46.680048+0000","last_peered":"2024-09-19T06:41:46.680048+0000","last_clean":"2024-09-19T06:41:46.680048+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:41:46.680048+0000","last_undegraded":"2024-09-19T06:41:46.680048+0000","last_fullsized":"2024-09-19T06:41:46.680048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609208+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:41:46.609208+0000","last_peered":"2024-09-19T06:41:46.609208+0000","last_clean":"2024-09-19T06:41:46.609208+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:41:46.609208+0000","last_undegraded":"2024-09-19T06:41:46.609208+0000","last_fullsized":"2024-09-19T06:41:46.609208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587384+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:41:46.587384+0000","last_peered":"2024-09-19T06:41:46.587384+0000","last_clean":"2024-09-19T06:41:46.587384+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:41:46.587384+0000","last_undegraded":"2024-09-19T06:41:46.587384+0000","last_fullsized":"2024-09-19T06:41:46.587384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933722+0000","last_change":"2024-09-19T06:40:58.143264+0000","last_active":"2024-09-19T06:41:46.933722+0000","last_peered":"2024-09-19T06:41:46.933722+0000","last_clean":"2024-09-19T06:41:46.933722+0000","last_became_active":"2024-09-19T06:40:58.135400+0000","last_became_peered":"2024-09-19T06:40:58.135400+0000","last_unstale":"2024-09-19T06:41:46.933722+0000","last_undegraded":"2024-09-19T06:41:46.933722+0000","last_fullsized":"2024-09-19T06:41:46.933722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:30.500765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860840+0000","last_change":"2024-09-19T06:40:58.134136+0000","last_active":"2024-09-19T06:41:46.860840+0000","last_peered":"2024-09-19T06:41:46.860840+0000","last_clean":"2024-09-19T06:41:46.860840+0000","last_became_active":"2024-09-19T06:40:58.133734+0000","last_became_peered":"2024-09-19T06:40:58.133734+0000","last_unstale":"2024-09-19T06:41:46.860840+0000","last_undegraded":"2024-09-19T06:41:46.860840+0000","last_fullsized":"2024-09-19T06:41:46.860840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:50.822670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607548+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:41:46.607548+0000","last_peered":"2024-09-19T06:41:46.607548+0000","last_clean":"2024-09-19T06:41:46.607548+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:41:46.607548+0000","last_undegraded":"2024-09-19T06:41:46.607548+0000","last_fullsized":"2024-09-19T06:41:46.607548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860373+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:41:46.860373+0000","last_peered":"2024-09-19T06:41:46.860373+0000","last_clean":"2024-09-19T06:41:46.860373+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:41:46.860373+0000","last_undegraded":"2024-09-19T06:41:46.860373+0000","last_fullsized":"2024-09-19T06:41:46.860373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679751+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:41:46.679751+0000","last_peered":"2024-09-19T06:41:46.679751+0000","last_clean":"2024-09-19T06:41:46.679751+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:41:46.679751+0000","last_undegraded":"2024-09-19T06:41:46.679751+0000","last_fullsized":"2024-09-19T06:41:46.679751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588476+0000","last_change":"2024-09-19T06:40:58.135978+0000","last_active":"2024-09-19T06:41:46.588476+0000","last_peered":"2024-09-19T06:41:46.588476+0000","last_clean":"2024-09-19T06:41:46.588476+0000","last_became_active":"2024-09-19T06:40:58.135581+0000","last_became_peered":"2024-09-19T06:40:58.135581+0000","last_unstale":"2024-09-19T06:41:46.588476+0000","last_undegraded":"2024-09-19T06:41:46.588476+0000","last_fullsized":"2024-09-19T06:41:46.588476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:54.532003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680159+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:41:46.680159+0000","last_peered":"2024-09-19T06:41:46.680159+0000","last_clean":"2024-09-19T06:41:46.680159+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:41:46.680159+0000","last_undegraded":"2024-09-19T06:41:46.680159+0000","last_fullsized":"2024-09-19T06:41:46.680159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933860+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:41:46.933860+0000","last_peered":"2024-09-19T06:41:46.933860+0000","last_clean":"2024-09-19T06:41:46.933860+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:41:46.933860+0000","last_undegraded":"2024-09-19T06:41:46.933860+0000","last_fullsized":"2024-09-19T06:41:46.933860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587529+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:41:46.587529+0000","last_peered":"2024-09-19T06:41:46.587529+0000","last_clean":"2024-09-19T06:41:46.587529+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:41:46.587529+0000","last_undegraded":"2024-09-19T06:41:46.587529+0000","last_fullsized":"2024-09-19T06:41:46.587529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.797821+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:41:46.797821+0000","last_peered":"2024-09-19T06:41:46.797821+0000","last_clean":"2024-09-19T06:41:46.797821+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:41:46.797821+0000","last_undegraded":"2024-09-19T06:41:46.797821+0000","last_fullsized":"2024-09-19T06:41:46.797821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860872+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:41:46.860872+0000","last_peered":"2024-09-19T06:41:46.860872+0000","last_clean":"2024-09-19T06:41:46.860872+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:41:46.860872+0000","last_undegraded":"2024-09-19T06:41:46.860872+0000","last_fullsized":"2024-09-19T06:41:46.860872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607210+0000","last_change":"2024-09-19T06:40:58.138283+0000","last_active":"2024-09-19T06:41:46.607210+0000","last_peered":"2024-09-19T06:41:46.607210+0000","last_clean":"2024-09-19T06:41:46.607210+0000","last_became_active":"2024-09-19T06:40:58.132079+0000","last_became_peered":"2024-09-19T06:40:58.132079+0000","last_unstale":"2024-09-19T06:41:46.607210+0000","last_undegraded":"2024-09-19T06:41:46.607210+0000","last_fullsized":"2024-09-19T06:41:46.607210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:12.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860532+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:41:46.860532+0000","last_peered":"2024-09-19T06:41:46.860532+0000","last_clean":"2024-09-19T06:41:46.860532+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:41:46.860532+0000","last_undegraded":"2024-09-19T06:41:46.860532+0000","last_fullsized":"2024-09-19T06:41:46.860532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607259+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:41:46.607259+0000","last_peered":"2024-09-19T06:41:46.607259+0000","last_clean":"2024-09-19T06:41:46.607259+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:41:46.607259+0000","last_undegraded":"2024-09-19T06:41:46.607259+0000","last_fullsized":"2024-09-19T06:41:46.607259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607408+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:41:46.607408+0000","last_peered":"2024-09-19T06:41:46.607408+0000","last_clean":"2024-09-19T06:41:46.607408+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:41:46.607408+0000","last_undegraded":"2024-09-19T06:41:46.607408+0000","last_fullsized":"2024-09-19T06:41:46.607408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860541+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:41:46.860541+0000","last_peered":"2024-09-19T06:41:46.860541+0000","last_clean":"2024-09-19T06:41:46.860541+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:41:46.860541+0000","last_undegraded":"2024-09-19T06:41:46.860541+0000","last_fullsized":"2024-09-19T06:41:46.860541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587269+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:41:46.587269+0000","last_peered":"2024-09-19T06:41:46.587269+0000","last_clean":"2024-09-19T06:41:46.587269+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:41:46.587269+0000","last_undegraded":"2024-09-19T06:41:46.587269+0000","last_fullsized":"2024-09-19T06:41:46.587269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":135,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.605518+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:41:46.605518+0000","last_peered":"2024-09-19T06:41:46.605518+0000","last_clean":"2024-09-19T06:41:46.605518+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:41:46.605518+0000","last_undegraded":"2024-09-19T06:41:46.605518+0000","last_fullsized":"2024-09-19T06:41:46.605518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677201+0000","last_change":"2024-09-19T06:40:58.132715+0000","last_active":"2024-09-19T06:41:46.677201+0000","last_peered":"2024-09-19T06:41:46.677201+0000","last_clean":"2024-09-19T06:41:46.677201+0000","last_became_active":"2024-09-19T06:40:58.132333+0000","last_became_peered":"2024-09-19T06:40:58.132333+0000","last_unstale":"2024-09-19T06:41:46.677201+0000","last_undegraded":"2024-09-19T06:41:46.677201+0000","last_fullsized":"2024-09-19T06:41:46.677201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:03.256998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860643+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:41:46.860643+0000","last_peered":"2024-09-19T06:41:46.860643+0000","last_clean":"2024-09-19T06:41:46.860643+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:41:46.860643+0000","last_undegraded":"2024-09-19T06:41:46.860643+0000","last_fullsized":"2024-09-19T06:41:46.860643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590285+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:41:46.590285+0000","last_peered":"2024-09-19T06:41:46.590285+0000","last_clean":"2024-09-19T06:41:46.590285+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:41:46.590285+0000","last_undegraded":"2024-09-19T06:41:46.590285+0000","last_fullsized":"2024-09-19T06:41:46.590285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677533+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:41:46.677533+0000","last_peered":"2024-09-19T06:41:46.677533+0000","last_clean":"2024-09-19T06:41:46.677533+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:41:46.677533+0000","last_undegraded":"2024-09-19T06:41:46.677533+0000","last_fullsized":"2024-09-19T06:41:46.677533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590332+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:41:46.590332+0000","last_peered":"2024-09-19T06:41:46.590332+0000","last_clean":"2024-09-19T06:41:46.590332+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:41:46.590332+0000","last_undegraded":"2024-09-19T06:41:46.590332+0000","last_fullsized":"2024-09-19T06:41:46.590332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933826+0000","last_change":"2024-09-19T06:40:58.140944+0000","last_active":"2024-09-19T06:41:46.933826+0000","last_peered":"2024-09-19T06:41:46.933826+0000","last_clean":"2024-09-19T06:41:46.933826+0000","last_became_active":"2024-09-19T06:40:58.132427+0000","last_became_peered":"2024-09-19T06:40:58.132427+0000","last_unstale":"2024-09-19T06:41:46.933826+0000","last_undegraded":"2024-09-19T06:41:46.933826+0000","last_fullsized":"2024-09-19T06:41:46.933826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:32.472242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587689+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:41:46.587689+0000","last_peered":"2024-09-19T06:41:46.587689+0000","last_clean":"2024-09-19T06:41:46.587689+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:41:46.587689+0000","last_undegraded":"2024-09-19T06:41:46.587689+0000","last_fullsized":"2024-09-19T06:41:46.587689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'47","reported_seq":94,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.800662+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:41:46.800662+0000","last_peered":"2024-09-19T06:41:46.800662+0000","last_clean":"2024-09-19T06:41:46.800662+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:41:46.800662+0000","last_undegraded":"2024-09-19T06:41:46.800662+0000","last_fullsized":"2024-09-19T06:41:46.800662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607761+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:41:46.607761+0000","last_peered":"2024-09-19T06:41:46.607761+0000","last_clean":"2024-09-19T06:41:46.607761+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:41:46.607761+0000","last_undegraded":"2024-09-19T06:41:46.607761+0000","last_fullsized":"2024-09-19T06:41:46.607761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607492+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:41:46.607492+0000","last_peered":"2024-09-19T06:41:46.607492+0000","last_clean":"2024-09-19T06:41:46.607492+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:41:46.607492+0000","last_undegraded":"2024-09-19T06:41:46.607492+0000","last_fullsized":"2024-09-19T06:41:46.607492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866926+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:41:46.866926+0000","last_peered":"2024-09-19T06:41:46.866926+0000","last_clean":"2024-09-19T06:41:46.866926+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:41:46.866926+0000","last_undegraded":"2024-09-19T06:41:46.866926+0000","last_fullsized":"2024-09-19T06:41:46.866926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609037+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:41:46.609037+0000","last_peered":"2024-09-19T06:41:46.609037+0000","last_clean":"2024-09-19T06:41:46.609037+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:41:46.609037+0000","last_undegraded":"2024-09-19T06:41:46.609037+0000","last_fullsized":"2024-09-19T06:41:46.609037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679748+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:41:46.679748+0000","last_peered":"2024-09-19T06:41:46.679748+0000","last_clean":"2024-09-19T06:41:46.679748+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:41:46.679748+0000","last_undegraded":"2024-09-19T06:41:46.679748+0000","last_fullsized":"2024-09-19T06:41:46.679748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607361+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:41:46.607361+0000","last_peered":"2024-09-19T06:41:46.607361+0000","last_clean":"2024-09-19T06:41:46.607361+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:41:46.607361+0000","last_undegraded":"2024-09-19T06:41:46.607361+0000","last_fullsized":"2024-09-19T06:41:46.607361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933699+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:41:46.933699+0000","last_peered":"2024-09-19T06:41:46.933699+0000","last_clean":"2024-09-19T06:41:46.933699+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:41:46.933699+0000","last_undegraded":"2024-09-19T06:41:46.933699+0000","last_fullsized":"2024-09-19T06:41:46.933699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609110+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:41:46.609110+0000","last_peered":"2024-09-19T06:41:46.609110+0000","last_clean":"2024-09-19T06:41:46.609110+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:41:46.609110+0000","last_undegraded":"2024-09-19T06:41:46.609110+0000","last_fullsized":"2024-09-19T06:41:46.609110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607487+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:41:46.607487+0000","last_peered":"2024-09-19T06:41:46.607487+0000","last_clean":"2024-09-19T06:41:46.607487+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:41:46.607487+0000","last_undegraded":"2024-09-19T06:41:46.607487+0000","last_fullsized":"2024-09-19T06:41:46.607487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679819+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:41:46.679819+0000","last_peered":"2024-09-19T06:41:46.679819+0000","last_clean":"2024-09-19T06:41:46.679819+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:41:46.679819+0000","last_undegraded":"2024-09-19T06:41:46.679819+0000","last_fullsized":"2024-09-19T06:41:46.679819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860513+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:41:46.860513+0000","last_peered":"2024-09-19T06:41:46.860513+0000","last_clean":"2024-09-19T06:41:46.860513+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:41:46.860513+0000","last_undegraded":"2024-09-19T06:41:46.860513+0000","last_fullsized":"2024-09-19T06:41:46.860513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588680+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:41:46.588680+0000","last_peered":"2024-09-19T06:41:46.588680+0000","last_clean":"2024-09-19T06:41:46.588680+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:41:46.588680+0000","last_undegraded":"2024-09-19T06:41:46.588680+0000","last_fullsized":"2024-09-19T06:41:46.588680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680018+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:41:46.680018+0000","last_peered":"2024-09-19T06:41:46.680018+0000","last_clean":"2024-09-19T06:41:46.680018+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:41:46.680018+0000","last_undegraded":"2024-09-19T06:41:46.680018+0000","last_fullsized":"2024-09-19T06:41:46.680018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609092+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:41:46.609092+0000","last_peered":"2024-09-19T06:41:46.609092+0000","last_clean":"2024-09-19T06:41:46.609092+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:41:46.609092+0000","last_undegraded":"2024-09-19T06:41:46.609092+0000","last_fullsized":"2024-09-19T06:41:46.609092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680073+0000","last_change":"2024-09-19T06:40:58.139755+0000","last_active":"2024-09-19T06:41:46.680073+0000","last_peered":"2024-09-19T06:41:46.680073+0000","last_clean":"2024-09-19T06:41:46.680073+0000","last_became_active":"2024-09-19T06:40:58.135411+0000","last_became_peered":"2024-09-19T06:40:58.135411+0000","last_unstale":"2024-09-19T06:41:46.680073+0000","last_undegraded":"2024-09-19T06:41:46.680073+0000","last_fullsized":"2024-09-19T06:41:46.680073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:38.951366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867192+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:41:46.867192+0000","last_peered":"2024-09-19T06:41:46.867192+0000","last_clean":"2024-09-19T06:41:46.867192+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:41:46.867192+0000","last_undegraded":"2024-09-19T06:41:46.867192+0000","last_fullsized":"2024-09-19T06:41:46.867192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607877+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:41:46.607877+0000","last_peered":"2024-09-19T06:41:46.607877+0000","last_clean":"2024-09-19T06:41:46.607877+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:41:46.607877+0000","last_undegraded":"2024-09-19T06:41:46.607877+0000","last_fullsized":"2024-09-19T06:41:46.607877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607731+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:41:46.607731+0000","last_peered":"2024-09-19T06:41:46.607731+0000","last_clean":"2024-09-19T06:41:46.607731+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:41:46.607731+0000","last_undegraded":"2024-09-19T06:41:46.607731+0000","last_fullsized":"2024-09-19T06:41:46.607731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677656+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:41:46.677656+0000","last_peered":"2024-09-19T06:41:46.677656+0000","last_clean":"2024-09-19T06:41:46.677656+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:41:46.677656+0000","last_undegraded":"2024-09-19T06:41:46.677656+0000","last_fullsized":"2024-09-19T06:41:46.677656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596374,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":751,"num_read_kb":476,"num_write":292,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1789122,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":437263,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":1236161,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":774142,"data_compressed":24564,"data_compressed_allocated":32768,"data_compressed_original":352256,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":446360,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1029778,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445851,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":1170299,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":447797,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":970978,"data_compressed":33961,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":511861,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":511571,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438137,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134170,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200032,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:41:52.067 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-19T06:41:52.067 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-19T06:41:52.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 -- 172.21.15.37:0/1288674396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1534101360 msgr2=0x7f153410d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:52.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 --2- 172.21.15.37:0/1288674396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1534101360 0x7f153410d890 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f152c009cd0 tx=0x7f152c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 -- 172.21.15.37:0/1288674396 shutdown_connections 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 --2- 172.21.15.37:0/1288674396 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1534101360 0x7f153410d890 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 --2- 172.21.15.37:0/1288674396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f15341009c0 0x7f1534100e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 --2- 172.21.15.37:0/1288674396 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15341069d0 0x7f1534106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 -- 172.21.15.37:0/1288674396 >> 172.21.15.37:0/1288674396 conn(0x7f15340fc6a0 msgr2=0x7f15340feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 -- 172.21.15.37:0/1288674396 shutdown_connections 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 -- 172.21.15.37:0/1288674396 wait complete. 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 Processor -- start 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.205+0000 7f153aed0640 1 -- start start 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153aed0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15341009c0 0x7f153419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153aed0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1534101360 0x7f153419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153aed0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15341069d0 0x7f153419eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:52.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153aed0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1534110e10 con 0x7f15341009c0 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153aed0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1534110c90 con 0x7f1534101360 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153aed0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1534110f90 con 0x7f15341069d0 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15341009c0 0x7f153419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15341009c0 0x7f153419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54832/0 (socket says 172.21.15.37:54832) 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 -- 172.21.15.37:0/797147326 learned_addr learned my addr 172.21.15.37:0/797147326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153946d640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15341069d0 0x7f153419eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1533fff640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1534101360 0x7f153419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 -- 172.21.15.37:0/797147326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15341069d0 msgr2=0x7f153419eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15341069d0 0x7f153419eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 -- 172.21.15.37:0/797147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1534101360 msgr2=0x7f153419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1534101360 0x7f153419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 -- 172.21.15.37:0/797147326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153419f5d0 con 0x7f15341009c0 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1538c6c640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15341009c0 0x7f153419a5e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f152400d770 tx=0x7f152400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:52.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1531ffb640 1 -- 172.21.15.37:0/797147326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524017070 con 0x7f15341009c0 2024-09-19T06:41:52.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1531ffb640 1 -- 172.21.15.37:0/797147326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1524005500 con 0x7f15341009c0 2024-09-19T06:41:52.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153419f8c0 con 0x7f15341009c0 2024-09-19T06:41:52.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.206+0000 7f1531ffb640 1 -- 172.21.15.37:0/797147326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1524004e00 con 0x7f15341009c0 2024-09-19T06:41:52.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.207+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15341a7150 con 0x7f15341009c0 2024-09-19T06:41:52.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.208+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1534100e20 con 0x7f15341009c0 2024-09-19T06:41:52.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.209+0000 7f1531ffb640 1 -- 172.21.15.37:0/797147326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f1524004fa0 con 0x7f15341009c0 2024-09-19T06:41:52.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.209+0000 7f1531ffb640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f151c0a34e0 0x7f151c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:52.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.210+0000 7f1533fff640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f151c0a34e0 0x7f151c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:52.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.210+0000 7f1531ffb640 1 -- 172.21.15.37:0/797147326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f15240cf9a0 con 0x7f15341009c0 2024-09-19T06:41:52.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.210+0000 7f1533fff640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f151c0a34e0 0x7f151c0a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15340feaa0 tx=0x7f1528005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:52.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.214+0000 7f1531ffb640 1 -- 172.21.15.37:0/797147326 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f152409c130 con 0x7f15341009c0 2024-09-19T06:41:52.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.386+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f153406aad0 con 0x7f151c0a34e0 2024-09-19T06:41:52.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.393+0000 7f1531ffb640 1 -- 172.21.15.37:0/797147326 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258000 (secure 0 0 0) 0x7f153406aad0 con 0x7f151c0a34e0 2024-09-19T06:41:52.398 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:52.399 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:41:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f151c0a34e0 msgr2=0x7f151c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f151c0a34e0 0x7f151c0a59a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f15340feaa0 tx=0x7f1528005f90 comp rx=0 tx=0).stop 2024-09-19T06:41:52.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15341009c0 msgr2=0x7f153419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15341009c0 0x7f153419a5e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f152400d770 tx=0x7f152400dc30 comp rx=0 tx=0).stop 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 shutdown_connections 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f15341069d0 0x7f153419eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1534101360 0x7f153419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f151c0a34e0 0x7f151c0a59a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 --2- 172.21.15.37:0/797147326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f15341009c0 0x7f153419a5e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 >> 172.21.15.37:0/797147326 conn(0x7f15340fc6a0 msgr2=0x7f153406a3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 shutdown_connections 2024-09-19T06:41:52.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:52.397+0000 7f153aed0640 1 -- 172.21.15.37:0/797147326 wait complete. 2024-09-19T06:41:52.410 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-09-19T06:41:50.600810+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1186976,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":887,"num_read_kb":595,"num_write":475,"num_write_kb":4377,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":867,"ondisk_log_size":867,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364700,"kb_used_data":11264,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577124,"statfs":{"total":1151940427776,"available":1151566974976,"internally_reserved":0,"allocated":11534336,"data_stored":8420198,"data_compressed":256268,"data_compressed_allocated":339968,"data_compressed_original":3129344,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990992"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934024+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:41:46.934024+0000","last_peered":"2024-09-19T06:41:46.934024+0000","last_clean":"2024-09-19T06:41:46.934024+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:41:46.934024+0000","last_undegraded":"2024-09-19T06:41:46.934024+0000","last_fullsized":"2024-09-19T06:41:46.934024+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866799+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:41:46.866799+0000","last_peered":"2024-09-19T06:41:46.866799+0000","last_clean":"2024-09-19T06:41:46.866799+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:41:46.866799+0000","last_undegraded":"2024-09-19T06:41:46.866799+0000","last_fullsized":"2024-09-19T06:41:46.866799+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866503+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:41:46.866503+0000","last_peered":"2024-09-19T06:41:46.866503+0000","last_clean":"2024-09-19T06:41:46.866503+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:41:46.866503+0000","last_undegraded":"2024-09-19T06:41:46.866503+0000","last_fullsized":"2024-09-19T06:41:46.866503+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607414+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:41:46.607414+0000","last_peered":"2024-09-19T06:41:46.607414+0000","last_clean":"2024-09-19T06:41:46.607414+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:41:46.607414+0000","last_undegraded":"2024-09-19T06:41:46.607414+0000","last_fullsized":"2024-09-19T06:41:46.607414+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679815+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:41:46.679815+0000","last_peered":"2024-09-19T06:41:46.679815+0000","last_clean":"2024-09-19T06:41:46.679815+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:41:46.679815+0000","last_undegraded":"2024-09-19T06:41:46.679815+0000","last_fullsized":"2024-09-19T06:41:46.679815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590191+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:41:46.590191+0000","last_peered":"2024-09-19T06:41:46.590191+0000","last_clean":"2024-09-19T06:41:46.590191+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:41:46.590191+0000","last_undegraded":"2024-09-19T06:41:46.590191+0000","last_fullsized":"2024-09-19T06:41:46.590191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860437+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:41:46.860437+0000","last_peered":"2024-09-19T06:41:46.860437+0000","last_clean":"2024-09-19T06:41:46.860437+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:41:46.860437+0000","last_undegraded":"2024-09-19T06:41:46.860437+0000","last_fullsized":"2024-09-19T06:41:46.860437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607193+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:41:46.607193+0000","last_peered":"2024-09-19T06:41:46.607193+0000","last_clean":"2024-09-19T06:41:46.607193+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:41:46.607193+0000","last_undegraded":"2024-09-19T06:41:46.607193+0000","last_fullsized":"2024-09-19T06:41:46.607193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609566+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:41:46.609566+0000","last_peered":"2024-09-19T06:41:46.609566+0000","last_clean":"2024-09-19T06:41:46.609566+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:41:46.609566+0000","last_undegraded":"2024-09-19T06:41:46.609566+0000","last_fullsized":"2024-09-19T06:41:46.609566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590543+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:41:46.590543+0000","last_peered":"2024-09-19T06:41:46.590543+0000","last_clean":"2024-09-19T06:41:46.590543+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:41:46.590543+0000","last_undegraded":"2024-09-19T06:41:46.590543+0000","last_fullsized":"2024-09-19T06:41:46.590543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860924+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:41:46.860924+0000","last_peered":"2024-09-19T06:41:46.860924+0000","last_clean":"2024-09-19T06:41:46.860924+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:41:46.860924+0000","last_undegraded":"2024-09-19T06:41:46.860924+0000","last_fullsized":"2024-09-19T06:41:46.860924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607557+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:41:46.607557+0000","last_peered":"2024-09-19T06:41:46.607557+0000","last_clean":"2024-09-19T06:41:46.607557+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:41:46.607557+0000","last_undegraded":"2024-09-19T06:41:46.607557+0000","last_fullsized":"2024-09-19T06:41:46.607557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587632+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:41:46.587632+0000","last_peered":"2024-09-19T06:41:46.587632+0000","last_clean":"2024-09-19T06:41:46.587632+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:41:46.587632+0000","last_undegraded":"2024-09-19T06:41:46.587632+0000","last_fullsized":"2024-09-19T06:41:46.587632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866079+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:41:46.866079+0000","last_peered":"2024-09-19T06:41:46.866079+0000","last_clean":"2024-09-19T06:41:46.866079+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:41:46.866079+0000","last_undegraded":"2024-09-19T06:41:46.866079+0000","last_fullsized":"2024-09-19T06:41:46.866079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.865755+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:41:46.865755+0000","last_peered":"2024-09-19T06:41:46.865755+0000","last_clean":"2024-09-19T06:41:46.865755+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:41:46.865755+0000","last_undegraded":"2024-09-19T06:41:46.865755+0000","last_fullsized":"2024-09-19T06:41:46.865755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609034+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:41:46.609034+0000","last_peered":"2024-09-19T06:41:46.609034+0000","last_clean":"2024-09-19T06:41:46.609034+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:41:46.609034+0000","last_undegraded":"2024-09-19T06:41:46.609034+0000","last_fullsized":"2024-09-19T06:41:46.609034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934022+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:41:46.934022+0000","last_peered":"2024-09-19T06:41:46.934022+0000","last_clean":"2024-09-19T06:41:46.934022+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:41:46.934022+0000","last_undegraded":"2024-09-19T06:41:46.934022+0000","last_fullsized":"2024-09-19T06:41:46.934022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677589+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:41:46.677589+0000","last_peered":"2024-09-19T06:41:46.677589+0000","last_clean":"2024-09-19T06:41:46.677589+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:41:46.677589+0000","last_undegraded":"2024-09-19T06:41:46.677589+0000","last_fullsized":"2024-09-19T06:41:46.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609334+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:41:46.609334+0000","last_peered":"2024-09-19T06:41:46.609334+0000","last_clean":"2024-09-19T06:41:46.609334+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:41:46.609334+0000","last_undegraded":"2024-09-19T06:41:46.609334+0000","last_fullsized":"2024-09-19T06:41:46.609334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933853+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:41:46.933853+0000","last_peered":"2024-09-19T06:41:46.933853+0000","last_clean":"2024-09-19T06:41:46.933853+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:41:46.933853+0000","last_undegraded":"2024-09-19T06:41:46.933853+0000","last_fullsized":"2024-09-19T06:41:46.933853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587643+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:41:46.587643+0000","last_peered":"2024-09-19T06:41:46.587643+0000","last_clean":"2024-09-19T06:41:46.587643+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:41:46.587643+0000","last_undegraded":"2024-09-19T06:41:46.587643+0000","last_fullsized":"2024-09-19T06:41:46.587643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609138+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:41:46.609138+0000","last_peered":"2024-09-19T06:41:46.609138+0000","last_clean":"2024-09-19T06:41:46.609138+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:41:46.609138+0000","last_undegraded":"2024-09-19T06:41:46.609138+0000","last_fullsized":"2024-09-19T06:41:46.609138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607201+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:41:46.607201+0000","last_peered":"2024-09-19T06:41:46.607201+0000","last_clean":"2024-09-19T06:41:46.607201+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:41:46.607201+0000","last_undegraded":"2024-09-19T06:41:46.607201+0000","last_fullsized":"2024-09-19T06:41:46.607201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588420+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:41:46.588420+0000","last_peered":"2024-09-19T06:41:46.588420+0000","last_clean":"2024-09-19T06:41:46.588420+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:41:46.588420+0000","last_undegraded":"2024-09-19T06:41:46.588420+0000","last_fullsized":"2024-09-19T06:41:46.588420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677514+0000","last_change":"2024-09-19T06:40:58.134245+0000","last_active":"2024-09-19T06:41:46.677514+0000","last_peered":"2024-09-19T06:41:46.677514+0000","last_clean":"2024-09-19T06:41:46.677514+0000","last_became_active":"2024-09-19T06:40:58.133514+0000","last_became_peered":"2024-09-19T06:40:58.133514+0000","last_unstale":"2024-09-19T06:41:46.677514+0000","last_undegraded":"2024-09-19T06:41:46.677514+0000","last_fullsized":"2024-09-19T06:41:46.677514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:23.267466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609405+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:41:46.609405+0000","last_peered":"2024-09-19T06:41:46.609405+0000","last_clean":"2024-09-19T06:41:46.609405+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:41:46.609405+0000","last_undegraded":"2024-09-19T06:41:46.609405+0000","last_fullsized":"2024-09-19T06:41:46.609405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933946+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:41:46.933946+0000","last_peered":"2024-09-19T06:41:46.933946+0000","last_clean":"2024-09-19T06:41:46.933946+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:41:46.933946+0000","last_undegraded":"2024-09-19T06:41:46.933946+0000","last_fullsized":"2024-09-19T06:41:46.933946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587547+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:41:46.587547+0000","last_peered":"2024-09-19T06:41:46.587547+0000","last_clean":"2024-09-19T06:41:46.587547+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:41:46.587547+0000","last_undegraded":"2024-09-19T06:41:46.587547+0000","last_fullsized":"2024-09-19T06:41:46.587547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866245+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:41:46.866245+0000","last_peered":"2024-09-19T06:41:46.866245+0000","last_clean":"2024-09-19T06:41:46.866245+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:41:46.866245+0000","last_undegraded":"2024-09-19T06:41:46.866245+0000","last_fullsized":"2024-09-19T06:41:46.866245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609008+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:41:46.609008+0000","last_peered":"2024-09-19T06:41:46.609008+0000","last_clean":"2024-09-19T06:41:46.609008+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:41:46.609008+0000","last_undegraded":"2024-09-19T06:41:46.609008+0000","last_fullsized":"2024-09-19T06:41:46.609008+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609041+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:41:46.609041+0000","last_peered":"2024-09-19T06:41:46.609041+0000","last_clean":"2024-09-19T06:41:46.609041+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:41:46.609041+0000","last_undegraded":"2024-09-19T06:41:46.609041+0000","last_fullsized":"2024-09-19T06:41:46.609041+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933663+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:41:46.933663+0000","last_peered":"2024-09-19T06:41:46.933663+0000","last_clean":"2024-09-19T06:41:46.933663+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:41:46.933663+0000","last_undegraded":"2024-09-19T06:41:46.933663+0000","last_fullsized":"2024-09-19T06:41:46.933663+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679766+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:41:46.679766+0000","last_peered":"2024-09-19T06:41:46.679766+0000","last_clean":"2024-09-19T06:41:46.679766+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:41:46.679766+0000","last_undegraded":"2024-09-19T06:41:46.679766+0000","last_fullsized":"2024-09-19T06:41:46.679766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":140,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860151+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:41:46.860151+0000","last_peered":"2024-09-19T06:41:46.860151+0000","last_clean":"2024-09-19T06:41:46.860151+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:41:46.860151+0000","last_undegraded":"2024-09-19T06:41:46.860151+0000","last_fullsized":"2024-09-19T06:41:46.860151+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588409+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:41:46.588409+0000","last_peered":"2024-09-19T06:41:46.588409+0000","last_clean":"2024-09-19T06:41:46.588409+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:41:46.588409+0000","last_undegraded":"2024-09-19T06:41:46.588409+0000","last_fullsized":"2024-09-19T06:41:46.588409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590296+0000","last_change":"2024-09-19T06:40:58.136626+0000","last_active":"2024-09-19T06:41:46.590296+0000","last_peered":"2024-09-19T06:41:46.590296+0000","last_clean":"2024-09-19T06:41:46.590296+0000","last_became_active":"2024-09-19T06:40:58.132752+0000","last_became_peered":"2024-09-19T06:40:58.132752+0000","last_unstale":"2024-09-19T06:41:46.590296+0000","last_undegraded":"2024-09-19T06:41:46.590296+0000","last_fullsized":"2024-09-19T06:41:46.590296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587271+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:41:46.587271+0000","last_peered":"2024-09-19T06:41:46.587271+0000","last_clean":"2024-09-19T06:41:46.587271+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:41:46.587271+0000","last_undegraded":"2024-09-19T06:41:46.587271+0000","last_fullsized":"2024-09-19T06:41:46.587271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607281+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:41:46.607281+0000","last_peered":"2024-09-19T06:41:46.607281+0000","last_clean":"2024-09-19T06:41:46.607281+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:41:46.607281+0000","last_undegraded":"2024-09-19T06:41:46.607281+0000","last_fullsized":"2024-09-19T06:41:46.607281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588546+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:41:46.588546+0000","last_peered":"2024-09-19T06:41:46.588546+0000","last_clean":"2024-09-19T06:41:46.588546+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:41:46.588546+0000","last_undegraded":"2024-09-19T06:41:46.588546+0000","last_fullsized":"2024-09-19T06:41:46.588546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677238+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:41:46.677238+0000","last_peered":"2024-09-19T06:41:46.677238+0000","last_clean":"2024-09-19T06:41:46.677238+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:41:46.677238+0000","last_undegraded":"2024-09-19T06:41:46.677238+0000","last_fullsized":"2024-09-19T06:41:46.677238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590558+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:41:46.590558+0000","last_peered":"2024-09-19T06:41:46.590558+0000","last_clean":"2024-09-19T06:41:46.590558+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:41:46.590558+0000","last_undegraded":"2024-09-19T06:41:46.590558+0000","last_fullsized":"2024-09-19T06:41:46.590558+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609442+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:41:46.609442+0000","last_peered":"2024-09-19T06:41:46.609442+0000","last_clean":"2024-09-19T06:41:46.609442+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:41:46.609442+0000","last_undegraded":"2024-09-19T06:41:46.609442+0000","last_fullsized":"2024-09-19T06:41:46.609442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860979+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:41:46.860979+0000","last_peered":"2024-09-19T06:41:46.860979+0000","last_clean":"2024-09-19T06:41:46.860979+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:41:46.860979+0000","last_undegraded":"2024-09-19T06:41:46.860979+0000","last_fullsized":"2024-09-19T06:41:46.860979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867221+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:41:46.867221+0000","last_peered":"2024-09-19T06:41:46.867221+0000","last_clean":"2024-09-19T06:41:46.867221+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:41:46.867221+0000","last_undegraded":"2024-09-19T06:41:46.867221+0000","last_fullsized":"2024-09-19T06:41:46.867221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588728+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:41:46.588728+0000","last_peered":"2024-09-19T06:41:46.588728+0000","last_clean":"2024-09-19T06:41:46.588728+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:41:46.588728+0000","last_undegraded":"2024-09-19T06:41:46.588728+0000","last_fullsized":"2024-09-19T06:41:46.588728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609314+0000","last_change":"2024-09-19T06:40:58.135595+0000","last_active":"2024-09-19T06:41:46.609314+0000","last_peered":"2024-09-19T06:41:46.609314+0000","last_clean":"2024-09-19T06:41:46.609314+0000","last_became_active":"2024-09-19T06:40:58.135031+0000","last_became_peered":"2024-09-19T06:40:58.135031+0000","last_unstale":"2024-09-19T06:41:46.609314+0000","last_undegraded":"2024-09-19T06:41:46.609314+0000","last_fullsized":"2024-09-19T06:41:46.609314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:53.542630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'34","reported_seq":278,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860101+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:41:46.860101+0000","last_peered":"2024-09-19T06:41:46.860101+0000","last_clean":"2024-09-19T06:41:46.860101+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:41:46.860101+0000","last_undegraded":"2024-09-19T06:41:46.860101+0000","last_fullsized":"2024-09-19T06:41:46.860101+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":65927,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":220,"num_read_kb":168,"num_write":35,"num_write_kb":80,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680048+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:41:46.680048+0000","last_peered":"2024-09-19T06:41:46.680048+0000","last_clean":"2024-09-19T06:41:46.680048+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:41:46.680048+0000","last_undegraded":"2024-09-19T06:41:46.680048+0000","last_fullsized":"2024-09-19T06:41:46.680048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609208+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:41:46.609208+0000","last_peered":"2024-09-19T06:41:46.609208+0000","last_clean":"2024-09-19T06:41:46.609208+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:41:46.609208+0000","last_undegraded":"2024-09-19T06:41:46.609208+0000","last_fullsized":"2024-09-19T06:41:46.609208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587384+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:41:46.587384+0000","last_peered":"2024-09-19T06:41:46.587384+0000","last_clean":"2024-09-19T06:41:46.587384+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:41:46.587384+0000","last_undegraded":"2024-09-19T06:41:46.587384+0000","last_fullsized":"2024-09-19T06:41:46.587384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933722+0000","last_change":"2024-09-19T06:40:58.143264+0000","last_active":"2024-09-19T06:41:46.933722+0000","last_peered":"2024-09-19T06:41:46.933722+0000","last_clean":"2024-09-19T06:41:46.933722+0000","last_became_active":"2024-09-19T06:40:58.135400+0000","last_became_peered":"2024-09-19T06:40:58.135400+0000","last_unstale":"2024-09-19T06:41:46.933722+0000","last_undegraded":"2024-09-19T06:41:46.933722+0000","last_fullsized":"2024-09-19T06:41:46.933722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:30.500765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860840+0000","last_change":"2024-09-19T06:40:58.134136+0000","last_active":"2024-09-19T06:41:46.860840+0000","last_peered":"2024-09-19T06:41:46.860840+0000","last_clean":"2024-09-19T06:41:46.860840+0000","last_became_active":"2024-09-19T06:40:58.133734+0000","last_became_peered":"2024-09-19T06:40:58.133734+0000","last_unstale":"2024-09-19T06:41:46.860840+0000","last_undegraded":"2024-09-19T06:41:46.860840+0000","last_fullsized":"2024-09-19T06:41:46.860840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:50.822670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'6","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607548+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:41:46.607548+0000","last_peered":"2024-09-19T06:41:46.607548+0000","last_clean":"2024-09-19T06:41:46.607548+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:41:46.607548+0000","last_undegraded":"2024-09-19T06:41:46.607548+0000","last_fullsized":"2024-09-19T06:41:46.607548+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1515,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860373+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:41:46.860373+0000","last_peered":"2024-09-19T06:41:46.860373+0000","last_clean":"2024-09-19T06:41:46.860373+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:41:46.860373+0000","last_undegraded":"2024-09-19T06:41:46.860373+0000","last_fullsized":"2024-09-19T06:41:46.860373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679751+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:41:46.679751+0000","last_peered":"2024-09-19T06:41:46.679751+0000","last_clean":"2024-09-19T06:41:46.679751+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:41:46.679751+0000","last_undegraded":"2024-09-19T06:41:46.679751+0000","last_fullsized":"2024-09-19T06:41:46.679751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588476+0000","last_change":"2024-09-19T06:40:58.135978+0000","last_active":"2024-09-19T06:41:46.588476+0000","last_peered":"2024-09-19T06:41:46.588476+0000","last_clean":"2024-09-19T06:41:46.588476+0000","last_became_active":"2024-09-19T06:40:58.135581+0000","last_became_peered":"2024-09-19T06:40:58.135581+0000","last_unstale":"2024-09-19T06:41:46.588476+0000","last_undegraded":"2024-09-19T06:41:46.588476+0000","last_fullsized":"2024-09-19T06:41:46.588476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:54.532003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680159+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:41:46.680159+0000","last_peered":"2024-09-19T06:41:46.680159+0000","last_clean":"2024-09-19T06:41:46.680159+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:41:46.680159+0000","last_undegraded":"2024-09-19T06:41:46.680159+0000","last_fullsized":"2024-09-19T06:41:46.680159+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933860+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:41:46.933860+0000","last_peered":"2024-09-19T06:41:46.933860+0000","last_clean":"2024-09-19T06:41:46.933860+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:41:46.933860+0000","last_undegraded":"2024-09-19T06:41:46.933860+0000","last_fullsized":"2024-09-19T06:41:46.933860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587529+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:41:46.587529+0000","last_peered":"2024-09-19T06:41:46.587529+0000","last_clean":"2024-09-19T06:41:46.587529+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:41:46.587529+0000","last_undegraded":"2024-09-19T06:41:46.587529+0000","last_fullsized":"2024-09-19T06:41:46.587529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'39","reported_seq":71,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.797821+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:41:46.797821+0000","last_peered":"2024-09-19T06:41:46.797821+0000","last_clean":"2024-09-19T06:41:46.797821+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:41:46.797821+0000","last_undegraded":"2024-09-19T06:41:46.797821+0000","last_fullsized":"2024-09-19T06:41:46.797821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860872+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:41:46.860872+0000","last_peered":"2024-09-19T06:41:46.860872+0000","last_clean":"2024-09-19T06:41:46.860872+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:41:46.860872+0000","last_undegraded":"2024-09-19T06:41:46.860872+0000","last_fullsized":"2024-09-19T06:41:46.860872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607210+0000","last_change":"2024-09-19T06:40:58.138283+0000","last_active":"2024-09-19T06:41:46.607210+0000","last_peered":"2024-09-19T06:41:46.607210+0000","last_clean":"2024-09-19T06:41:46.607210+0000","last_became_active":"2024-09-19T06:40:58.132079+0000","last_became_peered":"2024-09-19T06:40:58.132079+0000","last_unstale":"2024-09-19T06:41:46.607210+0000","last_undegraded":"2024-09-19T06:41:46.607210+0000","last_fullsized":"2024-09-19T06:41:46.607210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:12.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":346,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860532+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:41:46.860532+0000","last_peered":"2024-09-19T06:41:46.860532+0000","last_clean":"2024-09-19T06:41:46.860532+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:41:46.860532+0000","last_undegraded":"2024-09-19T06:41:46.860532+0000","last_fullsized":"2024-09-19T06:41:46.860532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607259+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:41:46.607259+0000","last_peered":"2024-09-19T06:41:46.607259+0000","last_clean":"2024-09-19T06:41:46.607259+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:41:46.607259+0000","last_undegraded":"2024-09-19T06:41:46.607259+0000","last_fullsized":"2024-09-19T06:41:46.607259+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607408+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:41:46.607408+0000","last_peered":"2024-09-19T06:41:46.607408+0000","last_clean":"2024-09-19T06:41:46.607408+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:41:46.607408+0000","last_undegraded":"2024-09-19T06:41:46.607408+0000","last_fullsized":"2024-09-19T06:41:46.607408+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860541+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:41:46.860541+0000","last_peered":"2024-09-19T06:41:46.860541+0000","last_clean":"2024-09-19T06:41:46.860541+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:41:46.860541+0000","last_undegraded":"2024-09-19T06:41:46.860541+0000","last_fullsized":"2024-09-19T06:41:46.860541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587269+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:41:46.587269+0000","last_peered":"2024-09-19T06:41:46.587269+0000","last_clean":"2024-09-19T06:41:46.587269+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:41:46.587269+0000","last_undegraded":"2024-09-19T06:41:46.587269+0000","last_fullsized":"2024-09-19T06:41:46.587269+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":135,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.605518+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:41:46.605518+0000","last_peered":"2024-09-19T06:41:46.605518+0000","last_clean":"2024-09-19T06:41:46.605518+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:41:46.605518+0000","last_undegraded":"2024-09-19T06:41:46.605518+0000","last_fullsized":"2024-09-19T06:41:46.605518+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":105,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677201+0000","last_change":"2024-09-19T06:40:58.132715+0000","last_active":"2024-09-19T06:41:46.677201+0000","last_peered":"2024-09-19T06:41:46.677201+0000","last_clean":"2024-09-19T06:41:46.677201+0000","last_became_active":"2024-09-19T06:40:58.132333+0000","last_became_peered":"2024-09-19T06:40:58.132333+0000","last_unstale":"2024-09-19T06:41:46.677201+0000","last_undegraded":"2024-09-19T06:41:46.677201+0000","last_fullsized":"2024-09-19T06:41:46.677201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:03.256998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860643+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:41:46.860643+0000","last_peered":"2024-09-19T06:41:46.860643+0000","last_clean":"2024-09-19T06:41:46.860643+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:41:46.860643+0000","last_undegraded":"2024-09-19T06:41:46.860643+0000","last_fullsized":"2024-09-19T06:41:46.860643+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590285+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:41:46.590285+0000","last_peered":"2024-09-19T06:41:46.590285+0000","last_clean":"2024-09-19T06:41:46.590285+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:41:46.590285+0000","last_undegraded":"2024-09-19T06:41:46.590285+0000","last_fullsized":"2024-09-19T06:41:46.590285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677533+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:41:46.677533+0000","last_peered":"2024-09-19T06:41:46.677533+0000","last_clean":"2024-09-19T06:41:46.677533+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:41:46.677533+0000","last_undegraded":"2024-09-19T06:41:46.677533+0000","last_fullsized":"2024-09-19T06:41:46.677533+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590332+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:41:46.590332+0000","last_peered":"2024-09-19T06:41:46.590332+0000","last_clean":"2024-09-19T06:41:46.590332+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:41:46.590332+0000","last_undegraded":"2024-09-19T06:41:46.590332+0000","last_fullsized":"2024-09-19T06:41:46.590332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933826+0000","last_change":"2024-09-19T06:40:58.140944+0000","last_active":"2024-09-19T06:41:46.933826+0000","last_peered":"2024-09-19T06:41:46.933826+0000","last_clean":"2024-09-19T06:41:46.933826+0000","last_became_active":"2024-09-19T06:40:58.132427+0000","last_became_peered":"2024-09-19T06:40:58.132427+0000","last_unstale":"2024-09-19T06:41:46.933826+0000","last_undegraded":"2024-09-19T06:41:46.933826+0000","last_fullsized":"2024-09-19T06:41:46.933826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:32.472242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587689+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:41:46.587689+0000","last_peered":"2024-09-19T06:41:46.587689+0000","last_clean":"2024-09-19T06:41:46.587689+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:41:46.587689+0000","last_undegraded":"2024-09-19T06:41:46.587689+0000","last_fullsized":"2024-09-19T06:41:46.587689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'47","reported_seq":94,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.800662+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:41:46.800662+0000","last_peered":"2024-09-19T06:41:46.800662+0000","last_clean":"2024-09-19T06:41:46.800662+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:41:46.800662+0000","last_undegraded":"2024-09-19T06:41:46.800662+0000","last_fullsized":"2024-09-19T06:41:46.800662+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607761+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:41:46.607761+0000","last_peered":"2024-09-19T06:41:46.607761+0000","last_clean":"2024-09-19T06:41:46.607761+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:41:46.607761+0000","last_undegraded":"2024-09-19T06:41:46.607761+0000","last_fullsized":"2024-09-19T06:41:46.607761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607492+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:41:46.607492+0000","last_peered":"2024-09-19T06:41:46.607492+0000","last_clean":"2024-09-19T06:41:46.607492+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:41:46.607492+0000","last_undegraded":"2024-09-19T06:41:46.607492+0000","last_fullsized":"2024-09-19T06:41:46.607492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866926+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:41:46.866926+0000","last_peered":"2024-09-19T06:41:46.866926+0000","last_clean":"2024-09-19T06:41:46.866926+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:41:46.866926+0000","last_undegraded":"2024-09-19T06:41:46.866926+0000","last_fullsized":"2024-09-19T06:41:46.866926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609037+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:41:46.609037+0000","last_peered":"2024-09-19T06:41:46.609037+0000","last_clean":"2024-09-19T06:41:46.609037+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:41:46.609037+0000","last_undegraded":"2024-09-19T06:41:46.609037+0000","last_fullsized":"2024-09-19T06:41:46.609037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679748+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:41:46.679748+0000","last_peered":"2024-09-19T06:41:46.679748+0000","last_clean":"2024-09-19T06:41:46.679748+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:41:46.679748+0000","last_undegraded":"2024-09-19T06:41:46.679748+0000","last_fullsized":"2024-09-19T06:41:46.679748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607361+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:41:46.607361+0000","last_peered":"2024-09-19T06:41:46.607361+0000","last_clean":"2024-09-19T06:41:46.607361+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:41:46.607361+0000","last_undegraded":"2024-09-19T06:41:46.607361+0000","last_fullsized":"2024-09-19T06:41:46.607361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933699+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:41:46.933699+0000","last_peered":"2024-09-19T06:41:46.933699+0000","last_clean":"2024-09-19T06:41:46.933699+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:41:46.933699+0000","last_undegraded":"2024-09-19T06:41:46.933699+0000","last_fullsized":"2024-09-19T06:41:46.933699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609110+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:41:46.609110+0000","last_peered":"2024-09-19T06:41:46.609110+0000","last_clean":"2024-09-19T06:41:46.609110+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:41:46.609110+0000","last_undegraded":"2024-09-19T06:41:46.609110+0000","last_fullsized":"2024-09-19T06:41:46.609110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607487+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:41:46.607487+0000","last_peered":"2024-09-19T06:41:46.607487+0000","last_clean":"2024-09-19T06:41:46.607487+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:41:46.607487+0000","last_undegraded":"2024-09-19T06:41:46.607487+0000","last_fullsized":"2024-09-19T06:41:46.607487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679819+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:41:46.679819+0000","last_peered":"2024-09-19T06:41:46.679819+0000","last_clean":"2024-09-19T06:41:46.679819+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:41:46.679819+0000","last_undegraded":"2024-09-19T06:41:46.679819+0000","last_fullsized":"2024-09-19T06:41:46.679819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860513+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:41:46.860513+0000","last_peered":"2024-09-19T06:41:46.860513+0000","last_clean":"2024-09-19T06:41:46.860513+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:41:46.860513+0000","last_undegraded":"2024-09-19T06:41:46.860513+0000","last_fullsized":"2024-09-19T06:41:46.860513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588680+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:41:46.588680+0000","last_peered":"2024-09-19T06:41:46.588680+0000","last_clean":"2024-09-19T06:41:46.588680+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:41:46.588680+0000","last_undegraded":"2024-09-19T06:41:46.588680+0000","last_fullsized":"2024-09-19T06:41:46.588680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680018+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:41:46.680018+0000","last_peered":"2024-09-19T06:41:46.680018+0000","last_clean":"2024-09-19T06:41:46.680018+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:41:46.680018+0000","last_undegraded":"2024-09-19T06:41:46.680018+0000","last_fullsized":"2024-09-19T06:41:46.680018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609092+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:41:46.609092+0000","last_peered":"2024-09-19T06:41:46.609092+0000","last_clean":"2024-09-19T06:41:46.609092+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:41:46.609092+0000","last_undegraded":"2024-09-19T06:41:46.609092+0000","last_fullsized":"2024-09-19T06:41:46.609092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680073+0000","last_change":"2024-09-19T06:40:58.139755+0000","last_active":"2024-09-19T06:41:46.680073+0000","last_peered":"2024-09-19T06:41:46.680073+0000","last_clean":"2024-09-19T06:41:46.680073+0000","last_became_active":"2024-09-19T06:40:58.135411+0000","last_became_peered":"2024-09-19T06:40:58.135411+0000","last_unstale":"2024-09-19T06:41:46.680073+0000","last_undegraded":"2024-09-19T06:41:46.680073+0000","last_fullsized":"2024-09-19T06:41:46.680073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:38.951366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867192+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:41:46.867192+0000","last_peered":"2024-09-19T06:41:46.867192+0000","last_clean":"2024-09-19T06:41:46.867192+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:41:46.867192+0000","last_undegraded":"2024-09-19T06:41:46.867192+0000","last_fullsized":"2024-09-19T06:41:46.867192+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607877+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:41:46.607877+0000","last_peered":"2024-09-19T06:41:46.607877+0000","last_clean":"2024-09-19T06:41:46.607877+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:41:46.607877+0000","last_undegraded":"2024-09-19T06:41:46.607877+0000","last_fullsized":"2024-09-19T06:41:46.607877+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607731+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:41:46.607731+0000","last_peered":"2024-09-19T06:41:46.607731+0000","last_clean":"2024-09-19T06:41:46.607731+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:41:46.607731+0000","last_undegraded":"2024-09-19T06:41:46.607731+0000","last_fullsized":"2024-09-19T06:41:46.607731+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677656+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:41:46.677656+0000","last_peered":"2024-09-19T06:41:46.677656+0000","last_clean":"2024-09-19T06:41:46.677656+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:41:46.677656+0000","last_undegraded":"2024-09-19T06:41:46.677656+0000","last_fullsized":"2024-09-19T06:41:46.677656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":596374,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":751,"num_read_kb":476,"num_write":292,"num_write_kb":1437,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":1789122,"data_compressed":75906,"data_compressed_allocated":98304,"data_compressed_original":1548288,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547236,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32064,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713088,"statfs":{"total":95995035648,"available":95962202112,"internally_reserved":0,"allocated":901120,"data_stored":437263,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776169,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28084,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717068,"statfs":{"total":95995035648,"available":95966277632,"internally_reserved":0,"allocated":1085440,"data_stored":1236161,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939695,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32112,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713040,"statfs":{"total":95995035648,"available":95962152960,"internally_reserved":0,"allocated":946176,"data_stored":774142,"data_compressed":24564,"data_compressed_allocated":32768,"data_compressed_original":352256,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168628,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":921600,"data_stored":446360,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397560,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27964,"kb_used_data":940,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717188,"statfs":{"total":95995035648,"available":95966400512,"internally_reserved":0,"allocated":962560,"data_stored":1029778,"data_compressed":41606,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561086,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445851,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790018,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28012,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717140,"statfs":{"total":95995035648,"available":95966351360,"internally_reserved":0,"allocated":1007616,"data_stored":1170299,"data_compressed":50658,"data_compressed_allocated":65536,"data_compressed_original":737280,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953544,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":447797,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182476,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32136,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713016,"statfs":{"total":95995035648,"available":95962128384,"internally_reserved":0,"allocated":974848,"data_stored":970978,"data_compressed":33961,"data_compressed_allocated":45056,"data_compressed_original":540672,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":999424,"data_stored":511861,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574935,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":978944,"data_stored":511571,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738460,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":438137,"data_compressed":5763,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":65927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":67789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134170,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":126976,"data_stored":200032,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:41:52.415 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:41:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:52.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:52 smithi037 ceph-mon[31211]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:52.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:52 smithi037 ceph-mon[31211]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:52.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:52 smithi037 ceph-mon[31211]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:52.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:52 smithi151 ceph-mon[35236]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:52.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:52 smithi151 ceph-mon[35236]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:52.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:52 smithi151 ceph-mon[35236]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:53.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:52 smithi096 ceph-mon[35004]: from='client.15258 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:53.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:52 smithi096 ceph-mon[35004]: from='client.15264 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:53.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:52 smithi096 ceph-mon[35004]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:41:53.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 -- 172.21.15.37:0/3616517277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 msgr2=0x7f13e410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:53.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 --2- 172.21.15.37:0/3616517277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e410d950 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f13dc009cd0 tx=0x7f13dc02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 -- 172.21.15.37:0/3616517277 shutdown_connections 2024-09-19T06:41:53.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 --2- 172.21.15.37:0/3616517277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e410d950 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 --2- 172.21.15.37:0/3616517277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e4100a80 0x7f13e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 --2- 172.21.15.37:0/3616517277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e4106a80 0x7f13e4106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 -- 172.21.15.37:0/3616517277 >> 172.21.15.37:0/3616517277 conn(0x7f13e40fc7a0 msgr2=0x7f13e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:53.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 -- 172.21.15.37:0/3616517277 shutdown_connections 2024-09-19T06:41:53.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.161+0000 7f13e8d46640 1 -- 172.21.15.37:0/3616517277 wait complete. 2024-09-19T06:41:53.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 Processor -- start 2024-09-19T06:41:53.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 -- start start 2024-09-19T06:41:53.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e4100a80 0x7f13e41972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:53.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e4197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e4106a80 0x7f13e419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4110e80 con 0x7f13e4101420 2024-09-19T06:41:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4110d00 con 0x7f13e4106a80 2024-09-19T06:41:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.162+0000 7f13e8d46640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4111000 con 0x7f13e4100a80 2024-09-19T06:41:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e4197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e4197820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54846/0 (socket says 172.21.15.37:54846) 2024-09-19T06:41:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 -- 172.21.15.37:0/2720102982 learned_addr learned my addr 172.21.15.37:0/2720102982 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13e2d9d640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e4106a80 0x7f13e419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13e259c640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e4100a80 0x7f13e41972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 -- 172.21.15.37:0/2720102982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e4100a80 msgr2=0x7f13e41972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e4100a80 0x7f13e41972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 -- 172.21.15.37:0/2720102982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e4106a80 msgr2=0x7f13e419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e4106a80 0x7f13e419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 -- 172.21.15.37:0/2720102982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13e419c290 con 0x7f13e4101420 2024-09-19T06:41:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13e2d9d640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e4106a80 0x7f13e419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13e259c640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e4100a80 0x7f13e41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13dbfff640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e4197820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009970 tx=0x7f13cc009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13db7fe640 1 -- 172.21.15.37:0/2720102982 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00d020 con 0x7f13e4101420 2024-09-19T06:41:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13e419c580 con 0x7f13e4101420 2024-09-19T06:41:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13db7fe640 1 -- 172.21.15.37:0/2720102982 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f13cc007d40 con 0x7f13e4101420 2024-09-19T06:41:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13db7fe640 1 -- 172.21.15.37:0/2720102982 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00ccc0 con 0x7f13e4101420 2024-09-19T06:41:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.163+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13e406a790 con 0x7f13e4101420 2024-09-19T06:41:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.164+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13e4100ee0 con 0x7f13e4101420 2024-09-19T06:41:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.166+0000 7f13db7fe640 1 -- 172.21.15.37:0/2720102982 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f13cc00c460 con 0x7f13e4101420 2024-09-19T06:41:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.167+0000 7f13db7fe640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13d00a34d0 0x7f13d00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:53.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.167+0000 7f13db7fe640 1 -- 172.21.15.37:0/2720102982 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f13cc0d19d0 con 0x7f13e4101420 2024-09-19T06:41:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.167+0000 7f13e259c640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13d00a34d0 0x7f13d00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.167+0000 7f13e259c640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13d00a34d0 0x7f13d00a5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13d4005fd0 tx=0x7f13d40094d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:53.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.170+0000 7f13db7fe640 1 -- 172.21.15.37:0/2720102982 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f13cc09e100 con 0x7f13e4101420 2024-09-19T06:41:53.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.382+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f13e406aa20 con 0x7f13e4101420 2024-09-19T06:41:53.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.382+0000 7f13db7fe640 1 -- 172.21.15.37:0/2720102982 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7f13cc09da00 con 0x7f13e4101420 2024-09-19T06:41:53.384 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:53.384 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:41:53.385 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13d00a34d0 msgr2=0x7f13d00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13d00a34d0 0x7f13d00a5990 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f13d4005fd0 tx=0x7f13d40094d0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 msgr2=0x7f13e4197820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e4197820 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009970 tx=0x7f13cc009e30 comp rx=0 tx=0).stop 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 shutdown_connections 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13e4106a80 0x7f13e419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13d00a34d0 0x7f13d00a5990 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13e4101420 0x7f13e4197820 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 --2- 172.21.15.37:0/2720102982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13e4100a80 0x7f13e41972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:53.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 >> 172.21.15.37:0/2720102982 conn(0x7f13e40fc7a0 msgr2=0x7f13e40fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:53.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 shutdown_connections 2024-09-19T06:41:53.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:53.385+0000 7f13e8d46640 1 -- 172.21.15.37:0/2720102982 wait complete. 2024-09-19T06:41:53.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:53 smithi037 ceph-mon[31211]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-19T06:41:53.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:53 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2720102982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:54.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:53 smithi096 ceph-mon[35004]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-19T06:41:54.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:53 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2720102982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:54.069 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:41:54.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:53 smithi151 ceph-mon[35236]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-19T06:41:54.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:53 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2720102982' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:41:54.328 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:54.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 -- 172.21.15.37:0/3549389665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 msgr2=0x7f47c8100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:54.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/3549389665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c8100860 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009900 tx=0x7f47c402a410 comp rx=0 tx=0).stop 2024-09-19T06:41:54.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 -- 172.21.15.37:0/3549389665 shutdown_connections 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/3549389665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47c81055b0 0x7f47c810d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/3549389665 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c8100e30 0x7f47c8104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/3549389665 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c8100860 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 -- 172.21.15.37:0/3549389665 >> 172.21.15.37:0/3549389665 conn(0x7f47c8077210 msgr2=0x7f47c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 -- 172.21.15.37:0/3549389665 shutdown_connections 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.795+0000 7f47cfcb5640 1 -- 172.21.15.37:0/3549389665 wait complete. 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 Processor -- start 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 -- start start 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c8100e30 0x7f47c8199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47c81055b0 0x7f47c8199f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:54.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8110630 con 0x7f47c8100480 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81104b0 con 0x7f47c81055b0 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cfcb5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81107b0 con 0x7f47c8100e30 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cda51640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cda51640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c81994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54864/0 (socket says 172.21.15.37:54864) 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cda51640 1 -- 172.21.15.37:0/2852633312 learned_addr learned my addr 172.21.15.37:0/2852633312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47cd250640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c8100e30 0x7f47c8199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.796+0000 7f47ce252640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47c81055b0 0x7f47c8199f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cda51640 1 -- 172.21.15.37:0/2852633312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c8100e30 msgr2=0x7f47c8199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cda51640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c8100e30 0x7f47c8199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cda51640 1 -- 172.21.15.37:0/2852633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47c81055b0 msgr2=0x7f47c8199f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cda51640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47c81055b0 0x7f47c8199f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cda51640 1 -- 172.21.15.37:0/2852633312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c81a48d0 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47ce252640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47c81055b0 0x7f47c8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cda51640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c81994d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005ab0 tx=0x7f47c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47beffd640 1 -- 172.21.15.37:0/2852633312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c403f030 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47beffd640 1 -- 172.21.15.37:0/2852633312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47c4003ce0 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c81a4b60 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47beffd640 1 -- 172.21.15.37:0/2852633312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c4035070 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.797+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81a5040 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.798+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c8100860 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.800+0000 7f47beffd640 1 -- 172.21.15.37:0/2852633312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f47c4042030 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.801+0000 7f47beffd640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f47b40a34d0 0x7f47b40a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.801+0000 7f47cd250640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f47b40a34d0 0x7f47b40a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.801+0000 7f47beffd640 1 -- 172.21.15.37:0/2852633312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f47c40f0ab0 con 0x7f47c8100480 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.801+0000 7f47cd250640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f47b40a34d0 0x7f47b40a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47c8104f80 tx=0x7f47b8009110 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:54.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.804+0000 7f47beffd640 1 -- 172.21.15.37:0/2852633312 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f47c40bd2b0 con 0x7f47c8100480 2024-09-19T06:41:54.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.980+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f47c81a52a0 con 0x7f47c8100480 2024-09-19T06:41:54.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.982+0000 7f47beffd640 1 -- 172.21.15.37:0/2852633312 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23170 (secure 0 0 0) 0x7f47c40bcbb0 con 0x7f47c8100480 2024-09-19T06:41:54.984 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:54.984 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:41:46.581293+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3406854118":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/1436144832":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6801/871019432":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1644258429":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/711456420":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6800/871019432":"2024-09-20T06:41:46.581265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:41:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f47b40a34d0 msgr2=0x7f47b40a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f47b40a34d0 0x7f47b40a5990 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47c8104f80 tx=0x7f47b8009110 comp rx=0 tx=0).stop 2024-09-19T06:41:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 msgr2=0x7f47c81994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c81994d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f47c4005ab0 tx=0x7f47c40042f0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 shutdown_connections 2024-09-19T06:41:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f47c81055b0 0x7f47c8199f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47c8100e30 0x7f47c8199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f47b40a34d0 0x7f47b40a5990 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 --2- 172.21.15.37:0/2852633312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f47c8100480 0x7f47c81994d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:54.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 >> 172.21.15.37:0/2852633312 conn(0x7f47c8077210 msgr2=0x7f47c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:54.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 shutdown_connections 2024-09-19T06:41:54.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:54.984+0000 7f47cfcb5640 1 -- 172.21.15.37:0/2852633312 wait complete. 2024-09-19T06:41:55.586 DEBUG:tasks.fwd_scrub:Yielding 2024-09-19T06:41:55.586 INFO:teuthology.run_tasks:Running task exec... 2024-09-19T06:41:55.597 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:41:55.597 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:41:55.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-09-19T06:41:55.598 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-19T06:41:55.598 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:41:55.598 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:41:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.707+0000 7f4454cf1640 1 -- 172.21.15.37:0/875908306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4450102ce0 msgr2=0x7f44501030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.707+0000 7f4454cf1640 1 --2- 172.21.15.37:0/875908306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4450102ce0 0x7f44501030c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f4440009900 tx=0x7f444002a480 comp rx=0 tx=0).stop 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 -- 172.21.15.37:0/875908306 shutdown_connections 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 --2- 172.21.15.37:0/875908306 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f445010c0b0 0x7f445010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 --2- 172.21.15.37:0/875908306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4450103600 0x7f445010bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 --2- 172.21.15.37:0/875908306 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4450102ce0 0x7f44501030c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 -- 172.21.15.37:0/875908306 >> 172.21.15.37:0/875908306 conn(0x7f4450077990 msgr2=0x7f44500fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 -- 172.21.15.37:0/875908306 shutdown_connections 2024-09-19T06:41:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 -- 172.21.15.37:0/875908306 wait complete. 2024-09-19T06:41:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 Processor -- start 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.708+0000 7f4454cf1640 1 -- start start 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f4454cf1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4450102ce0 0x7f445019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f4454cf1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4450103600 0x7f445019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f4454cf1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f445010c0b0 0x7f445019ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f4454cf1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111ef0 con 0x7f445010c0b0 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f4454cf1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4450111d70 con 0x7f4450102ce0 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f4454cf1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4450112070 con 0x7f4450103600 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f445010c0b0 0x7f445019ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f445010c0b0 0x7f445019ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54884/0 (socket says 172.21.15.37:54884) 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 -- 172.21.15.37:0/1643981118 learned_addr learned my addr 172.21.15.37:0/1643981118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:55.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444e59c640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4450102ce0 0x7f445019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444dd9b640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4450103600 0x7f445019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 -- 172.21.15.37:0/1643981118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4450103600 msgr2=0x7f445019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4450103600 0x7f445019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 -- 172.21.15.37:0/1643981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4450102ce0 msgr2=0x7f445019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4450102ce0 0x7f445019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 -- 172.21.15.37:0/1643981118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445019f570 con 0x7f445010c0b0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444e59c640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4450102ce0 0x7f445019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f444ed9d640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f445010c0b0 0x7f445019ee40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f444800b970 tx=0x7f444800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f443f7fe640 1 -- 172.21.15.37:0/1643981118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448019070 con 0x7f445010c0b0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f443f7fe640 1 -- 172.21.15.37:0/1643981118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f444800cb40 con 0x7f445010c0b0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f443f7fe640 1 -- 172.21.15.37:0/1643981118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448007650 con 0x7f445010c0b0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.709+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445019f860 con 0x7f445010c0b0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.710+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44501a70f0 con 0x7f445010c0b0 2024-09-19T06:41:55.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.711+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44501030c0 con 0x7f445010c0b0 2024-09-19T06:41:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.712+0000 7f443f7fe640 1 -- 172.21.15.37:0/1643981118 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f444801c070 con 0x7f445010c0b0 2024-09-19T06:41:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.713+0000 7f443f7fe640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44380a3600 0x7f44380a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.713+0000 7f444e59c640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44380a3600 0x7f44380a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:55.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.713+0000 7f443f7fe640 1 -- 172.21.15.37:0/1643981118 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f44480cf4e0 con 0x7f445010c0b0 2024-09-19T06:41:55.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.713+0000 7f444e59c640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44380a3600 0x7f44380a5ac0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440032c30 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.717+0000 7f443f7fe640 1 -- 172.21.15.37:0/1643981118 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f444809bd20 con 0x7f445010c0b0 2024-09-19T06:41:55.870 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:55.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.909+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f44501a73e0 con 0x7f445010c0b0 2024-09-19T06:41:55.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.913+0000 7f443f7fe640 1 -- 172.21.15.37:0/1643981118 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f444809b620 con 0x7f445010c0b0 2024-09-19T06:41:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.915+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44380a3600 msgr2=0x7f44380a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.915+0000 7f4454cf1640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44380a3600 0x7f44380a5ac0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440032c30 comp rx=0 tx=0).stop 2024-09-19T06:41:55.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f445010c0b0 msgr2=0x7f445019ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f445010c0b0 0x7f445019ee40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f444800b970 tx=0x7f444800be30 comp rx=0 tx=0).stop 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 shutdown_connections 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44380a3600 0x7f44380a5ac0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f445010c0b0 0x7f445019ee40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4450103600 0x7f445019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 --2- 172.21.15.37:0/1643981118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4450102ce0 0x7f445019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 >> 172.21.15.37:0/1643981118 conn(0x7f4450077990 msgr2=0x7f4450100860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 shutdown_connections 2024-09-19T06:41:55.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:55.916+0000 7f4454cf1640 1 -- 172.21.15.37:0/1643981118 wait complete. 2024-09-19T06:41:55.928 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:55 smithi037 ceph-mon[31211]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1 KiB/s wr, 23 op/s 2024-09-19T06:41:55.928 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:55 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2852633312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:55.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-09-19T06:41:56.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:55 smithi096 ceph-mon[35004]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1 KiB/s wr, 23 op/s 2024-09-19T06:41:56.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:55 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2852633312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/2729454444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a81069c0 msgr2=0x7fb0a8106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/2729454444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a81069c0 0x7fb0a8106da0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb098009930 tx=0x7fb09802a420 comp rx=0 tx=0).stop 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/2729454444 shutdown_connections 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/2729454444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0a8101350 0x7fb0a810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/2729454444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a81009b0 0x7fb0a8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/2729454444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a81069c0 0x7fb0a8106da0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/2729454444 >> 172.21.15.37:0/2729454444 conn(0x7fb0a8077990 msgr2=0x7fb0a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/2729454444 shutdown_connections 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/2729454444 wait complete. 2024-09-19T06:41:56.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 Processor -- start 2024-09-19T06:41:56.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.036+0000 7fb0ae2f1640 1 -- start start 2024-09-19T06:41:56.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0ae2f1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a81009b0 0x7fb0a81991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0ae2f1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a8101350 0x7fb0a81996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0ae2f1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0a81069c0 0x7fb0a81a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0ae2f1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8110e50 con 0x7fb0a8101350 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0ae2f1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8110cd0 con 0x7fb0a81069c0 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0ae2f1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8110fd0 con 0x7fb0a81009b0 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a8101350 0x7fb0a81996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a8101350 0x7fb0a81996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54906/0 (socket says 172.21.15.37:54906) 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 -- 172.21.15.37:0/89449887 learned_addr learned my addr 172.21.15.37:0/89449887 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0ac88e640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0a81069c0 0x7fb0a81a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a7fff640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a81009b0 0x7fb0a81991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 -- 172.21.15.37:0/89449887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a81009b0 msgr2=0x7fb0a81991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a81009b0 0x7fb0a81991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 -- 172.21.15.37:0/89449887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0a81069c0 msgr2=0x7fb0a81a4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0a81069c0 0x7fb0a81a4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 -- 172.21.15.37:0/89449887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a81a4760 con 0x7fb0a8101350 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.037+0000 7fb0a77fe640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a8101350 0x7fb0a81996e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d950 tx=0x7fb09c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.038+0000 7fb0a57fa640 1 -- 172.21.15.37:0/89449887 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c017070 con 0x7fb0a8101350 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.038+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a81a4930 con 0x7fb0a8101350 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.038+0000 7fb0a57fa640 1 -- 172.21.15.37:0/89449887 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb09c009c30 con 0x7fb0a8101350 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.038+0000 7fb0a57fa640 1 -- 172.21.15.37:0/89449887 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00f6b0 con 0x7fb0a8101350 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.038+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81a4e50 con 0x7fb0a8101350 2024-09-19T06:41:56.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.039+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a8100e10 con 0x7fb0a8101350 2024-09-19T06:41:56.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.040+0000 7fb0a57fa640 1 -- 172.21.15.37:0/89449887 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fb09c0109d0 con 0x7fb0a8101350 2024-09-19T06:41:56.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.040+0000 7fb0a57fa640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0900a3580 0x7fb0900a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.041+0000 7fb0a7fff640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0900a3580 0x7fb0900a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.041+0000 7fb0a57fa640 1 -- 172.21.15.37:0/89449887 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb09c0cfbd0 con 0x7fb0a8101350 2024-09-19T06:41:56.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.041+0000 7fb0a7fff640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0900a3580 0x7fb0900a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb098006010 tx=0x7fb098000bc0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.045+0000 7fb0a57fa640 1 -- 172.21.15.37:0/89449887 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb09c009de0 con 0x7fb0a8101350 2024-09-19T06:41:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:55 smithi151 ceph-mon[35236]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 1 KiB/s wr, 23 op/s 2024-09-19T06:41:56.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:55 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2852633312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:41:56.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.226+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb0a81a50e0 con 0x7fb0a8101350 2024-09-19T06:41:56.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.232+0000 7fb0a57fa640 1 -- 172.21.15.37:0/89449887 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v30)=0 v30) ==== 138+0+0 (secure 0 0 0) 0x7fb09c09bcb0 con 0x7fb0a8101350 2024-09-19T06:41:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0900a3580 msgr2=0x7fb0900a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0900a3580 0x7fb0900a5a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb098006010 tx=0x7fb098000bc0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a8101350 msgr2=0x7fb0a81996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a8101350 0x7fb0a81996e0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb09c00d950 tx=0x7fb09c00de10 comp rx=0 tx=0).stop 2024-09-19T06:41:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 shutdown_connections 2024-09-19T06:41:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0a81069c0 0x7fb0a81a4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0900a3580 0x7fb0900a5a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0a8101350 0x7fb0a81996e0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 --2- 172.21.15.37:0/89449887 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0a81009b0 0x7fb0a81991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 >> 172.21.15.37:0/89449887 conn(0x7fb0a8077990 msgr2=0x7fb0a80fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 shutdown_connections 2024-09-19T06:41:56.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.234+0000 7fb0ae2f1640 1 -- 172.21.15.37:0/89449887 wait complete. 2024-09-19T06:41:56.248 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-09-19T06:41:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 -- 172.21.15.37:0/357315551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2154102d00 msgr2=0x7f21541030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 --2- 172.21.15.37:0/357315551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2154102d00 0x7f21541030e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f2148009900 tx=0x7f214802a3f0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 -- 172.21.15.37:0/357315551 shutdown_connections 2024-09-19T06:41:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 --2- 172.21.15.37:0/357315551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f215410c120 0x7f215410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 --2- 172.21.15.37:0/357315551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2154103620 0x7f215410bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 --2- 172.21.15.37:0/357315551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2154102d00 0x7f21541030e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 -- 172.21.15.37:0/357315551 >> 172.21.15.37:0/357315551 conn(0x7f21540fc6b0 msgr2=0x7f21540fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 -- 172.21.15.37:0/357315551 shutdown_connections 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.357+0000 7f215beaf640 1 -- 172.21.15.37:0/357315551 wait complete. 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 Processor -- start 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 -- start start 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2154102d00 0x7f2154100950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2154103620 0x7f2154100e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f215410c120 0x7f21541013d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2154111eb0 con 0x7f215410c120 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2154111d30 con 0x7f2154102d00 2024-09-19T06:41:56.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215beaf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2154112030 con 0x7f2154103620 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215a44c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f215410c120 0x7f21541013d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215a44c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f215410c120 0x7f21541013d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54922/0 (socket says 172.21.15.37:54922) 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f215a44c640 1 -- 172.21.15.37:0/420417004 learned_addr learned my addr 172.21.15.37:0/420417004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.358+0000 7f2159c4b640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2154102d00 0x7f2154100950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215944a640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2154103620 0x7f2154100e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215a44c640 1 -- 172.21.15.37:0/420417004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2154103620 msgr2=0x7f2154100e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215a44c640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2154103620 0x7f2154100e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215a44c640 1 -- 172.21.15.37:0/420417004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2154102d00 msgr2=0x7f2154100950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215a44c640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2154102d00 0x7f2154100950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215a44c640 1 -- 172.21.15.37:0/420417004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21541a67a0 con 0x7f215410c120 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215a44c640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f215410c120 0x7f21541013d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f214c00cce0 tx=0x7f214c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f2146ffd640 1 -- 172.21.15.37:0/420417004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c00f450 con 0x7f215410c120 2024-09-19T06:41:56.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21541a6a90 con 0x7f215410c120 2024-09-19T06:41:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f2146ffd640 1 -- 172.21.15.37:0/420417004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f214c00fad0 con 0x7f215410c120 2024-09-19T06:41:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f2146ffd640 1 -- 172.21.15.37:0/420417004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f214c018c10 con 0x7f215410c120 2024-09-19T06:41:56.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.359+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21541a6f90 con 0x7f215410c120 2024-09-19T06:41:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.360+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21541030e0 con 0x7f215410c120 2024-09-19T06:41:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.361+0000 7f2146ffd640 1 -- 172.21.15.37:0/420417004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f214c00f5f0 con 0x7f215410c120 2024-09-19T06:41:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.362+0000 7f2146ffd640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21400a3580 0x7f21400a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.362+0000 7f2146ffd640 1 -- 172.21.15.37:0/420417004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f214c0d0600 con 0x7f215410c120 2024-09-19T06:41:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.362+0000 7f2159c4b640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21400a3580 0x7f21400a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.363+0000 7f2159c4b640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21400a3580 0x7f21400a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2148009230 tx=0x7f21480058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.366+0000 7f2146ffd640 1 -- 172.21.15.37:0/420417004 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f214c014030 con 0x7f215410c120 2024-09-19T06:41:56.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 -- 172.21.15.37:0/3109572763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014106f10 msgr2=0x7f001410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 --2- 172.21.15.37:0/3109572763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014106f10 0x7f001410ea90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f000c009ca0 tx=0x7f000c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 -- 172.21.15.37:0/3109572763 shutdown_connections 2024-09-19T06:41:56.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 --2- 172.21.15.37:0/3109572763 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014106f10 0x7f001410ea90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 --2- 172.21.15.37:0/3109572763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014102790 0x7f00141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 --2- 172.21.15.37:0/3109572763 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0014101de0 0x7f00141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 -- 172.21.15.37:0/3109572763 >> 172.21.15.37:0/3109572763 conn(0x7f00140fd8c0 msgr2=0x7f00140ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 -- 172.21.15.37:0/3109572763 shutdown_connections 2024-09-19T06:41:56.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.383+0000 7f0019a80640 1 -- 172.21.15.37:0/3109572763 wait complete. 2024-09-19T06:41:56.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 Processor -- start 2024-09-19T06:41:56.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 -- start start 2024-09-19T06:41:56.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014101de0 0x7f00140744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014102790 0x7f0014074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0014106f10 0x7f0014074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0014111f70 con 0x7f0014101de0 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0014111df0 con 0x7f0014102790 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0019a80640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00141120f0 con 0x7f0014106f10 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f00137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014101de0 0x7f00140744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f00137fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014101de0 0x7f00140744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54936/0 (socket says 172.21.15.37:54936) 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f00137fe640 1 -- 172.21.15.37:0/2451212721 learned_addr learned my addr 172.21.15.37:0/2451212721 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f000affd640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014102790 0x7f0014074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f0013fff640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0014106f10 0x7f0014074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f00137fe640 1 -- 172.21.15.37:0/2451212721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0014106f10 msgr2=0x7f0014074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.384+0000 7f00137fe640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0014106f10 0x7f0014074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f00137fe640 1 -- 172.21.15.37:0/2451212721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014102790 msgr2=0x7f0014074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f00137fe640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014102790 0x7f0014074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f00137fe640 1 -- 172.21.15.37:0/2451212721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00141a6800 con 0x7f0014101de0 2024-09-19T06:41:56.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f00137fe640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014101de0 0x7f00140744f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f000400d770 tx=0x7f000400db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f00117fa640 1 -- 172.21.15.37:0/2451212721 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004017070 con 0x7f0014101de0 2024-09-19T06:41:56.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f00117fa640 1 -- 172.21.15.37:0/2451212721 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0004009820 con 0x7f0014101de0 2024-09-19T06:41:56.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00141a6af0 con 0x7f0014101de0 2024-09-19T06:41:56.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f00117fa640 1 -- 172.21.15.37:0/2451212721 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0004004cd0 con 0x7f0014101de0 2024-09-19T06:41:56.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.385+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00141a6fe0 con 0x7f0014101de0 2024-09-19T06:41:56.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.387+0000 7f00117fa640 1 -- 172.21.15.37:0/2451212721 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f00040099d0 con 0x7f0014101de0 2024-09-19T06:41:56.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 Processor -- start 2024-09-19T06:41:56.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 -- start start 2024-09-19T06:41:56.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b86fa750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b8705010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8707940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e0b8657280 con 0x55e0b8705550 2024-09-19T06:41:56.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0b85feb10 con 0x55e0b86fa370 2024-09-19T06:41:56.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f001be59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e0b8653240 con 0x55e0b86fac90 2024-09-19T06:41:56.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f000affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8707940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f000affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8707940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54948/0 (socket says 172.21.15.37:54948) 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f0013fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b86fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f00137fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b8705010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.388+0000 7f000affd640 1 -- 172.21.15.37:0/1370755210 learned_addr learned my addr 172.21.15.37:0/1370755210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f00117fa640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f00000a34d0 0x7f00000a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 -- 172.21.15.37:0/1370755210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 msgr2=0x55e0b8705010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b8705010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 -- 172.21.15.37:0/1370755210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 msgr2=0x55e0b8707940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8707940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 -- 172.21.15.37:0/1370755210 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b860b1f0 con 0x55e0b86fa370 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f00117fa640 1 -- 172.21.15.37:0/2451212721 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f00040cfa70 con 0x7f0014101de0 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f000affd640 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8707940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f00000a34d0 0x7f00000a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b86fa750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f000c006010 tx=0x7f000c031bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89a31d3b411dfcc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0013fff640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f00000a34d0 0x7f00000a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f000c048000 tx=0x7f000c04d000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.389+0000 7f0009ffb640 1 -- 172.21.15.37:0/1370755210 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c041040 con 0x55e0b86fa370 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f0009ffb640 1 -- 172.21.15.37:0/1370755210 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c036070 con 0x55e0b86fa370 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f0009ffb640 1 -- 172.21.15.37:0/1370755210 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c02fb80 con 0x55e0b86fa370 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 -- 172.21.15.37:0/1370755210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 msgr2=0x55e0b86fa750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b86fa750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f000c006010 tx=0x7f000c031bb0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 -- 172.21.15.37:0/1370755210 shutdown_connections 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8707940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b8705010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 --2- 172.21.15.37:0/1370755210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b86fa750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 -- 172.21.15.37:0/1370755210 >> 172.21.15.37:0/1370755210 conn(0x55e0b86f6900 msgr2=0x55e0b86f6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 -- 172.21.15.37:0/1370755210 shutdown_connections 2024-09-19T06:41:56.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.390+0000 7f001be59740 1 -- 172.21.15.37:0/1370755210 wait complete. 2024-09-19T06:41:56.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.391+0000 7f001be59740 1 -- start start 2024-09-19T06:41:56.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.391+0000 7f001be59740 1 Processor -- start 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.391+0000 7f001be59740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b87024b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.391+0000 7f001be59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b87029f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.391+0000 7f001be59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8702f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.391+0000 7f001be59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e0b8708120 con 0x55e0b8705550 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.391+0000 7f001be59740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e0b8611f10 con 0x55e0b86fa370 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b87024b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f001be59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e0b87082a0 con 0x55e0b86fac90 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b87024b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56038/0 (socket says 172.21.15.37:56038) 2024-09-19T06:41:56.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 -- 172.21.15.37:0/436872160 learned_addr learned my addr 172.21.15.37:0/436872160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f001be59740 10 client.0 _ll_register_callbacks cb 0x55e0b85daa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f0013fff640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8702f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f000affd640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b87029f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 -- 172.21.15.37:0/436872160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 msgr2=0x55e0b87029f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b87029f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 -- 172.21.15.37:0/436872160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 msgr2=0x55e0b8702f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8702f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 -- 172.21.15.37:0/436872160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b83c86a0 con 0x55e0b86fa370 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f0013fff640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8702f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f00137fe640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b87024b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0004097780 tx=0x7f000400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=befbe4c69ae4c79e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7efff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7efff2ffd640 1 -- 172.21.15.37:0/436872160 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00040cfe60 con 0x55e0b86fa370 2024-09-19T06:41:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.392+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f000c036070 con 0x55e0b86fa370 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7efff2ffd640 1 -- 172.21.15.37:0/436872160 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f00040860b0 con 0x55e0b86fa370 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7efff2ffd640 1 -- 172.21.15.37:0/436872160 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0004020040 con 0x55e0b86fa370 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 10 client.24959 fetch_fsmap learned FSMap version 25 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0b8713df0 con 0x55e0b86fa370 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7efff2ffd640 1 -- 172.21.15.37:0/436872160 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f000c036070 con 0x55e0b86fa370 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7efff2ffd640 1 -- 172.21.15.37:0/436872160 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f00040cd2c0 con 0x55e0b86fa370 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 10 client.24959 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 10 client.24959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 20 client.24959 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8715200 0x55e0b8717630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 4 client.24959 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.393+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e0b86647f0 con 0x55e0b8715200 2024-09-19T06:41:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.394+0000 7f000affd640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8715200 0x55e0b8717630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.394+0000 7efff2ffd640 10 client.24959 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.394+0000 7f000affd640 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8715200 0x55e0b8717630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38cd265a65ee2339 server_cookie=cd92afbb7c57220e in_seq=0 out_seq=0 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7efff2ffd640 1 -- 172.21.15.37:0/436872160 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e0b86647f0 con 0x55e0b8715200 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7efff2ffd640 10 client.24959 handle_command_reply: tid=0 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 client.24959 shutdown 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 msgr2=0x55e0b87024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b87024b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f0004097780 tx=0x7f000400b480 comp rx=0 tx=0).stop 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 shutdown_connections 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8715200 0x55e0b8717630 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b8702f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b87029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 --2- 172.21.15.37:0/436872160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b87024b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 >> 172.21.15.37:0/436872160 conn(0x55e0b86f6900 msgr2=0x55e0b86ed270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7efff2ffd640 0 client.24959 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 shutdown_connections 2024-09-19T06:41:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 1 -- 172.21.15.37:0/436872160 wait complete. 2024-09-19T06:41:56.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.395+0000 7f001be59740 20 client.24959 trim_cache size 0 max 16384 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 Processor -- start 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 -- start start 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b871ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b86fac90 0x55e0b871d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b8705550 0x55e0b86ff730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00141adf50 con 0x55e0b86fac90 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00141ab820 con 0x55e0b86fa370 2024-09-19T06:41:56.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f001be59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00141a9690 con 0x55e0b8705550 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f000affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b86fac90 0x55e0b871d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f000affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b86fac90 0x55e0b871d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54960/0 (socket says 172.21.15.37:54960) 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f000affd640 1 -- 172.21.15.37:0/986199206 learned_addr learned my addr 172.21.15.37:0/986199206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f00137fe640 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b871ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f0013fff640 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b8705550 0x55e0b86ff730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f000affd640 1 -- 172.21.15.37:0/986199206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b8705550 msgr2=0x55e0b86ff730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f000affd640 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b8705550 0x55e0b86ff730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f000affd640 1 -- 172.21.15.37:0/986199206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 msgr2=0x55e0b871ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.413+0000 7f000affd640 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b871ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.414+0000 7f000affd640 1 -- 172.21.15.37:0/986199206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e0b86faac0 con 0x55e0b86fac90 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.414+0000 7f0013fff640 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b8705550 0x55e0b86ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.414+0000 7f000affd640 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b86fac90 0x55e0b871d3f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55e0b86f7c80 tx=0x7efffc010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=784cd93a1963080 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.414+0000 7f00097fa640 1 -- 172.21.15.37:0/986199206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc010e60 con 0x55e0b86fac90 2024-09-19T06:41:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.414+0000 7f00097fa640 1 -- 172.21.15.37:0/986199206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efffc0047a0 con 0x55e0b86fac90 2024-09-19T06:41:56.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.414+0000 7f00097fa640 1 -- 172.21.15.37:0/986199206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efffc01d930 con 0x55e0b86fac90 2024-09-19T06:41:56.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 -- 172.21.15.37:0/986199206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b86fac90 msgr2=0x55e0b871d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b86fac90 0x55e0b871d3f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x55e0b86f7c80 tx=0x7efffc010800 comp rx=0 tx=0).stop 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 -- 172.21.15.37:0/986199206 shutdown_connections 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b8705550 0x55e0b86ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b86fac90 0x55e0b871d3f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 --2- 172.21.15.37:0/986199206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b871ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 -- 172.21.15.37:0/986199206 >> 172.21.15.37:0/986199206 conn(0x55e0b86f6900 msgr2=0x55e0b86ed270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 -- 172.21.15.37:0/986199206 shutdown_connections 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 -- 172.21.15.37:0/986199206 wait complete. 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 -- start start 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.415+0000 7f001be59740 1 Processor -- start 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f001be59740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b870d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f001be59740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b870dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f001be59740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b870a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f001be59740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f00141a9ce0 con 0x55e0b8705550 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f001be59740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f00141ab150 con 0x55e0b86fa370 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f001be59740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f00141aa3b0 con 0x55e0b86fac90 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f001be59740 10 client.0 _ll_register_callbacks cb 0x7f0004005790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b870a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b870a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54966/0 (socket says 172.21.15.37:54966) 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 -- 172.21.15.37:0/3143993621 learned_addr learned my addr 172.21.15.37:0/3143993621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f00137fe640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b870d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f000affd640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b870dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 -- 172.21.15.37:0/3143993621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 msgr2=0x55e0b870dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b870dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 -- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 msgr2=0x55e0b870d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b870d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 -- 172.21.15.37:0/3143993621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e0b870ab50 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f00137fe640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b870d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7f0013fff640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b870a4c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f000c02a900 tx=0x7f000c02faa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d13696b3e8e9131f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.416+0000 7efff2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7efff2ffd640 1 -- 172.21.15.37:0/3143993621 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f000c041040 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7efff2ffd640 1 -- 172.21.15.37:0/3143993621 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f000c02fdf0 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efffc0047a0 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7efff2ffd640 1 -- 172.21.15.37:0/3143993621 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f000c02f310 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 10 client.15318 fetch_fsmap learned FSMap version 25 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e0b8780850 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7efff2ffd640 1 -- 172.21.15.37:0/3143993621 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efffc0047a0 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7efff2ffd640 1 -- 172.21.15.37:0/3143993621 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f000c030750 con 0x55e0b8705550 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 10 client.15318 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:41:56.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 10 client.15318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 20 client.15318 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8780b50 0x55e0b8782f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 4 client.15318 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.417+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e0b865ed00 con 0x55e0b8780b50 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.418+0000 7f00137fe640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8780b50 0x55e0b8782f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.418+0000 7f00137fe640 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8780b50 0x55e0b8782f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318851771318279 server_cookie=7d5cd14b3befeb29 in_seq=0 out_seq=0 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.418+0000 7efff2ffd640 10 client.15318 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7efff2ffd640 1 -- 172.21.15.37:0/3143993621 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e0b865ed00 con 0x55e0b8780b50 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7efff2ffd640 10 client.15318 handle_command_reply: tid=0 2024-09-19T06:41:56.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 client.15318 shutdown 2024-09-19T06:41:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 msgr2=0x55e0b870a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b870a4c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f000c02a900 tx=0x7f000c02faa0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 shutdown_connections 2024-09-19T06:41:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e0b8780b50 0x55e0b8782f40 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e0b8705550 0x55e0b870a4c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e0b86fac90 0x55e0b870dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7efff2ffd640 0 client.15318 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 --2- 172.21.15.37:0/3143993621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e0b86fa370 0x55e0b870d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 >> 172.21.15.37:0/3143993621 conn(0x55e0b86f6900 msgr2=0x55e0b86fbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 shutdown_connections 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.419+0000 7f001be59740 1 -- 172.21.15.37:0/3143993621 wait complete. 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f001be59740 20 client.15318 trim_cache size 0 max 16384 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "df3a94bb-98c3-47be-9da2-9b5ae969eac0", 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:41:56.426 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f00000a34d0 msgr2=0x7f00000a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f00000a34d0 0x7f00000a5990 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f000c048000 tx=0x7f000c04d000 comp rx=0 tx=0).stop 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014101de0 msgr2=0x7f00140744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014101de0 0x7f00140744f0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f000400d770 tx=0x7f000400db70 comp rx=0 tx=0).stop 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 shutdown_connections 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0014106f10 0x7f0014074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0014102790 0x7f0014074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f00000a34d0 0x7f00000a5990 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 --2- 172.21.15.37:0/2451212721 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0014101de0 0x7f00140744f0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 >> 172.21.15.37:0/2451212721 conn(0x7f00140fd8c0 msgr2=0x7f00140febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 shutdown_connections 2024-09-19T06:41:56.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.420+0000 7f0019a80640 1 -- 172.21.15.37:0/2451212721 wait complete. 2024-09-19T06:41:56.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.562+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f21541a72f0 con 0x7f215410c120 2024-09-19T06:41:56.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.567+0000 7f2146ffd640 1 -- 172.21.15.37:0/420417004 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v31)=0 v31) ==== 133+0+0 (secure 0 0 0) 0x7f214c09cd30 con 0x7f215410c120 2024-09-19T06:41:56.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21400a3580 msgr2=0x7f21400a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21400a3580 0x7f21400a5a40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f2148009230 tx=0x7f21480058d0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f215410c120 msgr2=0x7f21541013d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f215410c120 0x7f21541013d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f214c00cce0 tx=0x7f214c00a620 comp rx=0 tx=0).stop 2024-09-19T06:41:56.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 shutdown_connections 2024-09-19T06:41:56.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f21400a3580 0x7f21400a5a40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f215410c120 0x7f21541013d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2154103620 0x7f2154100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 --2- 172.21.15.37:0/420417004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2154102d00 0x7f2154100950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 >> 172.21.15.37:0/420417004 conn(0x7f21540fc6b0 msgr2=0x7f21540fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 shutdown_connections 2024-09-19T06:41:56.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.570+0000 7f215beaf640 1 -- 172.21.15.37:0/420417004 wait complete. 2024-09-19T06:41:56.584 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-19T06:41:56.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 -- 172.21.15.37:0/3193320418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca981020c0 msgr2=0x7fca9810d8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3193320418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca981020c0 0x7fca9810d8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fca90009ca0 tx=0x7fca9002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 -- 172.21.15.37:0/3193320418 shutdown_connections 2024-09-19T06:41:56.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3193320418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca981020c0 0x7fca9810d8f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3193320418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca98101740 0x7fca98101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3193320418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca98106b70 0x7fca98106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 -- 172.21.15.37:0/3193320418 >> 172.21.15.37:0/3193320418 conn(0x7fca98075ef0 msgr2=0x7fca98076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 -- 172.21.15.37:0/3193320418 shutdown_connections 2024-09-19T06:41:56.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 -- 172.21.15.37:0/3193320418 wait complete. 2024-09-19T06:41:56.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.692+0000 7fca9d40e640 1 Processor -- start 2024-09-19T06:41:56.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca9d40e640 1 -- start start 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca9d40e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca98101740 0x7fca98198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca9d40e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca981020c0 0x7fca98198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca9d40e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca98106b70 0x7fca9819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca9d40e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fca98111ff0 con 0x7fca98106b70 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca9d40e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca98111e70 con 0x7fca98101740 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca9d40e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fca98112170 con 0x7fca981020c0 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca98106b70 0x7fca9819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca98106b70 0x7fca9819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54986/0 (socket says 172.21.15.37:54986) 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 -- 172.21.15.37:0/3873325076 learned_addr learned my addr 172.21.15.37:0/3873325076 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca96ffd640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca98101740 0x7fca98198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca967fc640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca981020c0 0x7fca98198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 -- 172.21.15.37:0/3873325076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca981020c0 msgr2=0x7fca98198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca981020c0 0x7fca98198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 -- 172.21.15.37:0/3873325076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca98101740 msgr2=0x7fca98198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca98101740 0x7fca98198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.693+0000 7fca977fe640 1 -- 172.21.15.37:0/3873325076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9819d320 con 0x7fca98106b70 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.694+0000 7fca977fe640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca98106b70 0x7fca9819cbf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca90031a20 tx=0x7fca90031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.694+0000 7fca7ffff640 1 -- 172.21.15.37:0/3873325076 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90003c30 con 0x7fca98106b70 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.694+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca9819d5b0 con 0x7fca98106b70 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.694+0000 7fca7ffff640 1 -- 172.21.15.37:0/3873325076 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fca90003dd0 con 0x7fca98106b70 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.694+0000 7fca7ffff640 1 -- 172.21.15.37:0/3873325076 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9002fc40 con 0x7fca98106b70 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.694+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca981a4ea0 con 0x7fca98106b70 2024-09-19T06:41:56.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.695+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca98101b80 con 0x7fca98106b70 2024-09-19T06:41:56.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.696+0000 7fca7ffff640 1 -- 172.21.15.37:0/3873325076 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fca90004050 con 0x7fca98106b70 2024-09-19T06:41:56.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.697+0000 7fca7ffff640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fca800a3580 0x7fca800a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:56.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.697+0000 7fca96ffd640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fca800a3580 0x7fca800a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:56.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.697+0000 7fca7ffff640 1 -- 172.21.15.37:0/3873325076 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fca900f10b0 con 0x7fca98106b70 2024-09-19T06:41:56.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.697+0000 7fca96ffd640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fca800a3580 0x7fca800a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca88005fd0 tx=0x7fca8800b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:56.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.701+0000 7fca7ffff640 1 -- 172.21.15.37:0/3873325076 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fca900bdaa0 con 0x7fca98106b70 2024-09-19T06:41:56.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.898+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fca980ff790 con 0x7fca800a3580 2024-09-19T06:41:56.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.963+0000 7fca7ffff640 1 -- 172.21.15.37:0/3873325076 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fca980ff790 con 0x7fca800a3580 2024-09-19T06:41:56.965 INFO:teuthology.orchestra.run.smithi037.stdout:Schedule set for path / 2024-09-19T06:41:56.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fca800a3580 msgr2=0x7fca800a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fca800a3580 0x7fca800a5a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fca88005fd0 tx=0x7fca8800b390 comp rx=0 tx=0).stop 2024-09-19T06:41:56.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca98106b70 msgr2=0x7fca9819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:56.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca98106b70 0x7fca9819cbf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca90031a20 tx=0x7fca90031a50 comp rx=0 tx=0).stop 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 shutdown_connections 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fca800a3580 0x7fca800a5a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fca98106b70 0x7fca9819cbf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca981020c0 0x7fca98198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 --2- 172.21.15.37:0/3873325076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca98101740 0x7fca98198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.965+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 >> 172.21.15.37:0/3873325076 conn(0x7fca98075ef0 msgr2=0x7fca980ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.966+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 shutdown_connections 2024-09-19T06:41:56.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:56.966+0000 7fca9d40e640 1 -- 172.21.15.37:0/3873325076 wait complete. 2024-09-19T06:41:56.978 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-19T06:41:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 -- 172.21.15.37:0/2282150901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581009b0 msgr2=0x7ff458100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 --2- 172.21.15.37:0/2282150901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581009b0 0x7ff458100e10 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7ff440009930 tx=0x7ff44002a420 comp rx=0 tx=0).stop 2024-09-19T06:41:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 -- 172.21.15.37:0/2282150901 shutdown_connections 2024-09-19T06:41:57.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 --2- 172.21.15.37:0/2282150901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff458101350 0x7ff45810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 --2- 172.21.15.37:0/2282150901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581009b0 0x7ff458100e10 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 --2- 172.21.15.37:0/2282150901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff4581069c0 0x7ff458106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 -- 172.21.15.37:0/2282150901 >> 172.21.15.37:0/2282150901 conn(0x7ff458077990 msgr2=0x7ff4580fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 -- 172.21.15.37:0/2282150901 shutdown_connections 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.085+0000 7ff45e503640 1 -- 172.21.15.37:0/2282150901 wait complete. 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 Processor -- start 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 -- start start 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4581009b0 0x7ff458197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff458101350 0x7ff4581977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581069c0 0x7ff45819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff458110e00 con 0x7ff4581069c0 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff458110c80 con 0x7ff458101350 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.086+0000 7ff45e503640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff458110f80 con 0x7ff4581009b0 2024-09-19T06:41:57.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581069c0 0x7ff45819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581069c0 0x7ff45819bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55002/0 (socket says 172.21.15.37:55002) 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 -- 172.21.15.37:0/1756487921 learned_addr learned my addr 172.21.15.37:0/1756487921 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff4577fe640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff458101350 0x7ff4581977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff457fff640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4581009b0 0x7ff458197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 -- 172.21.15.37:0/1756487921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4581009b0 msgr2=0x7ff458197280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4581009b0 0x7ff458197280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 -- 172.21.15.37:0/1756487921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff458101350 msgr2=0x7ff4581977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff458101350 0x7ff4581977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 -- 172.21.15.37:0/1756487921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff45819c270 con 0x7ff4581069c0 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45caa0640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581069c0 0x7ff45819bb40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff45000eb10 tx=0x7ff45000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff4557fa640 1 -- 172.21.15.37:0/1756487921 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450004b20 con 0x7ff4581069c0 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff4557fa640 1 -- 172.21.15.37:0/1756487921 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff450004cc0 con 0x7ff4581069c0 2024-09-19T06:41:57.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff45819c560 con 0x7ff4581069c0 2024-09-19T06:41:57.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff4557fa640 1 -- 172.21.15.37:0/1756487921 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff450010760 con 0x7ff4581069c0 2024-09-19T06:41:57.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.087+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4581a3df0 con 0x7ff4581069c0 2024-09-19T06:41:57.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.088+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff458100e10 con 0x7ff4581069c0 2024-09-19T06:41:57.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.090+0000 7ff4557fa640 1 -- 172.21.15.37:0/1756487921 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7ff450005020 con 0x7ff4581069c0 2024-09-19T06:41:57.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.090+0000 7ff4557fa640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff4440a3600 0x7ff4440a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.091+0000 7ff4557fa640 1 -- 172.21.15.37:0/1756487921 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7ff4500d0700 con 0x7ff4581069c0 2024-09-19T06:41:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.091+0000 7ff457fff640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff4440a3600 0x7ff4440a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.091+0000 7ff457fff640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff4440a3600 0x7ff4440a5ac0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff448005e00 tx=0x7ff448005c50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.095+0000 7ff4557fa640 1 -- 172.21.15.37:0/1756487921 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff45009ce30 con 0x7ff4581069c0 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1643981118' entity='client.admin' 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/89449887' entity='client.admin' 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.204 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/420417004' entity='client.admin' 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.205 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:56 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.230 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:41:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1643981118' entity='client.admin' 2024-09-19T06:41:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/89449887' entity='client.admin' 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/420417004' entity='client.admin' 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:56 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.317+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7ff458062630 con 0x7ff4440a3600 2024-09-19T06:41:57.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1643981118' entity='client.admin' 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/89449887' entity='client.admin' 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/420417004' entity='client.admin' 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:41:57.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:56 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:41:57.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.385+0000 7ff4557fa640 1 -- 172.21.15.37:0/1756487921 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7ff458062630 con 0x7ff4440a3600 2024-09-19T06:41:57.387 INFO:teuthology.orchestra.run.smithi037.stdout:Retention added to path / 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff4440a3600 msgr2=0x7ff4440a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff4440a3600 0x7ff4440a5ac0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff448005e00 tx=0x7ff448005c50 comp rx=0 tx=0).stop 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581069c0 msgr2=0x7ff45819bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581069c0 0x7ff45819bb40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff45000eb10 tx=0x7ff45000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 shutdown_connections 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff4440a3600 0x7ff4440a5ac0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff4581069c0 0x7ff45819bb40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff458101350 0x7ff4581977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 --2- 172.21.15.37:0/1756487921 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff4581009b0 0x7ff458197280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 >> 172.21.15.37:0/1756487921 conn(0x7ff458077990 msgr2=0x7ff4581038d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:57.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 shutdown_connections 2024-09-19T06:41:57.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.387+0000 7ff45e503640 1 -- 172.21.15.37:0/1756487921 wait complete. 2024-09-19T06:41:57.400 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T06:41:57.513 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 -- 172.21.15.37:0/3421181612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb410c0b0 msgr2=0x7efcb410e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/3421181612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb410c0b0 0x7efcb410e550 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7efcac009ca0 tx=0x7efcac02a950 comp rx=0 tx=0).stop 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 -- 172.21.15.37:0/3421181612 shutdown_connections 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/3421181612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb410c0b0 0x7efcb410e550 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/3421181612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb4103600 0x7efcb410bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/3421181612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcb4102ce0 0x7efcb41030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 -- 172.21.15.37:0/3421181612 >> 172.21.15.37:0/3421181612 conn(0x7efcb4077990 msgr2=0x7efcb40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:57.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 -- 172.21.15.37:0/3421181612 shutdown_connections 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.522+0000 7efcbbd8d640 1 -- 172.21.15.37:0/3421181612 wait complete. 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 Processor -- start 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 -- start start 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcb4102ce0 0x7efcb4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb4103600 0x7efcb4198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb410c0b0 0x7efcb419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111ef0 con 0x7efcb4103600 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4111d70 con 0x7efcb410c0b0 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcbbd8d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efcb4112070 con 0x7efcb4102ce0 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcb9328640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb4103600 0x7efcb4198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcb9328640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb4103600 0x7efcb4198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55020/0 (socket says 172.21.15.37:55020) 2024-09-19T06:41:57.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcb9328640 1 -- 172.21.15.37:0/2403127072 learned_addr learned my addr 172.21.15.37:0/2403127072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcb9b29640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcb4102ce0 0x7efcb4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.523+0000 7efcba32a640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb410c0b0 0x7efcb419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcb9328640 1 -- 172.21.15.37:0/2403127072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcb4102ce0 msgr2=0x7efcb4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcb9328640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcb4102ce0 0x7efcb4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcb9328640 1 -- 172.21.15.37:0/2403127072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb410c0b0 msgr2=0x7efcb419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcb9328640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb410c0b0 0x7efcb419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcb9328640 1 -- 172.21.15.37:0/2403127072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcb419d2f0 con 0x7efcb4103600 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcb9328640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb4103600 0x7efcb4198840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efca400c890 tx=0x7efca400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcaaffd640 1 -- 172.21.15.37:0/2403127072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4004220 con 0x7efcb4103600 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcaaffd640 1 -- 172.21.15.37:0/2403127072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efca40043c0 con 0x7efcb4103600 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcb419d5e0 con 0x7efcb4103600 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcaaffd640 1 -- 172.21.15.37:0/2403127072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efca4004a00 con 0x7efcb4103600 2024-09-19T06:41:57.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.524+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcb41a4e70 con 0x7efcb4103600 2024-09-19T06:41:57.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.526+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcb41030c0 con 0x7efcb4103600 2024-09-19T06:41:57.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.527+0000 7efcaaffd640 1 -- 172.21.15.37:0/2403127072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7efca40079b0 con 0x7efcb4103600 2024-09-19T06:41:57.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.528+0000 7efcaaffd640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efca00a3600 0x7efca00a5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.529+0000 7efcaaffd640 1 -- 172.21.15.37:0/2403127072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7efca40cfb70 con 0x7efcb4103600 2024-09-19T06:41:57.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.529+0000 7efcb9b29640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efca00a3600 0x7efca00a5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.529+0000 7efcb9b29640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efca00a3600 0x7efca00a5ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009990 tx=0x7efcb0009110 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.533+0000 7efcaaffd640 1 -- 172.21.15.37:0/2403127072 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efca4014030 con 0x7efcb4103600 2024-09-19T06:41:57.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.744+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7efcb40fd680 con 0x7efca00a3600 2024-09-19T06:41:57.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.751+0000 7efcaaffd640 1 -- 172.21.15.37:0/2403127072 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7efcb40fd680 con 0x7efca00a3600 2024-09-19T06:41:57.753 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T06:41:56", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.753+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efca00a3600 msgr2=0x7efca00a5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.753+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efca00a3600 0x7efca00a5ac0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efcb0009990 tx=0x7efcb0009110 comp rx=0 tx=0).stop 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb4103600 msgr2=0x7efcb4198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb4103600 0x7efcb4198840 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7efca400c890 tx=0x7efca400cc90 comp rx=0 tx=0).stop 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 shutdown_connections 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efcb410c0b0 0x7efcb419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efca00a3600 0x7efca00a5ac0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efcb4103600 0x7efcb4198840 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 --2- 172.21.15.37:0/2403127072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efcb4102ce0 0x7efcb4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 >> 172.21.15.37:0/2403127072 conn(0x7efcb4077990 msgr2=0x7efcb40fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:57.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 shutdown_connections 2024-09-19T06:41:57.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.754+0000 7efcbbd8d640 1 -- 172.21.15.37:0/2403127072 wait complete. 2024-09-19T06:41:57.769 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T06:41:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 -- 172.21.15.37:0/3093806250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681009b0 msgr2=0x7f9768100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 --2- 172.21.15.37:0/3093806250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681009b0 0x7f9768100e10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f975c009930 tx=0x7f975c02a420 comp rx=0 tx=0).stop 2024-09-19T06:41:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 -- 172.21.15.37:0/3093806250 shutdown_connections 2024-09-19T06:41:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 --2- 172.21.15.37:0/3093806250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9768101350 0x7f976810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 --2- 172.21.15.37:0/3093806250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681009b0 0x7f9768100e10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 --2- 172.21.15.37:0/3093806250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97681069c0 0x7f9768106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 -- 172.21.15.37:0/3093806250 >> 172.21.15.37:0/3093806250 conn(0x7f9768077990 msgr2=0x7f97680fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 -- 172.21.15.37:0/3093806250 shutdown_connections 2024-09-19T06:41:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.878+0000 7f976ca80640 1 -- 172.21.15.37:0/3093806250 wait complete. 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 Processor -- start 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 -- start start 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97681009b0 0x7f97681972e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9768101350 0x7f9768197820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681069c0 0x7f976819bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9768110e50 con 0x7f97681069c0 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9768110cd0 con 0x7f9768101350 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.879+0000 7f976ca80640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9768110fd0 con 0x7f97681009b0 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681069c0 0x7f976819bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681069c0 0x7f976819bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55026/0 (socket says 172.21.15.37:55026) 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 -- 172.21.15.37:0/2257048839 learned_addr learned my addr 172.21.15.37:0/2257048839 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9765d9b640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9768101350 0x7f9768197820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f976659c640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97681009b0 0x7f97681972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 -- 172.21.15.37:0/2257048839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97681009b0 msgr2=0x7f97681972e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97681009b0 0x7f97681972e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 -- 172.21.15.37:0/2257048839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9768101350 msgr2=0x7f9768197820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9768101350 0x7f9768197820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 -- 172.21.15.37:0/2257048839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976819c2d0 con 0x7f97681069c0 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9765d9b640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9768101350 0x7f9768197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f9766d9d640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681069c0 0x7f976819bba0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f976000b970 tx=0x7f976000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f97577fe640 1 -- 172.21.15.37:0/2257048839 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760019070 con 0x7f97681069c0 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f97577fe640 1 -- 172.21.15.37:0/2257048839 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f976000cb40 con 0x7f97681069c0 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976819c5c0 con 0x7f97681069c0 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.880+0000 7f97577fe640 1 -- 172.21.15.37:0/2257048839 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9760007650 con 0x7f97681069c0 2024-09-19T06:41:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.881+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97681a3e50 con 0x7f97681069c0 2024-09-19T06:41:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.882+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9768100e10 con 0x7f97681069c0 2024-09-19T06:41:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.883+0000 7f97577fe640 1 -- 172.21.15.37:0/2257048839 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f976001c070 con 0x7f97681069c0 2024-09-19T06:41:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.883+0000 7f97577fe640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97500a3580 0x7f97500a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.884+0000 7f97577fe640 1 -- 172.21.15.37:0/2257048839 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f97600cf550 con 0x7f97681069c0 2024-09-19T06:41:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.884+0000 7f976659c640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97500a3580 0x7f97500a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.884+0000 7f976659c640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97500a3580 0x7f97500a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9758005fd0 tx=0x7f975800b590 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.888+0000 7f97577fe640 1 -- 172.21.15.37:0/2257048839 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f976009bdf0 con 0x7f97681069c0 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 -- 172.21.15.37:0/4215446524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 msgr2=0x7f7eb81031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/4215446524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81031d0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8009900 tx=0x7f7ea802a410 comp rx=0 tx=0).stop 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 -- 172.21.15.37:0/4215446524 shutdown_connections 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/4215446524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7eb810c2f0 0x7f7eb810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/4215446524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7eb81037a0 0x7f7eb810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/4215446524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81031d0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 -- 172.21.15.37:0/4215446524 >> 172.21.15.37:0/4215446524 conn(0x7f7eb80fc7d0 msgr2=0x7f7eb80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 -- 172.21.15.37:0/4215446524 shutdown_connections 2024-09-19T06:41:57.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.962+0000 7f7ebd38a640 1 -- 172.21.15.37:0/4215446524 wait complete. 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 Processor -- start 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 -- start start 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7eb81037a0 0x7f7eb81a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7eb810c2f0 0x7f7eb819a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb8112170 con 0x7f7eb8102df0 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb8111ff0 con 0x7f7eb810c2f0 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7ebd38a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81122f0 con 0x7f7eb81037a0 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81a05e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55048/0 (socket says 172.21.15.37:55048) 2024-09-19T06:41:57.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/1494942616 learned_addr learned my addr 172.21.15.37:0/1494942616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7eb77fe640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7eb810c2f0 0x7f7eb819a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.963+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7eb81037a0 0x7f7eb81a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7eb77fe640 1 -- 172.21.15.37:0/1494942616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7eb81037a0 msgr2=0x7f7eb81a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7eb77fe640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7eb81037a0 0x7f7eb81a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7eb77fe640 1 -- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 msgr2=0x7f7eb81a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7eb77fe640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7eb77fe640 1 -- 172.21.15.37:0/1494942616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7eb819af20 con 0x7f7eb810c2f0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7eb77fe640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7eb810c2f0 0x7f7eb819a6b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000be20 tx=0x7f7eb000bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7e9ffff640 1 -- 172.21.15.37:0/1494942616 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0019070 con 0x7f7eb810c2f0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7e9ffff640 1 -- 172.21.15.37:0/1494942616 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eb0014920 con 0x7f7eb810c2f0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7eb819b210 con 0x7f7eb810c2f0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7e9ffff640 1 -- 172.21.15.37:0/1494942616 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb0012920 con 0x7f7eb810c2f0 2024-09-19T06:41:57.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.964+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7eb81a7390 con 0x7f7eb810c2f0 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.967+0000 7f7ebf763740 1 Processor -- start 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7e9ffff640 1 -- 172.21.15.37:0/1494942616 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f7eb001c070 con 0x7f7eb810c2f0 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7ebf763740 1 -- start start 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afc6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd3a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7ebf763740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb8af2c2e0 con 0x55cb8afc6dc0 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7ebf763740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cb8aed3ce0 con 0x55cb8afc64a0 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7ebf763740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cb8af282a0 con 0x55cb8afd1680 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd1140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55054/0 (socket says 172.21.15.37:55054) 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/723634278 learned_addr learned my addr 172.21.15.37:0/723634278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:57.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afc6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/723634278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 msgr2=0x55cb8afd3a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd3a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/723634278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 msgr2=0x55cb8afc6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afc6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7e9ffff640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ea00a34d0 0x7f7ea00a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/723634278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8aeb8440 con 0x55cb8afc6dc0 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.968+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.969+0000 7f7eb77fe640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ea00a34d0 0x7f7ea00a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.969+0000 7f7e9ffff640 1 -- 172.21.15.37:0/1494942616 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f7eb00cf750 con 0x7f7eb810c2f0 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.969+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd1140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80316e0 tx=0x7f7ea8031710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b10b483af33bd4d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.969+0000 7f7e9d7fa640 1 -- 172.21.15.37:0/723634278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8040070 con 0x55cb8afc6dc0 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.969+0000 7f7e9d7fa640 1 -- 172.21.15.37:0/723634278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea80319f0 con 0x55cb8afc6dc0 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.969+0000 7f7e9d7fa640 1 -- 172.21.15.37:0/723634278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8035040 con 0x55cb8afc6dc0 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 -- 172.21.15.37:0/723634278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 msgr2=0x55cb8afd1140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd1140 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ea80316e0 tx=0x7f7ea8031710 comp rx=0 tx=0).stop 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7eb77fe640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ea00a34d0 0x7f7ea00a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00cce90 tx=0x7f7eb0010000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 -- 172.21.15.37:0/723634278 shutdown_connections 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd3a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd1140 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 --2- 172.21.15.37:0/723634278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afc6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 -- 172.21.15.37:0/723634278 >> 172.21.15.37:0/723634278 conn(0x55cb8afc2a30 msgr2=0x55cb8afc2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 -- 172.21.15.37:0/723634278 shutdown_connections 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 -- 172.21.15.37:0/723634278 wait complete. 2024-09-19T06:41:57.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 -- start start 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.970+0000 7f7ebf763740 1 Processor -- start 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afd0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afce730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7ebf763740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cb8afd4250 con 0x55cb8afc6dc0 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7ebf763740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cb8aee7050 con 0x55cb8afc64a0 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7ebf763740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cb8afd43d0 con 0x55cb8afd1680 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7ebf763740 10 client.0 _ll_register_callbacks cb 0x55cb8af27160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb67fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afd0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55064/0 (socket says 172.21.15.37:55064) 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/956661909 learned_addr learned my addr 172.21.15.37:0/956661909 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/956661909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 msgr2=0x55cb8afce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 msgr2=0x55cb8afd0f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afd0f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb6ffd640 1 -- 172.21.15.37:0/956661909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8ac9d3c0 con 0x55cb8afc6dc0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.971+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afd0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afce1f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006010 tx=0x7f7ea803db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8940a21fbfe1488 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7e9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7e9aff5640 1 -- 172.21.15.37:0/956661909 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ea804c070 con 0x55cb8afc6dc0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ea80319f0 con 0x55cb8afc6dc0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7e9aff5640 1 -- 172.21.15.37:0/956661909 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f7ea8044400 con 0x55cb8afc6dc0 2024-09-19T06:41:57.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7e9aff5640 1 -- 172.21.15.37:0/956661909 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7ea8058050 con 0x55cb8afc6dc0 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7ebf763740 10 client.15354 fetch_fsmap learned FSMap version 25 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb8afdfb40 con 0x55cb8afc6dc0 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.972+0000 7f7e9aff5640 1 -- 172.21.15.37:0/956661909 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ea80319f0 con 0x55cb8afc6dc0 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7e9aff5640 1 -- 172.21.15.37:0/956661909 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7ea803b760 con 0x55cb8afc6dc0 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7ebf763740 10 client.15354 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7ebf763740 10 client.15354 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7ebf763740 20 client.15354 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7ebf763740 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8afe0e90 0x55cb8afe3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7ebf763740 4 client.15354 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb8af39850 con 0x55cb8afe0e90 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.973+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8afe0e90 0x55cb8afe3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8afe0e90 0x55cb8afe3280 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81f13b8301318db3 server_cookie=d790c0f5629ba74 in_seq=0 out_seq=0 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7e9aff5640 10 client.15354 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7e9aff5640 1 -- 172.21.15.37:0/956661909 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cb8af39850 con 0x55cb8afe0e90 2024-09-19T06:41:58.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7e9aff5640 10 client.15354 handle_command_reply: tid=0 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7ebf763740 1 client.15354 shutdown 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 msgr2=0x55cb8afce1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7ebf763740 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afce1f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8006010 tx=0x7f7ea803db90 comp rx=0 tx=0).stop 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 shutdown_connections 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7ebf763740 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8afe0e90 0x55cb8afe3280 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.974+0000 7f7ebf763740 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.975+0000 7f7ebf763740 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc6dc0 0x55cb8afce1f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.975+0000 7f7ebf763740 1 --2- 172.21.15.37:0/956661909 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc64a0 0x55cb8afd0f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.975+0000 7f7e9aff5640 0 client.15354 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.975+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 >> 172.21.15.37:0/956661909 conn(0x55cb8afc2a30 msgr2=0x55cb8afc1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.975+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 shutdown_connections 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.975+0000 7f7ebf763740 1 -- 172.21.15.37:0/956661909 wait complete. 2024-09-19T06:41:58.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:57.975+0000 7f7ebf763740 20 client.15354 trim_cache size 0 max 16384 2024-09-19T06:41:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 Processor -- start 2024-09-19T06:41:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 -- start start 2024-09-19T06:41:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afc64a0 0x55cb8afdd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afd1680 0x55cb8afd0160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81b24b0 con 0x55cb8afd1680 2024-09-19T06:41:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81afd80 con 0x55cb8afc6dc0 2024-09-19T06:41:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7ebf763740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81adbd0 con 0x55cb8afc64a0 2024-09-19T06:41:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afd1680 0x55cb8afd0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afd1680 0x55cb8afd0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55074/0 (socket says 172.21.15.37:55074) 2024-09-19T06:41:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7eb67fc640 1 -- 172.21.15.37:0/913984803 learned_addr learned my addr 172.21.15.37:0/913984803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afc64a0 0x55cb8afdd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.009+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7eb67fc640 1 -- 172.21.15.37:0/913984803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afc64a0 msgr2=0x55cb8afdd4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afc64a0 0x55cb8afdd4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7eb67fc640 1 -- 172.21.15.37:0/913984803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 msgr2=0x55cb8afccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afccac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7eb67fc640 1 -- 172.21.15.37:0/913984803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8afcd1a0 con 0x55cb8afd1680 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afd1680 0x55cb8afd0160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000c00 tx=0x7f7eac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cdfeaf24df5570d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7e9cff9640 1 -- 172.21.15.37:0/913984803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac025070 con 0x55cb8afd1680 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7e9cff9640 1 -- 172.21.15.37:0/913984803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac026040 con 0x55cb8afd1680 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.010+0000 7f7e9cff9640 1 -- 172.21.15.37:0/913984803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac004e40 con 0x55cb8afd1680 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 -- 172.21.15.37:0/913984803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afd1680 msgr2=0x55cb8afd0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afd1680 0x55cb8afd0160 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7eac000c00 tx=0x7f7eac0046c0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 -- 172.21.15.37:0/913984803 shutdown_connections 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afd1680 0x55cb8afd0160 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 --2- 172.21.15.37:0/913984803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afc64a0 0x55cb8afdd4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 -- 172.21.15.37:0/913984803 >> 172.21.15.37:0/913984803 conn(0x55cb8afc2a30 msgr2=0x55cb8afcfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 -- 172.21.15.37:0/913984803 shutdown_connections 2024-09-19T06:41:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.011+0000 7f7ebf763740 1 -- 172.21.15.37:0/913984803 wait complete. 2024-09-19T06:41:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 -- start start 2024-09-19T06:41:58.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 Processor -- start 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc64a0 0x55cb8afd9f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81ae240 con 0x55cb8afc64a0 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81af6b0 con 0x55cb8afc6dc0 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7eb81ae910 con 0x55cb8afd1680 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7ebf763740 10 client.0 _ll_register_callbacks cb 0x7f7ea80315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.012+0000 7f7eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc64a0 0x55cb8afd9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc64a0 0x55cb8afd9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55076/0 (socket says 172.21.15.37:55076) 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 -- 172.21.15.37:0/2139163765 learned_addr learned my addr 172.21.15.37:0/2139163765 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 -- 172.21.15.37:0/2139163765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 msgr2=0x55cb8afd5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 -- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 msgr2=0x55cb8afd52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 -- 172.21.15.37:0/2139163765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb8afd12b0 con 0x55cb8afc64a0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7eb67fc640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc64a0 0x55cb8afd9f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00b6b0 tx=0x7f7eac004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd93dd4796fcf087 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7e9aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7e9aff5640 1 -- 172.21.15.37:0/2139163765 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eac036060 con 0x55cb8afc64a0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7e9aff5640 1 -- 172.21.15.37:0/2139163765 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f7eac012250 con 0x55cb8afc64a0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.013+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7eac026040 con 0x55cb8afc64a0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7e9aff5640 1 -- 172.21.15.37:0/2139163765 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7eac025040 con 0x55cb8afc64a0 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 10 client.15360 fetch_fsmap learned FSMap version 25 2024-09-19T06:41:58.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cb8b04b7f0 con 0x55cb8afc64a0 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7e9aff5640 1 -- 172.21.15.37:0/2139163765 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7eac021930 con 0x55cb8afc64a0 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7e9aff5640 1 -- 172.21.15.37:0/2139163765 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f7eac02b850 con 0x55cb8afc64a0 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 10 client.15360 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 10 client.15360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 20 client.15360 populate_metadata read hostname 'smithi037' 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8b04be40 0x55cb8b04e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 4 client.15360 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f7eac026040 con 0x55cb8b04be40 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.014+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8b04be40 0x55cb8b04e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7e9aff5640 10 client.15360 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7eb6ffd640 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8b04be40 0x55cb8b04e260 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8627688871e4661c server_cookie=e95417d57a84e5b0 in_seq=0 out_seq=0 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7e9aff5640 1 -- 172.21.15.37:0/2139163765 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7f7eac026040 con 0x55cb8b04be40 2024-09-19T06:41:58.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7e9aff5640 10 client.15360 handle_command_reply: tid=0 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 client.15360 shutdown 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc64a0 msgr2=0x55cb8afd9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc64a0 0x55cb8afd9f60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7eac00b6b0 tx=0x7f7eac004ac0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 shutdown_connections 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cb8b04be40 0x55cb8b04e260 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cb8afd1680 0x55cb8afd5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cb8afc6dc0 0x55cb8afd52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7e9aff5640 0 client.15360 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 --2- 172.21.15.37:0/2139163765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cb8afc64a0 0x55cb8afd9f60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 >> 172.21.15.37:0/2139163765 conn(0x55cb8afc2a30 msgr2=0x55cb8afcfa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 shutdown_connections 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.015+0000 7f7ebf763740 1 -- 172.21.15.37:0/2139163765 wait complete. 2024-09-19T06:41:58.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebf763740 20 client.15360 trim_cache size 0 max 16384 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: "df3a94bb-98c3-47be-9da2-9b5ae969eac0": { 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "df3a94bb-98c3-47be-9da2-9b5ae969eac0", 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:41:58.022 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:41:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ea00a34d0 msgr2=0x7f7ea00a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ea00a34d0 0x7f7ea00a5990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7eb00cce90 tx=0x7f7eb0010000 comp rx=0 tx=0).stop 2024-09-19T06:41:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7eb810c2f0 msgr2=0x7f7eb819a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7eb810c2f0 0x7f7eb819a6b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7eb000be20 tx=0x7f7eb000bf10 comp rx=0 tx=0).stop 2024-09-19T06:41:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 shutdown_connections 2024-09-19T06:41:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7eb810c2f0 0x7f7eb819a6b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7eb81037a0 0x7f7eb81a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ea00a34d0 0x7f7ea00a5990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 --2- 172.21.15.37:0/1494942616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7eb8102df0 0x7f7eb81a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.016+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 >> 172.21.15.37:0/1494942616 conn(0x7f7eb80fc7d0 msgr2=0x7f7eb810d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.017+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 shutdown_connections 2024-09-19T06:41:58.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.017+0000 7f7ebd38a640 1 -- 172.21.15.37:0/1494942616 wait complete. 2024-09-19T06:41:58.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.098+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f9768103ec0 con 0x7f97500a3580 2024-09-19T06:41:58.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.105+0000 7f97577fe640 1 -- 172.21.15.37:0/2257048839 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f9768103ec0 con 0x7f97500a3580 2024-09-19T06:41:58.107 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-09-19T06:41:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97500a3580 msgr2=0x7f97500a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:58.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97500a3580 0x7f97500a5a40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9758005fd0 tx=0x7f975800b590 comp rx=0 tx=0).stop 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681069c0 msgr2=0x7f976819bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681069c0 0x7f976819bba0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f976000b970 tx=0x7f976000be30 comp rx=0 tx=0).stop 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 shutdown_connections 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97500a3580 0x7f97500a5a40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97681069c0 0x7f976819bba0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9768101350 0x7f9768197820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 --2- 172.21.15.37:0/2257048839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97681009b0 0x7f97681972e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.107+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 >> 172.21.15.37:0/2257048839 conn(0x7f9768077990 msgr2=0x7f97681037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.108+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 shutdown_connections 2024-09-19T06:41:58.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:58.108+0000 7f976ca80640 1 -- 172.21.15.37:0/2257048839 wait complete. 2024-09-19T06:41:58.120 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-19T06:41:58.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:57 smithi037 ceph-mon[31211]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 796 B/s wr, 18 op/s 2024-09-19T06:41:58.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:57 smithi037 ceph-mon[31211]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:58.143 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:57 smithi037 ceph-mon[31211]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:58.148 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-19T06:41:58.159 INFO:teuthology.run_tasks:Running task background_exec... 2024-09-19T06:41:58.176 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:41:58.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-09-19T06:41:58.191 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-19T06:41:58.205 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-19T06:41:58.205 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:41:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:57 smithi096 ceph-mon[35004]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 796 B/s wr, 18 op/s 2024-09-19T06:41:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:57 smithi096 ceph-mon[35004]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:57 smithi096 ceph-mon[35004]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:57 smithi151 ceph-mon[35236]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 796 B/s wr, 18 op/s 2024-09-19T06:41:58.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:57 smithi151 ceph-mon[35236]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:58.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:57 smithi151 ceph-mon[35236]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:58.819 INFO:tasks.cephfs.filesystem:scrub status for tag:df3a94bb-98c3-47be-9da2-9b5ae969eac0 - {'path': '/', 'tag': 'df3a94bb-98c3-47be-9da2-9b5ae969eac0', 'options': 'recursive,force'} 2024-09-19T06:41:58.904 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:41:59.041 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:58 smithi037 ceph-mon[31211]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:59.041 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:58 smithi037 ceph-mon[31211]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:59.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:58 smithi096 ceph-mon[35004]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:59.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:58 smithi096 ceph-mon[35004]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:59.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:58 smithi151 ceph-mon[35236]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:59.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:58 smithi151 ceph-mon[35236]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:41:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 -- 172.21.15.37:0/4151017527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128104010 msgr2=0x7f81281043f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 --2- 172.21.15.37:0/4151017527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128104010 0x7f81281043f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8118009900 tx=0x7f811802a410 comp rx=0 tx=0).stop 2024-09-19T06:41:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 -- 172.21.15.37:0/4151017527 shutdown_connections 2024-09-19T06:41:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 --2- 172.21.15.37:0/4151017527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128109140 0x7f8128110cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 --2- 172.21.15.37:0/4151017527 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81281049c0 0x7f8128108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 --2- 172.21.15.37:0/4151017527 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128104010 0x7f81281043f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 -- 172.21.15.37:0/4151017527 >> 172.21.15.37:0/4151017527 conn(0x7f81280ffa50 msgr2=0x7f8128101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 -- 172.21.15.37:0/4151017527 shutdown_connections 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.375+0000 7f812d8eb640 1 -- 172.21.15.37:0/4151017527 wait complete. 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.376+0000 7f812d8eb640 1 Processor -- start 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.376+0000 7f812d8eb640 1 -- start start 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f812d8eb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128104010 0x7f812819a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f812d8eb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81281049c0 0x7f812819ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f812d8eb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128109140 0x7f812819ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f812d8eb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81281141c0 con 0x7f8128109140 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f812d8eb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8128114040 con 0x7f8128104010 2024-09-19T06:41:59.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f812d8eb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8128114340 con 0x7f81281049c0 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128109140 0x7f812819ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128109140 0x7f812819ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55100/0 (socket says 172.21.15.37:55100) 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 -- 172.21.15.37:0/1940560186 learned_addr learned my addr 172.21.15.37:0/1940560186 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f8126ffd640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128104010 0x7f812819a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f811e7fc640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81281049c0 0x7f812819ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 -- 172.21.15.37:0/1940560186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81281049c0 msgr2=0x7f812819ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81281049c0 0x7f812819ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 -- 172.21.15.37:0/1940560186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128104010 msgr2=0x7f812819a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128104010 0x7f812819a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 -- 172.21.15.37:0/1940560186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812819f670 con 0x7f8128109140 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.377+0000 7f81277fe640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128109140 0x7f812819ef90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f812000eaa0 tx=0x7f812000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:59.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.378+0000 7f812c8e9640 1 -- 172.21.15.37:0/1940560186 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f812000cdb0 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.378+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f812819f960 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.378+0000 7f812c8e9640 1 -- 172.21.15.37:0/1940560186 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8120005020 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.378+0000 7f812c8e9640 1 -- 172.21.15.37:0/1940560186 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81200116b0 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.378+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81281a7240 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.379+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81281043f0 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.380+0000 7f812c8e9640 1 -- 172.21.15.37:0/1940560186 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7f8120005560 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.381+0000 7f812c8e9640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f81140a34d0 0x7f81140a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.381+0000 7f812c8e9640 1 -- 172.21.15.37:0/1940560186 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f81200d0610 con 0x7f8128109140 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.381+0000 7f8126ffd640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f81140a34d0 0x7f81140a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.381+0000 7f8126ffd640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f81140a34d0 0x7f81140a5990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8118009230 tx=0x7f81180058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:41:59.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.384+0000 7f812c8e9640 1 -- 172.21.15.37:0/1940560186 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8120016030 con 0x7f8128109140 2024-09-19T06:41:59.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.557+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8128062630 con 0x7f81140a34d0 2024-09-19T06:41:59.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.564+0000 7f812c8e9640 1 -- 172.21.15.37:0/1940560186 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258036 (secure 0 0 0) 0x7f8128062630 con 0x7f81140a34d0 2024-09-19T06:41:59.567 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:41:59.568 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:41:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f81140a34d0 msgr2=0x7f81140a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f81140a34d0 0x7f81140a5990 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8118009230 tx=0x7f81180058d0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128109140 msgr2=0x7f812819ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:41:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128109140 0x7f812819ef90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f812000eaa0 tx=0x7f812000ef60 comp rx=0 tx=0).stop 2024-09-19T06:41:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 shutdown_connections 2024-09-19T06:41:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f81140a34d0 0x7f81140a5990 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8128109140 0x7f812819ef90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f81281049c0 0x7f812819ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 --2- 172.21.15.37:0/1940560186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8128104010 0x7f812819a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:41:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 >> 172.21.15.37:0/1940560186 conn(0x7f81280ffa50 msgr2=0x7f8128100df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:41:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 shutdown_connections 2024-09-19T06:41:59.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:41:59.568+0000 7f812d8eb640 1 -- 172.21.15.37:0/1940560186 wait complete. 2024-09-19T06:42:00.158 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:41:59 smithi037 ceph-mon[31211]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-19T06:42:00.186 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-19T06:41:58.605093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388052,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1288,"num_read_kb":1064,"num_write":621,"num_write_kb":5300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":959,"ondisk_log_size":959,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364848,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576976,"statfs":{"total":1151940427776,"available":1151566823424,"internally_reserved":0,"allocated":11685888,"data_stored":9511884,"data_compressed":313886,"data_compressed_allocated":417792,"data_compressed_original":4165632,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201076,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":401,"num_read_kb":469,"num_write":146,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.995274"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934075+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:41:46.934075+0000","last_peered":"2024-09-19T06:41:46.934075+0000","last_clean":"2024-09-19T06:41:46.934075+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:41:46.934075+0000","last_undegraded":"2024-09-19T06:41:46.934075+0000","last_fullsized":"2024-09-19T06:41:46.934075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866847+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:41:46.866847+0000","last_peered":"2024-09-19T06:41:46.866847+0000","last_clean":"2024-09-19T06:41:46.866847+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:41:46.866847+0000","last_undegraded":"2024-09-19T06:41:46.866847+0000","last_fullsized":"2024-09-19T06:41:46.866847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866554+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:41:46.866554+0000","last_peered":"2024-09-19T06:41:46.866554+0000","last_clean":"2024-09-19T06:41:46.866554+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:41:46.866554+0000","last_undegraded":"2024-09-19T06:41:46.866554+0000","last_fullsized":"2024-09-19T06:41:46.866554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607461+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:41:46.607461+0000","last_peered":"2024-09-19T06:41:46.607461+0000","last_clean":"2024-09-19T06:41:46.607461+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:41:46.607461+0000","last_undegraded":"2024-09-19T06:41:46.607461+0000","last_fullsized":"2024-09-19T06:41:46.607461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679815+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:41:46.679815+0000","last_peered":"2024-09-19T06:41:46.679815+0000","last_clean":"2024-09-19T06:41:46.679815+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:41:46.679815+0000","last_undegraded":"2024-09-19T06:41:46.679815+0000","last_fullsized":"2024-09-19T06:41:46.679815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590191+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:41:46.590191+0000","last_peered":"2024-09-19T06:41:46.590191+0000","last_clean":"2024-09-19T06:41:46.590191+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:41:46.590191+0000","last_undegraded":"2024-09-19T06:41:46.590191+0000","last_fullsized":"2024-09-19T06:41:46.590191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860437+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:41:46.860437+0000","last_peered":"2024-09-19T06:41:46.860437+0000","last_clean":"2024-09-19T06:41:46.860437+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:41:46.860437+0000","last_undegraded":"2024-09-19T06:41:46.860437+0000","last_fullsized":"2024-09-19T06:41:46.860437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607193+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:41:46.607193+0000","last_peered":"2024-09-19T06:41:46.607193+0000","last_clean":"2024-09-19T06:41:46.607193+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:41:46.607193+0000","last_undegraded":"2024-09-19T06:41:46.607193+0000","last_fullsized":"2024-09-19T06:41:46.607193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609566+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:41:46.609566+0000","last_peered":"2024-09-19T06:41:46.609566+0000","last_clean":"2024-09-19T06:41:46.609566+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:41:46.609566+0000","last_undegraded":"2024-09-19T06:41:46.609566+0000","last_fullsized":"2024-09-19T06:41:46.609566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590543+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:41:46.590543+0000","last_peered":"2024-09-19T06:41:46.590543+0000","last_clean":"2024-09-19T06:41:46.590543+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:41:46.590543+0000","last_undegraded":"2024-09-19T06:41:46.590543+0000","last_fullsized":"2024-09-19T06:41:46.590543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860924+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:41:46.860924+0000","last_peered":"2024-09-19T06:41:46.860924+0000","last_clean":"2024-09-19T06:41:46.860924+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:41:46.860924+0000","last_undegraded":"2024-09-19T06:41:46.860924+0000","last_fullsized":"2024-09-19T06:41:46.860924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607557+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:41:46.607557+0000","last_peered":"2024-09-19T06:41:46.607557+0000","last_clean":"2024-09-19T06:41:46.607557+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:41:46.607557+0000","last_undegraded":"2024-09-19T06:41:46.607557+0000","last_fullsized":"2024-09-19T06:41:46.607557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587632+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:41:46.587632+0000","last_peered":"2024-09-19T06:41:46.587632+0000","last_clean":"2024-09-19T06:41:46.587632+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:41:46.587632+0000","last_undegraded":"2024-09-19T06:41:46.587632+0000","last_fullsized":"2024-09-19T06:41:46.587632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866079+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:41:46.866079+0000","last_peered":"2024-09-19T06:41:46.866079+0000","last_clean":"2024-09-19T06:41:46.866079+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:41:46.866079+0000","last_undegraded":"2024-09-19T06:41:46.866079+0000","last_fullsized":"2024-09-19T06:41:46.866079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.865755+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:41:46.865755+0000","last_peered":"2024-09-19T06:41:46.865755+0000","last_clean":"2024-09-19T06:41:46.865755+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:41:46.865755+0000","last_undegraded":"2024-09-19T06:41:46.865755+0000","last_fullsized":"2024-09-19T06:41:46.865755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609034+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:41:46.609034+0000","last_peered":"2024-09-19T06:41:46.609034+0000","last_clean":"2024-09-19T06:41:46.609034+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:41:46.609034+0000","last_undegraded":"2024-09-19T06:41:46.609034+0000","last_fullsized":"2024-09-19T06:41:46.609034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934022+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:41:46.934022+0000","last_peered":"2024-09-19T06:41:46.934022+0000","last_clean":"2024-09-19T06:41:46.934022+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:41:46.934022+0000","last_undegraded":"2024-09-19T06:41:46.934022+0000","last_fullsized":"2024-09-19T06:41:46.934022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677589+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:41:46.677589+0000","last_peered":"2024-09-19T06:41:46.677589+0000","last_clean":"2024-09-19T06:41:46.677589+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:41:46.677589+0000","last_undegraded":"2024-09-19T06:41:46.677589+0000","last_fullsized":"2024-09-19T06:41:46.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609334+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:41:46.609334+0000","last_peered":"2024-09-19T06:41:46.609334+0000","last_clean":"2024-09-19T06:41:46.609334+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:41:46.609334+0000","last_undegraded":"2024-09-19T06:41:46.609334+0000","last_fullsized":"2024-09-19T06:41:46.609334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933853+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:41:46.933853+0000","last_peered":"2024-09-19T06:41:46.933853+0000","last_clean":"2024-09-19T06:41:46.933853+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:41:46.933853+0000","last_undegraded":"2024-09-19T06:41:46.933853+0000","last_fullsized":"2024-09-19T06:41:46.933853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587643+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:41:46.587643+0000","last_peered":"2024-09-19T06:41:46.587643+0000","last_clean":"2024-09-19T06:41:46.587643+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:41:46.587643+0000","last_undegraded":"2024-09-19T06:41:46.587643+0000","last_fullsized":"2024-09-19T06:41:46.587643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609138+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:41:46.609138+0000","last_peered":"2024-09-19T06:41:46.609138+0000","last_clean":"2024-09-19T06:41:46.609138+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:41:46.609138+0000","last_undegraded":"2024-09-19T06:41:46.609138+0000","last_fullsized":"2024-09-19T06:41:46.609138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607201+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:41:46.607201+0000","last_peered":"2024-09-19T06:41:46.607201+0000","last_clean":"2024-09-19T06:41:46.607201+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:41:46.607201+0000","last_undegraded":"2024-09-19T06:41:46.607201+0000","last_fullsized":"2024-09-19T06:41:46.607201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588420+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:41:46.588420+0000","last_peered":"2024-09-19T06:41:46.588420+0000","last_clean":"2024-09-19T06:41:46.588420+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:41:46.588420+0000","last_undegraded":"2024-09-19T06:41:46.588420+0000","last_fullsized":"2024-09-19T06:41:46.588420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677514+0000","last_change":"2024-09-19T06:40:58.134245+0000","last_active":"2024-09-19T06:41:46.677514+0000","last_peered":"2024-09-19T06:41:46.677514+0000","last_clean":"2024-09-19T06:41:46.677514+0000","last_became_active":"2024-09-19T06:40:58.133514+0000","last_became_peered":"2024-09-19T06:40:58.133514+0000","last_unstale":"2024-09-19T06:41:46.677514+0000","last_undegraded":"2024-09-19T06:41:46.677514+0000","last_fullsized":"2024-09-19T06:41:46.677514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:23.267466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609405+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:41:46.609405+0000","last_peered":"2024-09-19T06:41:46.609405+0000","last_clean":"2024-09-19T06:41:46.609405+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:41:46.609405+0000","last_undegraded":"2024-09-19T06:41:46.609405+0000","last_fullsized":"2024-09-19T06:41:46.609405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933946+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:41:46.933946+0000","last_peered":"2024-09-19T06:41:46.933946+0000","last_clean":"2024-09-19T06:41:46.933946+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:41:46.933946+0000","last_undegraded":"2024-09-19T06:41:46.933946+0000","last_fullsized":"2024-09-19T06:41:46.933946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587547+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:41:46.587547+0000","last_peered":"2024-09-19T06:41:46.587547+0000","last_clean":"2024-09-19T06:41:46.587547+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:41:46.587547+0000","last_undegraded":"2024-09-19T06:41:46.587547+0000","last_fullsized":"2024-09-19T06:41:46.587547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866245+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:41:46.866245+0000","last_peered":"2024-09-19T06:41:46.866245+0000","last_clean":"2024-09-19T06:41:46.866245+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:41:46.866245+0000","last_undegraded":"2024-09-19T06:41:46.866245+0000","last_fullsized":"2024-09-19T06:41:46.866245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609084+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:41:46.609084+0000","last_peered":"2024-09-19T06:41:46.609084+0000","last_clean":"2024-09-19T06:41:46.609084+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:41:46.609084+0000","last_undegraded":"2024-09-19T06:41:46.609084+0000","last_fullsized":"2024-09-19T06:41:46.609084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609093+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:41:46.609093+0000","last_peered":"2024-09-19T06:41:46.609093+0000","last_clean":"2024-09-19T06:41:46.609093+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:41:46.609093+0000","last_undegraded":"2024-09-19T06:41:46.609093+0000","last_fullsized":"2024-09-19T06:41:46.609093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933721+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:41:46.933721+0000","last_peered":"2024-09-19T06:41:46.933721+0000","last_clean":"2024-09-19T06:41:46.933721+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:41:46.933721+0000","last_undegraded":"2024-09-19T06:41:46.933721+0000","last_fullsized":"2024-09-19T06:41:46.933721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679766+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:41:46.679766+0000","last_peered":"2024-09-19T06:41:46.679766+0000","last_clean":"2024-09-19T06:41:46.679766+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:41:46.679766+0000","last_undegraded":"2024-09-19T06:41:46.679766+0000","last_fullsized":"2024-09-19T06:41:46.679766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:55.619074+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:41:55.619074+0000","last_peered":"2024-09-19T06:41:55.619074+0000","last_clean":"2024-09-19T06:41:55.619074+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:41:55.619074+0000","last_undegraded":"2024-09-19T06:41:55.619074+0000","last_fullsized":"2024-09-19T06:41:55.619074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588409+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:41:46.588409+0000","last_peered":"2024-09-19T06:41:46.588409+0000","last_clean":"2024-09-19T06:41:46.588409+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:41:46.588409+0000","last_undegraded":"2024-09-19T06:41:46.588409+0000","last_fullsized":"2024-09-19T06:41:46.588409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590296+0000","last_change":"2024-09-19T06:40:58.136626+0000","last_active":"2024-09-19T06:41:46.590296+0000","last_peered":"2024-09-19T06:41:46.590296+0000","last_clean":"2024-09-19T06:41:46.590296+0000","last_became_active":"2024-09-19T06:40:58.132752+0000","last_became_peered":"2024-09-19T06:40:58.132752+0000","last_unstale":"2024-09-19T06:41:46.590296+0000","last_undegraded":"2024-09-19T06:41:46.590296+0000","last_fullsized":"2024-09-19T06:41:46.590296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587271+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:41:46.587271+0000","last_peered":"2024-09-19T06:41:46.587271+0000","last_clean":"2024-09-19T06:41:46.587271+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:41:46.587271+0000","last_undegraded":"2024-09-19T06:41:46.587271+0000","last_fullsized":"2024-09-19T06:41:46.587271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607281+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:41:46.607281+0000","last_peered":"2024-09-19T06:41:46.607281+0000","last_clean":"2024-09-19T06:41:46.607281+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:41:46.607281+0000","last_undegraded":"2024-09-19T06:41:46.607281+0000","last_fullsized":"2024-09-19T06:41:46.607281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588546+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:41:46.588546+0000","last_peered":"2024-09-19T06:41:46.588546+0000","last_clean":"2024-09-19T06:41:46.588546+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:41:46.588546+0000","last_undegraded":"2024-09-19T06:41:46.588546+0000","last_fullsized":"2024-09-19T06:41:46.588546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677238+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:41:46.677238+0000","last_peered":"2024-09-19T06:41:46.677238+0000","last_clean":"2024-09-19T06:41:46.677238+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:41:46.677238+0000","last_undegraded":"2024-09-19T06:41:46.677238+0000","last_fullsized":"2024-09-19T06:41:46.677238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:55.618471+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:41:55.618471+0000","last_peered":"2024-09-19T06:41:55.618471+0000","last_clean":"2024-09-19T06:41:55.618471+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:41:55.618471+0000","last_undegraded":"2024-09-19T06:41:55.618471+0000","last_fullsized":"2024-09-19T06:41:55.618471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609442+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:41:46.609442+0000","last_peered":"2024-09-19T06:41:46.609442+0000","last_clean":"2024-09-19T06:41:46.609442+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:41:46.609442+0000","last_undegraded":"2024-09-19T06:41:46.609442+0000","last_fullsized":"2024-09-19T06:41:46.609442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860979+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:41:46.860979+0000","last_peered":"2024-09-19T06:41:46.860979+0000","last_clean":"2024-09-19T06:41:46.860979+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:41:46.860979+0000","last_undegraded":"2024-09-19T06:41:46.860979+0000","last_fullsized":"2024-09-19T06:41:46.860979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867221+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:41:46.867221+0000","last_peered":"2024-09-19T06:41:46.867221+0000","last_clean":"2024-09-19T06:41:46.867221+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:41:46.867221+0000","last_undegraded":"2024-09-19T06:41:46.867221+0000","last_fullsized":"2024-09-19T06:41:46.867221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588728+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:41:46.588728+0000","last_peered":"2024-09-19T06:41:46.588728+0000","last_clean":"2024-09-19T06:41:46.588728+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:41:46.588728+0000","last_undegraded":"2024-09-19T06:41:46.588728+0000","last_fullsized":"2024-09-19T06:41:46.588728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609314+0000","last_change":"2024-09-19T06:40:58.135595+0000","last_active":"2024-09-19T06:41:46.609314+0000","last_peered":"2024-09-19T06:41:46.609314+0000","last_clean":"2024-09-19T06:41:46.609314+0000","last_became_active":"2024-09-19T06:40:58.135031+0000","last_became_peered":"2024-09-19T06:40:58.135031+0000","last_unstale":"2024-09-19T06:41:46.609314+0000","last_undegraded":"2024-09-19T06:41:46.609314+0000","last_fullsized":"2024-09-19T06:41:46.609314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:53.542630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":306,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:56.620569+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:41:56.620569+0000","last_peered":"2024-09-19T06:41:56.620569+0000","last_clean":"2024-09-19T06:41:56.620569+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:41:56.620569+0000","last_undegraded":"2024-09-19T06:41:56.620569+0000","last_fullsized":"2024-09-19T06:41:56.620569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":192,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680048+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:41:46.680048+0000","last_peered":"2024-09-19T06:41:46.680048+0000","last_clean":"2024-09-19T06:41:46.680048+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:41:46.680048+0000","last_undegraded":"2024-09-19T06:41:46.680048+0000","last_fullsized":"2024-09-19T06:41:46.680048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609208+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:41:46.609208+0000","last_peered":"2024-09-19T06:41:46.609208+0000","last_clean":"2024-09-19T06:41:46.609208+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:41:46.609208+0000","last_undegraded":"2024-09-19T06:41:46.609208+0000","last_fullsized":"2024-09-19T06:41:46.609208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587384+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:41:46.587384+0000","last_peered":"2024-09-19T06:41:46.587384+0000","last_clean":"2024-09-19T06:41:46.587384+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:41:46.587384+0000","last_undegraded":"2024-09-19T06:41:46.587384+0000","last_fullsized":"2024-09-19T06:41:46.587384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933722+0000","last_change":"2024-09-19T06:40:58.143264+0000","last_active":"2024-09-19T06:41:46.933722+0000","last_peered":"2024-09-19T06:41:46.933722+0000","last_clean":"2024-09-19T06:41:46.933722+0000","last_became_active":"2024-09-19T06:40:58.135400+0000","last_became_peered":"2024-09-19T06:40:58.135400+0000","last_unstale":"2024-09-19T06:41:46.933722+0000","last_undegraded":"2024-09-19T06:41:46.933722+0000","last_fullsized":"2024-09-19T06:41:46.933722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:30.500765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860840+0000","last_change":"2024-09-19T06:40:58.134136+0000","last_active":"2024-09-19T06:41:46.860840+0000","last_peered":"2024-09-19T06:41:46.860840+0000","last_clean":"2024-09-19T06:41:46.860840+0000","last_became_active":"2024-09-19T06:40:58.133734+0000","last_became_peered":"2024-09-19T06:40:58.133734+0000","last_unstale":"2024-09-19T06:41:46.860840+0000","last_undegraded":"2024-09-19T06:41:46.860840+0000","last_fullsized":"2024-09-19T06:41:46.860840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:50.822670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:55.618680+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:41:55.618680+0000","last_peered":"2024-09-19T06:41:55.618680+0000","last_clean":"2024-09-19T06:41:55.618680+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:41:55.618680+0000","last_undegraded":"2024-09-19T06:41:55.618680+0000","last_fullsized":"2024-09-19T06:41:55.618680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860373+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:41:46.860373+0000","last_peered":"2024-09-19T06:41:46.860373+0000","last_clean":"2024-09-19T06:41:46.860373+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:41:46.860373+0000","last_undegraded":"2024-09-19T06:41:46.860373+0000","last_fullsized":"2024-09-19T06:41:46.860373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679751+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:41:46.679751+0000","last_peered":"2024-09-19T06:41:46.679751+0000","last_clean":"2024-09-19T06:41:46.679751+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:41:46.679751+0000","last_undegraded":"2024-09-19T06:41:46.679751+0000","last_fullsized":"2024-09-19T06:41:46.679751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588476+0000","last_change":"2024-09-19T06:40:58.135978+0000","last_active":"2024-09-19T06:41:46.588476+0000","last_peered":"2024-09-19T06:41:46.588476+0000","last_clean":"2024-09-19T06:41:46.588476+0000","last_became_active":"2024-09-19T06:40:58.135581+0000","last_became_peered":"2024-09-19T06:40:58.135581+0000","last_unstale":"2024-09-19T06:41:46.588476+0000","last_undegraded":"2024-09-19T06:41:46.588476+0000","last_fullsized":"2024-09-19T06:41:46.588476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:54.532003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:47.456507+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:41:47.456507+0000","last_peered":"2024-09-19T06:41:47.456507+0000","last_clean":"2024-09-19T06:41:47.456507+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:41:47.456507+0000","last_undegraded":"2024-09-19T06:41:47.456507+0000","last_fullsized":"2024-09-19T06:41:47.456507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933860+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:41:46.933860+0000","last_peered":"2024-09-19T06:41:46.933860+0000","last_clean":"2024-09-19T06:41:46.933860+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:41:46.933860+0000","last_undegraded":"2024-09-19T06:41:46.933860+0000","last_fullsized":"2024-09-19T06:41:46.933860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587529+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:41:46.587529+0000","last_peered":"2024-09-19T06:41:46.587529+0000","last_clean":"2024-09-19T06:41:46.587529+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:41:46.587529+0000","last_undegraded":"2024-09-19T06:41:46.587529+0000","last_fullsized":"2024-09-19T06:41:46.587529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'65","reported_seq":118,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:57.377629+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:41:57.377629+0000","last_peered":"2024-09-19T06:41:57.377629+0000","last_clean":"2024-09-19T06:41:57.377629+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:41:57.377629+0000","last_undegraded":"2024-09-19T06:41:57.377629+0000","last_fullsized":"2024-09-19T06:41:57.377629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860872+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:41:46.860872+0000","last_peered":"2024-09-19T06:41:46.860872+0000","last_clean":"2024-09-19T06:41:46.860872+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:41:46.860872+0000","last_undegraded":"2024-09-19T06:41:46.860872+0000","last_fullsized":"2024-09-19T06:41:46.860872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607210+0000","last_change":"2024-09-19T06:40:58.138283+0000","last_active":"2024-09-19T06:41:46.607210+0000","last_peered":"2024-09-19T06:41:46.607210+0000","last_clean":"2024-09-19T06:41:46.607210+0000","last_became_active":"2024-09-19T06:40:58.132079+0000","last_became_peered":"2024-09-19T06:40:58.132079+0000","last_unstale":"2024-09-19T06:41:46.607210+0000","last_undegraded":"2024-09-19T06:41:46.607210+0000","last_fullsized":"2024-09-19T06:41:46.607210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:12.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":390,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:48.697056+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:41:48.697056+0000","last_peered":"2024-09-19T06:41:48.697056+0000","last_clean":"2024-09-19T06:41:48.697056+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:41:48.697056+0000","last_undegraded":"2024-09-19T06:41:48.697056+0000","last_fullsized":"2024-09-19T06:41:48.697056+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607328+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:41:46.607328+0000","last_peered":"2024-09-19T06:41:46.607328+0000","last_clean":"2024-09-19T06:41:46.607328+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:41:46.607328+0000","last_undegraded":"2024-09-19T06:41:46.607328+0000","last_fullsized":"2024-09-19T06:41:46.607328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607462+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:41:46.607462+0000","last_peered":"2024-09-19T06:41:46.607462+0000","last_clean":"2024-09-19T06:41:46.607462+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:41:46.607462+0000","last_undegraded":"2024-09-19T06:41:46.607462+0000","last_fullsized":"2024-09-19T06:41:46.607462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860594+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:41:46.860594+0000","last_peered":"2024-09-19T06:41:46.860594+0000","last_clean":"2024-09-19T06:41:46.860594+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:41:46.860594+0000","last_undegraded":"2024-09-19T06:41:46.860594+0000","last_fullsized":"2024-09-19T06:41:46.860594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587338+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:41:46.587338+0000","last_peered":"2024-09-19T06:41:46.587338+0000","last_clean":"2024-09-19T06:41:46.587338+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:41:46.587338+0000","last_undegraded":"2024-09-19T06:41:46.587338+0000","last_fullsized":"2024-09-19T06:41:46.587338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:56.619617+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:41:56.619617+0000","last_peered":"2024-09-19T06:41:56.619617+0000","last_clean":"2024-09-19T06:41:56.619617+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:41:56.619617+0000","last_undegraded":"2024-09-19T06:41:56.619617+0000","last_fullsized":"2024-09-19T06:41:56.619617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677201+0000","last_change":"2024-09-19T06:40:58.132715+0000","last_active":"2024-09-19T06:41:46.677201+0000","last_peered":"2024-09-19T06:41:46.677201+0000","last_clean":"2024-09-19T06:41:46.677201+0000","last_became_active":"2024-09-19T06:40:58.132333+0000","last_became_peered":"2024-09-19T06:40:58.132333+0000","last_unstale":"2024-09-19T06:41:46.677201+0000","last_undegraded":"2024-09-19T06:41:46.677201+0000","last_fullsized":"2024-09-19T06:41:46.677201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:03.256998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860671+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:41:46.860671+0000","last_peered":"2024-09-19T06:41:46.860671+0000","last_clean":"2024-09-19T06:41:46.860671+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:41:46.860671+0000","last_undegraded":"2024-09-19T06:41:46.860671+0000","last_fullsized":"2024-09-19T06:41:46.860671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590285+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:41:46.590285+0000","last_peered":"2024-09-19T06:41:46.590285+0000","last_clean":"2024-09-19T06:41:46.590285+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:41:46.590285+0000","last_undegraded":"2024-09-19T06:41:46.590285+0000","last_fullsized":"2024-09-19T06:41:46.590285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677581+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:41:46.677581+0000","last_peered":"2024-09-19T06:41:46.677581+0000","last_clean":"2024-09-19T06:41:46.677581+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:41:46.677581+0000","last_undegraded":"2024-09-19T06:41:46.677581+0000","last_fullsized":"2024-09-19T06:41:46.677581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590332+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:41:46.590332+0000","last_peered":"2024-09-19T06:41:46.590332+0000","last_clean":"2024-09-19T06:41:46.590332+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:41:46.590332+0000","last_undegraded":"2024-09-19T06:41:46.590332+0000","last_fullsized":"2024-09-19T06:41:46.590332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933826+0000","last_change":"2024-09-19T06:40:58.140944+0000","last_active":"2024-09-19T06:41:46.933826+0000","last_peered":"2024-09-19T06:41:46.933826+0000","last_clean":"2024-09-19T06:41:46.933826+0000","last_became_active":"2024-09-19T06:40:58.132427+0000","last_became_peered":"2024-09-19T06:40:58.132427+0000","last_unstale":"2024-09-19T06:41:46.933826+0000","last_undegraded":"2024-09-19T06:41:46.933826+0000","last_fullsized":"2024-09-19T06:41:46.933826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:32.472242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587689+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:41:46.587689+0000","last_peered":"2024-09-19T06:41:46.587689+0000","last_clean":"2024-09-19T06:41:46.587689+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:41:46.587689+0000","last_undegraded":"2024-09-19T06:41:46.587689+0000","last_fullsized":"2024-09-19T06:41:46.587689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'97","reported_seq":208,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:57.385896+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:41:57.385896+0000","last_peered":"2024-09-19T06:41:57.385896+0000","last_clean":"2024-09-19T06:41:57.385896+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:41:57.385896+0000","last_undegraded":"2024-09-19T06:41:57.385896+0000","last_fullsized":"2024-09-19T06:41:57.385896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":177,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607761+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:41:46.607761+0000","last_peered":"2024-09-19T06:41:46.607761+0000","last_clean":"2024-09-19T06:41:46.607761+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:41:46.607761+0000","last_undegraded":"2024-09-19T06:41:46.607761+0000","last_fullsized":"2024-09-19T06:41:46.607761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607492+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:41:46.607492+0000","last_peered":"2024-09-19T06:41:46.607492+0000","last_clean":"2024-09-19T06:41:46.607492+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:41:46.607492+0000","last_undegraded":"2024-09-19T06:41:46.607492+0000","last_fullsized":"2024-09-19T06:41:46.607492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866926+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:41:46.866926+0000","last_peered":"2024-09-19T06:41:46.866926+0000","last_clean":"2024-09-19T06:41:46.866926+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:41:46.866926+0000","last_undegraded":"2024-09-19T06:41:46.866926+0000","last_fullsized":"2024-09-19T06:41:46.866926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609037+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:41:46.609037+0000","last_peered":"2024-09-19T06:41:46.609037+0000","last_clean":"2024-09-19T06:41:46.609037+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:41:46.609037+0000","last_undegraded":"2024-09-19T06:41:46.609037+0000","last_fullsized":"2024-09-19T06:41:46.609037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679748+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:41:46.679748+0000","last_peered":"2024-09-19T06:41:46.679748+0000","last_clean":"2024-09-19T06:41:46.679748+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:41:46.679748+0000","last_undegraded":"2024-09-19T06:41:46.679748+0000","last_fullsized":"2024-09-19T06:41:46.679748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607361+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:41:46.607361+0000","last_peered":"2024-09-19T06:41:46.607361+0000","last_clean":"2024-09-19T06:41:46.607361+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:41:46.607361+0000","last_undegraded":"2024-09-19T06:41:46.607361+0000","last_fullsized":"2024-09-19T06:41:46.607361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933699+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:41:46.933699+0000","last_peered":"2024-09-19T06:41:46.933699+0000","last_clean":"2024-09-19T06:41:46.933699+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:41:46.933699+0000","last_undegraded":"2024-09-19T06:41:46.933699+0000","last_fullsized":"2024-09-19T06:41:46.933699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609110+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:41:46.609110+0000","last_peered":"2024-09-19T06:41:46.609110+0000","last_clean":"2024-09-19T06:41:46.609110+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:41:46.609110+0000","last_undegraded":"2024-09-19T06:41:46.609110+0000","last_fullsized":"2024-09-19T06:41:46.609110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607487+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:41:46.607487+0000","last_peered":"2024-09-19T06:41:46.607487+0000","last_clean":"2024-09-19T06:41:46.607487+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:41:46.607487+0000","last_undegraded":"2024-09-19T06:41:46.607487+0000","last_fullsized":"2024-09-19T06:41:46.607487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679819+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:41:46.679819+0000","last_peered":"2024-09-19T06:41:46.679819+0000","last_clean":"2024-09-19T06:41:46.679819+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:41:46.679819+0000","last_undegraded":"2024-09-19T06:41:46.679819+0000","last_fullsized":"2024-09-19T06:41:46.679819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860513+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:41:46.860513+0000","last_peered":"2024-09-19T06:41:46.860513+0000","last_clean":"2024-09-19T06:41:46.860513+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:41:46.860513+0000","last_undegraded":"2024-09-19T06:41:46.860513+0000","last_fullsized":"2024-09-19T06:41:46.860513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588680+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:41:46.588680+0000","last_peered":"2024-09-19T06:41:46.588680+0000","last_clean":"2024-09-19T06:41:46.588680+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:41:46.588680+0000","last_undegraded":"2024-09-19T06:41:46.588680+0000","last_fullsized":"2024-09-19T06:41:46.588680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680018+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:41:46.680018+0000","last_peered":"2024-09-19T06:41:46.680018+0000","last_clean":"2024-09-19T06:41:46.680018+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:41:46.680018+0000","last_undegraded":"2024-09-19T06:41:46.680018+0000","last_fullsized":"2024-09-19T06:41:46.680018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609092+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:41:46.609092+0000","last_peered":"2024-09-19T06:41:46.609092+0000","last_clean":"2024-09-19T06:41:46.609092+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:41:46.609092+0000","last_undegraded":"2024-09-19T06:41:46.609092+0000","last_fullsized":"2024-09-19T06:41:46.609092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680073+0000","last_change":"2024-09-19T06:40:58.139755+0000","last_active":"2024-09-19T06:41:46.680073+0000","last_peered":"2024-09-19T06:41:46.680073+0000","last_clean":"2024-09-19T06:41:46.680073+0000","last_became_active":"2024-09-19T06:40:58.135411+0000","last_became_peered":"2024-09-19T06:40:58.135411+0000","last_unstale":"2024-09-19T06:41:46.680073+0000","last_undegraded":"2024-09-19T06:41:46.680073+0000","last_fullsized":"2024-09-19T06:41:46.680073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:38.951366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867245+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:41:46.867245+0000","last_peered":"2024-09-19T06:41:46.867245+0000","last_clean":"2024-09-19T06:41:46.867245+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:41:46.867245+0000","last_undegraded":"2024-09-19T06:41:46.867245+0000","last_fullsized":"2024-09-19T06:41:46.867245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607906+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:41:46.607906+0000","last_peered":"2024-09-19T06:41:46.607906+0000","last_clean":"2024-09-19T06:41:46.607906+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:41:46.607906+0000","last_undegraded":"2024-09-19T06:41:46.607906+0000","last_fullsized":"2024-09-19T06:41:46.607906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607785+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:41:46.607785+0000","last_peered":"2024-09-19T06:41:46.607785+0000","last_clean":"2024-09-19T06:41:46.607785+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:41:46.607785+0000","last_undegraded":"2024-09-19T06:41:46.607785+0000","last_fullsized":"2024-09-19T06:41:46.607785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:52.602852+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:41:52.602852+0000","last_peered":"2024-09-19T06:41:52.602852+0000","last_clean":"2024-09-19T06:41:52.602852+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:41:52.602852+0000","last_undegraded":"2024-09-19T06:41:52.602852+0000","last_fullsized":"2024-09-19T06:41:52.602852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797450,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1051,"num_read_kb":734,"num_write":418,"num_write_kb":2360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2392350,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":446248,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":1437237,"data_compressed":59704,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":839678,"data_compressed":27805,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":839576,"data_compressed":27805,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1040335,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445851,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1366931,"data_compressed":59342,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":448190,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1167610,"data_compressed":42774,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":517484,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":515622,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":447122,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330802,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":401108,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:42:00.190 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:42:00.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:41:59 smithi096 ceph-mon[35004]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-19T06:42:00.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:41:59 smithi151 ceph-mon[35236]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 84 KiB/s wr, 49 op/s 2024-09-19T06:42:00.460 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:00.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:42:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:42:00.396+0000 7f15260b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:42:00.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:42:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:42:00.396+0000 7f15260b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:42:00.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:42:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:42:00.396+0000 7f15260b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:42:00.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:42:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:42:00.396+0000 7f15260b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:42:00.525 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:42:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x[31435]: 2024-09-19T06:42:00.396+0000 7f15260b1640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-19T06:42:00.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.899+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/3682825079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28102790 msgr2=0x7fbb281067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.899+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/3682825079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28102790 0x7fbb281067e0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fbb18009900 tx=0x7fbb1802a410 comp rx=0 tx=0).stop 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/3682825079 shutdown_connections 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/3682825079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb28106f10 0x7fbb2810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/3682825079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28102790 0x7fbb281067e0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/3682825079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb28101de0 0x7fbb281021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/3682825079 >> 172.21.15.37:0/3682825079 conn(0x7fbb280fd8c0 msgr2=0x7fbb280ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/3682825079 shutdown_connections 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/3682825079 wait complete. 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 Processor -- start 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.900+0000 7fbb2f9dc640 1 -- start start 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2f9dc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb28101de0 0x7fbb280744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2f9dc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb28102790 0x7fbb28074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2f9dc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28106f10 0x7fbb28074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2f9dc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28111f90 con 0x7fbb28106f10 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2f9dc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28111e10 con 0x7fbb28101de0 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2f9dc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb28112110 con 0x7fbb28102790 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28106f10 0x7fbb28074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28106f10 0x7fbb28074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41864/0 (socket says 172.21.15.37:41864) 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 -- 172.21.15.37:0/823178616 learned_addr learned my addr 172.21.15.37:0/823178616 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2d778640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb28101de0 0x7fbb280744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2cf77640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb28102790 0x7fbb28074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 -- 172.21.15.37:0/823178616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb28102790 msgr2=0x7fbb28074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb28102790 0x7fbb28074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 -- 172.21.15.37:0/823178616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb28101de0 msgr2=0x7fbb280744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb28101de0 0x7fbb280744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 -- 172.21.15.37:0/823178616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb281a6950 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.901+0000 7fbb2df79640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28106f10 0x7fbb28074f70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000eaa0 tx=0x7fbb2000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.902+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb20004d20 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.902+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb281a6c40 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.902+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb20005020 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.902+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb200116d0 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.902+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb281a7180 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.903+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb281021c0 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.904+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 24) ==== 138582+0+0 (secure 0 0 0) 0x7fbb200051d0 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.904+0000 7fbb1e7fc640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbb140a34d0 0x7fbb140a5990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.905+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fbb200d0240 con 0x7fbb28106f10 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.905+0000 7fbb2d778640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbb140a34d0 0x7fbb140a5990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:00.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.905+0000 7fbb2d778640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbb140a34d0 0x7fbb140a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009990 tx=0x7fbb24009110 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:00.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.909+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbb20016030 con 0x7fbb28106f10 2024-09-19T06:42:00.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:00.935+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fbb20098940 con 0x7fbb28106f10 2024-09-19T06:42:01.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.086+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbb28062630 con 0x7fbb140a34d0 2024-09-19T06:42:01.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.092+0000 7fbb1e7fc640 1 -- 172.21.15.37:0/823178616 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258037 (secure 0 0 0) 0x7fbb28062630 con 0x7fbb140a34d0 2024-09-19T06:42:01.096 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:01.097 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbb140a34d0 msgr2=0x7fbb140a5990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbb140a34d0 0x7fbb140a5990 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbb24009990 tx=0x7fbb24009110 comp rx=0 tx=0).stop 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28106f10 msgr2=0x7fbb28074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28106f10 0x7fbb28074f70 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000eaa0 tx=0x7fbb2000eea0 comp rx=0 tx=0).stop 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 shutdown_connections 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbb140a34d0 0x7fbb140a5990 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb28106f10 0x7fbb28074f70 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb28102790 0x7fbb28074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 --2- 172.21.15.37:0/823178616 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb28101de0 0x7fbb280744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 >> 172.21.15.37:0/823178616 conn(0x7fbb280fd8c0 msgr2=0x7fbb280fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 shutdown_connections 2024-09-19T06:42:01.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:01.095+0000 7fbb2f9dc640 1 -- 172.21.15.37:0/823178616 wait complete. 2024-09-19T06:42:01.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:00 smithi096 ceph-mon[35004]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:01.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:00 smithi037 ceph-mon[31211]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:01.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:00 smithi151 ceph-mon[35236]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:01.907 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-19T06:42:00.605826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1388052,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1288,"num_read_kb":1064,"num_write":621,"num_write_kb":5300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":959,"ondisk_log_size":959,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364848,"kb_used_data":11412,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124576976,"statfs":{"total":1151940427776,"available":1151566823424,"internally_reserved":0,"allocated":11685888,"data_stored":9511884,"data_compressed":313886,"data_compressed_allocated":417792,"data_compressed_original":4165632,"omap_allocated":19143,"internal_metadata":361674041},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":201076,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":401,"num_read_kb":469,"num_write":146,"num_write_kb":923,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005399"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934075+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:41:46.934075+0000","last_peered":"2024-09-19T06:41:46.934075+0000","last_clean":"2024-09-19T06:41:46.934075+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:41:46.934075+0000","last_undegraded":"2024-09-19T06:41:46.934075+0000","last_fullsized":"2024-09-19T06:41:46.934075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866847+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:41:46.866847+0000","last_peered":"2024-09-19T06:41:46.866847+0000","last_clean":"2024-09-19T06:41:46.866847+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:41:46.866847+0000","last_undegraded":"2024-09-19T06:41:46.866847+0000","last_fullsized":"2024-09-19T06:41:46.866847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866554+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:41:46.866554+0000","last_peered":"2024-09-19T06:41:46.866554+0000","last_clean":"2024-09-19T06:41:46.866554+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:41:46.866554+0000","last_undegraded":"2024-09-19T06:41:46.866554+0000","last_fullsized":"2024-09-19T06:41:46.866554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607461+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:41:46.607461+0000","last_peered":"2024-09-19T06:41:46.607461+0000","last_clean":"2024-09-19T06:41:46.607461+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:41:46.607461+0000","last_undegraded":"2024-09-19T06:41:46.607461+0000","last_fullsized":"2024-09-19T06:41:46.607461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679815+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:41:46.679815+0000","last_peered":"2024-09-19T06:41:46.679815+0000","last_clean":"2024-09-19T06:41:46.679815+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:41:46.679815+0000","last_undegraded":"2024-09-19T06:41:46.679815+0000","last_fullsized":"2024-09-19T06:41:46.679815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590191+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:41:46.590191+0000","last_peered":"2024-09-19T06:41:46.590191+0000","last_clean":"2024-09-19T06:41:46.590191+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:41:46.590191+0000","last_undegraded":"2024-09-19T06:41:46.590191+0000","last_fullsized":"2024-09-19T06:41:46.590191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860437+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:41:46.860437+0000","last_peered":"2024-09-19T06:41:46.860437+0000","last_clean":"2024-09-19T06:41:46.860437+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:41:46.860437+0000","last_undegraded":"2024-09-19T06:41:46.860437+0000","last_fullsized":"2024-09-19T06:41:46.860437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607193+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:41:46.607193+0000","last_peered":"2024-09-19T06:41:46.607193+0000","last_clean":"2024-09-19T06:41:46.607193+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:41:46.607193+0000","last_undegraded":"2024-09-19T06:41:46.607193+0000","last_fullsized":"2024-09-19T06:41:46.607193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609566+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:41:46.609566+0000","last_peered":"2024-09-19T06:41:46.609566+0000","last_clean":"2024-09-19T06:41:46.609566+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:41:46.609566+0000","last_undegraded":"2024-09-19T06:41:46.609566+0000","last_fullsized":"2024-09-19T06:41:46.609566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590543+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:41:46.590543+0000","last_peered":"2024-09-19T06:41:46.590543+0000","last_clean":"2024-09-19T06:41:46.590543+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:41:46.590543+0000","last_undegraded":"2024-09-19T06:41:46.590543+0000","last_fullsized":"2024-09-19T06:41:46.590543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860924+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:41:46.860924+0000","last_peered":"2024-09-19T06:41:46.860924+0000","last_clean":"2024-09-19T06:41:46.860924+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:41:46.860924+0000","last_undegraded":"2024-09-19T06:41:46.860924+0000","last_fullsized":"2024-09-19T06:41:46.860924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607557+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:41:46.607557+0000","last_peered":"2024-09-19T06:41:46.607557+0000","last_clean":"2024-09-19T06:41:46.607557+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:41:46.607557+0000","last_undegraded":"2024-09-19T06:41:46.607557+0000","last_fullsized":"2024-09-19T06:41:46.607557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587632+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:41:46.587632+0000","last_peered":"2024-09-19T06:41:46.587632+0000","last_clean":"2024-09-19T06:41:46.587632+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:41:46.587632+0000","last_undegraded":"2024-09-19T06:41:46.587632+0000","last_fullsized":"2024-09-19T06:41:46.587632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866079+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:41:46.866079+0000","last_peered":"2024-09-19T06:41:46.866079+0000","last_clean":"2024-09-19T06:41:46.866079+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:41:46.866079+0000","last_undegraded":"2024-09-19T06:41:46.866079+0000","last_fullsized":"2024-09-19T06:41:46.866079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.865755+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:41:46.865755+0000","last_peered":"2024-09-19T06:41:46.865755+0000","last_clean":"2024-09-19T06:41:46.865755+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:41:46.865755+0000","last_undegraded":"2024-09-19T06:41:46.865755+0000","last_fullsized":"2024-09-19T06:41:46.865755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609034+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:41:46.609034+0000","last_peered":"2024-09-19T06:41:46.609034+0000","last_clean":"2024-09-19T06:41:46.609034+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:41:46.609034+0000","last_undegraded":"2024-09-19T06:41:46.609034+0000","last_fullsized":"2024-09-19T06:41:46.609034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934022+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:41:46.934022+0000","last_peered":"2024-09-19T06:41:46.934022+0000","last_clean":"2024-09-19T06:41:46.934022+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:41:46.934022+0000","last_undegraded":"2024-09-19T06:41:46.934022+0000","last_fullsized":"2024-09-19T06:41:46.934022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677589+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:41:46.677589+0000","last_peered":"2024-09-19T06:41:46.677589+0000","last_clean":"2024-09-19T06:41:46.677589+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:41:46.677589+0000","last_undegraded":"2024-09-19T06:41:46.677589+0000","last_fullsized":"2024-09-19T06:41:46.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609334+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:41:46.609334+0000","last_peered":"2024-09-19T06:41:46.609334+0000","last_clean":"2024-09-19T06:41:46.609334+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:41:46.609334+0000","last_undegraded":"2024-09-19T06:41:46.609334+0000","last_fullsized":"2024-09-19T06:41:46.609334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933853+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:41:46.933853+0000","last_peered":"2024-09-19T06:41:46.933853+0000","last_clean":"2024-09-19T06:41:46.933853+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:41:46.933853+0000","last_undegraded":"2024-09-19T06:41:46.933853+0000","last_fullsized":"2024-09-19T06:41:46.933853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587643+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:41:46.587643+0000","last_peered":"2024-09-19T06:41:46.587643+0000","last_clean":"2024-09-19T06:41:46.587643+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:41:46.587643+0000","last_undegraded":"2024-09-19T06:41:46.587643+0000","last_fullsized":"2024-09-19T06:41:46.587643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609138+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:41:46.609138+0000","last_peered":"2024-09-19T06:41:46.609138+0000","last_clean":"2024-09-19T06:41:46.609138+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:41:46.609138+0000","last_undegraded":"2024-09-19T06:41:46.609138+0000","last_fullsized":"2024-09-19T06:41:46.609138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607201+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:41:46.607201+0000","last_peered":"2024-09-19T06:41:46.607201+0000","last_clean":"2024-09-19T06:41:46.607201+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:41:46.607201+0000","last_undegraded":"2024-09-19T06:41:46.607201+0000","last_fullsized":"2024-09-19T06:41:46.607201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588420+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:41:46.588420+0000","last_peered":"2024-09-19T06:41:46.588420+0000","last_clean":"2024-09-19T06:41:46.588420+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:41:46.588420+0000","last_undegraded":"2024-09-19T06:41:46.588420+0000","last_fullsized":"2024-09-19T06:41:46.588420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677514+0000","last_change":"2024-09-19T06:40:58.134245+0000","last_active":"2024-09-19T06:41:46.677514+0000","last_peered":"2024-09-19T06:41:46.677514+0000","last_clean":"2024-09-19T06:41:46.677514+0000","last_became_active":"2024-09-19T06:40:58.133514+0000","last_became_peered":"2024-09-19T06:40:58.133514+0000","last_unstale":"2024-09-19T06:41:46.677514+0000","last_undegraded":"2024-09-19T06:41:46.677514+0000","last_fullsized":"2024-09-19T06:41:46.677514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:23.267466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609405+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:41:46.609405+0000","last_peered":"2024-09-19T06:41:46.609405+0000","last_clean":"2024-09-19T06:41:46.609405+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:41:46.609405+0000","last_undegraded":"2024-09-19T06:41:46.609405+0000","last_fullsized":"2024-09-19T06:41:46.609405+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933946+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:41:46.933946+0000","last_peered":"2024-09-19T06:41:46.933946+0000","last_clean":"2024-09-19T06:41:46.933946+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:41:46.933946+0000","last_undegraded":"2024-09-19T06:41:46.933946+0000","last_fullsized":"2024-09-19T06:41:46.933946+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587547+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:41:46.587547+0000","last_peered":"2024-09-19T06:41:46.587547+0000","last_clean":"2024-09-19T06:41:46.587547+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:41:46.587547+0000","last_undegraded":"2024-09-19T06:41:46.587547+0000","last_fullsized":"2024-09-19T06:41:46.587547+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866245+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:41:46.866245+0000","last_peered":"2024-09-19T06:41:46.866245+0000","last_clean":"2024-09-19T06:41:46.866245+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:41:46.866245+0000","last_undegraded":"2024-09-19T06:41:46.866245+0000","last_fullsized":"2024-09-19T06:41:46.866245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609084+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:41:46.609084+0000","last_peered":"2024-09-19T06:41:46.609084+0000","last_clean":"2024-09-19T06:41:46.609084+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:41:46.609084+0000","last_undegraded":"2024-09-19T06:41:46.609084+0000","last_fullsized":"2024-09-19T06:41:46.609084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609093+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:41:46.609093+0000","last_peered":"2024-09-19T06:41:46.609093+0000","last_clean":"2024-09-19T06:41:46.609093+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:41:46.609093+0000","last_undegraded":"2024-09-19T06:41:46.609093+0000","last_fullsized":"2024-09-19T06:41:46.609093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933721+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:41:46.933721+0000","last_peered":"2024-09-19T06:41:46.933721+0000","last_clean":"2024-09-19T06:41:46.933721+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:41:46.933721+0000","last_undegraded":"2024-09-19T06:41:46.933721+0000","last_fullsized":"2024-09-19T06:41:46.933721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679766+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:41:46.679766+0000","last_peered":"2024-09-19T06:41:46.679766+0000","last_clean":"2024-09-19T06:41:46.679766+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:41:46.679766+0000","last_undegraded":"2024-09-19T06:41:46.679766+0000","last_fullsized":"2024-09-19T06:41:46.679766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:55.619074+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:41:55.619074+0000","last_peered":"2024-09-19T06:41:55.619074+0000","last_clean":"2024-09-19T06:41:55.619074+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:41:55.619074+0000","last_undegraded":"2024-09-19T06:41:55.619074+0000","last_fullsized":"2024-09-19T06:41:55.619074+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":119,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588409+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:41:46.588409+0000","last_peered":"2024-09-19T06:41:46.588409+0000","last_clean":"2024-09-19T06:41:46.588409+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:41:46.588409+0000","last_undegraded":"2024-09-19T06:41:46.588409+0000","last_fullsized":"2024-09-19T06:41:46.588409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590296+0000","last_change":"2024-09-19T06:40:58.136626+0000","last_active":"2024-09-19T06:41:46.590296+0000","last_peered":"2024-09-19T06:41:46.590296+0000","last_clean":"2024-09-19T06:41:46.590296+0000","last_became_active":"2024-09-19T06:40:58.132752+0000","last_became_peered":"2024-09-19T06:40:58.132752+0000","last_unstale":"2024-09-19T06:41:46.590296+0000","last_undegraded":"2024-09-19T06:41:46.590296+0000","last_fullsized":"2024-09-19T06:41:46.590296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587271+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:41:46.587271+0000","last_peered":"2024-09-19T06:41:46.587271+0000","last_clean":"2024-09-19T06:41:46.587271+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:41:46.587271+0000","last_undegraded":"2024-09-19T06:41:46.587271+0000","last_fullsized":"2024-09-19T06:41:46.587271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607281+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:41:46.607281+0000","last_peered":"2024-09-19T06:41:46.607281+0000","last_clean":"2024-09-19T06:41:46.607281+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:41:46.607281+0000","last_undegraded":"2024-09-19T06:41:46.607281+0000","last_fullsized":"2024-09-19T06:41:46.607281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588546+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:41:46.588546+0000","last_peered":"2024-09-19T06:41:46.588546+0000","last_clean":"2024-09-19T06:41:46.588546+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:41:46.588546+0000","last_undegraded":"2024-09-19T06:41:46.588546+0000","last_fullsized":"2024-09-19T06:41:46.588546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677238+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:41:46.677238+0000","last_peered":"2024-09-19T06:41:46.677238+0000","last_clean":"2024-09-19T06:41:46.677238+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:41:46.677238+0000","last_undegraded":"2024-09-19T06:41:46.677238+0000","last_fullsized":"2024-09-19T06:41:46.677238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":143,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:55.618471+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:41:55.618471+0000","last_peered":"2024-09-19T06:41:55.618471+0000","last_clean":"2024-09-19T06:41:55.618471+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:41:55.618471+0000","last_undegraded":"2024-09-19T06:41:55.618471+0000","last_fullsized":"2024-09-19T06:41:55.618471+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609442+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:41:46.609442+0000","last_peered":"2024-09-19T06:41:46.609442+0000","last_clean":"2024-09-19T06:41:46.609442+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:41:46.609442+0000","last_undegraded":"2024-09-19T06:41:46.609442+0000","last_fullsized":"2024-09-19T06:41:46.609442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860979+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:41:46.860979+0000","last_peered":"2024-09-19T06:41:46.860979+0000","last_clean":"2024-09-19T06:41:46.860979+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:41:46.860979+0000","last_undegraded":"2024-09-19T06:41:46.860979+0000","last_fullsized":"2024-09-19T06:41:46.860979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867221+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:41:46.867221+0000","last_peered":"2024-09-19T06:41:46.867221+0000","last_clean":"2024-09-19T06:41:46.867221+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:41:46.867221+0000","last_undegraded":"2024-09-19T06:41:46.867221+0000","last_fullsized":"2024-09-19T06:41:46.867221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588728+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:41:46.588728+0000","last_peered":"2024-09-19T06:41:46.588728+0000","last_clean":"2024-09-19T06:41:46.588728+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:41:46.588728+0000","last_undegraded":"2024-09-19T06:41:46.588728+0000","last_fullsized":"2024-09-19T06:41:46.588728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609314+0000","last_change":"2024-09-19T06:40:58.135595+0000","last_active":"2024-09-19T06:41:46.609314+0000","last_peered":"2024-09-19T06:41:46.609314+0000","last_clean":"2024-09-19T06:41:46.609314+0000","last_became_active":"2024-09-19T06:40:58.135031+0000","last_became_peered":"2024-09-19T06:40:58.135031+0000","last_unstale":"2024-09-19T06:41:46.609314+0000","last_undegraded":"2024-09-19T06:41:46.609314+0000","last_fullsized":"2024-09-19T06:41:46.609314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:53.542630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'37","reported_seq":306,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:56.620569+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:41:56.620569+0000","last_peered":"2024-09-19T06:41:56.620569+0000","last_clean":"2024-09-19T06:41:56.620569+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:41:56.620569+0000","last_undegraded":"2024-09-19T06:41:56.620569+0000","last_fullsized":"2024-09-19T06:41:56.620569+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69978,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":245,"num_read_kb":192,"num_write":38,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680048+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:41:46.680048+0000","last_peered":"2024-09-19T06:41:46.680048+0000","last_clean":"2024-09-19T06:41:46.680048+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:41:46.680048+0000","last_undegraded":"2024-09-19T06:41:46.680048+0000","last_fullsized":"2024-09-19T06:41:46.680048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609208+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:41:46.609208+0000","last_peered":"2024-09-19T06:41:46.609208+0000","last_clean":"2024-09-19T06:41:46.609208+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:41:46.609208+0000","last_undegraded":"2024-09-19T06:41:46.609208+0000","last_fullsized":"2024-09-19T06:41:46.609208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587384+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:41:46.587384+0000","last_peered":"2024-09-19T06:41:46.587384+0000","last_clean":"2024-09-19T06:41:46.587384+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:41:46.587384+0000","last_undegraded":"2024-09-19T06:41:46.587384+0000","last_fullsized":"2024-09-19T06:41:46.587384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933722+0000","last_change":"2024-09-19T06:40:58.143264+0000","last_active":"2024-09-19T06:41:46.933722+0000","last_peered":"2024-09-19T06:41:46.933722+0000","last_clean":"2024-09-19T06:41:46.933722+0000","last_became_active":"2024-09-19T06:40:58.135400+0000","last_became_peered":"2024-09-19T06:40:58.135400+0000","last_unstale":"2024-09-19T06:41:46.933722+0000","last_undegraded":"2024-09-19T06:41:46.933722+0000","last_fullsized":"2024-09-19T06:41:46.933722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:30.500765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860840+0000","last_change":"2024-09-19T06:40:58.134136+0000","last_active":"2024-09-19T06:41:46.860840+0000","last_peered":"2024-09-19T06:41:46.860840+0000","last_clean":"2024-09-19T06:41:46.860840+0000","last_became_active":"2024-09-19T06:40:58.133734+0000","last_became_peered":"2024-09-19T06:40:58.133734+0000","last_unstale":"2024-09-19T06:41:46.860840+0000","last_undegraded":"2024-09-19T06:41:46.860840+0000","last_fullsized":"2024-09-19T06:41:46.860840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:50.822670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:55.618680+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:41:55.618680+0000","last_peered":"2024-09-19T06:41:55.618680+0000","last_clean":"2024-09-19T06:41:55.618680+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:41:55.618680+0000","last_undegraded":"2024-09-19T06:41:55.618680+0000","last_fullsized":"2024-09-19T06:41:55.618680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860373+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:41:46.860373+0000","last_peered":"2024-09-19T06:41:46.860373+0000","last_clean":"2024-09-19T06:41:46.860373+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:41:46.860373+0000","last_undegraded":"2024-09-19T06:41:46.860373+0000","last_fullsized":"2024-09-19T06:41:46.860373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679751+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:41:46.679751+0000","last_peered":"2024-09-19T06:41:46.679751+0000","last_clean":"2024-09-19T06:41:46.679751+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:41:46.679751+0000","last_undegraded":"2024-09-19T06:41:46.679751+0000","last_fullsized":"2024-09-19T06:41:46.679751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588476+0000","last_change":"2024-09-19T06:40:58.135978+0000","last_active":"2024-09-19T06:41:46.588476+0000","last_peered":"2024-09-19T06:41:46.588476+0000","last_clean":"2024-09-19T06:41:46.588476+0000","last_became_active":"2024-09-19T06:40:58.135581+0000","last_became_peered":"2024-09-19T06:40:58.135581+0000","last_unstale":"2024-09-19T06:41:46.588476+0000","last_undegraded":"2024-09-19T06:41:46.588476+0000","last_fullsized":"2024-09-19T06:41:46.588476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:54.532003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:47.456507+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:41:47.456507+0000","last_peered":"2024-09-19T06:41:47.456507+0000","last_clean":"2024-09-19T06:41:47.456507+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:41:47.456507+0000","last_undegraded":"2024-09-19T06:41:47.456507+0000","last_fullsized":"2024-09-19T06:41:47.456507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933860+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:41:46.933860+0000","last_peered":"2024-09-19T06:41:46.933860+0000","last_clean":"2024-09-19T06:41:46.933860+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:41:46.933860+0000","last_undegraded":"2024-09-19T06:41:46.933860+0000","last_fullsized":"2024-09-19T06:41:46.933860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587529+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:41:46.587529+0000","last_peered":"2024-09-19T06:41:46.587529+0000","last_clean":"2024-09-19T06:41:46.587529+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:41:46.587529+0000","last_undegraded":"2024-09-19T06:41:46.587529+0000","last_fullsized":"2024-09-19T06:41:46.587529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'65","reported_seq":118,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:57.377629+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:41:57.377629+0000","last_peered":"2024-09-19T06:41:57.377629+0000","last_clean":"2024-09-19T06:41:57.377629+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:41:57.377629+0000","last_undegraded":"2024-09-19T06:41:57.377629+0000","last_fullsized":"2024-09-19T06:41:57.377629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":32,"num_read_kb":31,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860872+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:41:46.860872+0000","last_peered":"2024-09-19T06:41:46.860872+0000","last_clean":"2024-09-19T06:41:46.860872+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:41:46.860872+0000","last_undegraded":"2024-09-19T06:41:46.860872+0000","last_fullsized":"2024-09-19T06:41:46.860872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607210+0000","last_change":"2024-09-19T06:40:58.138283+0000","last_active":"2024-09-19T06:41:46.607210+0000","last_peered":"2024-09-19T06:41:46.607210+0000","last_clean":"2024-09-19T06:41:46.607210+0000","last_became_active":"2024-09-19T06:40:58.132079+0000","last_became_peered":"2024-09-19T06:40:58.132079+0000","last_unstale":"2024-09-19T06:41:46.607210+0000","last_undegraded":"2024-09-19T06:41:46.607210+0000","last_fullsized":"2024-09-19T06:41:46.607210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:12.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":390,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:48.697056+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:41:48.697056+0000","last_peered":"2024-09-19T06:41:48.697056+0000","last_clean":"2024-09-19T06:41:48.697056+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:41:48.697056+0000","last_undegraded":"2024-09-19T06:41:48.697056+0000","last_fullsized":"2024-09-19T06:41:48.697056+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607328+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:41:46.607328+0000","last_peered":"2024-09-19T06:41:46.607328+0000","last_clean":"2024-09-19T06:41:46.607328+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:41:46.607328+0000","last_undegraded":"2024-09-19T06:41:46.607328+0000","last_fullsized":"2024-09-19T06:41:46.607328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607462+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:41:46.607462+0000","last_peered":"2024-09-19T06:41:46.607462+0000","last_clean":"2024-09-19T06:41:46.607462+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:41:46.607462+0000","last_undegraded":"2024-09-19T06:41:46.607462+0000","last_fullsized":"2024-09-19T06:41:46.607462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860594+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:41:46.860594+0000","last_peered":"2024-09-19T06:41:46.860594+0000","last_clean":"2024-09-19T06:41:46.860594+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:41:46.860594+0000","last_undegraded":"2024-09-19T06:41:46.860594+0000","last_fullsized":"2024-09-19T06:41:46.860594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587338+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:41:46.587338+0000","last_peered":"2024-09-19T06:41:46.587338+0000","last_clean":"2024-09-19T06:41:46.587338+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:41:46.587338+0000","last_undegraded":"2024-09-19T06:41:46.587338+0000","last_fullsized":"2024-09-19T06:41:46.587338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":146,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:56.619617+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:41:56.619617+0000","last_peered":"2024-09-19T06:41:56.619617+0000","last_clean":"2024-09-19T06:41:56.619617+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:41:56.619617+0000","last_undegraded":"2024-09-19T06:41:56.619617+0000","last_fullsized":"2024-09-19T06:41:56.619617+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677201+0000","last_change":"2024-09-19T06:40:58.132715+0000","last_active":"2024-09-19T06:41:46.677201+0000","last_peered":"2024-09-19T06:41:46.677201+0000","last_clean":"2024-09-19T06:41:46.677201+0000","last_became_active":"2024-09-19T06:40:58.132333+0000","last_became_peered":"2024-09-19T06:40:58.132333+0000","last_unstale":"2024-09-19T06:41:46.677201+0000","last_undegraded":"2024-09-19T06:41:46.677201+0000","last_fullsized":"2024-09-19T06:41:46.677201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:03.256998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860671+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:41:46.860671+0000","last_peered":"2024-09-19T06:41:46.860671+0000","last_clean":"2024-09-19T06:41:46.860671+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:41:46.860671+0000","last_undegraded":"2024-09-19T06:41:46.860671+0000","last_fullsized":"2024-09-19T06:41:46.860671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590285+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:41:46.590285+0000","last_peered":"2024-09-19T06:41:46.590285+0000","last_clean":"2024-09-19T06:41:46.590285+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:41:46.590285+0000","last_undegraded":"2024-09-19T06:41:46.590285+0000","last_fullsized":"2024-09-19T06:41:46.590285+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677581+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:41:46.677581+0000","last_peered":"2024-09-19T06:41:46.677581+0000","last_clean":"2024-09-19T06:41:46.677581+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:41:46.677581+0000","last_undegraded":"2024-09-19T06:41:46.677581+0000","last_fullsized":"2024-09-19T06:41:46.677581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590332+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:41:46.590332+0000","last_peered":"2024-09-19T06:41:46.590332+0000","last_clean":"2024-09-19T06:41:46.590332+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:41:46.590332+0000","last_undegraded":"2024-09-19T06:41:46.590332+0000","last_fullsized":"2024-09-19T06:41:46.590332+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933826+0000","last_change":"2024-09-19T06:40:58.140944+0000","last_active":"2024-09-19T06:41:46.933826+0000","last_peered":"2024-09-19T06:41:46.933826+0000","last_clean":"2024-09-19T06:41:46.933826+0000","last_became_active":"2024-09-19T06:40:58.132427+0000","last_became_peered":"2024-09-19T06:40:58.132427+0000","last_unstale":"2024-09-19T06:41:46.933826+0000","last_undegraded":"2024-09-19T06:41:46.933826+0000","last_fullsized":"2024-09-19T06:41:46.933826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:32.472242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587689+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:41:46.587689+0000","last_peered":"2024-09-19T06:41:46.587689+0000","last_clean":"2024-09-19T06:41:46.587689+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:41:46.587689+0000","last_undegraded":"2024-09-19T06:41:46.587689+0000","last_fullsized":"2024-09-19T06:41:46.587689+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'97","reported_seq":208,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:57.385896+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:41:57.385896+0000","last_peered":"2024-09-19T06:41:57.385896+0000","last_clean":"2024-09-19T06:41:57.385896+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:41:57.385896+0000","last_undegraded":"2024-09-19T06:41:57.385896+0000","last_fullsized":"2024-09-19T06:41:57.385896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":97,"log_dups_size":0,"ondisk_log_size":97,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":259,"num_write":177,"num_write_kb":1286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607761+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:41:46.607761+0000","last_peered":"2024-09-19T06:41:46.607761+0000","last_clean":"2024-09-19T06:41:46.607761+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:41:46.607761+0000","last_undegraded":"2024-09-19T06:41:46.607761+0000","last_fullsized":"2024-09-19T06:41:46.607761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607492+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:41:46.607492+0000","last_peered":"2024-09-19T06:41:46.607492+0000","last_clean":"2024-09-19T06:41:46.607492+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:41:46.607492+0000","last_undegraded":"2024-09-19T06:41:46.607492+0000","last_fullsized":"2024-09-19T06:41:46.607492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866926+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:41:46.866926+0000","last_peered":"2024-09-19T06:41:46.866926+0000","last_clean":"2024-09-19T06:41:46.866926+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:41:46.866926+0000","last_undegraded":"2024-09-19T06:41:46.866926+0000","last_fullsized":"2024-09-19T06:41:46.866926+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609037+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:41:46.609037+0000","last_peered":"2024-09-19T06:41:46.609037+0000","last_clean":"2024-09-19T06:41:46.609037+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:41:46.609037+0000","last_undegraded":"2024-09-19T06:41:46.609037+0000","last_fullsized":"2024-09-19T06:41:46.609037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679748+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:41:46.679748+0000","last_peered":"2024-09-19T06:41:46.679748+0000","last_clean":"2024-09-19T06:41:46.679748+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:41:46.679748+0000","last_undegraded":"2024-09-19T06:41:46.679748+0000","last_fullsized":"2024-09-19T06:41:46.679748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607361+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:41:46.607361+0000","last_peered":"2024-09-19T06:41:46.607361+0000","last_clean":"2024-09-19T06:41:46.607361+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:41:46.607361+0000","last_undegraded":"2024-09-19T06:41:46.607361+0000","last_fullsized":"2024-09-19T06:41:46.607361+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933699+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:41:46.933699+0000","last_peered":"2024-09-19T06:41:46.933699+0000","last_clean":"2024-09-19T06:41:46.933699+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:41:46.933699+0000","last_undegraded":"2024-09-19T06:41:46.933699+0000","last_fullsized":"2024-09-19T06:41:46.933699+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609110+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:41:46.609110+0000","last_peered":"2024-09-19T06:41:46.609110+0000","last_clean":"2024-09-19T06:41:46.609110+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:41:46.609110+0000","last_undegraded":"2024-09-19T06:41:46.609110+0000","last_fullsized":"2024-09-19T06:41:46.609110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607487+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:41:46.607487+0000","last_peered":"2024-09-19T06:41:46.607487+0000","last_clean":"2024-09-19T06:41:46.607487+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:41:46.607487+0000","last_undegraded":"2024-09-19T06:41:46.607487+0000","last_fullsized":"2024-09-19T06:41:46.607487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679819+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:41:46.679819+0000","last_peered":"2024-09-19T06:41:46.679819+0000","last_clean":"2024-09-19T06:41:46.679819+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:41:46.679819+0000","last_undegraded":"2024-09-19T06:41:46.679819+0000","last_fullsized":"2024-09-19T06:41:46.679819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860513+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:41:46.860513+0000","last_peered":"2024-09-19T06:41:46.860513+0000","last_clean":"2024-09-19T06:41:46.860513+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:41:46.860513+0000","last_undegraded":"2024-09-19T06:41:46.860513+0000","last_fullsized":"2024-09-19T06:41:46.860513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588680+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:41:46.588680+0000","last_peered":"2024-09-19T06:41:46.588680+0000","last_clean":"2024-09-19T06:41:46.588680+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:41:46.588680+0000","last_undegraded":"2024-09-19T06:41:46.588680+0000","last_fullsized":"2024-09-19T06:41:46.588680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680018+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:41:46.680018+0000","last_peered":"2024-09-19T06:41:46.680018+0000","last_clean":"2024-09-19T06:41:46.680018+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:41:46.680018+0000","last_undegraded":"2024-09-19T06:41:46.680018+0000","last_fullsized":"2024-09-19T06:41:46.680018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609092+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:41:46.609092+0000","last_peered":"2024-09-19T06:41:46.609092+0000","last_clean":"2024-09-19T06:41:46.609092+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:41:46.609092+0000","last_undegraded":"2024-09-19T06:41:46.609092+0000","last_fullsized":"2024-09-19T06:41:46.609092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680073+0000","last_change":"2024-09-19T06:40:58.139755+0000","last_active":"2024-09-19T06:41:46.680073+0000","last_peered":"2024-09-19T06:41:46.680073+0000","last_clean":"2024-09-19T06:41:46.680073+0000","last_became_active":"2024-09-19T06:40:58.135411+0000","last_became_peered":"2024-09-19T06:40:58.135411+0000","last_unstale":"2024-09-19T06:41:46.680073+0000","last_undegraded":"2024-09-19T06:41:46.680073+0000","last_fullsized":"2024-09-19T06:41:46.680073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:38.951366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867245+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:41:46.867245+0000","last_peered":"2024-09-19T06:41:46.867245+0000","last_clean":"2024-09-19T06:41:46.867245+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:41:46.867245+0000","last_undegraded":"2024-09-19T06:41:46.867245+0000","last_fullsized":"2024-09-19T06:41:46.867245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607906+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:41:46.607906+0000","last_peered":"2024-09-19T06:41:46.607906+0000","last_clean":"2024-09-19T06:41:46.607906+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:41:46.607906+0000","last_undegraded":"2024-09-19T06:41:46.607906+0000","last_fullsized":"2024-09-19T06:41:46.607906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607785+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:41:46.607785+0000","last_peered":"2024-09-19T06:41:46.607785+0000","last_clean":"2024-09-19T06:41:46.607785+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:41:46.607785+0000","last_undegraded":"2024-09-19T06:41:46.607785+0000","last_fullsized":"2024-09-19T06:41:46.607785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:52.602852+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:41:52.602852+0000","last_peered":"2024-09-19T06:41:52.602852+0000","last_clean":"2024-09-19T06:41:52.602852+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:41:52.602852+0000","last_undegraded":"2024-09-19T06:41:52.602852+0000","last_fullsized":"2024-09-19T06:41:52.602852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":797450,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1051,"num_read_kb":734,"num_write":418,"num_write_kb":2360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":651264,"data_stored":2392350,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547238,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":446248,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776171,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1114112,"data_stored":1437237,"data_compressed":59704,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939697,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32116,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713036,"statfs":{"total":95995035648,"available":95962148864,"internally_reserved":0,"allocated":950272,"data_stored":839678,"data_compressed":27805,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":839576,"data_compressed":27805,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1040335,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561088,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445851,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790020,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":1366931,"data_compressed":59342,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":448190,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182478,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":1167610,"data_compressed":42774,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411412,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":517484,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":515622,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738462,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":447122,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330802,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":401108,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:42:01.911 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-19T06:42:01.911 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-19T06:42:02.171 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:01 smithi096 ceph-mon[35004]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T06:42:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:01 smithi096 ceph-mon[35004]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-09-19T06:42:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:01 smithi096 ceph-mon[35004]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:01 smithi037 ceph-mon[31211]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T06:42:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:01 smithi037 ceph-mon[31211]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-09-19T06:42:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:01 smithi037 ceph-mon[31211]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:01 smithi151 ceph-mon[35236]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 45 op/s 2024-09-19T06:42:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:01 smithi151 ceph-mon[35236]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-09-19T06:42:02.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:01 smithi151 ceph-mon[35236]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:02.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 -- 172.21.15.37:0/1196714985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 msgr2=0x7f1194107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:02.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 --2- 172.21.15.37:0/1196714985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 0x7f1194107b90 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f117c009930 tx=0x7f117c02a420 comp rx=0 tx=0).stop 2024-09-19T06:42:02.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 -- 172.21.15.37:0/1196714985 shutdown_connections 2024-09-19T06:42:02.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 --2- 172.21.15.37:0/1196714985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11941082a0 0x7f119410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 --2- 172.21.15.37:0/1196714985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 0x7f1194107b90 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 --2- 172.21.15.37:0/1196714985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1194102d30 0x7f1194103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 -- 172.21.15.37:0/1196714985 >> 172.21.15.37:0/1196714985 conn(0x7f11940fc7d0 msgr2=0x7f11940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 -- 172.21.15.37:0/1196714985 shutdown_connections 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.603+0000 7f1198a5f640 1 -- 172.21.15.37:0/1196714985 wait complete. 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.604+0000 7f1198a5f640 1 Processor -- start 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.604+0000 7f1198a5f640 1 -- start start 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1198a5f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1194102d30 0x7f1194103c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1198a5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 0x7f1194104160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1198a5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11941082a0 0x7f11941046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1198a5f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11941120b0 con 0x7f11941036e0 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1198a5f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1194111f30 con 0x7f1194102d30 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1198a5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1194112230 con 0x7f11941082a0 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1194102d30 0x7f1194103c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:02.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1189d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 0x7f1194104160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1194102d30 0x7f1194103c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44086/0 (socket says 172.21.15.37:44086) 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 -- 172.21.15.37:0/2803193543 learned_addr learned my addr 172.21.15.37:0/2803193543 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1192d9d640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11941082a0 0x7f11941046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 -- 172.21.15.37:0/2803193543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11941082a0 msgr2=0x7f11941046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11941082a0 0x7f11941046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 -- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 msgr2=0x7f1194104160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 0x7f1194104160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f119259c640 1 -- 172.21.15.37:0/2803193543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11941aaed0 con 0x7f1194102d30 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.605+0000 7f1192d9d640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11941082a0 0x7f11941046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.606+0000 7f119259c640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1194102d30 0x7f1194103c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f118400d7d0 tx=0x7f118400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.606+0000 7f118bfff640 1 -- 172.21.15.37:0/2803193543 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184017070 con 0x7f1194102d30 2024-09-19T06:42:02.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.606+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11941ab1c0 con 0x7f1194102d30 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.606+0000 7f118bfff640 1 -- 172.21.15.37:0/2803193543 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1184005500 con 0x7f1194102d30 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.606+0000 7f118bfff640 1 -- 172.21.15.37:0/2803193543 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1184004e00 con 0x7f1194102d30 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.606+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11941ab6d0 con 0x7f1194102d30 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.607+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f119406a850 con 0x7f1194102d30 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.609+0000 7f118bfff640 1 -- 172.21.15.37:0/2803193543 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f1184009c30 con 0x7f1194102d30 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.609+0000 7f118bfff640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11800a3560 0x7f11800a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.609+0000 7f1189d9b640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11800a3560 0x7f11800a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.609+0000 7f118bfff640 1 -- 172.21.15.37:0/2803193543 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f11840cfee0 con 0x7f1194102d30 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.610+0000 7f1189d9b640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11800a3560 0x7f11800a5a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f117c02a940 tx=0x7f117c005a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:02.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.612+0000 7f118bfff640 1 -- 172.21.15.37:0/2803193543 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f118409c680 con 0x7f1194102d30 2024-09-19T06:42:02.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.786+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1194062630 con 0x7f11800a3560 2024-09-19T06:42:02.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.792+0000 7f118bfff640 1 -- 172.21.15.37:0/2803193543 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258045 (secure 0 0 0) 0x7f1194062630 con 0x7f11800a3560 2024-09-19T06:42:02.796 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:02.797 INFO:teuthology.orchestra.run.smithi037.stderr:dumped all 2024-09-19T06:42:02.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11800a3560 msgr2=0x7f11800a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11800a3560 0x7f11800a5a20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f117c02a940 tx=0x7f117c005a40 comp rx=0 tx=0).stop 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1194102d30 msgr2=0x7f1194103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1194102d30 0x7f1194103c20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f118400d7d0 tx=0x7f118400dc90 comp rx=0 tx=0).stop 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 shutdown_connections 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11941082a0 0x7f11941046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11800a3560 0x7f11800a5a20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11941036e0 0x7f1194104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 --2- 172.21.15.37:0/2803193543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1194102d30 0x7f1194103c20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 >> 172.21.15.37:0/2803193543 conn(0x7f11940fc7d0 msgr2=0x7f11940fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 shutdown_connections 2024-09-19T06:42:02.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:02.795+0000 7f1198a5f640 1 -- 172.21.15.37:0/2803193543 wait complete. 2024-09-19T06:42:03.518 INFO:teuthology.orchestra.run.smithi037.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-09-19T06:42:02.606162+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1389576,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1458,"num_read_kb":1212,"num_write":688,"num_write_kb":5567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1181340,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1000,"ondisk_log_size":1000,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369028,"kb_used_data":11420,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572796,"statfs":{"total":1151940427776,"available":1151562543104,"internally_reserved":0,"allocated":11694080,"data_stored":9515325,"data_compressed":314781,"data_compressed_allocated":417792,"data_compressed_original":4165632,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":202600,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":571,"num_read_kb":617,"num_write":213,"num_write_kb":1190,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005352"},"pg_stats":[{"pgid":"3.3f","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934075+0000","last_change":"2024-09-19T06:39:52.393774+0000","last_active":"2024-09-19T06:41:46.934075+0000","last_peered":"2024-09-19T06:41:46.934075+0000","last_clean":"2024-09-19T06:41:46.934075+0000","last_became_active":"2024-09-19T06:39:52.393399+0000","last_became_peered":"2024-09-19T06:39:52.393399+0000","last_unstale":"2024-09-19T06:41:46.934075+0000","last_undegraded":"2024-09-19T06:41:46.934075+0000","last_fullsized":"2024-09-19T06:41:46.934075+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:47:23.543248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866847+0000","last_change":"2024-09-19T06:39:52.393854+0000","last_active":"2024-09-19T06:41:46.866847+0000","last_peered":"2024-09-19T06:41:46.866847+0000","last_clean":"2024-09-19T06:41:46.866847+0000","last_became_active":"2024-09-19T06:39:52.393493+0000","last_became_peered":"2024-09-19T06:39:52.393493+0000","last_unstale":"2024-09-19T06:41:46.866847+0000","last_undegraded":"2024-09-19T06:41:46.866847+0000","last_fullsized":"2024-09-19T06:41:46.866847+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:21:29.650059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866554+0000","last_change":"2024-09-19T06:39:52.400844+0000","last_active":"2024-09-19T06:41:46.866554+0000","last_peered":"2024-09-19T06:41:46.866554+0000","last_clean":"2024-09-19T06:41:46.866554+0000","last_became_active":"2024-09-19T06:39:52.400329+0000","last_became_peered":"2024-09-19T06:39:52.400329+0000","last_unstale":"2024-09-19T06:41:46.866554+0000","last_undegraded":"2024-09-19T06:41:46.866554+0000","last_fullsized":"2024-09-19T06:41:46.866554+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:54:43.385035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607461+0000","last_change":"2024-09-19T06:39:52.397599+0000","last_active":"2024-09-19T06:41:46.607461+0000","last_peered":"2024-09-19T06:41:46.607461+0000","last_clean":"2024-09-19T06:41:46.607461+0000","last_became_active":"2024-09-19T06:39:52.397274+0000","last_became_peered":"2024-09-19T06:39:52.397274+0000","last_unstale":"2024-09-19T06:41:46.607461+0000","last_undegraded":"2024-09-19T06:41:46.607461+0000","last_fullsized":"2024-09-19T06:41:46.607461+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:30:31.660641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"83'4","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679815+0000","last_change":"2024-09-19T06:39:52.388984+0000","last_active":"2024-09-19T06:41:46.679815+0000","last_peered":"2024-09-19T06:41:46.679815+0000","last_clean":"2024-09-19T06:41:46.679815+0000","last_became_active":"2024-09-19T06:39:52.388642+0000","last_became_peered":"2024-09-19T06:39:52.388642+0000","last_unstale":"2024-09-19T06:41:46.679815+0000","last_undegraded":"2024-09-19T06:41:46.679815+0000","last_fullsized":"2024-09-19T06:41:46.679815+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:26:08.720713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590191+0000","last_change":"2024-09-19T06:39:52.396937+0000","last_active":"2024-09-19T06:41:46.590191+0000","last_peered":"2024-09-19T06:41:46.590191+0000","last_clean":"2024-09-19T06:41:46.590191+0000","last_became_active":"2024-09-19T06:39:52.396601+0000","last_became_peered":"2024-09-19T06:39:52.396601+0000","last_unstale":"2024-09-19T06:41:46.590191+0000","last_undegraded":"2024-09-19T06:41:46.590191+0000","last_fullsized":"2024-09-19T06:41:46.590191+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:35:48.712660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860437+0000","last_change":"2024-09-19T06:39:52.389450+0000","last_active":"2024-09-19T06:41:46.860437+0000","last_peered":"2024-09-19T06:41:46.860437+0000","last_clean":"2024-09-19T06:41:46.860437+0000","last_became_active":"2024-09-19T06:39:52.388816+0000","last_became_peered":"2024-09-19T06:39:52.388816+0000","last_unstale":"2024-09-19T06:41:46.860437+0000","last_undegraded":"2024-09-19T06:41:46.860437+0000","last_fullsized":"2024-09-19T06:41:46.860437+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:09:37.474689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607193+0000","last_change":"2024-09-19T06:39:52.380188+0000","last_active":"2024-09-19T06:41:46.607193+0000","last_peered":"2024-09-19T06:41:46.607193+0000","last_clean":"2024-09-19T06:41:46.607193+0000","last_became_active":"2024-09-19T06:39:52.379054+0000","last_became_peered":"2024-09-19T06:39:52.379054+0000","last_unstale":"2024-09-19T06:41:46.607193+0000","last_undegraded":"2024-09-19T06:41:46.607193+0000","last_fullsized":"2024-09-19T06:41:46.607193+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:44:47.448161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609566+0000","last_change":"2024-09-19T06:39:52.391012+0000","last_active":"2024-09-19T06:41:46.609566+0000","last_peered":"2024-09-19T06:41:46.609566+0000","last_clean":"2024-09-19T06:41:46.609566+0000","last_became_active":"2024-09-19T06:39:52.390442+0000","last_became_peered":"2024-09-19T06:39:52.390442+0000","last_unstale":"2024-09-19T06:41:46.609566+0000","last_undegraded":"2024-09-19T06:41:46.609566+0000","last_fullsized":"2024-09-19T06:41:46.609566+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:33:24.883369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"83'16","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590543+0000","last_change":"2024-09-19T06:39:52.399326+0000","last_active":"2024-09-19T06:41:46.590543+0000","last_peered":"2024-09-19T06:41:46.590543+0000","last_clean":"2024-09-19T06:41:46.590543+0000","last_became_active":"2024-09-19T06:39:52.399051+0000","last_became_peered":"2024-09-19T06:39:52.399051+0000","last_unstale":"2024-09-19T06:41:46.590543+0000","last_undegraded":"2024-09-19T06:41:46.590543+0000","last_fullsized":"2024-09-19T06:41:46.590543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:15:52.195756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860924+0000","last_change":"2024-09-19T06:39:52.397081+0000","last_active":"2024-09-19T06:41:46.860924+0000","last_peered":"2024-09-19T06:41:46.860924+0000","last_clean":"2024-09-19T06:41:46.860924+0000","last_became_active":"2024-09-19T06:39:52.396720+0000","last_became_peered":"2024-09-19T06:39:52.396720+0000","last_unstale":"2024-09-19T06:41:46.860924+0000","last_undegraded":"2024-09-19T06:41:46.860924+0000","last_fullsized":"2024-09-19T06:41:46.860924+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:16.596287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607557+0000","last_change":"2024-09-19T06:39:52.392892+0000","last_active":"2024-09-19T06:41:46.607557+0000","last_peered":"2024-09-19T06:41:46.607557+0000","last_clean":"2024-09-19T06:41:46.607557+0000","last_became_active":"2024-09-19T06:39:52.392221+0000","last_became_peered":"2024-09-19T06:39:52.392221+0000","last_unstale":"2024-09-19T06:41:46.607557+0000","last_undegraded":"2024-09-19T06:41:46.607557+0000","last_fullsized":"2024-09-19T06:41:46.607557+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:57:51.895275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587632+0000","last_change":"2024-09-19T06:39:52.387642+0000","last_active":"2024-09-19T06:41:46.587632+0000","last_peered":"2024-09-19T06:41:46.587632+0000","last_clean":"2024-09-19T06:41:46.587632+0000","last_became_active":"2024-09-19T06:39:52.387283+0000","last_became_peered":"2024-09-19T06:39:52.387283+0000","last_unstale":"2024-09-19T06:41:46.587632+0000","last_undegraded":"2024-09-19T06:41:46.587632+0000","last_fullsized":"2024-09-19T06:41:46.587632+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:33:22.392952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866079+0000","last_change":"2024-09-19T06:39:52.391409+0000","last_active":"2024-09-19T06:41:46.866079+0000","last_peered":"2024-09-19T06:41:46.866079+0000","last_clean":"2024-09-19T06:41:46.866079+0000","last_became_active":"2024-09-19T06:39:52.390519+0000","last_became_peered":"2024-09-19T06:39:52.390519+0000","last_unstale":"2024-09-19T06:41:46.866079+0000","last_undegraded":"2024-09-19T06:41:46.866079+0000","last_fullsized":"2024-09-19T06:41:46.866079+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:02:20.772920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.865755+0000","last_change":"2024-09-19T06:39:52.399796+0000","last_active":"2024-09-19T06:41:46.865755+0000","last_peered":"2024-09-19T06:41:46.865755+0000","last_clean":"2024-09-19T06:41:46.865755+0000","last_became_active":"2024-09-19T06:39:52.399508+0000","last_became_peered":"2024-09-19T06:39:52.399508+0000","last_unstale":"2024-09-19T06:41:46.865755+0000","last_undegraded":"2024-09-19T06:41:46.865755+0000","last_fullsized":"2024-09-19T06:41:46.865755+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T06:50:51.428774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609034+0000","last_change":"2024-09-19T06:39:52.403057+0000","last_active":"2024-09-19T06:41:46.609034+0000","last_peered":"2024-09-19T06:41:46.609034+0000","last_clean":"2024-09-19T06:41:46.609034+0000","last_became_active":"2024-09-19T06:39:52.400951+0000","last_became_peered":"2024-09-19T06:39:52.400951+0000","last_unstale":"2024-09-19T06:41:46.609034+0000","last_undegraded":"2024-09-19T06:41:46.609034+0000","last_fullsized":"2024-09-19T06:41:46.609034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:48:02.620433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.934022+0000","last_change":"2024-09-19T06:39:52.393783+0000","last_active":"2024-09-19T06:41:46.934022+0000","last_peered":"2024-09-19T06:41:46.934022+0000","last_clean":"2024-09-19T06:41:46.934022+0000","last_became_active":"2024-09-19T06:39:52.393413+0000","last_became_peered":"2024-09-19T06:39:52.393413+0000","last_unstale":"2024-09-19T06:41:46.934022+0000","last_undegraded":"2024-09-19T06:41:46.934022+0000","last_fullsized":"2024-09-19T06:41:46.934022+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:37:18.630985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677589+0000","last_change":"2024-09-19T06:39:52.394141+0000","last_active":"2024-09-19T06:41:46.677589+0000","last_peered":"2024-09-19T06:41:46.677589+0000","last_clean":"2024-09-19T06:41:46.677589+0000","last_became_active":"2024-09-19T06:39:52.393842+0000","last_became_peered":"2024-09-19T06:39:52.393842+0000","last_unstale":"2024-09-19T06:41:46.677589+0000","last_undegraded":"2024-09-19T06:41:46.677589+0000","last_fullsized":"2024-09-19T06:41:46.677589+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:00:03.104507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609334+0000","last_change":"2024-09-19T06:39:52.398172+0000","last_active":"2024-09-19T06:41:46.609334+0000","last_peered":"2024-09-19T06:41:46.609334+0000","last_clean":"2024-09-19T06:41:46.609334+0000","last_became_active":"2024-09-19T06:39:52.397890+0000","last_became_peered":"2024-09-19T06:39:52.397890+0000","last_unstale":"2024-09-19T06:41:46.609334+0000","last_undegraded":"2024-09-19T06:41:46.609334+0000","last_fullsized":"2024-09-19T06:41:46.609334+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:46:32.986761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933853+0000","last_change":"2024-09-19T06:39:52.394783+0000","last_active":"2024-09-19T06:41:46.933853+0000","last_peered":"2024-09-19T06:41:46.933853+0000","last_clean":"2024-09-19T06:41:46.933853+0000","last_became_active":"2024-09-19T06:39:52.394449+0000","last_became_peered":"2024-09-19T06:39:52.394449+0000","last_unstale":"2024-09-19T06:41:46.933853+0000","last_undegraded":"2024-09-19T06:41:46.933853+0000","last_fullsized":"2024-09-19T06:41:46.933853+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:29:26.589176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587643+0000","last_change":"2024-09-19T06:39:52.395201+0000","last_active":"2024-09-19T06:41:46.587643+0000","last_peered":"2024-09-19T06:41:46.587643+0000","last_clean":"2024-09-19T06:41:46.587643+0000","last_became_active":"2024-09-19T06:39:52.394888+0000","last_became_peered":"2024-09-19T06:39:52.394888+0000","last_unstale":"2024-09-19T06:41:46.587643+0000","last_undegraded":"2024-09-19T06:41:46.587643+0000","last_fullsized":"2024-09-19T06:41:46.587643+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:19.654052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609138+0000","last_change":"2024-09-19T06:39:52.403387+0000","last_active":"2024-09-19T06:41:46.609138+0000","last_peered":"2024-09-19T06:41:46.609138+0000","last_clean":"2024-09-19T06:41:46.609138+0000","last_became_active":"2024-09-19T06:39:52.403070+0000","last_became_peered":"2024-09-19T06:39:52.403070+0000","last_unstale":"2024-09-19T06:41:46.609138+0000","last_undegraded":"2024-09-19T06:41:46.609138+0000","last_fullsized":"2024-09-19T06:41:46.609138+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:17:43.062907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607201+0000","last_change":"2024-09-19T06:39:52.399880+0000","last_active":"2024-09-19T06:41:46.607201+0000","last_peered":"2024-09-19T06:41:46.607201+0000","last_clean":"2024-09-19T06:41:46.607201+0000","last_became_active":"2024-09-19T06:39:52.399568+0000","last_became_peered":"2024-09-19T06:39:52.399568+0000","last_unstale":"2024-09-19T06:41:46.607201+0000","last_undegraded":"2024-09-19T06:41:46.607201+0000","last_fullsized":"2024-09-19T06:41:46.607201+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:26:19.746838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588420+0000","last_change":"2024-09-19T06:39:52.382543+0000","last_active":"2024-09-19T06:41:46.588420+0000","last_peered":"2024-09-19T06:41:46.588420+0000","last_clean":"2024-09-19T06:41:46.588420+0000","last_became_active":"2024-09-19T06:39:52.382155+0000","last_became_peered":"2024-09-19T06:39:52.382155+0000","last_unstale":"2024-09-19T06:41:46.588420+0000","last_undegraded":"2024-09-19T06:41:46.588420+0000","last_fullsized":"2024-09-19T06:41:46.588420+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:13:41.395639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"83'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677514+0000","last_change":"2024-09-19T06:40:58.134245+0000","last_active":"2024-09-19T06:41:46.677514+0000","last_peered":"2024-09-19T06:41:46.677514+0000","last_clean":"2024-09-19T06:41:46.677514+0000","last_became_active":"2024-09-19T06:40:58.133514+0000","last_became_peered":"2024-09-19T06:40:58.133514+0000","last_unstale":"2024-09-19T06:41:46.677514+0000","last_undegraded":"2024-09-19T06:41:46.677514+0000","last_fullsized":"2024-09-19T06:41:46.677514+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T16:07:23.267466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,2],"acting":[9,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.26","version":"83'15","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609431+0000","last_change":"2024-09-19T06:39:52.394976+0000","last_active":"2024-09-19T06:41:46.609431+0000","last_peered":"2024-09-19T06:41:46.609431+0000","last_clean":"2024-09-19T06:41:46.609431+0000","last_became_active":"2024-09-19T06:39:52.394646+0000","last_became_peered":"2024-09-19T06:39:52.394646+0000","last_unstale":"2024-09-19T06:41:46.609431+0000","last_undegraded":"2024-09-19T06:41:46.609431+0000","last_fullsized":"2024-09-19T06:41:46.609431+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:34:20.844968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933992+0000","last_change":"2024-09-19T06:39:52.393274+0000","last_active":"2024-09-19T06:41:46.933992+0000","last_peered":"2024-09-19T06:41:46.933992+0000","last_clean":"2024-09-19T06:41:46.933992+0000","last_became_active":"2024-09-19T06:39:52.392769+0000","last_became_peered":"2024-09-19T06:39:52.392769+0000","last_unstale":"2024-09-19T06:41:46.933992+0000","last_undegraded":"2024-09-19T06:41:46.933992+0000","last_fullsized":"2024-09-19T06:41:46.933992+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:07:17.314749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"83'4","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587598+0000","last_change":"2024-09-19T06:39:52.388317+0000","last_active":"2024-09-19T06:41:46.587598+0000","last_peered":"2024-09-19T06:41:46.587598+0000","last_clean":"2024-09-19T06:41:46.587598+0000","last_became_active":"2024-09-19T06:39:52.387996+0000","last_became_peered":"2024-09-19T06:39:52.387996+0000","last_unstale":"2024-09-19T06:41:46.587598+0000","last_undegraded":"2024-09-19T06:41:46.587598+0000","last_fullsized":"2024-09-19T06:41:46.587598+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:19:50.269478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866293+0000","last_change":"2024-09-19T06:39:52.395453+0000","last_active":"2024-09-19T06:41:46.866293+0000","last_peered":"2024-09-19T06:41:46.866293+0000","last_clean":"2024-09-19T06:41:46.866293+0000","last_became_active":"2024-09-19T06:39:52.395097+0000","last_became_peered":"2024-09-19T06:39:52.395097+0000","last_unstale":"2024-09-19T06:41:46.866293+0000","last_undegraded":"2024-09-19T06:41:46.866293+0000","last_fullsized":"2024-09-19T06:41:46.866293+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:36:45.844180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609084+0000","last_change":"2024-09-19T06:39:52.398148+0000","last_active":"2024-09-19T06:41:46.609084+0000","last_peered":"2024-09-19T06:41:46.609084+0000","last_clean":"2024-09-19T06:41:46.609084+0000","last_became_active":"2024-09-19T06:39:52.397802+0000","last_became_peered":"2024-09-19T06:39:52.397802+0000","last_unstale":"2024-09-19T06:41:46.609084+0000","last_undegraded":"2024-09-19T06:41:46.609084+0000","last_fullsized":"2024-09-19T06:41:46.609084+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:56:14.771805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"83'12","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609093+0000","last_change":"2024-09-19T06:39:52.394474+0000","last_active":"2024-09-19T06:41:46.609093+0000","last_peered":"2024-09-19T06:41:46.609093+0000","last_clean":"2024-09-19T06:41:46.609093+0000","last_became_active":"2024-09-19T06:39:52.394087+0000","last_became_peered":"2024-09-19T06:39:52.394087+0000","last_unstale":"2024-09-19T06:41:46.609093+0000","last_undegraded":"2024-09-19T06:41:46.609093+0000","last_fullsized":"2024-09-19T06:41:46.609093+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:41:24.381575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933721+0000","last_change":"2024-09-19T06:39:52.397193+0000","last_active":"2024-09-19T06:41:46.933721+0000","last_peered":"2024-09-19T06:41:46.933721+0000","last_clean":"2024-09-19T06:41:46.933721+0000","last_became_active":"2024-09-19T06:39:52.396836+0000","last_became_peered":"2024-09-19T06:39:52.396836+0000","last_unstale":"2024-09-19T06:41:46.933721+0000","last_undegraded":"2024-09-19T06:41:46.933721+0000","last_fullsized":"2024-09-19T06:41:46.933721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:38:42.518255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679766+0000","last_change":"2024-09-19T06:39:52.392185+0000","last_active":"2024-09-19T06:41:46.679766+0000","last_peered":"2024-09-19T06:41:46.679766+0000","last_clean":"2024-09-19T06:41:46.679766+0000","last_became_active":"2024-09-19T06:39:52.391764+0000","last_became_peered":"2024-09-19T06:39:52.391764+0000","last_unstale":"2024-09-19T06:41:46.679766+0000","last_undegraded":"2024-09-19T06:41:46.679766+0000","last_fullsized":"2024-09-19T06:41:46.679766+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:38:59.847691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"84'8","reported_seq":156,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:42:00.626755+0000","last_change":"2024-09-19T06:39:50.315274+0000","last_active":"2024-09-19T06:42:00.626755+0000","last_peered":"2024-09-19T06:42:00.626755+0000","last_clean":"2024-09-19T06:42:00.626755+0000","last_became_active":"2024-09-19T06:39:50.314872+0000","last_became_peered":"2024-09-19T06:39:50.314872+0000","last_unstale":"2024-09-19T06:42:00.626755+0000","last_undegraded":"2024-09-19T06:42:00.626755+0000","last_fullsized":"2024-09-19T06:42:00.626755+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:00:54.399616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1716,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":124,"num_read_kb":4,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588409+0000","last_change":"2024-09-19T06:39:52.388130+0000","last_active":"2024-09-19T06:41:46.588409+0000","last_peered":"2024-09-19T06:41:46.588409+0000","last_clean":"2024-09-19T06:41:46.588409+0000","last_became_active":"2024-09-19T06:39:52.387812+0000","last_became_peered":"2024-09-19T06:39:52.387812+0000","last_unstale":"2024-09-19T06:41:46.588409+0000","last_undegraded":"2024-09-19T06:41:46.588409+0000","last_fullsized":"2024-09-19T06:41:46.588409+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:11:35.079027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"81'3","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590296+0000","last_change":"2024-09-19T06:40:58.136626+0000","last_active":"2024-09-19T06:41:46.590296+0000","last_peered":"2024-09-19T06:41:46.590296+0000","last_clean":"2024-09-19T06:41:46.590296+0000","last_became_active":"2024-09-19T06:40:58.132752+0000","last_became_peered":"2024-09-19T06:40:58.132752+0000","last_unstale":"2024-09-19T06:41:46.590296+0000","last_undegraded":"2024-09-19T06:41:46.590296+0000","last_fullsized":"2024-09-19T06:41:46.590296+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:50:21.283329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,6],"acting":[10,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587271+0000","last_change":"2024-09-19T06:39:50.312899+0000","last_active":"2024-09-19T06:41:46.587271+0000","last_peered":"2024-09-19T06:41:46.587271+0000","last_clean":"2024-09-19T06:41:46.587271+0000","last_became_active":"2024-09-19T06:39:50.312586+0000","last_became_peered":"2024-09-19T06:39:50.312586+0000","last_unstale":"2024-09-19T06:41:46.587271+0000","last_undegraded":"2024-09-19T06:41:46.587271+0000","last_fullsized":"2024-09-19T06:41:46.587271+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:12:01.385483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607281+0000","last_change":"2024-09-19T06:39:52.387894+0000","last_active":"2024-09-19T06:41:46.607281+0000","last_peered":"2024-09-19T06:41:46.607281+0000","last_clean":"2024-09-19T06:41:46.607281+0000","last_became_active":"2024-09-19T06:39:52.387522+0000","last_became_peered":"2024-09-19T06:39:52.387522+0000","last_unstale":"2024-09-19T06:41:46.607281+0000","last_undegraded":"2024-09-19T06:41:46.607281+0000","last_fullsized":"2024-09-19T06:41:46.607281+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:35:22.509530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"81'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588546+0000","last_change":"2024-09-19T06:39:50.312212+0000","last_active":"2024-09-19T06:41:46.588546+0000","last_peered":"2024-09-19T06:41:46.588546+0000","last_clean":"2024-09-19T06:41:46.588546+0000","last_became_active":"2024-09-19T06:39:50.311606+0000","last_became_peered":"2024-09-19T06:39:50.311606+0000","last_unstale":"2024-09-19T06:41:46.588546+0000","last_undegraded":"2024-09-19T06:41:46.588546+0000","last_fullsized":"2024-09-19T06:41:46.588546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:43:01.117303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"83'11","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677238+0000","last_change":"2024-09-19T06:39:52.381766+0000","last_active":"2024-09-19T06:41:46.677238+0000","last_peered":"2024-09-19T06:41:46.677238+0000","last_clean":"2024-09-19T06:41:46.677238+0000","last_became_active":"2024-09-19T06:39:52.381475+0000","last_became_peered":"2024-09-19T06:39:52.381475+0000","last_unstale":"2024-09-19T06:41:46.677238+0000","last_undegraded":"2024-09-19T06:41:46.677238+0000","last_fullsized":"2024-09-19T06:41:46.677238+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:52:32.827393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"83'4","reported_seq":148,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:42:00.626224+0000","last_change":"2024-09-19T06:39:50.319720+0000","last_active":"2024-09-19T06:42:00.626224+0000","last_peered":"2024-09-19T06:42:00.626224+0000","last_clean":"2024-09-19T06:42:00.626224+0000","last_became_active":"2024-09-19T06:39:50.319384+0000","last_became_peered":"2024-09-19T06:39:50.319384+0000","last_unstale":"2024-09-19T06:42:00.626224+0000","last_undegraded":"2024-09-19T06:42:00.626224+0000","last_fullsized":"2024-09-19T06:42:00.626224+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:04:38.193472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"83'10","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609442+0000","last_change":"2024-09-19T06:39:52.400814+0000","last_active":"2024-09-19T06:41:46.609442+0000","last_peered":"2024-09-19T06:41:46.609442+0000","last_clean":"2024-09-19T06:41:46.609442+0000","last_became_active":"2024-09-19T06:39:52.400416+0000","last_became_peered":"2024-09-19T06:39:52.400416+0000","last_unstale":"2024-09-19T06:41:46.609442+0000","last_undegraded":"2024-09-19T06:41:46.609442+0000","last_fullsized":"2024-09-19T06:41:46.609442+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:18:37.445501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860979+0000","last_change":"2024-09-19T06:39:50.315575+0000","last_active":"2024-09-19T06:41:46.860979+0000","last_peered":"2024-09-19T06:41:46.860979+0000","last_clean":"2024-09-19T06:41:46.860979+0000","last_became_active":"2024-09-19T06:39:50.315167+0000","last_became_peered":"2024-09-19T06:39:50.315167+0000","last_unstale":"2024-09-19T06:41:46.860979+0000","last_undegraded":"2024-09-19T06:41:46.860979+0000","last_fullsized":"2024-09-19T06:41:46.860979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:18:30.970104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867221+0000","last_change":"2024-09-19T06:39:52.397297+0000","last_active":"2024-09-19T06:41:46.867221+0000","last_peered":"2024-09-19T06:41:46.867221+0000","last_clean":"2024-09-19T06:41:46.867221+0000","last_became_active":"2024-09-19T06:39:52.397091+0000","last_became_peered":"2024-09-19T06:39:52.397091+0000","last_unstale":"2024-09-19T06:41:46.867221+0000","last_undegraded":"2024-09-19T06:41:46.867221+0000","last_fullsized":"2024-09-19T06:41:46.867221+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:02:29.731909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588728+0000","last_change":"2024-09-19T06:39:50.507542+0000","last_active":"2024-09-19T06:41:46.588728+0000","last_peered":"2024-09-19T06:41:46.588728+0000","last_clean":"2024-09-19T06:41:46.588728+0000","last_became_active":"2024-09-19T06:39:50.507033+0000","last_became_peered":"2024-09-19T06:39:50.507033+0000","last_unstale":"2024-09-19T06:41:46.588728+0000","last_undegraded":"2024-09-19T06:41:46.588728+0000","last_fullsized":"2024-09-19T06:41:46.588728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:01:45.175229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"83'11","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609314+0000","last_change":"2024-09-19T06:40:58.135595+0000","last_active":"2024-09-19T06:41:46.609314+0000","last_peered":"2024-09-19T06:41:46.609314+0000","last_clean":"2024-09-19T06:41:46.609314+0000","last_became_active":"2024-09-19T06:40:58.135031+0000","last_became_peered":"2024-09-19T06:40:58.135031+0000","last_unstale":"2024-09-19T06:41:46.609314+0000","last_undegraded":"2024-09-19T06:41:46.609314+0000","last_fullsized":"2024-09-19T06:41:46.609314+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:23:53.542630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"84'42","reported_seq":323,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:42:01.628836+0000","last_change":"2024-09-19T06:39:50.319067+0000","last_active":"2024-09-19T06:42:01.628836+0000","last_peered":"2024-09-19T06:42:01.628836+0000","last_clean":"2024-09-19T06:42:01.628836+0000","last_became_active":"2024-09-19T06:39:50.318697+0000","last_became_peered":"2024-09-19T06:39:50.318697+0000","last_unstale":"2024-09-19T06:42:01.628836+0000","last_undegraded":"2024-09-19T06:42:01.628836+0000","last_fullsized":"2024-09-19T06:42:01.628836+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:10:08.457305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71502,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":257,"num_read_kb":200,"num_write":43,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680048+0000","last_change":"2024-09-19T06:39:52.397107+0000","last_active":"2024-09-19T06:41:46.680048+0000","last_peered":"2024-09-19T06:41:46.680048+0000","last_clean":"2024-09-19T06:41:46.680048+0000","last_became_active":"2024-09-19T06:39:52.396793+0000","last_became_peered":"2024-09-19T06:39:52.396793+0000","last_unstale":"2024-09-19T06:41:46.680048+0000","last_undegraded":"2024-09-19T06:41:46.680048+0000","last_fullsized":"2024-09-19T06:41:46.680048+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:27:09.403923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"81'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609208+0000","last_change":"2024-09-19T06:39:50.317396+0000","last_active":"2024-09-19T06:41:46.609208+0000","last_peered":"2024-09-19T06:41:46.609208+0000","last_clean":"2024-09-19T06:41:46.609208+0000","last_became_active":"2024-09-19T06:39:50.316949+0000","last_became_peered":"2024-09-19T06:39:50.316949+0000","last_unstale":"2024-09-19T06:41:46.609208+0000","last_undegraded":"2024-09-19T06:41:46.609208+0000","last_fullsized":"2024-09-19T06:41:46.609208+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:06:50.281106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587384+0000","last_change":"2024-09-19T06:39:52.393133+0000","last_active":"2024-09-19T06:41:46.587384+0000","last_peered":"2024-09-19T06:41:46.587384+0000","last_clean":"2024-09-19T06:41:46.587384+0000","last_became_active":"2024-09-19T06:39:52.392568+0000","last_became_peered":"2024-09-19T06:39:52.392568+0000","last_unstale":"2024-09-19T06:41:46.587384+0000","last_undegraded":"2024-09-19T06:41:46.587384+0000","last_fullsized":"2024-09-19T06:41:46.587384+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:42:10.139197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"81'3","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933722+0000","last_change":"2024-09-19T06:40:58.143264+0000","last_active":"2024-09-19T06:41:46.933722+0000","last_peered":"2024-09-19T06:41:46.933722+0000","last_clean":"2024-09-19T06:41:46.933722+0000","last_became_active":"2024-09-19T06:40:58.135400+0000","last_became_peered":"2024-09-19T06:40:58.135400+0000","last_unstale":"2024-09-19T06:41:46.933722+0000","last_undegraded":"2024-09-19T06:41:46.933722+0000","last_fullsized":"2024-09-19T06:41:46.933722+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:08:30.500765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,7],"acting":[11,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.13","version":"83'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860840+0000","last_change":"2024-09-19T06:40:58.134136+0000","last_active":"2024-09-19T06:41:46.860840+0000","last_peered":"2024-09-19T06:41:46.860840+0000","last_clean":"2024-09-19T06:41:46.860840+0000","last_became_active":"2024-09-19T06:40:58.133734+0000","last_became_peered":"2024-09-19T06:40:58.133734+0000","last_unstale":"2024-09-19T06:41:46.860840+0000","last_undegraded":"2024-09-19T06:41:46.860840+0000","last_fullsized":"2024-09-19T06:41:46.860840+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:51:50.822670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"84'7","reported_seq":152,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:55.618680+0000","last_change":"2024-09-19T06:39:50.316757+0000","last_active":"2024-09-19T06:41:55.618680+0000","last_peered":"2024-09-19T06:41:55.618680+0000","last_clean":"2024-09-19T06:41:55.618680+0000","last_became_active":"2024-09-19T06:39:50.316343+0000","last_became_peered":"2024-09-19T06:39:50.316343+0000","last_unstale":"2024-09-19T06:41:55.618680+0000","last_undegraded":"2024-09-19T06:41:55.618680+0000","last_fullsized":"2024-09-19T06:41:55.618680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:58:55.050176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1908,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":122,"num_read_kb":7,"num_write":6,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"83'6","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860373+0000","last_change":"2024-09-19T06:39:52.400037+0000","last_active":"2024-09-19T06:41:46.860373+0000","last_peered":"2024-09-19T06:41:46.860373+0000","last_clean":"2024-09-19T06:41:46.860373+0000","last_became_active":"2024-09-19T06:39:52.398202+0000","last_became_peered":"2024-09-19T06:39:52.398202+0000","last_unstale":"2024-09-19T06:41:46.860373+0000","last_undegraded":"2024-09-19T06:41:46.860373+0000","last_fullsized":"2024-09-19T06:41:46.860373+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:38:39.695212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"81'15","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679751+0000","last_change":"2024-09-19T06:39:50.507344+0000","last_active":"2024-09-19T06:41:46.679751+0000","last_peered":"2024-09-19T06:41:46.679751+0000","last_clean":"2024-09-19T06:41:46.679751+0000","last_became_active":"2024-09-19T06:39:50.506975+0000","last_became_peered":"2024-09-19T06:39:50.506975+0000","last_unstale":"2024-09-19T06:41:46.679751+0000","last_undegraded":"2024-09-19T06:41:46.679751+0000","last_fullsized":"2024-09-19T06:41:46.679751+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:15:18.333881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"83'6","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588476+0000","last_change":"2024-09-19T06:40:58.135978+0000","last_active":"2024-09-19T06:41:46.588476+0000","last_peered":"2024-09-19T06:41:46.588476+0000","last_clean":"2024-09-19T06:41:46.588476+0000","last_became_active":"2024-09-19T06:40:58.135581+0000","last_became_peered":"2024-09-19T06:40:58.135581+0000","last_unstale":"2024-09-19T06:41:46.588476+0000","last_undegraded":"2024-09-19T06:41:46.588476+0000","last_fullsized":"2024-09-19T06:41:46.588476+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:33:54.532003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:47.456507+0000","last_change":"2024-09-19T06:39:50.312902+0000","last_active":"2024-09-19T06:41:47.456507+0000","last_peered":"2024-09-19T06:41:47.456507+0000","last_clean":"2024-09-19T06:41:47.456507+0000","last_became_active":"2024-09-19T06:39:50.312535+0000","last_became_peered":"2024-09-19T06:39:50.312535+0000","last_unstale":"2024-09-19T06:41:47.456507+0000","last_undegraded":"2024-09-19T06:41:47.456507+0000","last_fullsized":"2024-09-19T06:41:47.456507+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:52:19.807865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933860+0000","last_change":"2024-09-19T06:39:52.394827+0000","last_active":"2024-09-19T06:41:46.933860+0000","last_peered":"2024-09-19T06:41:46.933860+0000","last_clean":"2024-09-19T06:41:46.933860+0000","last_became_active":"2024-09-19T06:39:52.394514+0000","last_became_peered":"2024-09-19T06:39:52.394514+0000","last_unstale":"2024-09-19T06:41:46.933860+0000","last_undegraded":"2024-09-19T06:41:46.933860+0000","last_fullsized":"2024-09-19T06:41:46.933860+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:09:19.190345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587529+0000","last_change":"2024-09-19T06:39:50.314823+0000","last_active":"2024-09-19T06:41:46.587529+0000","last_peered":"2024-09-19T06:41:46.587529+0000","last_clean":"2024-09-19T06:41:46.587529+0000","last_became_active":"2024-09-19T06:39:50.314416+0000","last_became_peered":"2024-09-19T06:39:50.314416+0000","last_unstale":"2024-09-19T06:41:46.587529+0000","last_undegraded":"2024-09-19T06:41:46.587529+0000","last_fullsized":"2024-09-19T06:41:46.587529+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:29:31.904480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.396836+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.396496+0000","last_became_peered":"2024-09-19T06:39:52.396496+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:42:22.848803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'73","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:42:00.460308+0000","last_change":"2024-09-19T06:39:50.317086+0000","last_active":"2024-09-19T06:42:00.460308+0000","last_peered":"2024-09-19T06:42:00.460308+0000","last_clean":"2024-09-19T06:42:00.460308+0000","last_became_active":"2024-09-19T06:39:50.316654+0000","last_became_peered":"2024-09-19T06:39:50.316654+0000","last_unstale":"2024-09-19T06:42:00.460308+0000","last_undegraded":"2024-09-19T06:42:00.460308+0000","last_fullsized":"2024-09-19T06:42:00.460308+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:55:56.539698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860872+0000","last_change":"2024-09-19T06:39:52.399552+0000","last_active":"2024-09-19T06:41:46.860872+0000","last_peered":"2024-09-19T06:41:46.860872+0000","last_clean":"2024-09-19T06:41:46.860872+0000","last_became_active":"2024-09-19T06:39:52.399185+0000","last_became_peered":"2024-09-19T06:39:52.399185+0000","last_unstale":"2024-09-19T06:41:46.860872+0000","last_undegraded":"2024-09-19T06:41:46.860872+0000","last_fullsized":"2024-09-19T06:41:46.860872+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:11:02.705419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"81'3","reported_seq":19,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607210+0000","last_change":"2024-09-19T06:40:58.138283+0000","last_active":"2024-09-19T06:41:46.607210+0000","last_peered":"2024-09-19T06:41:46.607210+0000","last_clean":"2024-09-19T06:41:46.607210+0000","last_became_active":"2024-09-19T06:40:58.132079+0000","last_became_peered":"2024-09-19T06:40:58.132079+0000","last_unstale":"2024-09-19T06:41:46.607210+0000","last_undegraded":"2024-09-19T06:41:46.607210+0000","last_fullsized":"2024-09-19T06:41:46.607210+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:12.018193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,0],"acting":[4,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"83'7","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.861068+0000","last_change":"2024-09-19T06:39:52.394211+0000","last_active":"2024-09-19T06:41:46.861068+0000","last_peered":"2024-09-19T06:41:46.861068+0000","last_clean":"2024-09-19T06:41:46.861068+0000","last_became_active":"2024-09-19T06:39:52.393897+0000","last_became_peered":"2024-09-19T06:39:52.393897+0000","last_unstale":"2024-09-19T06:41:46.861068+0000","last_undegraded":"2024-09-19T06:41:46.861068+0000","last_fullsized":"2024-09-19T06:41:46.861068+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:30:38.180838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"84'148","reported_seq":390,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:48.697056+0000","last_change":"2024-09-19T06:38:40.026127+0000","last_active":"2024-09-19T06:41:48.697056+0000","last_peered":"2024-09-19T06:41:48.697056+0000","last_clean":"2024-09-19T06:41:48.697056+0000","last_became_active":"2024-09-19T06:38:40.025545+0000","last_became_peered":"2024-09-19T06:38:40.025545+0000","last_unstale":"2024-09-19T06:41:48.697056+0000","last_undegraded":"2024-09-19T06:41:48.697056+0000","last_fullsized":"2024-09-19T06:41:48.697056+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:35:14.623839+0000","last_clean_scrub_stamp":"2024-09-19T06:35:14.623839+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:05:04.111626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"81'3","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607328+0000","last_change":"2024-09-19T06:39:50.316897+0000","last_active":"2024-09-19T06:41:46.607328+0000","last_peered":"2024-09-19T06:41:46.607328+0000","last_clean":"2024-09-19T06:41:46.607328+0000","last_became_active":"2024-09-19T06:39:50.316473+0000","last_became_peered":"2024-09-19T06:39:50.316473+0000","last_unstale":"2024-09-19T06:41:46.607328+0000","last_undegraded":"2024-09-19T06:41:46.607328+0000","last_fullsized":"2024-09-19T06:41:46.607328+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:13:05.573479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"83'8","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607462+0000","last_change":"2024-09-19T06:39:52.388396+0000","last_active":"2024-09-19T06:41:46.607462+0000","last_peered":"2024-09-19T06:41:46.607462+0000","last_clean":"2024-09-19T06:41:46.607462+0000","last_became_active":"2024-09-19T06:39:52.388003+0000","last_became_peered":"2024-09-19T06:39:52.388003+0000","last_unstale":"2024-09-19T06:41:46.607462+0000","last_undegraded":"2024-09-19T06:41:46.607462+0000","last_fullsized":"2024-09-19T06:41:46.607462+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:28:44.599913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860594+0000","last_change":"2024-09-19T06:39:50.314571+0000","last_active":"2024-09-19T06:41:46.860594+0000","last_peered":"2024-09-19T06:41:46.860594+0000","last_clean":"2024-09-19T06:41:46.860594+0000","last_became_active":"2024-09-19T06:39:50.314180+0000","last_became_peered":"2024-09-19T06:39:50.314180+0000","last_unstale":"2024-09-19T06:41:46.860594+0000","last_undegraded":"2024-09-19T06:41:46.860594+0000","last_fullsized":"2024-09-19T06:41:46.860594+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:22:44.994655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587338+0000","last_change":"2024-09-19T06:39:52.399865+0000","last_active":"2024-09-19T06:41:46.587338+0000","last_peered":"2024-09-19T06:41:46.587338+0000","last_clean":"2024-09-19T06:41:46.587338+0000","last_became_active":"2024-09-19T06:39:52.399574+0000","last_became_peered":"2024-09-19T06:39:52.399574+0000","last_unstale":"2024-09-19T06:41:46.587338+0000","last_undegraded":"2024-09-19T06:41:46.587338+0000","last_fullsized":"2024-09-19T06:41:46.587338+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:43:37.044384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"84'6","reported_seq":151,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:42:00.625333+0000","last_change":"2024-09-19T06:39:50.506810+0000","last_active":"2024-09-19T06:42:00.625333+0000","last_peered":"2024-09-19T06:42:00.625333+0000","last_clean":"2024-09-19T06:42:00.625333+0000","last_became_active":"2024-09-19T06:39:50.506503+0000","last_became_peered":"2024-09-19T06:39:50.506503+0000","last_unstale":"2024-09-19T06:42:00.625333+0000","last_undegraded":"2024-09-19T06:42:00.625333+0000","last_fullsized":"2024-09-19T06:42:00.625333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:56:25.617424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"83'8","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677201+0000","last_change":"2024-09-19T06:40:58.132715+0000","last_active":"2024-09-19T06:41:46.677201+0000","last_peered":"2024-09-19T06:41:46.677201+0000","last_clean":"2024-09-19T06:41:46.677201+0000","last_became_active":"2024-09-19T06:40:58.132333+0000","last_became_peered":"2024-09-19T06:40:58.132333+0000","last_unstale":"2024-09-19T06:41:46.677201+0000","last_undegraded":"2024-09-19T06:41:46.677201+0000","last_fullsized":"2024-09-19T06:41:46.677201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:07:03.256998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"81'1","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860671+0000","last_change":"2024-09-19T06:39:50.313379+0000","last_active":"2024-09-19T06:41:46.860671+0000","last_peered":"2024-09-19T06:41:46.860671+0000","last_clean":"2024-09-19T06:41:46.860671+0000","last_became_active":"2024-09-19T06:39:50.312438+0000","last_became_peered":"2024-09-19T06:39:50.312438+0000","last_unstale":"2024-09-19T06:41:46.860671+0000","last_undegraded":"2024-09-19T06:41:46.860671+0000","last_fullsized":"2024-09-19T06:41:46.860671+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:01:30.658272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590335+0000","last_change":"2024-09-19T06:39:52.396737+0000","last_active":"2024-09-19T06:41:46.590335+0000","last_peered":"2024-09-19T06:41:46.590335+0000","last_clean":"2024-09-19T06:41:46.590335+0000","last_became_active":"2024-09-19T06:39:52.396207+0000","last_became_peered":"2024-09-19T06:39:52.396207+0000","last_unstale":"2024-09-19T06:41:46.590335+0000","last_undegraded":"2024-09-19T06:41:46.590335+0000","last_fullsized":"2024-09-19T06:41:46.590335+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:16:20.640929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.677581+0000","last_change":"2024-09-19T06:39:50.509319+0000","last_active":"2024-09-19T06:41:46.677581+0000","last_peered":"2024-09-19T06:41:46.677581+0000","last_clean":"2024-09-19T06:41:46.677581+0000","last_became_active":"2024-09-19T06:39:50.508965+0000","last_became_peered":"2024-09-19T06:39:50.508965+0000","last_unstale":"2024-09-19T06:41:46.677581+0000","last_undegraded":"2024-09-19T06:41:46.677581+0000","last_fullsized":"2024-09-19T06:41:46.677581+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:04:39.190245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.590381+0000","last_change":"2024-09-19T06:39:52.387965+0000","last_active":"2024-09-19T06:41:46.590381+0000","last_peered":"2024-09-19T06:41:46.590381+0000","last_clean":"2024-09-19T06:41:46.590381+0000","last_became_active":"2024-09-19T06:39:52.387602+0000","last_became_peered":"2024-09-19T06:39:52.387602+0000","last_unstale":"2024-09-19T06:41:46.590381+0000","last_undegraded":"2024-09-19T06:41:46.590381+0000","last_fullsized":"2024-09-19T06:41:46.590381+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T11:46:49.463402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"81'1","reported_seq":17,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933826+0000","last_change":"2024-09-19T06:40:58.140944+0000","last_active":"2024-09-19T06:41:46.933826+0000","last_peered":"2024-09-19T06:41:46.933826+0000","last_clean":"2024-09-19T06:41:46.933826+0000","last_became_active":"2024-09-19T06:40:58.132427+0000","last_became_peered":"2024-09-19T06:40:58.132427+0000","last_unstale":"2024-09-19T06:41:46.933826+0000","last_undegraded":"2024-09-19T06:41:46.933826+0000","last_fullsized":"2024-09-19T06:41:46.933826+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:01:32.472242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.587739+0000","last_change":"2024-09-19T06:39:52.401484+0000","last_active":"2024-09-19T06:41:46.587739+0000","last_peered":"2024-09-19T06:41:46.587739+0000","last_clean":"2024-09-19T06:41:46.587739+0000","last_became_active":"2024-09-19T06:39:52.401128+0000","last_became_peered":"2024-09-19T06:39:52.401128+0000","last_unstale":"2024-09-19T06:41:46.587739+0000","last_undegraded":"2024-09-19T06:41:46.587739+0000","last_fullsized":"2024-09-19T06:41:46.587739+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:01:54.284725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"84'125","reported_seq":275,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:42:00.465126+0000","last_change":"2024-09-19T06:39:50.315227+0000","last_active":"2024-09-19T06:42:00.465126+0000","last_peered":"2024-09-19T06:42:00.465126+0000","last_clean":"2024-09-19T06:42:00.465126+0000","last_became_active":"2024-09-19T06:39:50.314798+0000","last_became_peered":"2024-09-19T06:39:50.314798+0000","last_unstale":"2024-09-19T06:42:00.465126+0000","last_undegraded":"2024-09-19T06:42:00.465126+0000","last_fullsized":"2024-09-19T06:42:00.465126+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":125,"log_dups_size":0,"ondisk_log_size":125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:02:11.198269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":231,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607812+0000","last_change":"2024-09-19T06:39:52.389980+0000","last_active":"2024-09-19T06:41:46.607812+0000","last_peered":"2024-09-19T06:41:46.607812+0000","last_clean":"2024-09-19T06:41:46.607812+0000","last_became_active":"2024-09-19T06:39:52.389677+0000","last_became_peered":"2024-09-19T06:39:52.389677+0000","last_unstale":"2024-09-19T06:41:46.607812+0000","last_undegraded":"2024-09-19T06:41:46.607812+0000","last_fullsized":"2024-09-19T06:41:46.607812+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:55:51.691791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"81'4","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607492+0000","last_change":"2024-09-19T06:39:50.397870+0000","last_active":"2024-09-19T06:41:46.607492+0000","last_peered":"2024-09-19T06:41:46.607492+0000","last_clean":"2024-09-19T06:41:46.607492+0000","last_became_active":"2024-09-19T06:39:50.397591+0000","last_became_peered":"2024-09-19T06:39:50.397591+0000","last_unstale":"2024-09-19T06:41:46.607492+0000","last_undegraded":"2024-09-19T06:41:46.607492+0000","last_fullsized":"2024-09-19T06:41:46.607492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:08:27.239784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"83'7","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.866974+0000","last_change":"2024-09-19T06:39:52.395522+0000","last_active":"2024-09-19T06:41:46.866974+0000","last_peered":"2024-09-19T06:41:46.866974+0000","last_clean":"2024-09-19T06:41:46.866974+0000","last_became_active":"2024-09-19T06:39:52.395224+0000","last_became_peered":"2024-09-19T06:39:52.395224+0000","last_unstale":"2024-09-19T06:41:46.866974+0000","last_undegraded":"2024-09-19T06:41:46.866974+0000","last_fullsized":"2024-09-19T06:41:46.866974+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T09:11:38.428767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609037+0000","last_change":"2024-09-19T06:39:50.507407+0000","last_active":"2024-09-19T06:41:46.609037+0000","last_peered":"2024-09-19T06:41:46.609037+0000","last_clean":"2024-09-19T06:41:46.609037+0000","last_became_active":"2024-09-19T06:39:50.507076+0000","last_became_peered":"2024-09-19T06:39:50.507076+0000","last_unstale":"2024-09-19T06:41:46.609037+0000","last_undegraded":"2024-09-19T06:41:46.609037+0000","last_fullsized":"2024-09-19T06:41:46.609037+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:20:34.646749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"83'2","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679748+0000","last_change":"2024-09-19T06:39:52.391654+0000","last_active":"2024-09-19T06:41:46.679748+0000","last_peered":"2024-09-19T06:41:46.679748+0000","last_clean":"2024-09-19T06:41:46.679748+0000","last_became_active":"2024-09-19T06:39:52.391255+0000","last_became_peered":"2024-09-19T06:39:52.391255+0000","last_unstale":"2024-09-19T06:41:46.679748+0000","last_undegraded":"2024-09-19T06:41:46.679748+0000","last_fullsized":"2024-09-19T06:41:46.679748+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T08:18:33.081931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"81'3","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607416+0000","last_change":"2024-09-19T06:39:50.316564+0000","last_active":"2024-09-19T06:41:46.607416+0000","last_peered":"2024-09-19T06:41:46.607416+0000","last_clean":"2024-09-19T06:41:46.607416+0000","last_became_active":"2024-09-19T06:39:50.316275+0000","last_became_peered":"2024-09-19T06:39:50.316275+0000","last_unstale":"2024-09-19T06:41:46.607416+0000","last_undegraded":"2024-09-19T06:41:46.607416+0000","last_fullsized":"2024-09-19T06:41:46.607416+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T07:24:36.479912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"83'9","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.933751+0000","last_change":"2024-09-19T06:39:52.397971+0000","last_active":"2024-09-19T06:41:46.933751+0000","last_peered":"2024-09-19T06:41:46.933751+0000","last_clean":"2024-09-19T06:41:46.933751+0000","last_became_active":"2024-09-19T06:39:52.397670+0000","last_became_peered":"2024-09-19T06:39:52.397670+0000","last_unstale":"2024-09-19T06:41:46.933751+0000","last_undegraded":"2024-09-19T06:41:46.933751+0000","last_fullsized":"2024-09-19T06:41:46.933751+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:30:37.585587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"84'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609110+0000","last_change":"2024-09-19T06:39:50.315138+0000","last_active":"2024-09-19T06:41:46.609110+0000","last_peered":"2024-09-19T06:41:46.609110+0000","last_clean":"2024-09-19T06:41:46.609110+0000","last_became_active":"2024-09-19T06:39:50.314730+0000","last_became_peered":"2024-09-19T06:39:50.314730+0000","last_unstale":"2024-09-19T06:41:46.609110+0000","last_undegraded":"2024-09-19T06:41:46.609110+0000","last_fullsized":"2024-09-19T06:41:46.609110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:05:49.824274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"83'8","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607487+0000","last_change":"2024-09-19T06:39:52.385845+0000","last_active":"2024-09-19T06:41:46.607487+0000","last_peered":"2024-09-19T06:41:46.607487+0000","last_clean":"2024-09-19T06:41:46.607487+0000","last_became_active":"2024-09-19T06:39:52.385537+0000","last_became_peered":"2024-09-19T06:39:52.385537+0000","last_unstale":"2024-09-19T06:41:46.607487+0000","last_undegraded":"2024-09-19T06:41:46.607487+0000","last_fullsized":"2024-09-19T06:41:46.607487+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:41:28.562718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"81'4","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.679819+0000","last_change":"2024-09-19T06:39:50.316236+0000","last_active":"2024-09-19T06:41:46.679819+0000","last_peered":"2024-09-19T06:41:46.679819+0000","last_clean":"2024-09-19T06:41:46.679819+0000","last_became_active":"2024-09-19T06:39:50.315948+0000","last_became_peered":"2024-09-19T06:39:50.315948+0000","last_unstale":"2024-09-19T06:41:46.679819+0000","last_undegraded":"2024-09-19T06:41:46.679819+0000","last_fullsized":"2024-09-19T06:41:46.679819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T13:38:57.932321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"83'5","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.860513+0000","last_change":"2024-09-19T06:39:52.395379+0000","last_active":"2024-09-19T06:41:46.860513+0000","last_peered":"2024-09-19T06:41:46.860513+0000","last_clean":"2024-09-19T06:41:46.860513+0000","last_became_active":"2024-09-19T06:39:52.395079+0000","last_became_peered":"2024-09-19T06:39:52.395079+0000","last_unstale":"2024-09-19T06:41:46.860513+0000","last_undegraded":"2024-09-19T06:41:46.860513+0000","last_fullsized":"2024-09-19T06:41:46.860513+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T15:16:59.689819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"83'15","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.588680+0000","last_change":"2024-09-19T06:39:52.391050+0000","last_active":"2024-09-19T06:41:46.588680+0000","last_peered":"2024-09-19T06:41:46.588680+0000","last_clean":"2024-09-19T06:41:46.588680+0000","last_became_active":"2024-09-19T06:39:52.390689+0000","last_became_peered":"2024-09-19T06:39:52.390689+0000","last_unstale":"2024-09-19T06:41:46.588680+0000","last_undegraded":"2024-09-19T06:41:46.588680+0000","last_fullsized":"2024-09-19T06:41:46.588680+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:41:41.612974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"81'2","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680018+0000","last_change":"2024-09-19T06:39:50.312505+0000","last_active":"2024-09-19T06:41:46.680018+0000","last_peered":"2024-09-19T06:41:46.680018+0000","last_clean":"2024-09-19T06:41:46.680018+0000","last_became_active":"2024-09-19T06:39:50.312112+0000","last_became_peered":"2024-09-19T06:39:50.312112+0000","last_unstale":"2024-09-19T06:41:46.680018+0000","last_undegraded":"2024-09-19T06:41:46.680018+0000","last_fullsized":"2024-09-19T06:41:46.680018+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T14:16:01.125735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"83'9","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.609092+0000","last_change":"2024-09-19T06:39:52.391337+0000","last_active":"2024-09-19T06:41:46.609092+0000","last_peered":"2024-09-19T06:41:46.609092+0000","last_clean":"2024-09-19T06:41:46.609092+0000","last_became_active":"2024-09-19T06:39:52.390928+0000","last_became_peered":"2024-09-19T06:39:52.390928+0000","last_unstale":"2024-09-19T06:41:46.609092+0000","last_undegraded":"2024-09-19T06:41:46.609092+0000","last_fullsized":"2024-09-19T06:41:46.609092+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:37:46.502434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"81'4","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.680073+0000","last_change":"2024-09-19T06:40:58.139755+0000","last_active":"2024-09-19T06:41:46.680073+0000","last_peered":"2024-09-19T06:41:46.680073+0000","last_clean":"2024-09-19T06:41:46.680073+0000","last_became_active":"2024-09-19T06:40:58.135411+0000","last_became_peered":"2024-09-19T06:40:58.135411+0000","last_unstale":"2024-09-19T06:41:46.680073+0000","last_undegraded":"2024-09-19T06:41:46.680073+0000","last_fullsized":"2024-09-19T06:41:46.680073+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T10:16:38.951366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"83'6","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.867245+0000","last_change":"2024-09-19T06:39:52.393913+0000","last_active":"2024-09-19T06:41:46.867245+0000","last_peered":"2024-09-19T06:41:46.867245+0000","last_clean":"2024-09-19T06:41:46.867245+0000","last_became_active":"2024-09-19T06:39:52.393596+0000","last_became_peered":"2024-09-19T06:39:52.393596+0000","last_unstale":"2024-09-19T06:41:46.867245+0000","last_undegraded":"2024-09-19T06:41:46.867245+0000","last_fullsized":"2024-09-19T06:41:46.867245+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T18:39:16.190220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"81'2","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607906+0000","last_change":"2024-09-19T06:39:50.506798+0000","last_active":"2024-09-19T06:41:46.607906+0000","last_peered":"2024-09-19T06:41:46.607906+0000","last_clean":"2024-09-19T06:41:46.607906+0000","last_became_active":"2024-09-19T06:39:50.506456+0000","last_became_peered":"2024-09-19T06:39:50.506456+0000","last_unstale":"2024-09-19T06:41:46.607906+0000","last_undegraded":"2024-09-19T06:41:46.607906+0000","last_fullsized":"2024-09-19T06:41:46.607906+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:27:50.244765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"83'5","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:46.607785+0000","last_change":"2024-09-19T06:39:52.400055+0000","last_active":"2024-09-19T06:41:46.607785+0000","last_peered":"2024-09-19T06:41:46.607785+0000","last_clean":"2024-09-19T06:41:46.607785+0000","last_became_active":"2024-09-19T06:39:52.399505+0000","last_became_peered":"2024-09-19T06:39:52.399505+0000","last_unstale":"2024-09-19T06:41:46.607785+0000","last_undegraded":"2024-09-19T06:41:46.607785+0000","last_fullsized":"2024-09-19T06:41:46.607785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:51.369852+0000","last_clean_scrub_stamp":"2024-09-19T06:39:51.369852+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T17:31:27.382796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"81'9","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-09-19T06:41:52.602852+0000","last_change":"2024-09-19T06:39:50.311858+0000","last_active":"2024-09-19T06:41:52.602852+0000","last_peered":"2024-09-19T06:41:52.602852+0000","last_clean":"2024-09-19T06:41:52.602852+0000","last_became_active":"2024-09-19T06:39:50.311647+0000","last_became_peered":"2024-09-19T06:39:50.311647+0000","last_unstale":"2024-09-19T06:41:52.602852+0000","last_undegraded":"2024-09-19T06:41:52.602852+0000","last_fullsized":"2024-09-19T06:41:52.602852+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-19T06:39:49.297604+0000","last_clean_scrub_stamp":"2024-09-19T06:39:49.297604+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T12:48:37.493140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":798974,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1221,"num_read_kb":882,"num_write":485,"num_write_kb":2627,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":604,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":659456,"data_stored":2395791,"data_compressed":103487,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107529,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547239,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":446248,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":68,"seq":292057776172,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1118208,"data_stored":1439154,"data_compressed":59704,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":63,"seq":270582939698,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36272,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708880,"statfs":{"total":95995035648,"available":95957893120,"internally_reserved":0,"allocated":950272,"data_stored":839678,"data_compressed":27955,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":56,"seq":240518168630,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32108,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713044,"statfs":{"total":95995035648,"available":95962157056,"internally_reserved":0,"allocated":946176,"data_stored":839576,"data_compressed":27805,"data_compressed_allocated":36864,"data_compressed_original":417792,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397562,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":974848,"data_stored":1040335,"data_compressed":44502,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561089,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32080,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713072,"statfs":{"total":95995035648,"available":95962185728,"internally_reserved":0,"allocated":913408,"data_stored":445851,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790021,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1032192,"data_stored":1366931,"data_compressed":59704,"data_compressed_allocated":77824,"data_compressed_original":921600,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953546,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":921600,"data_stored":448190,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182479,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":999424,"data_stored":1167610,"data_compressed":43157,"data_compressed_allocated":57344,"data_compressed_original":724992,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411413,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1007616,"data_stored":519008,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574937,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":983040,"data_stored":515622,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738463,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":933888,"data_stored":447122,"data_compressed":8659,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35843,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":69978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":721732,"data_compressed":34498,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":330802,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":393932,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":159744,"data_stored":403025,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-19T06:42:03.522 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:42:03.780 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:04.181 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:03 smithi037 ceph-mon[31211]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 65 op/s 2024-09-19T06:42:04.182 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:03 smithi037 ceph-mon[31211]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:03 smithi096 ceph-mon[35004]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 65 op/s 2024-09-19T06:42:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:03 smithi096 ceph-mon[35004]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:04.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 -- 172.21.15.37:0/323042272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc101de0 msgr2=0x7f59dc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:04.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 --2- 172.21.15.37:0/323042272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc101de0 0x7f59dc1021c0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f59cc009900 tx=0x7f59cc02a3f0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 -- 172.21.15.37:0/323042272 shutdown_connections 2024-09-19T06:42:04.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 --2- 172.21.15.37:0/323042272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc106f10 0x7f59dc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 --2- 172.21.15.37:0/323042272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59dc102790 0x7f59dc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 --2- 172.21.15.37:0/323042272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc101de0 0x7f59dc1021c0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 -- 172.21.15.37:0/323042272 >> 172.21.15.37:0/323042272 conn(0x7f59dc0fd8c0 msgr2=0x7f59dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.262+0000 7f59e2428640 1 -- 172.21.15.37:0/323042272 shutdown_connections 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.263+0000 7f59e2428640 1 -- 172.21.15.37:0/323042272 wait complete. 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.263+0000 7f59e2428640 1 Processor -- start 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.263+0000 7f59e2428640 1 -- start start 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.263+0000 7f59e2428640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59dc101de0 0x7f59dc198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.263+0000 7f59e2428640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc102790 0x7f59dc198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e2428640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc106f10 0x7f59dc19cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e2428640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc111f30 con 0x7f59dc106f10 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e2428640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc111db0 con 0x7f59dc101de0 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e2428640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59dc1120b0 con 0x7f59dc102790 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc106f10 0x7f59dc19cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc106f10 0x7f59dc19cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41910/0 (socket says 172.21.15.37:41910) 2024-09-19T06:42:04.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 -- 172.21.15.37:0/2564753407 learned_addr learned my addr 172.21.15.37:0/2564753407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59dbfff640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59dc101de0 0x7f59dc198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59db7fe640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc102790 0x7f59dc198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 -- 172.21.15.37:0/2564753407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc102790 msgr2=0x7f59dc198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc102790 0x7f59dc198850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 -- 172.21.15.37:0/2564753407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59dc101de0 msgr2=0x7f59dc198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59dc101de0 0x7f59dc198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 -- 172.21.15.37:0/2564753407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59dc19d2c0 con 0x7f59dc106f10 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59dbfff640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59dc101de0 0x7f59dc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e09c5640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc106f10 0x7f59dc19cbe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59d400cce0 tx=0x7f59d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59d97fa640 1 -- 172.21.15.37:0/2564753407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d400f450 con 0x7f59dc106f10 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59d97fa640 1 -- 172.21.15.37:0/2564753407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f59d400fad0 con 0x7f59dc106f10 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59dc19d5b0 con 0x7f59dc106f10 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59d97fa640 1 -- 172.21.15.37:0/2564753407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59d4018c10 con 0x7f59dc106f10 2024-09-19T06:42:04.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.264+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59dc1a4e90 con 0x7f59dc106f10 2024-09-19T06:42:04.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.265+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59dc1021c0 con 0x7f59dc106f10 2024-09-19T06:42:04.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.267+0000 7f59d97fa640 1 -- 172.21.15.37:0/2564753407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f59d400f5f0 con 0x7f59dc106f10 2024-09-19T06:42:04.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.267+0000 7f59d97fa640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59c40a35e0 0x7f59c40a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:04.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.268+0000 7f59dbfff640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59c40a35e0 0x7f59c40a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:04.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.268+0000 7f59d97fa640 1 -- 172.21.15.37:0/2564753407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f59d40d06d0 con 0x7f59dc106f10 2024-09-19T06:42:04.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.268+0000 7f59dbfff640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59c40a35e0 0x7f59c40a5aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59cc000c00 tx=0x7f59cc032d50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:04.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.271+0000 7f59d97fa640 1 -- 172.21.15.37:0/2564753407 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f59d4014030 con 0x7f59dc106f10 2024-09-19T06:42:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:03 smithi151 ceph-mon[35236]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 65 op/s 2024-09-19T06:42:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:03 smithi151 ceph-mon[35236]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-19T06:42:04.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.483+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59dc1a5130 con 0x7f59dc106f10 2024-09-19T06:42:04.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.484+0000 7f59d97fa640 1 -- 172.21.15.37:0/2564753407 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7f59d409cf80 con 0x7f59dc106f10 2024-09-19T06:42:04.487 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:04.487 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:42:04.487 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:42:04.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59c40a35e0 msgr2=0x7f59c40a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59c40a35e0 0x7f59c40a5aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f59cc000c00 tx=0x7f59cc032d50 comp rx=0 tx=0).stop 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc106f10 msgr2=0x7f59dc19cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc106f10 0x7f59dc19cbe0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f59d400cce0 tx=0x7f59d400a620 comp rx=0 tx=0).stop 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 shutdown_connections 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59c40a35e0 0x7f59c40a5aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f59dc106f10 0x7f59dc19cbe0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f59dc102790 0x7f59dc198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 --2- 172.21.15.37:0/2564753407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f59dc101de0 0x7f59dc198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 >> 172.21.15.37:0/2564753407 conn(0x7f59dc0fd8c0 msgr2=0x7f59dc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 shutdown_connections 2024-09-19T06:42:04.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:04.486+0000 7f59e2428640 1 -- 172.21.15.37:0/2564753407 wait complete. 2024-09-19T06:42:05.166 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:42:05.190 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:04 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2564753407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:05.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:04 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2564753407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:05.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:04 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2564753407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:05.421 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:05.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 -- 172.21.15.37:0/4146630087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14100a80 msgr2=0x7fbe14100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4146630087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14100a80 0x7fbe14100ee0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009930 tx=0x7fbdfc02a420 comp rx=0 tx=0).stop 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 -- 172.21.15.37:0/4146630087 shutdown_connections 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4146630087 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe14101420 0x7fbe1410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4146630087 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14100a80 0x7fbe14100ee0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4146630087 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe14106a80 0x7fbe14106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 -- 172.21.15.37:0/4146630087 >> 172.21.15.37:0/4146630087 conn(0x7fbe140fc7a0 msgr2=0x7fbe140febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 -- 172.21.15.37:0/4146630087 shutdown_connections 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.863+0000 7fbe18b26640 1 -- 172.21.15.37:0/4146630087 wait complete. 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 Processor -- start 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 -- start start 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe14100a80 0x7fbe1419ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:05.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14101420 0x7fbe1419f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe14106a80 0x7fbe141a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14110e80 con 0x7fbe14101420 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14110d00 con 0x7fbe14106a80 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe18b26640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbe14111000 con 0x7fbe14100a80 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe11d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14101420 0x7fbe1419f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe11d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14101420 0x7fbe1419f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41922/0 (socket says 172.21.15.37:41922) 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe11d9b640 1 -- 172.21.15.37:0/4008087297 learned_addr learned my addr 172.21.15.37:0/4008087297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe12d9d640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe14106a80 0x7fbe141a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.864+0000 7fbe1259c640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe14100a80 0x7fbe1419ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe11d9b640 1 -- 172.21.15.37:0/4008087297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe14100a80 msgr2=0x7fbe1419ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe11d9b640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe14100a80 0x7fbe1419ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe11d9b640 1 -- 172.21.15.37:0/4008087297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe14106a80 msgr2=0x7fbe141a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:05.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe11d9b640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe14106a80 0x7fbe141a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe11d9b640 1 -- 172.21.15.37:0/4008087297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe141a3a70 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe11d9b640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14101420 0x7fbe1419f230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009230 tx=0x7fbdfc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe0b7fe640 1 -- 172.21.15.37:0/4008087297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc03f030 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe0b7fe640 1 -- 172.21.15.37:0/4008087297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbdfc003ad0 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe141a3d00 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe0b7fe640 1 -- 172.21.15.37:0/4008087297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc035070 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.865+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe141ab620 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.866+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe14100ee0 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.867+0000 7fbe0b7fe640 1 -- 172.21.15.37:0/4008087297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fbdfc042030 con 0x7fbe14101420 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.868+0000 7fbe0b7fe640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbe000a3560 0x7fbe000a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.868+0000 7fbe1259c640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbe000a3560 0x7fbe000a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:05.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.869+0000 7fbe0b7fe640 1 -- 172.21.15.37:0/4008087297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fbdfc0f0500 con 0x7fbe14101420 2024-09-19T06:42:05.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.869+0000 7fbe1259c640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbe000a3560 0x7fbe000a5a20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe04005fd0 tx=0x7fbe0400b510 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:05.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:05.872+0000 7fbe0b7fe640 1 -- 172.21.15.37:0/4008087297 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbdfc0bcca0 con 0x7fbe14101420 2024-09-19T06:42:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.084+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbe141ab7f0 con 0x7fbe14101420 2024-09-19T06:42:06.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.085+0000 7fbe0b7fe640 1 -- 172.21.15.37:0/4008087297 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7fbdfc0bc5a0 con 0x7fbe14101420 2024-09-19T06:42:06.087 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:06.087 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:42:06.087 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbe000a3560 msgr2=0x7fbe000a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbe000a3560 0x7fbe000a5a20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fbe04005fd0 tx=0x7fbe0400b510 comp rx=0 tx=0).stop 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14101420 msgr2=0x7fbe1419f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14101420 0x7fbe1419f230 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fbdfc009230 tx=0x7fbdfc004270 comp rx=0 tx=0).stop 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 shutdown_connections 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe14106a80 0x7fbe141a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbe000a3560 0x7fbe000a5a20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbe14101420 0x7fbe1419f230 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 --2- 172.21.15.37:0/4008087297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbe14100a80 0x7fbe1419ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 >> 172.21.15.37:0/4008087297 conn(0x7fbe140fc7a0 msgr2=0x7fbe1406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 shutdown_connections 2024-09-19T06:42:06.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:06.087+0000 7fbe18b26640 1 -- 172.21.15.37:0/4008087297 wait complete. 2024-09-19T06:42:06.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:05 smithi096 ceph-mon[35004]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 65 op/s 2024-09-19T06:42:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:05 smithi037 ceph-mon[31211]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 65 op/s 2024-09-19T06:42:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:05 smithi151 ceph-mon[35236]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 99 KiB/s wr, 65 op/s 2024-09-19T06:42:06.751 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:42:07.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:06 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4008087297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:07.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:06 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:42:07.050 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:07.264 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:06 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4008087297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:07.264 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:06 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:42:07.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:06 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4008087297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:07.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:06 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 -- 172.21.15.37:0/1960699606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba58103660 msgr2=0x7fba5810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/1960699606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba58103660 0x7fba5810bb50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fba44009900 tx=0x7fba4402a480 comp rx=0 tx=0).stop 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 -- 172.21.15.37:0/1960699606 shutdown_connections 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/1960699606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba5810c170 0x7fba5810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/1960699606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba58103660 0x7fba5810bb50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/1960699606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba58102d40 0x7fba58103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 -- 172.21.15.37:0/1960699606 >> 172.21.15.37:0/1960699606 conn(0x7fba580fc7a0 msgr2=0x7fba580febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:07.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 -- 172.21.15.37:0/1960699606 shutdown_connections 2024-09-19T06:42:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.506+0000 7fba5f1b8640 1 -- 172.21.15.37:0/1960699606 wait complete. 2024-09-19T06:42:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 Processor -- start 2024-09-19T06:42:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 -- start start 2024-09-19T06:42:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba58102d40 0x7fba581984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba58103660 0x7fba58198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:07.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5810c170 0x7fba5819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fba58111f70 con 0x7fba5810c170 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba58111df0 con 0x7fba58103660 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5f1b8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fba581120f0 con 0x7fba58102d40 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5d755640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5810c170 0x7fba5819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5d755640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5810c170 0x7fba5819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41936/0 (socket says 172.21.15.37:41936) 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba57fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba58103660 0x7fba58198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5d755640 1 -- 172.21.15.37:0/4264199810 learned_addr learned my addr 172.21.15.37:0/4264199810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.507+0000 7fba5cf54640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba58102d40 0x7fba581984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:07.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5d755640 1 -- 172.21.15.37:0/4264199810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba58102d40 msgr2=0x7fba581984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5d755640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba58102d40 0x7fba581984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5d755640 1 -- 172.21.15.37:0/4264199810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba58103660 msgr2=0x7fba58198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5d755640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba58103660 0x7fba58198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5d755640 1 -- 172.21.15.37:0/4264199810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5819d480 con 0x7fba5810c170 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5cf54640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba58102d40 0x7fba581984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5d755640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5810c170 0x7fba5819cda0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fba5000eb10 tx=0x7fba5000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba55ffb640 1 -- 172.21.15.37:0/4264199810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50004b20 con 0x7fba5810c170 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba55ffb640 1 -- 172.21.15.37:0/4264199810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fba50004cc0 con 0x7fba5810c170 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5819d770 con 0x7fba5810c170 2024-09-19T06:42:07.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba55ffb640 1 -- 172.21.15.37:0/4264199810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba50010760 con 0x7fba5810c170 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.508+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba581a5050 con 0x7fba5810c170 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.509+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba58103120 con 0x7fba5810c170 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.510+0000 7fba55ffb640 1 -- 172.21.15.37:0/4264199810 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fba50005020 con 0x7fba5810c170 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.511+0000 7fba55ffb640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fba480a3560 0x7fba480a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.511+0000 7fba55ffb640 1 -- 172.21.15.37:0/4264199810 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fba500d0590 con 0x7fba5810c170 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.511+0000 7fba5cf54640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fba480a3560 0x7fba480a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.511+0000 7fba5cf54640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fba480a3560 0x7fba480a5a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fba4c005fd0 tx=0x7fba4c005ea0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:07.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.515+0000 7fba55ffb640 1 -- 172.21.15.37:0/4264199810 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fba5009cd30 con 0x7fba5810c170 2024-09-19T06:42:07.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.692+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba58107240 con 0x7fba5810c170 2024-09-19T06:42:07.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.695+0000 7fba55ffb640 1 -- 172.21.15.37:0/4264199810 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+23170 (secure 0 0 0) 0x7fba5009c630 con 0x7fba5810c170 2024-09-19T06:42:07.697 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:07.697 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":84,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:41:46.581293+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3406854118":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/1436144832":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6801/871019432":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1644258429":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/711456420":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6800/871019432":"2024-09-20T06:41:46.581265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fba480a3560 msgr2=0x7fba480a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fba480a3560 0x7fba480a5a20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fba4c005fd0 tx=0x7fba4c005ea0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5810c170 msgr2=0x7fba5819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5810c170 0x7fba5819cda0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fba5000eb10 tx=0x7fba5000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 shutdown_connections 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fba480a3560 0x7fba480a5a20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fba5810c170 0x7fba5819cda0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba58103660 0x7fba58198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 --2- 172.21.15.37:0/4264199810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fba58102d40 0x7fba581984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 >> 172.21.15.37:0/4264199810 conn(0x7fba580fc7a0 msgr2=0x7fba580fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:07.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 shutdown_connections 2024-09-19T06:42:07.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:07.697+0000 7fba5f1b8640 1 -- 172.21.15.37:0/4264199810 wait complete. 2024-09-19T06:42:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:07 smithi096 ceph-mon[35004]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-09-19T06:42:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:07 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4264199810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:42:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:07 smithi037 ceph-mon[31211]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-09-19T06:42:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:07 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4264199810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:42:08.285 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:07 smithi151 ceph-mon[35236]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 99 KiB/s wr, 52 op/s 2024-09-19T06:42:08.285 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:07 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4264199810' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:42:08.337 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-19T06:42:08.596 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:09.712 INFO:teuthology.orchestra.run.smithi037.stdout:{"run_dir":"/var/run/ceph"} 2024-09-19T06:42:09.712 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-19T06:42:09.712 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 1747 2024-09-19T06:42:09.713 DEBUG:tasks.quiescer:Yielding 2024-09-19T06:42:09.713 INFO:teuthology.run_tasks:Running task workunit... 2024-09-19T06:42:09.727 INFO:tasks.workunit:Pulling workunits from ref 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:42:09.728 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-19T06:42:09.728 INFO:tasks.workunit:timeout=3h 2024-09-19T06:42:09.728 INFO:tasks.workunit:cleanup=True 2024-09-19T06:42:09.729 DEBUG:teuthology.orchestra.run.smithi037:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-19T06:42:09.729 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:42:09.747 INFO:teuthology.orchestra.run.smithi037.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-19T06:42:09.747 INFO:teuthology.orchestra.run.smithi037.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-19T06:42:09.747 INFO:teuthology.orchestra.run.smithi037.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-19T06:42:09.747 INFO:teuthology.orchestra.run.smithi037.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-19T06:42:09.748 INFO:teuthology.orchestra.run.smithi037.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-19T06:42:09.748 INFO:teuthology.orchestra.run.smithi037.stdout:Access: 2024-09-19 06:40:10.970737681 +0000 2024-09-19T06:42:09.748 INFO:teuthology.orchestra.run.smithi037.stdout:Modify: 2024-09-19 06:40:10.970737681 +0000 2024-09-19T06:42:09.748 INFO:teuthology.orchestra.run.smithi037.stdout:Change: 2024-09-19 06:41:44.620881705 +0000 2024-09-19T06:42:09.748 INFO:teuthology.orchestra.run.smithi037.stdout: Birth: 2024-09-19 06:40:10.970737681 +0000 2024-09-19T06:42:09.748 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-19T06:42:09.748 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-19T06:42:09.821 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6fad5dd73c93545e52f57f681e140a9c3529c11e 2024-09-19T06:42:09.878 INFO:tasks.workunit.client.0.smithi037.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-19T06:42:09.991 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:10.238 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:09 smithi037 ceph-mon[31211]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-09-19T06:42:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:09 smithi096 ceph-mon[35004]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-09-19T06:42:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:09 smithi151 ceph-mon[35236]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 54 op/s 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/4056005883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 msgr2=0x7f7c84107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/4056005883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84107b80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7c78009930 tx=0x7f7c7802a420 comp rx=0 tx=0).stop 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/4056005883 shutdown_connections 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/4056005883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c84108290 0x7f7c8410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/4056005883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84107b80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/4056005883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c84102d90 0x7f7c84103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/4056005883 >> 172.21.15.37:0/4056005883 conn(0x7f7c840fc7d0 msgr2=0x7f7c840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/4056005883 shutdown_connections 2024-09-19T06:42:10.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.444+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/4056005883 wait complete. 2024-09-19T06:42:10.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 Processor -- start 2024-09-19T06:42:10.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 -- start start 2024-09-19T06:42:10.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c84102d90 0x7f7c841985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:10.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:10.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c84108290 0x7f7c8419ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84112050 con 0x7f7c84103740 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c84111ed0 con 0x7f7c84108290 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8c0c7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7c841121d0 con 0x7f7c84102d90 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c89662640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c89662640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84198ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54700/0 (socket says 172.21.15.37:54700) 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c89662640 1 -- 172.21.15.37:0/3908869518 learned_addr learned my addr 172.21.15.37:0/3908869518 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c89e63640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c84102d90 0x7f7c841985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.445+0000 7f7c8a664640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c84108290 0x7f7c8419ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c89662640 1 -- 172.21.15.37:0/3908869518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c84102d90 msgr2=0x7f7c841985a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c89662640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c84102d90 0x7f7c841985a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c89662640 1 -- 172.21.15.37:0/3908869518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c84108290 msgr2=0x7f7c8419ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:10.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c89662640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c84108290 0x7f7c8419ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c89662640 1 -- 172.21.15.37:0/3908869518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c8419d550 con 0x7f7c84103740 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c89e63640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c84102d90 0x7f7c841985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c89662640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84198ae0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c78005940 tx=0x7f7c780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c76ffd640 1 -- 172.21.15.37:0/3908869518 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7803f030 con 0x7f7c84103740 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c8419d7e0 con 0x7f7c84103740 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c76ffd640 1 -- 172.21.15.37:0/3908869518 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c78003ce0 con 0x7f7c84103740 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c76ffd640 1 -- 172.21.15.37:0/3908869518 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c78035070 con 0x7f7c84103740 2024-09-19T06:42:10.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.446+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c84100920 con 0x7f7c84103740 2024-09-19T06:42:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.447+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c8406a7f0 con 0x7f7c84103740 2024-09-19T06:42:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.449+0000 7f7c76ffd640 1 -- 172.21.15.37:0/3908869518 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f7c78042030 con 0x7f7c84103740 2024-09-19T06:42:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.449+0000 7f7c76ffd640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7c700a3560 0x7f7c700a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.450+0000 7f7c89e63640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7c700a3560 0x7f7c700a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:10.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.450+0000 7f7c76ffd640 1 -- 172.21.15.37:0/3908869518 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f7c780f0520 con 0x7f7c84103740 2024-09-19T06:42:10.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.450+0000 7f7c89e63640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7c700a3560 0x7f7c700a5a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c800058c0 tx=0x7f7c80005850 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:10.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.454+0000 7f7c76ffd640 1 -- 172.21.15.37:0/3908869518 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7c780bccc0 con 0x7f7c84103740 2024-09-19T06:42:10.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.672+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7c84199900 con 0x7f7c84103740 2024-09-19T06:42:10.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.672+0000 7f7c76ffd640 1 -- 172.21.15.37:0/3908869518 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7f7c780bc5c0 con 0x7f7c84103740 2024-09-19T06:42:10.674 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:10.675 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:42:10.675 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:42:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.674+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7c700a3560 msgr2=0x7f7c700a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.674+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7c700a3560 0x7f7c700a5a20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7c800058c0 tx=0x7f7c80005850 comp rx=0 tx=0).stop 2024-09-19T06:42:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.674+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 msgr2=0x7f7c84198ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.674+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84198ae0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7c78005940 tx=0x7f7c780042f0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 shutdown_connections 2024-09-19T06:42:10.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c84108290 0x7f7c8419ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7c700a3560 0x7f7c700a5a20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7c84103740 0x7f7c84198ae0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 --2- 172.21.15.37:0/3908869518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7c84102d90 0x7f7c841985a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:10.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 >> 172.21.15.37:0/3908869518 conn(0x7f7c840fc7d0 msgr2=0x7f7c84105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:10.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 shutdown_connections 2024-09-19T06:42:10.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:10.675+0000 7f7c8c0c7640 1 -- 172.21.15.37:0/3908869518 wait complete. 2024-09-19T06:42:11.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:10 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3908869518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:11.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:10 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3908869518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 -- 172.21.15.37:0/3665182488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 msgr2=0x7f0c40102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/3665182488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c40102ff0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0c30009900 tx=0x7f0c3002a410 comp rx=0 tx=0).stop 2024-09-19T06:42:11.326 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 -- 172.21.15.37:0/3665182488 shutdown_connections 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/3665182488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c4010c110 0x7f0c4010e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/3665182488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c401035c0 0x7f0c4010baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/3665182488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c40102ff0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 -- 172.21.15.37:0/3665182488 >> 172.21.15.37:0/3665182488 conn(0x7f0c400fc6f0 msgr2=0x7f0c400feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 -- 172.21.15.37:0/3665182488 shutdown_connections 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.324+0000 7f0c47f6d640 1 -- 172.21.15.37:0/3665182488 wait complete. 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 Processor -- start 2024-09-19T06:42:11.327 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 -- start start 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c401982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c401035c0 0x7f0c40198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c4010c110 0x7f0c4019cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111ec0 con 0x7f0c40102c10 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40111d40 con 0x7f0c401035c0 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c47f6d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c40112040 con 0x7f0c4010c110 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c45d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c401982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c45d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c401982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54730/0 (socket says 172.21.15.37:54730) 2024-09-19T06:42:11.329 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c45d09640 1 -- 172.21.15.37:0/1035124783 learned_addr learned my addr 172.21.15.37:0/1035124783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c4650a640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c4010c110 0x7f0c4019cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.325+0000 7f0c45508640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c401035c0 0x7f0c40198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c45d09640 1 -- 172.21.15.37:0/1035124783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c4010c110 msgr2=0x7f0c4019cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c45d09640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c4010c110 0x7f0c4019cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c45d09640 1 -- 172.21.15.37:0/1035124783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c401035c0 msgr2=0x7f0c40198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c45d09640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c401035c0 0x7f0c40198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c45d09640 1 -- 172.21.15.37:0/1035124783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c4019d2e0 con 0x7f0c40102c10 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c45d09640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c401982f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0c30006010 tx=0x7f0c300042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c36ffd640 1 -- 172.21.15.37:0/1035124783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3003f030 con 0x7f0c40102c10 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c36ffd640 1 -- 172.21.15.37:0/1035124783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c30003ce0 con 0x7f0c40102c10 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c4019d570 con 0x7f0c40102c10 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c36ffd640 1 -- 172.21.15.37:0/1035124783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30035070 con 0x7f0c40102c10 2024-09-19T06:42:11.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.326+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c401a4e60 con 0x7f0c40102c10 2024-09-19T06:42:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c36ffd640 1 -- 172.21.15.37:0/1035124783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f0c30042030 con 0x7f0c40102c10 2024-09-19T06:42:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 Processor -- start 2024-09-19T06:42:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 -- start start 2024-09-19T06:42:11.336 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a9116dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a9125b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91260c0 0x55b0a91284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0a907b2f0 con 0x55b0a91260c0 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9023270 con 0x55b0a9117310 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.329+0000 7f0c4adec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b0a90772b0 con 0x55b0a91169f0 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c36ffd640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c2c0a3610 0x7f0c2c0a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91260c0 0x55b0a91284b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45d09640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a9125b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91260c0 0x55b0a91284b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54742/0 (socket says 172.21.15.37:54742) 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 -- 172.21.15.37:0/3811888944 learned_addr learned my addr 172.21.15.37:0/3811888944 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c4650a640 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a9116dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.337 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c4650a640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c2c0a3610 0x7f0c2c0a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c36ffd640 1 -- 172.21.15.37:0/1035124783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0c300f0500 con 0x7f0c40102c10 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 -- 172.21.15.37:0/3811888944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 msgr2=0x55b0a9116dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a9116dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 -- 172.21.15.37:0/3811888944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 msgr2=0x55b0a9125b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a9125b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 -- 172.21.15.37:0/3811888944 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a9022340 con 0x55b0a91260c0 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45d09640 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a9125b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c45508640 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91260c0 0x55b0a91284b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00bc80 tx=0x7f0c3c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1062ed995c2e8d55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c4650a640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c2c0a3610 0x7f0c2c0a5ad0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009fd0 tx=0x7f0c38009600 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c2bfff640 1 -- 172.21.15.37:0/3811888944 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c015070 con 0x55b0a91260c0 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c2bfff640 1 -- 172.21.15.37:0/3811888944 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c01a940 con 0x55b0a91260c0 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.330+0000 7f0c2bfff640 1 -- 172.21.15.37:0/3811888944 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c011840 con 0x55b0a91260c0 2024-09-19T06:42:11.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 -- 172.21.15.37:0/3811888944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91260c0 msgr2=0x55b0a91284b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91260c0 0x55b0a91284b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00bc80 tx=0x7f0c3c00f730 comp rx=0 tx=0).stop 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 -- 172.21.15.37:0/3811888944 shutdown_connections 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91260c0 0x55b0a91284b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a9125b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 --2- 172.21.15.37:0/3811888944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a9116dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 -- 172.21.15.37:0/3811888944 >> 172.21.15.37:0/3811888944 conn(0x55b0a9113060 msgr2=0x55b0a9113440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 -- 172.21.15.37:0/3811888944 shutdown_connections 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.331+0000 7f0c4adec740 1 -- 172.21.15.37:0/3811888944 wait complete. 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 -- start start 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 Processor -- start 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9122ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a9117310 0x55b0a9123530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a9123a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9128d20 con 0x55b0a91169f0 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9004290 con 0x55b0a91260c0 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b0a9128ea0 con 0x55b0a9117310 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c4adec740 10 client.0 _ll_register_callbacks cb 0x55b0a9035140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c45d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9122ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c45d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9122ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54752/0 (socket says 172.21.15.37:54752) 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.332+0000 7f0c45d09640 1 -- 172.21.15.37:0/1461112833 learned_addr learned my addr 172.21.15.37:0/1461112833 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c4650a640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a9123a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c45508640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a9117310 0x55b0a9123530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c45d09640 1 -- 172.21.15.37:0/1461112833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a9117310 msgr2=0x55b0a9123530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c45d09640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a9117310 0x55b0a9123530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c45d09640 1 -- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 msgr2=0x55b0a9123a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c45d09640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a9123a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c45d09640 1 -- 172.21.15.37:0/1461112833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a8d8e210 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c4650a640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a9123a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c45d09640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9122ff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c300edb00 tx=0x7f0c300bc4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42bb24970dfebbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c297fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c297fa640 1 -- 172.21.15.37:0/1461112833 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30049040 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c297fa640 1 -- 172.21.15.37:0/1461112833 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0c300ba960 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c3c01a940 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c297fa640 1 -- 172.21.15.37:0/1461112833 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c3003bcb0 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c4adec740 10 client.15420 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.333+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0a8e5a620 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c297fa640 1 -- 172.21.15.37:0/1461112833 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3c01a940 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c297fa640 1 -- 172.21.15.37:0/1461112833 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0c300b8cd0 con 0x55b0a91169f0 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c4adec740 10 client.15420 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:11.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c4adec740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c4adec740 20 client.15420 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c4adec740 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a9136080 0x55b0a91384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c4adec740 4 client.15420 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b0a9088a20 con 0x55b0a9136080 2024-09-19T06:42:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.334+0000 7f0c45508640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a9136080 0x55b0a91384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.335+0000 7f0c297fa640 10 client.15420 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:11.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.335+0000 7f0c45508640 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a9136080 0x55b0a91384f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40f6954bf5ae138e server_cookie=d233795119f8ec84 in_seq=0 out_seq=0 2024-09-19T06:42:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.335+0000 7f0c297fa640 1 -- 172.21.15.37:0/1461112833 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b0a9088a20 con 0x55b0a9136080 2024-09-19T06:42:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.335+0000 7f0c297fa640 10 client.15420 handle_command_reply: tid=0 2024-09-19T06:42:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 client.15420 shutdown 2024-09-19T06:42:11.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 msgr2=0x55b0a9122ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9122ff0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0c300edb00 tx=0x7f0c300bc4f0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 shutdown_connections 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a9136080 0x55b0a91384f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a9123a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c297fa640 0 client.15420 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a9117310 0x55b0a9123530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 --2- 172.21.15.37:0/1461112833 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9122ff0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 >> 172.21.15.37:0/1461112833 conn(0x55b0a9113060 msgr2=0x55b0a9118dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 shutdown_connections 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 1 -- 172.21.15.37:0/1461112833 wait complete. 2024-09-19T06:42:11.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.336+0000 7f0c4adec740 20 client.15420 trim_cache size 0 max 16384 2024-09-19T06:42:11.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-19T06:39:53.022194+0000', 'modified': '2024-09-19T06:41:45.350782+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24497, 'mds_1': 24470, 'mds_2': 24449}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24428': {'gid': 24428, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/672218237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 672218237}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 672218237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6837/3741792814', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6836', 'nonce': 3741792814}, {'type': 'v1', 'addr': '172.21.15.151:6837', 'nonce': 3741792814}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24445': {'gid': 24445, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.151:6839/3546164673', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.151:6838', 'nonce': 3546164673}, {'type': 'v1', 'addr': '172.21.15.151:6839', 'nonce': 3546164673}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24449': {'gid': 24449, 'name': 'e', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.96:6835/946199433', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 946199433}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 946199433}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24470': {'gid': 24470, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.96:6837/3655112770', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 3655112770}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 3655112770}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24497': {'gid': 24497, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.96:6839/1066552330', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 1066552330}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 1066552330}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24497, 'qdb_cluster': [24497, 24470, 24449]} 2024-09-19T06:42:11.346 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-19T06:42:11.346 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-19T06:42:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.350+0000 7f0c4adec740 1 Processor -- start 2024-09-19T06:42:11.353 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.350+0000 7f0c4adec740 1 -- start start 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9118cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a913ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91260c0 0x55b0a913e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c4adec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401aded0 con 0x55b0a91169f0 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c4adec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401ab7a0 con 0x55b0a9117310 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c4adec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401a9610 con 0x55b0a91260c0 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9118cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9118cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54766/0 (socket says 172.21.15.37:54766) 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 -- 172.21.15.37:0/715996205 learned_addr learned my addr 172.21.15.37:0/715996205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45508640 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a913ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 -- 172.21.15.37:0/715996205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91260c0 msgr2=0x55b0a913e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91260c0 0x55b0a913e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 -- 172.21.15.37:0/715996205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 msgr2=0x55b0a913ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a913ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 -- 172.21.15.37:0/715996205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0a9112e90 con 0x55b0a91169f0 2024-09-19T06:42:11.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45508640 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a913ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c45d09640 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9118cb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c300c0a90 tx=0x7f0c300bc260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=473d808b108f98da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c2b7fe640 1 -- 172.21.15.37:0/715996205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c30049070 con 0x55b0a91169f0 2024-09-19T06:42:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c2b7fe640 1 -- 172.21.15.37:0/715996205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c3003b580 con 0x55b0a91169f0 2024-09-19T06:42:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.351+0000 7f0c2b7fe640 1 -- 172.21.15.37:0/715996205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c300044e0 con 0x55b0a91169f0 2024-09-19T06:42:11.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 -- 172.21.15.37:0/715996205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 msgr2=0x55b0a9118cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9118cb0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c300c0a90 tx=0x7f0c300bc260 comp rx=0 tx=0).stop 2024-09-19T06:42:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 -- 172.21.15.37:0/715996205 shutdown_connections 2024-09-19T06:42:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91260c0 0x55b0a913e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a9117310 0x55b0a913ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 --2- 172.21.15.37:0/715996205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a91169f0 0x55b0a9118cb0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 -- 172.21.15.37:0/715996205 >> 172.21.15.37:0/715996205 conn(0x55b0a9113060 msgr2=0x55b0a91185a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:11.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 -- 172.21.15.37:0/715996205 shutdown_connections 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.352+0000 7f0c4adec740 1 -- 172.21.15.37:0/715996205 wait complete. 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 -- start start 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 Processor -- start 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a912a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a9117310 0x55b0a912a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a912ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401a9c60 con 0x55b0a9117310 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401ab0d0 con 0x55b0a91260c0 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0c401aa330 con 0x55b0a91169f0 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c4adec740 10 client.0 _ll_register_callbacks cb 0x7f0c30003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c45508640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a9117310 0x55b0a912a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c45508640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a9117310 0x55b0a912a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54768/0 (socket says 172.21.15.37:54768) 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.353+0000 7f0c45508640 1 -- 172.21.15.37:0/792417522 learned_addr learned my addr 172.21.15.37:0/792417522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c4650a640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a912ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45d09640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a912a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45508640 1 -- 172.21.15.37:0/792417522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 msgr2=0x55b0a912a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45508640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a912a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45508640 1 -- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 msgr2=0x55b0a912ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45508640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a912ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45508640 1 -- 172.21.15.37:0/792417522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0a911dd50 con 0x55b0a9117310 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45d09640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a912a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c45508640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a9117310 0x55b0a912a600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00fc30 tx=0x7f0c3c011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05a0f35afd41bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:11.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c297fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c297fa640 1 -- 172.21.15.37:0/792417522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c015040 con 0x55b0a9117310 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c297fa640 1 -- 172.21.15.37:0/792417522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0c3c004440 con 0x55b0a9117310 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b0a9082d70 con 0x55b0a9117310 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c297fa640 1 -- 172.21.15.37:0/792417522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0c3c005040 con 0x55b0a9117310 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c4adec740 10 client.15426 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.354+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b0a91a10b0 con 0x55b0a9117310 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c297fa640 1 -- 172.21.15.37:0/792417522 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55b0a9082d70 con 0x55b0a9117310 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c297fa640 1 -- 172.21.15.37:0/792417522 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0c3c0049b0 con 0x55b0a9117310 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c4adec740 10 client.15426 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c4adec740 10 client.15426 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:11.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c4adec740 20 client.15426 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c4adec740 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a91a1550 0x55b0a91a3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c4adec740 4 client.15426 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:42:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b0a8fff900 con 0x55b0a91a1550 2024-09-19T06:42:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.355+0000 7f0c45d09640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a91a1550 0x55b0a91a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.356+0000 7f0c297fa640 10 client.15426 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:11.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.356+0000 7f0c45d09640 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a91a1550 0x55b0a91a3940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70200e0cd0767204 server_cookie=b531531c9dcc213 in_seq=0 out_seq=0 2024-09-19T06:42:11.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:10 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3908869518' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c297fa640 1 -- 172.21.15.37:0/792417522 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b0a8fff900 con 0x55b0a91a1550 2024-09-19T06:42:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c297fa640 10 client.15426 handle_command_reply: tid=0 2024-09-19T06:42:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 client.15426 shutdown 2024-09-19T06:42:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a9117310 msgr2=0x55b0a912a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a9117310 0x55b0a912a600 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c00fc30 tx=0x7f0c3c011440 comp rx=0 tx=0).stop 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 shutdown_connections 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b0a91a1550 0x55b0a91a3940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b0a91260c0 0x55b0a912ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b0a9117310 0x55b0a912a600 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 --2- 172.21.15.37:0/792417522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b0a91169f0 0x55b0a912a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 >> 172.21.15.37:0/792417522 conn(0x55b0a9113060 msgr2=0x55b0a9118dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c297fa640 0 client.15426 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 shutdown_connections 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 1 -- 172.21.15.37:0/792417522 wait complete. 2024-09-19T06:42:11.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.368+0000 7f0c4adec740 20 client.15426 trim_cache size 0 max 16384 2024-09-19T06:42:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:42:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:42:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:42:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:42:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c2c0a3610 msgr2=0x7f0c2c0a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c2c0a3610 0x7f0c2c0a5ad0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0c38009fd0 tx=0x7f0c38009600 comp rx=0 tx=0).stop 2024-09-19T06:42:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 msgr2=0x7f0c401982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c401982f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f0c30006010 tx=0x7f0c300042f0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 shutdown_connections 2024-09-19T06:42:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0c4010c110 0x7f0c4019cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0c401035c0 0x7f0c40198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c2c0a3610 0x7f0c2c0a5ad0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 --2- 172.21.15.37:0/1035124783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0c40102c10 0x7f0c401982f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 >> 172.21.15.37:0/1035124783 conn(0x7f0c400fc6f0 msgr2=0x7f0c400fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 shutdown_connections 2024-09-19T06:42:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:11.369+0000 7f0c47f6d640 1 -- 172.21.15.37:0/1035124783 wait complete. 2024-09-19T06:42:11.610 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:11.996 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:11 smithi037 ceph-mon[31211]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T06:42:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 -- 172.21.15.37:0/3555242100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 msgr2=0x7fb9a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/3555242100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41067e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fb998009900 tx=0x7fb99802a410 comp rx=0 tx=0).stop 2024-09-19T06:42:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 -- 172.21.15.37:0/3555242100 shutdown_connections 2024-09-19T06:42:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/3555242100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9a4106f10 0x7fb9a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/3555242100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41067e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/3555242100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4101de0 0x7fb9a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 -- 172.21.15.37:0/3555242100 >> 172.21.15.37:0/3555242100 conn(0x7fb9a40fd8c0 msgr2=0x7fb9a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 -- 172.21.15.37:0/3555242100 shutdown_connections 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 -- 172.21.15.37:0/3555242100 wait complete. 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 Processor -- start 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.078+0000 7fb9a9b62640 1 -- start start 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a9b62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4101de0 0x7fb9a4198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a9b62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a9b62640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9a4106f10 0x7fb9a419cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a9b62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4111f80 con 0x7fb9a4102790 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a9b62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4111e00 con 0x7fb9a4101de0 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a9b62640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a4112100 con 0x7fb9a4106f10 2024-09-19T06:42:12.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54780/0 (socket says 172.21.15.37:54780) 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/2020000206 learned_addr learned my addr 172.21.15.37:0/2020000206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4101de0 0x7fb9a4198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/2020000206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9a4106f10 msgr2=0x7fb9a419cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9a4106f10 0x7fb9a419cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/2020000206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4101de0 msgr2=0x7fb9a4198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4101de0 0x7fb9a4198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/2020000206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a419d410 con 0x7fb9a4102790 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41989a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb998004770 tx=0x7fb998003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.079+0000 7fb9a0ff9640 1 -- 172.21.15.37:0/2020000206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99803f070 con 0x7fb9a4102790 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.080+0000 7fb9a0ff9640 1 -- 172.21.15.37:0/2020000206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb998031ca0 con 0x7fb9a4102790 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.080+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a419d6a0 con 0x7fb9a4102790 2024-09-19T06:42:12.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.080+0000 7fb9a0ff9640 1 -- 172.21.15.37:0/2020000206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998035030 con 0x7fb9a4102790 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.080+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a41a4fe0 con 0x7fb9a4102790 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.082+0000 7fb9a0ff9640 1 -- 172.21.15.37:0/2020000206 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fb998042070 con 0x7fb9a4102790 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.082+0000 7fb9abf3b740 1 Processor -- start 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.082+0000 7fb9abf3b740 1 -- start start 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.082+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6e9d0 0x55abfaa6edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.082+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa79670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.082+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa79bb0 0x55abfaa7bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abfa9d5090 con 0x55abfaa6f2f0 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55abfa97ca70 con 0x55abfaa79bb0 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abfa9d1050 con 0x55abfaa6e9d0 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a0ff9640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb98c0a35e0 0x7fb98c0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa79670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa79670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54796/0 (socket says 172.21.15.37:54796) 2024-09-19T06:42:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 -- 172.21.15.37:0/901130409 learned_addr learned my addr 172.21.15.37:0/901130409 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa79bb0 0x55abfaa7bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb98c0a35e0 0x7fb98c0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6e9d0 0x55abfaa6edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 -- 172.21.15.37:0/901130409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6e9d0 msgr2=0x55abfaa6edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a0ff9640 1 -- 172.21.15.37:0/2020000206 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb9980f0740 con 0x7fb9a4102790 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6e9d0 0x55abfaa6edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 -- 172.21.15.37:0/901130409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa79bb0 msgr2=0x55abfaa7bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa79bb0 0x55abfaa7bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a37fe640 1 -- 172.21.15.37:0/901130409 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfa989130 con 0x55abfaa6f2f0 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6e9d0 0x55abfaa6edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.083+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa79bb0 0x55abfaa7bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.084+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa79670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb99400bd10 tx=0x7fb9940107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2c3bad6559711 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.084+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb98c0a35e0 0x7fb98c0a5aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0092f0 tx=0x7fb99c009420 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.084+0000 7fb989ffb640 1 -- 172.21.15.37:0/901130409 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994004030 con 0x55abfaa6f2f0 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.084+0000 7fb989ffb640 1 -- 172.21.15.37:0/901130409 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9940041d0 con 0x55abfaa6f2f0 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.084+0000 7fb989ffb640 1 -- 172.21.15.37:0/901130409 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994011840 con 0x55abfaa6f2f0 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 -- 172.21.15.37:0/901130409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 msgr2=0x55abfaa79670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa79670 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb99400bd10 tx=0x7fb9940107a0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 -- 172.21.15.37:0/901130409 shutdown_connections 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa79bb0 0x55abfaa7bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa79670 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/901130409 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6e9d0 0x55abfaa6edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 -- 172.21.15.37:0/901130409 >> 172.21.15.37:0/901130409 conn(0x55abfaa6af60 msgr2=0x55abfaa6b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 -- 172.21.15.37:0/901130409 shutdown_connections 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 -- 172.21.15.37:0/901130409 wait complete. 2024-09-19T06:42:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 -- start start 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.085+0000 7fb9abf3b740 1 Processor -- start 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa79480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6f2f0 0x55abfaa767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa79bb0 0x55abfaa76d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa7c780 con 0x55abfaa79bb0 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55abfa98ff20 con 0x55abfaa6e9d0 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa7c900 con 0x55abfaa6f2f0 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9abf3b740 10 client.0 _ll_register_callbacks cb 0x55abfa959030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa79bb0 0x55abfaa76d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa79bb0 0x55abfaa76d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54798/0 (socket says 172.21.15.37:54798) 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 -- 172.21.15.37:0/3266272017 learned_addr learned my addr 172.21.15.37:0/3266272017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6f2f0 0x55abfaa767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa79480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 -- 172.21.15.37:0/3266272017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6f2f0 msgr2=0x55abfaa767d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6f2f0 0x55abfaa767d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 -- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 msgr2=0x55abfaa79480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa79480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 -- 172.21.15.37:0/3266272017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfa7462b0 con 0x55abfaa79bb0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa79bb0 0x55abfaa76d10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009720 tx=0x7fb99c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bede075e49f64dca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.086+0000 7fb982ffd640 1 -- 172.21.15.37:0/3266272017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb99c021070 con 0x55abfaa79bb0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb982ffd640 1 -- 172.21.15.37:0/3266272017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb99c010ae0 con 0x55abfaa79bb0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9940041d0 con 0x55abfaa79bb0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb982ffd640 1 -- 172.21.15.37:0/3266272017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb99c02d050 con 0x55abfaa79bb0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 10 client.15438 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55abfaa880a0 con 0x55abfaa79bb0 2024-09-19T06:42:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb982ffd640 1 -- 172.21.15.37:0/3266272017 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb9940041d0 con 0x55abfaa79bb0 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb982ffd640 1 -- 172.21.15.37:0/3266272017 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb99c01a040 con 0x55abfaa79bb0 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 10 client.15438 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 10 client.15438 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 20 client.15438 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaa8a6d0 0x55abfaa8bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 4 client.15438 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abfa9e2600 con 0x55abfaa8a6d0 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.087+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaa8a6d0 0x55abfaa8bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.088+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaa8a6d0 0x55abfaa8bb40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a75f59542c488727 server_cookie=4c9d260d8fe73d96 in_seq=0 out_seq=0 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.088+0000 7fb982ffd640 10 client.15438 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb982ffd640 1 -- 172.21.15.37:0/3266272017 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abfa9e2600 con 0x55abfaa8a6d0 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb982ffd640 10 client.15438 handle_command_reply: tid=0 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 client.15438 shutdown 2024-09-19T06:42:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa79bb0 msgr2=0x55abfaa76d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa79bb0 0x55abfaa76d10 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb99c009720 tx=0x7fb99c004a40 comp rx=0 tx=0).stop 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 shutdown_connections 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaa8a6d0 0x55abfaa8bb40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa79bb0 0x55abfaa76d10 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa6f2f0 0x55abfaa767d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3266272017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa79480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 >> 172.21.15.37:0/3266272017 conn(0x55abfaa6af60 msgr2=0x55abfaa70c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb982ffd640 0 client.15438 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 shutdown_connections 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3266272017 wait complete. 2024-09-19T06:42:12.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.089+0000 7fb9abf3b740 20 client.15438 trim_cache size 0 max 16384 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 Processor -- start 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 -- start start 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa76790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa89e80 con 0x55abfaa6f2f0 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa8a550 con 0x55abfaa6e9d0 2024-09-19T06:42:12.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa8ac20 con 0x55abfaa79bb0 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54806/0 (socket says 172.21.15.37:54806) 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3634016707 learned_addr learned my addr 172.21.15.37:0/3634016707 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa76790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.107+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3634016707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 msgr2=0x55abfaa76790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa76790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3634016707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 msgr2=0x55abfaa8a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3634016707 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abfaa6e800 con 0x55abfaa6f2f0 2024-09-19T06:42:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8ada0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb9980b8c40 tx=0x7fb998049dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8430eb113e99c5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9897fa640 1 -- 172.21.15.37:0/3634016707 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998049040 con 0x55abfaa6f2f0 2024-09-19T06:42:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9897fa640 1 -- 172.21.15.37:0/3634016707 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb998031e50 con 0x55abfaa6f2f0 2024-09-19T06:42:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.108+0000 7fb9897fa640 1 -- 172.21.15.37:0/3634016707 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980c5450 con 0x55abfaa6f2f0 2024-09-19T06:42:12.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3634016707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 msgr2=0x55abfaa8ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8ada0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fb9980b8c40 tx=0x7fb998049dd0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3634016707 shutdown_connections 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa76790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8ada0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3634016707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3634016707 >> 172.21.15.37:0/3634016707 conn(0x55abfaa6af60 msgr2=0x55abfaa75d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3634016707 shutdown_connections 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3634016707 wait complete. 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 -- start start 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.109+0000 7fb9abf3b740 1 Processor -- start 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa8f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa8b9c0 con 0x55abfaa6f2f0 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa8b2f0 con 0x55abfaa6e9d0 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abfaa8c090 con 0x55abfaa79bb0 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 10 client.0 _ll_register_callbacks cb 0x7fb99c009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54816/0 (socket says 172.21.15.37:54816) 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3575954357 learned_addr learned my addr 172.21.15.37:0/3575954357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa8f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3575954357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 msgr2=0x55abfaa8f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa8f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 msgr2=0x55abfaa8ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 -- 172.21.15.37:0/3575954357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abfaaf2dd0 con 0x55abfaa6f2f0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a3fff640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa8f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9a2ffd640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8f290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9980a7110 tx=0x7fb9980b8b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8028b607e14478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb982ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb982ffd640 1 -- 172.21.15.37:0/3575954357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9980d9070 con 0x55abfaa6f2f0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb982ffd640 1 -- 172.21.15.37:0/3575954357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb9980bca40 con 0x55abfaa6f2f0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.110+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb998031e50 con 0x55abfaa6f2f0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb982ffd640 1 -- 172.21.15.37:0/3575954357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb998049040 con 0x55abfaa6f2f0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 10 client.15444 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55abfaaf3f90 con 0x55abfaa6f2f0 2024-09-19T06:42:12.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb982ffd640 1 -- 172.21.15.37:0/3575954357 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb998031e50 con 0x55abfaa6f2f0 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb982ffd640 1 -- 172.21.15.37:0/3575954357 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb9980c4fc0 con 0x55abfaa6f2f0 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 10 client.15444 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 10 client.15444 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 20 client.15444 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaaf8880 0x55abfaaf7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 4 client.15444 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "status"}) -- 0x55abfa9dcb10 con 0x55abfaaf8880 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.111+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaaf8880 0x55abfaaf7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb982ffd640 10 client.15444 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9a37fe640 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaaf8880 0x55abfaaf7590 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc398ae9fabb479d server_cookie=45bff9cee6542f55 in_seq=0 out_seq=0 2024-09-19T06:42:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb982ffd640 1 -- 172.21.15.37:0/3575954357 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55abfa9dcb10 con 0x55abfaaf8880 2024-09-19T06:42:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb982ffd640 10 client.15444 handle_command_reply: tid=0 2024-09-19T06:42:12.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 client.15444 shutdown 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 msgr2=0x55abfaa8f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8f290 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fb9980a7110 tx=0x7fb9980b8b90 comp rx=0 tx=0).stop 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 shutdown_connections 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abfaaf8880 0x55abfaaf7590 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abfaa79bb0 0x55abfaa8f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abfaa6f2f0 0x55abfaa8f290 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 --2- 172.21.15.37:0/3575954357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abfaa6e9d0 0x55abfaa8ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb982ffd640 0 client.15444 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 >> 172.21.15.37:0/3575954357 conn(0x55abfaa6af60 msgr2=0x55abfaa76790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 shutdown_connections 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 1 -- 172.21.15.37:0/3575954357 wait complete. 2024-09-19T06:42:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.112+0000 7fb9abf3b740 20 client.15444 trim_cache size 0 max 16384 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 2, 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24449, 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 130.674569983, 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 84, 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:42:12.120 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 169.08631510999999 2024-09-19T06:42:12.121 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb98c0a35e0 msgr2=0x7fb98c0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb98c0a35e0 0x7fb98c0a5aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb99c0092f0 tx=0x7fb99c009420 comp rx=0 tx=0).stop 2024-09-19T06:42:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 msgr2=0x7fb9a41989a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41989a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fb998004770 tx=0x7fb998003c10 comp rx=0 tx=0).stop 2024-09-19T06:42:12.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 shutdown_connections 2024-09-19T06:42:12.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb9a4106f10 0x7fb9a419cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb98c0a35e0 0x7fb98c0a5aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb9a4102790 0x7fb9a41989a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 --2- 172.21.15.37:0/2020000206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9a4101de0 0x7fb9a4198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:12.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 >> 172.21.15.37:0/2020000206 conn(0x7fb9a40fd8c0 msgr2=0x7fb9a40fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:12.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 shutdown_connections 2024-09-19T06:42:12.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:12.113+0000 7fb9a9b62640 1 -- 172.21.15.37:0/2020000206 wait complete. 2024-09-19T06:42:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:11 smithi096 ceph-mon[35004]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T06:42:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:11 smithi151 ceph-mon[35236]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-09-19T06:42:12.739 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-19T06:42:13.037 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 -- 172.21.15.37:0/3616290784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 msgr2=0x7f47741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 --2- 172.21.15.37:0/3616290784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741021c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4770009900 tx=0x7f477002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 -- 172.21.15.37:0/3616290784 shutdown_connections 2024-09-19T06:42:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 --2- 172.21.15.37:0/3616290784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4774106f10 0x7f477410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 --2- 172.21.15.37:0/3616290784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4774102790 0x7f47741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 --2- 172.21.15.37:0/3616290784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741021c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 -- 172.21.15.37:0/3616290784 >> 172.21.15.37:0/3616290784 conn(0x7f47740fd8c0 msgr2=0x7f47740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 -- 172.21.15.37:0/3616290784 shutdown_connections 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.520+0000 7f477aa99640 1 -- 172.21.15.37:0/3616290784 wait complete. 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 Processor -- start 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 -- start start 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4774102790 0x7f47741988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4774106f10 0x7f477419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4774111f70 con 0x7f4774101de0 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4774111df0 con 0x7f4774102790 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f477aa99640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f47741120f0 con 0x7f4774106f10 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54836/0 (socket says 172.21.15.37:54836) 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 -- 172.21.15.37:0/1564298699 learned_addr learned my addr 172.21.15.37:0/1564298699 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f476bfff640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4774102790 0x7f47741988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4779036640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4774106f10 0x7f477419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 -- 172.21.15.37:0/1564298699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4774106f10 msgr2=0x7f477419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4774106f10 0x7f477419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 -- 172.21.15.37:0/1564298699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4774102790 msgr2=0x7f47741988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4774102790 0x7f47741988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4778835640 1 -- 172.21.15.37:0/1564298699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477419d360 con 0x7f4774101de0 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.521+0000 7f4779036640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4774106f10 0x7f477419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.522+0000 7f4778835640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741983b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4770005ab0 tx=0x7f47700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.522+0000 7f4769ffb640 1 -- 172.21.15.37:0/1564298699 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477003f030 con 0x7f4774101de0 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.522+0000 7f4769ffb640 1 -- 172.21.15.37:0/1564298699 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4770003ce0 con 0x7f4774101de0 2024-09-19T06:42:13.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.522+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477419d5f0 con 0x7f4774101de0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.522+0000 7f4769ffb640 1 -- 172.21.15.37:0/1564298699 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770035070 con 0x7f4774101de0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.522+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47741a4f30 con 0x7f4774101de0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.524+0000 7f4769ffb640 1 -- 172.21.15.37:0/1564298699 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f4770042030 con 0x7f4774101de0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.524+0000 7f477ce72740 1 Processor -- start 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.524+0000 7f477ce72740 1 -- start start 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f477ce72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ac48d0 0x5565e8ac4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f477ce72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f477ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f477ce72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565e8a2b090 con 0x5565e8ac51f0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f477ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565e89d2a70 con 0x5565e8ad3fc0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f477ce72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5565e8a27050 con 0x5565e8ac48d0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54850/0 (socket says 172.21.15.37:54850) 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 -- 172.21.15.37:0/1945106000 learned_addr learned my addr 172.21.15.37:0/1945106000 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4779036640 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ac48d0 0x5565e8ac4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f476bfff640 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4769ffb640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f475c0a3560 0x7f475c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 -- 172.21.15.37:0/1945106000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ac48d0 msgr2=0x5565e8ac4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ac48d0 0x5565e8ac4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 -- 172.21.15.37:0/1945106000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ad3fc0 msgr2=0x5565e8ad63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4778835640 1 -- 172.21.15.37:0/1945106000 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e89df130 con 0x5565e8ac51f0 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.525+0000 7f4779036640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f475c0a3560 0x7f475c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.526+0000 7f4769ffb640 1 -- 172.21.15.37:0/1564298699 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f47700f0900 con 0x7f4774101de0 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.526+0000 7f4778835640 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad3a80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f47700eddb0 tx=0x7f477008b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7920fce964f65c7d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.526+0000 7f475affd640 1 -- 172.21.15.37:0/1945106000 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770049070 con 0x5565e8ac51f0 2024-09-19T06:42:13.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.526+0000 7f4779036640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f475c0a3560 0x7f475c0a5a20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f476c007f70 tx=0x7f476c004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.526+0000 7f475affd640 1 -- 172.21.15.37:0/1945106000 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47700bd0a0 con 0x5565e8ac51f0 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.526+0000 7f475affd640 1 -- 172.21.15.37:0/1945106000 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47700bc4e0 con 0x5565e8ac51f0 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 -- 172.21.15.37:0/1945106000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 msgr2=0x5565e8ad3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad3a80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f47700eddb0 tx=0x7f477008b690 comp rx=0 tx=0).stop 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 -- 172.21.15.37:0/1945106000 shutdown_connections 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad3a80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 --2- 172.21.15.37:0/1945106000 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ac48d0 0x5565e8ac4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 -- 172.21.15.37:0/1945106000 >> 172.21.15.37:0/1945106000 conn(0x5565e8ac0f60 msgr2=0x5565e8ac1340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 -- 172.21.15.37:0/1945106000 shutdown_connections 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 -- 172.21.15.37:0/1945106000 wait complete. 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 -- start start 2024-09-19T06:42:13.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.527+0000 7f477ce72740 1 Processor -- start 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac48d0 0x5565e8ad3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ad69b0 con 0x5565e8ac51f0 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565e89e5f20 con 0x5565e8ac48d0 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ad6b30 con 0x5565e8ad3fc0 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 10 client.0 _ll_register_callbacks cb 0x5565e89af030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54852/0 (socket says 172.21.15.37:54852) 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 -- 172.21.15.37:0/1168778823 learned_addr learned my addr 172.21.15.37:0/1168778823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f4778835640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac48d0 0x5565e8ad3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f4779036640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 -- 172.21.15.37:0/1168778823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 msgr2=0x5565e8ad10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 -- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac48d0 msgr2=0x5565e8ad3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac48d0 0x5565e8ad3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 -- 172.21.15.37:0/1168778823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e879c2b0 con 0x5565e8ac51f0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f4778835640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac48d0 0x5565e8ad3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f476bfff640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad0ba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f47640126d0 tx=0x7f4764012b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b53d816edd1f1fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f4753fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f4753fff640 1 -- 172.21.15.37:0/1168778823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476401f060 con 0x5565e8ac51f0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f4753fff640 1 -- 172.21.15.37:0/1168778823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f4764010dc0 con 0x5565e8ac51f0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.528+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47700bd0a0 con 0x5565e8ac51f0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f4753fff640 1 -- 172.21.15.37:0/1168778823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f476401a430 con 0x5565e8ac51f0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 10 client.15456 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565e8ae2250 con 0x5565e8ac51f0 2024-09-19T06:42:13.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f4753fff640 1 -- 172.21.15.37:0/1168778823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47700bd0a0 con 0x5565e8ac51f0 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f4753fff640 1 -- 172.21.15.37:0/1168778823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4764022590 con 0x5565e8ac51f0 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 10 client.15456 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 10 client.15456 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 20 client.15456 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8ae48e0 0x5565e8ae5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 4 client.15456 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565e8a38600 con 0x5565e8ae48e0 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.529+0000 7f4778835640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8ae48e0 0x5565e8ae5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.530+0000 7f4778835640 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8ae48e0 0x5565e8ae5d50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df47a46f33de0c40 server_cookie=6301477e45f5d238 in_seq=0 out_seq=0 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.530+0000 7f4753fff640 10 client.15456 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.530+0000 7f4753fff640 1 -- 172.21.15.37:0/1168778823 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5565e8a38600 con 0x5565e8ae48e0 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.530+0000 7f4753fff640 10 client.15456 handle_command_reply: tid=0 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.530+0000 7f477ce72740 1 client.15456 shutdown 2024-09-19T06:42:13.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 msgr2=0x5565e8ad0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad0ba0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f47640126d0 tx=0x7f4764012b90 comp rx=0 tx=0).stop 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 shutdown_connections 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8ae48e0 0x5565e8ae5d50 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ad10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac51f0 0x5565e8ad0ba0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 --2- 172.21.15.37:0/1168778823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac48d0 0x5565e8ad3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f4753fff640 0 client.15456 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 >> 172.21.15.37:0/1168778823 conn(0x5565e8ac0f60 msgr2=0x5565e8ac6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 shutdown_connections 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 1 -- 172.21.15.37:0/1168778823 wait complete. 2024-09-19T06:42:13.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.531+0000 7f477ce72740 20 client.15456 trim_cache size 0 max 16384 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 Processor -- start 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 -- start start 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae5d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8adfb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8acc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ae4090 con 0x5565e8ac48d0 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ae4760 con 0x5565e8ac51f0 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f477ce72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ae4e30 con 0x5565e8ad3fc0 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54868/0 (socket says 172.21.15.37:54868) 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 -- 172.21.15.37:0/1179338323 learned_addr learned my addr 172.21.15.37:0/1179338323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f476bfff640 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8adfb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 -- 172.21.15.37:0/1179338323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 msgr2=0x5565e8acc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8acc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 -- 172.21.15.37:0/1179338323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 msgr2=0x5565e8adfb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8adfb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.549+0000 7f4778835640 1 -- 172.21.15.37:0/1179338323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565e8ae0510 con 0x5565e8ac48d0 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f4778835640 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae5d50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f47700edf50 tx=0x7f4770009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4a76e2bc5c8d128 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f475a7fc640 1 -- 172.21.15.37:0/1179338323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770049040 con 0x5565e8ac48d0 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f475a7fc640 1 -- 172.21.15.37:0/1179338323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47700f6300 con 0x5565e8ac48d0 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f475a7fc640 1 -- 172.21.15.37:0/1179338323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47700baf60 con 0x5565e8ac48d0 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f477ce72740 1 -- 172.21.15.37:0/1179338323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 msgr2=0x5565e8ae5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f477ce72740 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae5d50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f47700edf50 tx=0x7f4770009560 comp rx=0 tx=0).stop 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f477ce72740 1 -- 172.21.15.37:0/1179338323 shutdown_connections 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f477ce72740 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8acc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f477ce72740 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8adfb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f477ce72740 1 --2- 172.21.15.37:0/1179338323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae5d50 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.550+0000 7f477ce72740 1 -- 172.21.15.37:0/1179338323 >> 172.21.15.37:0/1179338323 conn(0x5565e8ac0f60 msgr2=0x5565e8acba20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 -- 172.21.15.37:0/1179338323 shutdown_connections 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 -- 172.21.15.37:0/1179338323 wait complete. 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 -- start start 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 Processor -- start 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8ae7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ae79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ae5bd0 con 0x5565e8ac48d0 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ae5500 con 0x5565e8ac51f0 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5565e8ae62a0 con 0x5565e8ad3fc0 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.551+0000 7f477ce72740 10 client.0 _ll_register_callbacks cb 0x7f476400a590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:13.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae6f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54880/0 (socket says 172.21.15.37:54880) 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 -- 172.21.15.37:0/34691570 learned_addr learned my addr 172.21.15.37:0/34691570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4779036640 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ae79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f476bfff640 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8ae7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 -- 172.21.15.37:0/34691570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 msgr2=0x5565e8ae79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ae79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 -- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 msgr2=0x5565e8ae7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8ae7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 -- 172.21.15.37:0/34691570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565e8b4d140 con 0x5565e8ac48d0 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4778835640 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae6f50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f47700bc930 tx=0x7f47700058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ccf97928ebbb22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4753fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:13.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4753fff640 1 -- 172.21.15.37:0/34691570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4770049070 con 0x5565e8ac48d0 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47700f6300 con 0x5565e8ac48d0 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4753fff640 1 -- 172.21.15.37:0/34691570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f477003bd70 con 0x5565e8ac48d0 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f4753fff640 1 -- 172.21.15.37:0/34691570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f47700bb150 con 0x5565e8ac48d0 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f477ce72740 10 client.15462 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.552+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5565e8b4e440 con 0x5565e8ac48d0 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f4753fff640 1 -- 172.21.15.37:0/34691570 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47700f6300 con 0x5565e8ac48d0 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f4753fff640 1 -- 172.21.15.37:0/34691570 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f47700b9010 con 0x5565e8ac48d0 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f477ce72740 10 client.15462 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f477ce72740 10 client.15462 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f477ce72740 20 client.15462 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f477ce72740 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8b52ad0 0x5565e8b517e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f477ce72740 4 client.15462 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:42:13.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "status"}) -- 0x5565e8a32b10 con 0x5565e8b52ad0 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f476bfff640 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8b52ad0 0x5565e8b517e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f476bfff640 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8b52ad0 0x5565e8b517e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a96b47b6ded13c7 server_cookie=5fca79104c7bd920 in_seq=0 out_seq=0 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.553+0000 7f4753fff640 10 client.15462 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f4753fff640 1 -- 172.21.15.37:0/34691570 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5565e8a32b10 con 0x5565e8b52ad0 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f4753fff640 10 client.15462 handle_command_reply: tid=0 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 client.15462 shutdown 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 msgr2=0x5565e8ae6f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae6f50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f47700bc930 tx=0x7f47700058b0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 shutdown_connections 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5565e8b52ad0 0x5565e8b517e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5565e8ad3fc0 0x5565e8ae79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f4753fff640 0 client.15462 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5565e8ac51f0 0x5565e8ae7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 --2- 172.21.15.37:0/34691570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5565e8ac48d0 0x5565e8ae6f50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 >> 172.21.15.37:0/34691570 conn(0x5565e8ac0f60 msgr2=0x5565e8acc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 shutdown_connections 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 1 -- 172.21.15.37:0/34691570 wait complete. 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.554+0000 7f477ce72740 20 client.15462 trim_cache size 0 max 16384 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 1, 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24470, 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:42:13.567 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 133.12636568799999, 2024-09-19T06:42:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-09-19T06:42:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 84, 2024-09-19T06:42:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:42:13.568 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 162.797355031 2024-09-19T06:42:13.568 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:13.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f475c0a3560 msgr2=0x7f475c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f475c0a3560 0x7f475c0a5a20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f476c007f70 tx=0x7f476c004a40 comp rx=0 tx=0).stop 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 msgr2=0x7f47741983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741983b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4770005ab0 tx=0x7f47700042f0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 shutdown_connections 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4774106f10 0x7f477419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4774102790 0x7f47741988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f475c0a3560 0x7f475c0a5a20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 --2- 172.21.15.37:0/1564298699 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4774101de0 0x7f47741983b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 >> 172.21.15.37:0/1564298699 conn(0x7f47740fd8c0 msgr2=0x7f47740fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 shutdown_connections 2024-09-19T06:42:13.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:13.555+0000 7f477aa99640 1 -- 172.21.15.37:0/1564298699 wait complete. 2024-09-19T06:42:14.211 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:13 smithi037 ceph-mon[31211]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-19T06:42:14.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-19T06:42:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:13 smithi096 ceph-mon[35004]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-19T06:42:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:13 smithi151 ceph-mon[35236]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-09-19T06:42:14.499 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:14.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.970+0000 7fb79d489640 1 -- 172.21.15.37:0/1275858629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 msgr2=0x7fb79810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:14.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.970+0000 7fb79d489640 1 --2- 172.21.15.37:0/1275858629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79810e610 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb790009ca0 tx=0x7fb79002a950 comp rx=0 tx=0).stop 2024-09-19T06:42:14.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.970+0000 7fb79d489640 1 -- 172.21.15.37:0/1275858629 shutdown_connections 2024-09-19T06:42:14.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.970+0000 7fb79d489640 1 --2- 172.21.15.37:0/1275858629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79810e610 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:14.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.970+0000 7fb79d489640 1 --2- 172.21.15.37:0/1275858629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb798103660 0x7fb79810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:14.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.970+0000 7fb79d489640 1 --2- 172.21.15.37:0/1275858629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb798102d40 0x7fb798103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:14.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.970+0000 7fb79d489640 1 -- 172.21.15.37:0/1275858629 >> 172.21.15.37:0/1275858629 conn(0x7fb7980fc7a0 msgr2=0x7fb7980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.971+0000 7fb79d489640 1 -- 172.21.15.37:0/1275858629 shutdown_connections 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.971+0000 7fb79d489640 1 -- 172.21.15.37:0/1275858629 wait complete. 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.971+0000 7fb79d489640 1 Processor -- start 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.971+0000 7fb79d489640 1 -- start start 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb79d489640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb798102d40 0x7fb798198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb79d489640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb798103660 0x7fb7981988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb79d489640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb79d489640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb798111f80 con 0x7fb79810c170 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb79d489640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb798111e00 con 0x7fb798103660 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb79d489640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb798112100 con 0x7fb798102d40 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79819cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54902/0 (socket says 172.21.15.37:54902) 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 -- 172.21.15.37:0/3526310769 learned_addr learned my addr 172.21.15.37:0/3526310769 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:15.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7967fc640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb798103660 0x7fb7981988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb796ffd640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb798102d40 0x7fb798198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 -- 172.21.15.37:0/3526310769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb798102d40 msgr2=0x7fb798198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb798102d40 0x7fb798198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 -- 172.21.15.37:0/3526310769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb798103660 msgr2=0x7fb7981988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb798103660 0x7fb7981988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 -- 172.21.15.37:0/3526310769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb79819d310 con 0x7fb79810c170 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb7977fe640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79819cc30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb790009dc0 tx=0x7fb790003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb77ffff640 1 -- 172.21.15.37:0/3526310769 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb790034040 con 0x7fb79810c170 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb77ffff640 1 -- 172.21.15.37:0/3526310769 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb79002fe20 con 0x7fb79810c170 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.972+0000 7fb77ffff640 1 -- 172.21.15.37:0/3526310769 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7900047f0 con 0x7fb79810c170 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.973+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb79819d5a0 con 0x7fb79810c170 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.973+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7981a4ee0 con 0x7fb79810c170 2024-09-19T06:42:15.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 Processor -- start 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb77ffff640 1 -- 172.21.15.37:0/3526310769 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fb790037020 con 0x7fb79810c170 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 -- start start 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab32db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab3d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e9aa99090 con 0x560e9ab3dbb0 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e9aa40a70 con 0x560e9ab329d0 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.975+0000 7fb79f862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e9aa95050 con 0x560e9ab332f0 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54916/0 (socket says 172.21.15.37:54916) 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 -- 172.21.15.37:0/627234263 learned_addr learned my addr 172.21.15.37:0/627234263 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:15.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab32db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb796ffd640 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab3d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb77ffff640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7800a3560 0x7fb7800a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 -- 172.21.15.37:0/627234263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 msgr2=0x560e9ab3d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab3d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 -- 172.21.15.37:0/627234263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 msgr2=0x560e9ab32db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab32db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 -- 172.21.15.37:0/627234263 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9aa4d130 con 0x560e9ab3dbb0 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb77ffff640 1 -- 172.21.15.37:0/3526310769 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb7900f0eb0 con 0x7fb79810c170 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7977fe640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7800a3560 0x7fb7800a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb7967fc640 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ffa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0148c0 tx=0x7fb78c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ab2d73b1beed64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb77d7fa640 1 -- 172.21.15.37:0/627234263 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0142d0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb77d7fa640 1 -- 172.21.15.37:0/627234263 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb78c00fc40 con 0x560e9ab3dbb0 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.976+0000 7fb77d7fa640 1 -- 172.21.15.37:0/627234263 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c012ad0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 -- 172.21.15.37:0/627234263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 msgr2=0x560e9ab3ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb7977fe640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7800a3560 0x7fb7800a5a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7900ee530 tx=0x7fb790031f60 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ffa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb78c0148c0 tx=0x7fb78c00c490 comp rx=0 tx=0).stop 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 -- 172.21.15.37:0/627234263 shutdown_connections 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ffa0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab3d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 --2- 172.21.15.37:0/627234263 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab32db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 -- 172.21.15.37:0/627234263 >> 172.21.15.37:0/627234263 conn(0x560e9ab2ef60 msgr2=0x560e9ab2f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 -- 172.21.15.37:0/627234263 shutdown_connections 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.977+0000 7fb79f862740 1 -- 172.21.15.37:0/627234263 wait complete. 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 -- start start 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 Processor -- start 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 0x560e9ab3d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab332f0 0x560e9ab3a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab40780 con 0x560e9ab3dbb0 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e9aa53f20 con 0x560e9ab332f0 2024-09-19T06:42:15.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab40900 con 0x560e9ab329d0 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb79f862740 10 client.0 _ll_register_callbacks cb 0x560e9aa1d030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54918/0 (socket says 172.21.15.37:54918) 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 -- 172.21.15.37:0/2258721965 learned_addr learned my addr 172.21.15.37:0/2258721965 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb796ffd640 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab332f0 0x560e9ab3a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 0x560e9ab3d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 -- 172.21.15.37:0/2258721965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 msgr2=0x560e9ab3d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 0x560e9ab3d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 -- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab332f0 msgr2=0x560e9ab3a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab332f0 0x560e9ab3a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.978+0000 7fb7967fc640 1 -- 172.21.15.37:0/2258721965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9a80a2b0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb7967fc640 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ad30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb78c012770 tx=0x7fb78c012670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795565b745be123f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb77aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb77aff5640 1 -- 172.21.15.37:0/2258721965 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0140a0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb77aff5640 1 -- 172.21.15.37:0/2258721965 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb78c014240 con 0x560e9ab3dbb0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb78c00fc40 con 0x560e9ab3dbb0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb77aff5640 1 -- 172.21.15.37:0/2258721965 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb78c00f730 con 0x560e9ab3dbb0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 10 client.15474 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e9ab4c0f0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb77aff5640 1 -- 172.21.15.37:0/2258721965 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb78c00fc40 con 0x560e9ab3dbb0 2024-09-19T06:42:15.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb77aff5640 1 -- 172.21.15.37:0/2258721965 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb78c011640 con 0x560e9ab3dbb0 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 10 client.15474 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 10 client.15474 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 20 client.15474 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9ab4e1e0 0x560e9ab4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 4 client.15474 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.979+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e9aaa6600 con 0x560e9ab4e1e0 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.980+0000 7fb796ffd640 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9ab4e1e0 0x560e9ab4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.980+0000 7fb77aff5640 10 client.15474 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.980+0000 7fb796ffd640 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9ab4e1e0 0x560e9ab4d440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=492d1867ba875613 server_cookie=5af71f57c07a7b58 in_seq=0 out_seq=0 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb77aff5640 1 -- 172.21.15.37:0/2258721965 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e9aaa6600 con 0x560e9ab4e1e0 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb77aff5640 10 client.15474 handle_command_reply: tid=0 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 client.15474 shutdown 2024-09-19T06:42:15.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 msgr2=0x560e9ab3ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ad30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb78c012770 tx=0x7fb78c012670 comp rx=0 tx=0).stop 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 shutdown_connections 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9ab4e1e0 0x560e9ab4d440 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9ab3ad30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab332f0 0x560e9ab3a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb77aff5640 0 client.15474 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 --2- 172.21.15.37:0/2258721965 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 0x560e9ab3d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 >> 172.21.15.37:0/2258721965 conn(0x560e9ab2ef60 msgr2=0x560e9ab34c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 shutdown_connections 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 1 -- 172.21.15.37:0/2258721965 wait complete. 2024-09-19T06:42:15.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.981+0000 7fb79f862740 20 client.15474 trim_cache size 0 max 16384 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 Processor -- start 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 -- start start 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 0x560e9ab4d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab332f0 0x560e9ab57d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab3dbb0 0x560e9ab39100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab4d990 con 0x560e9ab332f0 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab4e060 con 0x560e9ab3dbb0 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb79f862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab4e730 con 0x560e9ab329d0 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb796ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab332f0 0x560e9ab57d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb796ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab332f0 0x560e9ab57d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54926/0 (socket says 172.21.15.37:54926) 2024-09-19T06:42:15.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb796ffd640 1 -- 172.21.15.37:0/2531095002 learned_addr learned my addr 172.21.15.37:0/2531095002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:14.999+0000 7fb7967fc640 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab3dbb0 0x560e9ab39100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb796ffd640 1 -- 172.21.15.37:0/2531095002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 msgr2=0x560e9ab4d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb796ffd640 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 0x560e9ab4d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb796ffd640 1 -- 172.21.15.37:0/2531095002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab3dbb0 msgr2=0x560e9ab39100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb796ffd640 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab3dbb0 0x560e9ab39100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb796ffd640 1 -- 172.21.15.37:0/2531095002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e9ab3d0c0 con 0x560e9ab332f0 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb7967fc640 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab3dbb0 0x560e9ab39100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb796ffd640 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab332f0 0x560e9ab57d20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x560e9ab3be30 tx=0x7fb78800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89ce1bc74dcd356b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb77cff9640 1 -- 172.21.15.37:0/2531095002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb788014070 con 0x560e9ab332f0 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb77cff9640 1 -- 172.21.15.37:0/2531095002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb788003d60 con 0x560e9ab332f0 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.000+0000 7fb77cff9640 1 -- 172.21.15.37:0/2531095002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78801c3b0 con 0x560e9ab332f0 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- 172.21.15.37:0/2531095002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab332f0 msgr2=0x560e9ab57d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab332f0 0x560e9ab57d20 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x560e9ab3be30 tx=0x7fb78800cdb0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- 172.21.15.37:0/2531095002 shutdown_connections 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab3dbb0 0x560e9ab39100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab332f0 0x560e9ab57d20 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 --2- 172.21.15.37:0/2531095002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab329d0 0x560e9ab4d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- 172.21.15.37:0/2531095002 >> 172.21.15.37:0/2531095002 conn(0x560e9ab2ef60 msgr2=0x560e9ab29ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- 172.21.15.37:0/2531095002 shutdown_connections 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- 172.21.15.37:0/2531095002 wait complete. 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- start start 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 Processor -- start 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab37680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab37bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9abb3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab4f4d0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab4ee00 con 0x560e9ab329d0 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e9ab4fba0 con 0x560e9ab332f0 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb79f862740 10 client.0 _ll_register_callbacks cb 0x7fb78c00ce00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.001+0000 7fb7967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9abb3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9abb3aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54942/0 (socket says 172.21.15.37:54942) 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 -- 172.21.15.37:0/388242994 learned_addr learned my addr 172.21.15.37:0/388242994 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb796ffd640 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab37bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab37680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 -- 172.21.15.37:0/388242994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 msgr2=0x560e9ab37bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab37bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 -- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 msgr2=0x560e9ab37680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab37680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 -- 172.21.15.37:0/388242994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e9abb4140 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb7967fc640 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9abb3aa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb78c012740 tx=0x7fb78c016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18450af3d8906114 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb77aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb77aff5640 1 -- 172.21.15.37:0/388242994 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0142d0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb77aff5640 1 -- 172.21.15.37:0/388242994 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb78c00f8d0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb788003d60 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb77aff5640 1 -- 172.21.15.37:0/388242994 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb78c00fad0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb79f862740 10 client.15480 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.002+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560e9abbc6d0 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb77aff5640 1 -- 172.21.15.37:0/388242994 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb788003d60 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb77aff5640 1 -- 172.21.15.37:0/388242994 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb78c037430 con 0x560e9ab3dbb0 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb79f862740 10 client.15480 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb79f862740 10 client.15480 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb79f862740 20 client.15480 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:15.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb79f862740 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9abc1210 0x560e9abbff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb79f862740 4 client.15480 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "status"}) -- 0x560e9aaa0b10 con 0x560e9abc1210 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb796ffd640 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9abc1210 0x560e9abbff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb77aff5640 10 client.15480 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.003+0000 7fb796ffd640 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9abc1210 0x560e9abbff20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa2b418ff311765f server_cookie=129beaf2aa4c5656 in_seq=0 out_seq=0 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb77aff5640 1 -- 172.21.15.37:0/388242994 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x560e9aaa0b10 con 0x560e9abc1210 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb77aff5640 10 client.15480 handle_command_reply: tid=0 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 client.15480 shutdown 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 msgr2=0x560e9abb3aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9abb3aa0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb78c012740 tx=0x7fb78c016040 comp rx=0 tx=0).stop 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 shutdown_connections 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e9abc1210 0x560e9abbff20 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e9ab3dbb0 0x560e9abb3aa0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e9ab332f0 0x560e9ab37bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 --2- 172.21.15.37:0/388242994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e9ab329d0 0x560e9ab37680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 >> 172.21.15.37:0/388242994 conn(0x560e9ab2ef60 msgr2=0x560e9ab390c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb77aff5640 0 client.15480 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 shutdown_connections 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 1 -- 172.21.15.37:0/388242994 wait complete. 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.004+0000 7fb79f862740 20 client.15480 trim_cache size 0 max 16384 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "cluster_fsid": "b715dee2-7650-11ef-bceb-c7b262605968", 2024-09-19T06:42:15.020 INFO:teuthology.orchestra.run.smithi037.stdout: "whoami": 0, 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "id": 24497, 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "want_state": "up:active", 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "state": "up:active", 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "fs_name": "cephfs", 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "rank_uptime": 141.598002877, 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "mdsmap_epoch": 25, 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch": 84, 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_epoch_barrier": 83, 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout: "uptime": 153.40245142099999 2024-09-19T06:42:15.021 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:15.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7800a3560 msgr2=0x7fb7800a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7800a3560 0x7fb7800a5a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb7900ee530 tx=0x7fb790031f60 comp rx=0 tx=0).stop 2024-09-19T06:42:15.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 msgr2=0x7fb79819cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:15.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79819cc30 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb790009dc0 tx=0x7fb790003f10 comp rx=0 tx=0).stop 2024-09-19T06:42:15.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 shutdown_connections 2024-09-19T06:42:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7800a3560 0x7fb7800a5a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb79810c170 0x7fb79819cc30 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb798103660 0x7fb7981988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 --2- 172.21.15.37:0/3526310769 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb798102d40 0x7fb798198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 >> 172.21.15.37:0/3526310769 conn(0x7fb7980fc7a0 msgr2=0x7fb7980fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 shutdown_connections 2024-09-19T06:42:15.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:15.005+0000 7fb79d489640 1 -- 172.21.15.37:0/3526310769 wait complete. 2024-09-19T06:42:15.703 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-19T06:42:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:15 smithi096 ceph-mon[35004]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-19T06:42:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:15 smithi037 ceph-mon[31211]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-19T06:42:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:15 smithi151 ceph-mon[35236]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.1 KiB/s wr, 10 op/s 2024-09-19T06:42:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:17 smithi096 ceph-mon[35004]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T06:42:18.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:17 smithi037 ceph-mon[31211]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T06:42:18.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:17 smithi151 ceph-mon[35236]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-09-19T06:42:20.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:19 smithi096 ceph-mon[35004]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-19T06:42:20.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:19 smithi037 ceph-mon[31211]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-19T06:42:20.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:19 smithi151 ceph-mon[35236]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-19T06:42:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:21 smithi096 ceph-mon[35004]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-19T06:42:22.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:21 smithi037 ceph-mon[31211]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-19T06:42:22.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:21 smithi151 ceph-mon[35236]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.9 KiB/s wr, 10 op/s 2024-09-19T06:42:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:23 smithi096 ceph-mon[35004]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-19T06:42:24.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:23 smithi037 ceph-mon[31211]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-19T06:42:24.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:23 smithi151 ceph-mon[35236]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-09-19T06:42:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 -- 172.21.15.37:0/2183284058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 msgr2=0x7fbc9c10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 --2- 172.21.15.37:0/2183284058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c10e9b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009cd0 tx=0x7fbc9402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 -- 172.21.15.37:0/2183284058 shutdown_connections 2024-09-19T06:42:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 --2- 172.21.15.37:0/2183284058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c10e9b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 --2- 172.21.15.37:0/2183284058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c101ae0 0x7fbc9c101f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 --2- 172.21.15.37:0/2183284058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9c107b80 0x7fbc9c107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.476 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 -- 172.21.15.37:0/2183284058 >> 172.21.15.37:0/2183284058 conn(0x7fbc9c0fd7e0 msgr2=0x7fbc9c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 -- 172.21.15.37:0/2183284058 shutdown_connections 2024-09-19T06:42:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 -- 172.21.15.37:0/2183284058 wait complete. 2024-09-19T06:42:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 Processor -- start 2024-09-19T06:42:24.477 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.475+0000 7fbca29b7640 1 -- start start 2024-09-19T06:42:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca29b7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c101ae0 0x7fbc9c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca29b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca29b7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9c107b80 0x7fbc9c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca29b7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111f30 con 0x7fbc9c102480 2024-09-19T06:42:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca29b7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c111db0 con 0x7fbc9c107b80 2024-09-19T06:42:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca29b7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1120b0 con 0x7fbc9c101ae0 2024-09-19T06:42:24.479 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51736/0 (socket says 172.21.15.37:51736) 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3863607322 learned_addr learned my addr 172.21.15.37:0/3863607322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c101ae0 0x7fbc9c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9c107b80 0x7fbc9c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3863607322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c101ae0 msgr2=0x7fbc9c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c101ae0 0x7fbc9c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3863607322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9c107b80 msgr2=0x7fbc9c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9c107b80 0x7fbc9c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3863607322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc9c19d2f0 con 0x7fbc9c102480 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9c107b80 0x7fbc9c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.476+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c198840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000e9f0 tx=0x7fbc9000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.477+0000 7fbc997fa640 1 -- 172.21.15.37:0/3863607322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000cd80 con 0x7fbc9c102480 2024-09-19T06:42:24.480 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.477+0000 7fbc997fa640 1 -- 172.21.15.37:0/3863607322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc90005500 con 0x7fbc9c102480 2024-09-19T06:42:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.477+0000 7fbc997fa640 1 -- 172.21.15.37:0/3863607322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc90010690 con 0x7fbc9c102480 2024-09-19T06:42:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.477+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc9c19d5e0 con 0x7fbc9c102480 2024-09-19T06:42:24.481 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.477+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc9c1a4e70 con 0x7fbc9c102480 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.479+0000 7fbc997fa640 1 -- 172.21.15.37:0/3863607322 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fbc900046c0 con 0x7fbc9c102480 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.479+0000 7fbca582b740 1 Processor -- start 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.479+0000 7fbca582b740 1 -- start start 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca582b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c1ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca582b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c28770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca582b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c2b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca582b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5619b2b822f0 con 0x5619b2c1e410 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca582b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5619b2b2a270 con 0x5619b2c1daf0 2024-09-19T06:42:24.487 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca582b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5619b2b7e2b0 con 0x5619b2c28cb0 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c28770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c28770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51744/0 (socket says 172.21.15.37:51744) 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 -- 172.21.15.37:0/4176287250 learned_addr learned my addr 172.21.15.37:0/4176287250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca0f54640 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c1ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c2b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc997fa640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc840a3690 0x7fbc840a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 -- 172.21.15.37:0/4176287250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 msgr2=0x5619b2c2b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c2b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 -- 172.21.15.37:0/4176287250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 msgr2=0x5619b2c1ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c1ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 -- 172.21.15.37:0/4176287250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b2b29340 con 0x5619b2c1e410 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc997fa640 1 -- 172.21.15.37:0/3863607322 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fbc900d0420 con 0x7fbc9c102480 2024-09-19T06:42:24.488 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc840a3690 0x7fbc840a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.480+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c28770 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00bc50 tx=0x7fbc8c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6d9035687ad49e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbc827fc640 1 -- 172.21.15.37:0/4176287250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c01c480 con 0x5619b2c1e410 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbc827fc640 1 -- 172.21.15.37:0/4176287250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c01cb00 con 0x5619b2c1e410 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbc827fc640 1 -- 172.21.15.37:0/4176287250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c011b90 con 0x5619b2c1e410 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 -- 172.21.15.37:0/4176287250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 msgr2=0x5619b2c28770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c28770 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00bc50 tx=0x7fbc8c0107a0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc840a3690 0x7fbc840a5b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009430 tx=0x7fbc9403f040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 -- 172.21.15.37:0/4176287250 shutdown_connections 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c2b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.489 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c28770 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 --2- 172.21.15.37:0/4176287250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c1ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 -- 172.21.15.37:0/4176287250 >> 172.21.15.37:0/4176287250 conn(0x5619b2c1a060 msgr2=0x5619b2c1a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 -- 172.21.15.37:0/4176287250 shutdown_connections 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.481+0000 7fbca582b740 1 -- 172.21.15.37:0/4176287250 wait complete. 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 -- start start 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 Processor -- start 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1daf0 0x5619b2c25c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1e410 0x5619b2c26140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5619b2c2b880 con 0x5619b2c1daf0 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5619b2b0b290 con 0x5619b2c1e410 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5619b2c2ba00 con 0x5619b2c28cb0 2024-09-19T06:42:24.490 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbca582b740 10 client.0 _ll_register_callbacks cb 0x5619b2b3c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.482+0000 7fbc9bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1daf0 0x5619b2c25c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1daf0 0x5619b2c25c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51758/0 (socket says 172.21.15.37:51758) 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 -- 172.21.15.37:0/3825053739 learned_addr learned my addr 172.21.15.37:0/3825053739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1e410 0x5619b2c26140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 -- 172.21.15.37:0/3825053739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 msgr2=0x5619b2c26680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c26680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 -- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1e410 msgr2=0x5619b2c26140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1e410 0x5619b2c26140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 -- 172.21.15.37:0/3825053739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b2895210 con 0x5619b2c1daf0 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1daf0 0x5619b2c25c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00bc20 tx=0x7fbc8c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb918cdd532b71d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:24.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3825053739 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc8c01c690 con 0x5619b2c1daf0 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc8c01cb00 con 0x5619b2c1daf0 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3825053739 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fbc8c01c830 con 0x5619b2c1daf0 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3825053739 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc8c004020 con 0x5619b2c1daf0 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbca582b740 10 client.15492 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.483+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619b2961620 con 0x5619b2c1daf0 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3825053739 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc8c01cb00 con 0x5619b2c1daf0 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3825053739 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc8c0037b0 con 0x5619b2c1daf0 2024-09-19T06:42:24.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbca582b740 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbca582b740 10 client.15492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbca582b740 20 client.15492 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbca582b740 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2c38c10 0x5619b2c3b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbca582b740 4 client.15492 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619b2b8fa20 con 0x5619b2c38c10 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.484+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2c38c10 0x5619b2c3b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.485+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2c38c10 0x5619b2c3b000 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8a3b226a37f4574 server_cookie=e9ca1034fab79044 in_seq=0 out_seq=0 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.485+0000 7fbc7b7fe640 10 client.15492 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.485+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3825053739 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619b2b8fa20 con 0x5619b2c38c10 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.485+0000 7fbc7b7fe640 10 client.15492 handle_command_reply: tid=0 2024-09-19T06:42:24.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.485+0000 7fbca582b740 1 client.15492 shutdown 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1daf0 msgr2=0x5619b2c25c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1daf0 0x5619b2c25c00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fbc8c00bc20 tx=0x7fbc8c005850 comp rx=0 tx=0).stop 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 shutdown_connections 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2c38c10 0x5619b2c3b000 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c26680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1e410 0x5619b2c26140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 --2- 172.21.15.37:0/3825053739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1daf0 0x5619b2c25c00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 >> 172.21.15.37:0/3825053739 conn(0x5619b2c1a060 msgr2=0x5619b2c1fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbc7b7fe640 0 client.15492 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 shutdown_connections 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 1 -- 172.21.15.37:0/3825053739 wait complete. 2024-09-19T06:42:24.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.486+0000 7fbca582b740 20 client.15492 trim_cache size 0 max 16384 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.500+0000 7fbca582b740 1 Processor -- start 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.500+0000 7fbca582b740 1 -- start start 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca582b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c1daf0 0x5619b2c264b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca582b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c27b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca582b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c28cb0 0x5619b2c405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca582b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1b0170 con 0x5619b2c1e410 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca582b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ada40 con 0x5619b2c28cb0 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca582b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ab890 con 0x5619b2c1daf0 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c27b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c27b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51768/0 (socket says 172.21.15.37:51768) 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3968528836 learned_addr learned my addr 172.21.15.37:0/3968528836 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3968528836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c1daf0 msgr2=0x5619b2c264b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c1daf0 0x5619b2c264b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c28cb0 0x5619b2c405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c1daf0 0x5619b2c264b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3968528836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c28cb0 msgr2=0x5619b2c405a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c28cb0 0x5619b2c405a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3968528836 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619b2c3b370 con 0x5619b2c1e410 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c28cb0 0x5619b2c405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c27b70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbc9009c1c0 tx=0x7fbc900d28b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f147bc9c7444645e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc81ffb640 1 -- 172.21.15.37:0/3968528836 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc9009cbc0 con 0x5619b2c1e410 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc81ffb640 1 -- 172.21.15.37:0/3968528836 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9009c4c0 con 0x5619b2c1e410 2024-09-19T06:42:24.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.501+0000 7fbc81ffb640 1 -- 172.21.15.37:0/3968528836 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900a4f00 con 0x5619b2c1e410 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 -- 172.21.15.37:0/3968528836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 msgr2=0x5619b2c27b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c27b70 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbc9009c1c0 tx=0x7fbc900d28b0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 -- 172.21.15.37:0/3968528836 shutdown_connections 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c28cb0 0x5619b2c405a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c27b70 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 --2- 172.21.15.37:0/3968528836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c1daf0 0x5619b2c264b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 -- 172.21.15.37:0/3968528836 >> 172.21.15.37:0/3968528836 conn(0x5619b2c1a060 msgr2=0x5619b2c27460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 -- 172.21.15.37:0/3968528836 shutdown_connections 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 -- 172.21.15.37:0/3968528836 wait complete. 2024-09-19T06:42:24.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.502+0000 7fbca582b740 1 -- start start 2024-09-19T06:42:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 1 Processor -- start 2024-09-19T06:42:24.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c2c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c2c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c9b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1abf00 con 0x5619b2c1e410 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ad370 con 0x5619b2c1daf0 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9c1ac5d0 con 0x5619b2c28cb0 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca582b740 10 client.0 _ll_register_callbacks cb 0x7fbc90005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c2c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c2c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51776/0 (socket says 172.21.15.37:51776) 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3171453079 learned_addr learned my addr 172.21.15.37:0/3171453079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c2c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c9b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3171453079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 msgr2=0x5619b2c9b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c9b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 msgr2=0x5619b2c2c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c2c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9b7fe640 1 -- 172.21.15.37:0/3171453079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619b2c9bb70 con 0x5619b2c1e410 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbca0f54640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c9b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.503+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c2c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbc9b7fe640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c2c940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbc90005b80 tx=0x7fbc900a0870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4def908e21f8df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:24.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbc7b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3171453079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc900b9020 con 0x5619b2c1e410 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3171453079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fbc900b5230 con 0x5619b2c1e410 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc9009c4c0 con 0x5619b2c1e410 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3171453079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc900b5430 con 0x5619b2c1e410 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 10 client.15498 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619b2ca2e80 con 0x5619b2c1e410 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3171453079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc9009c4c0 con 0x5619b2c1e410 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3171453079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fbc90098210 con 0x5619b2c1e410 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 10 client.15498 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 10 client.15498 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 20 client.15498 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:24.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2ca3580 0x5619b2ca59a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 4 client.15498 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:42:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.504+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5619b2b89d70 con 0x5619b2ca3580 2024-09-19T06:42:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.505+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2ca3580 0x5619b2ca59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.505+0000 7fbc7b7fe640 10 client.15498 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:24.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.505+0000 7fbc9bfff640 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2ca3580 0x5619b2ca59a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98c360a0a2eeb9c8 server_cookie=d2a243e36ba872cf in_seq=0 out_seq=0 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbc7b7fe640 1 -- 172.21.15.37:0/3171453079 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5619b2b89d70 con 0x5619b2ca3580 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbc7b7fe640 10 client.15498 handle_command_reply: tid=0 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 client.15498 shutdown 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 msgr2=0x5619b2c2c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c2c940 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbc90005b80 tx=0x7fbc900a0870 comp rx=0 tx=0).stop 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 shutdown_connections 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619b2ca3580 0x5619b2ca59a0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619b2c28cb0 0x5619b2c9b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619b2c1e410 0x5619b2c2c940 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 --2- 172.21.15.37:0/3171453079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619b2c1daf0 0x5619b2c2c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbc7b7fe640 0 client.15498 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 >> 172.21.15.37:0/3171453079 conn(0x5619b2c1a060 msgr2=0x5619b2c191c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 shutdown_connections 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 1 -- 172.21.15.37:0/3171453079 wait complete. 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.511+0000 7fbca582b740 20 client.15498 trim_cache size 0 max 16384 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:42:24.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:42:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc840a3690 msgr2=0x7fbc840a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc840a3690 0x7fbc840a5b50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009430 tx=0x7fbc9403f040 comp rx=0 tx=0).stop 2024-09-19T06:42:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 msgr2=0x7fbc9c198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:24.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c198840 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000e9f0 tx=0x7fbc9000edf0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 shutdown_connections 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc9c107b80 0x7fbc9c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc840a3690 0x7fbc840a5b50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc9c102480 0x7fbc9c198840 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 --2- 172.21.15.37:0/3863607322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc9c101ae0 0x7fbc9c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 >> 172.21.15.37:0/3863607322 conn(0x7fbc9c0fd7e0 msgr2=0x7fbc9c0fdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 shutdown_connections 2024-09-19T06:42:24.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:24.512+0000 7fbca29b7640 1 -- 172.21.15.37:0/3863607322 wait complete. 2024-09-19T06:42:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:25 smithi096 ceph-mon[35004]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T06:42:26.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:25 smithi037 ceph-mon[31211]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T06:42:26.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:25 smithi151 ceph-mon[35236]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-19T06:42:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:27 smithi096 ceph-mon[35004]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T06:42:28.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:27 smithi037 ceph-mon[31211]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T06:42:28.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:27 smithi151 ceph-mon[35236]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-19T06:42:28.819 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:42:29.093 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 -- 172.21.15.37:0/2488692945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78102ef0 msgr2=0x7fcc7810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 --2- 172.21.15.37:0/2488692945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78102ef0 0x7fcc7810da40 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009ca0 tx=0x7fcc7002a950 comp rx=0 tx=0).stop 2024-09-19T06:42:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 -- 172.21.15.37:0/2488692945 shutdown_connections 2024-09-19T06:42:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 --2- 172.21.15.37:0/2488692945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78102ef0 0x7fcc7810da40 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 --2- 172.21.15.37:0/2488692945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc78102570 0x7fcc781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 --2- 172.21.15.37:0/2488692945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78100e20 0x7fcc78101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 -- 172.21.15.37:0/2488692945 >> 172.21.15.37:0/2488692945 conn(0x7fcc780fc7d0 msgr2=0x7fcc780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 -- 172.21.15.37:0/2488692945 shutdown_connections 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.600+0000 7fcc805ac640 1 -- 172.21.15.37:0/2488692945 wait complete. 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 Processor -- start 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 -- start start 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78100e20 0x7fcc780fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc78102570 0x7fcc780fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78102ef0 0x7fcc780fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78110f80 con 0x7fcc78100e20 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78110e00 con 0x7fcc78102ef0 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc805ac640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc78111100 con 0x7fcc78102570 2024-09-19T06:42:29.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.601+0000 7fcc7e348640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78100e20 0x7fcc780fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78100e20 0x7fcc780fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54466/0 (socket says 172.21.15.37:54466) 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 -- 172.21.15.37:0/4236462536 learned_addr learned my addr 172.21.15.37:0/4236462536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7db47640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc78102570 0x7fcc780fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78102ef0 0x7fcc780fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 -- 172.21.15.37:0/4236462536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc78102570 msgr2=0x7fcc780fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc78102570 0x7fcc780fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 -- 172.21.15.37:0/4236462536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78102ef0 msgr2=0x7fcc780fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78102ef0 0x7fcc780fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 -- 172.21.15.37:0/4236462536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc781a6980 con 0x7fcc78100e20 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78102ef0 0x7fcc780fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc7e348640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78100e20 0x7fcc780fcc80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400d950 tx=0x7fcc7400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc6f7fe640 1 -- 172.21.15.37:0/4236462536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc74017030 con 0x7fcc78100e20 2024-09-19T06:42:29.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc781a6c70 con 0x7fcc78100e20 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc6f7fe640 1 -- 172.21.15.37:0/4236462536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc74004020 con 0x7fcc78100e20 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc6f7fe640 1 -- 172.21.15.37:0/4236462536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc740047a0 con 0x7fcc78100e20 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.602+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc781a71b0 con 0x7fcc78100e20 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.605+0000 7fcc82985740 1 Processor -- start 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.605+0000 7fcc82985740 1 -- start start 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.605+0000 7fcc6f7fe640 1 -- 172.21.15.37:0/4236462536 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fcc740041d0 con 0x7fcc78100e20 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.605+0000 7fcc82985740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e561401880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.605+0000 7fcc82985740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e56140c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc82985740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e56140ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc82985740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e5613672e0 con 0x55e56140c680 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc82985740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e56130ece0 con 0x55e5614014a0 2024-09-19T06:42:29.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc82985740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5613632a0 con 0x55e561401dc0 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e56140ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e56140ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54468/0 (socket says 172.21.15.37:54468) 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 -- 172.21.15.37:0/2776562483 learned_addr learned my addr 172.21.15.37:0/2776562483 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7e348640 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e56140c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e561401880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 -- 172.21.15.37:0/2776562483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 msgr2=0x55e56140c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e56140c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 -- 172.21.15.37:0/2776562483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 msgr2=0x55e561401880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e561401880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 -- 172.21.15.37:0/2776562483 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5612f3440 con 0x55e56140c680 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc7db47640 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e56140ea70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800bc30 tx=0x7fcc680107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b717cfd01dac8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.606+0000 7fcc6f7fe640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcc640a3560 0x7fcc640a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc6cff9640 1 -- 172.21.15.37:0/2776562483 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc6801c480 con 0x55e56140c680 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcc640a3560 0x7fcc640a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc6f7fe640 1 -- 172.21.15.37:0/4236462536 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fcc740cf9e0 con 0x7fcc78100e20 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc6cff9640 1 -- 172.21.15.37:0/2776562483 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6801cb00 con 0x55e56140c680 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc6cff9640 1 -- 172.21.15.37:0/2776562483 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc68011b50 con 0x55e56140c680 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc82985740 1 -- 172.21.15.37:0/2776562483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 msgr2=0x55e56140ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc82985740 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e56140ea70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fcc6800bc30 tx=0x7fcc680107a0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcc640a3560 0x7fcc640a5a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005cc0 tx=0x7fcc70005c50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc82985740 1 -- 172.21.15.37:0/2776562483 shutdown_connections 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.607+0000 7fcc82985740 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e56140ea70 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e56140c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 --2- 172.21.15.37:0/2776562483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e561401880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 -- 172.21.15.37:0/2776562483 >> 172.21.15.37:0/2776562483 conn(0x55e5613fda30 msgr2=0x55e5613fde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:29.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 -- 172.21.15.37:0/2776562483 shutdown_connections 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 -- 172.21.15.37:0/2776562483 wait complete. 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 -- start start 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 Processor -- start 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e56140bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e5614091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561409730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e56140f250 con 0x55e5614014a0 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e561322050 con 0x55e561401dc0 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e56140f3d0 con 0x55e56140c680 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc82985740 10 client.0 _ll_register_callbacks cb 0x55e561362160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.608+0000 7fcc7e348640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e56140bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e56140bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54480/0 (socket says 172.21.15.37:54480) 2024-09-19T06:42:29.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 -- 172.21.15.37:0/108761079 learned_addr learned my addr 172.21.15.37:0/108761079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7db47640 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e5614091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561409730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 -- 172.21.15.37:0/108761079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 msgr2=0x55e561409730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561409730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 -- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 msgr2=0x55e5614091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e5614091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 -- 172.21.15.37:0/108761079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5610d83c0 con 0x55e5614014a0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc7e348640 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e56140bf40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409ba10 tx=0x7fcc74097e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0e5bd2bb238b3b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc61ffb640 1 -- 172.21.15.37:0/108761079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7409c180 con 0x55e5614014a0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc61ffb640 1 -- 172.21.15.37:0/108761079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fcc7409aed0 con 0x55e5614014a0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc6801cb00 con 0x55e5614014a0 2024-09-19T06:42:29.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc61ffb640 1 -- 172.21.15.37:0/108761079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc7409ba80 con 0x55e5614014a0 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc82985740 10 client.15510 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.609+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e56141abd0 con 0x55e5614014a0 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc61ffb640 1 -- 172.21.15.37:0/108761079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc6801cb00 con 0x55e5614014a0 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc61ffb640 1 -- 172.21.15.37:0/108761079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc7409a940 con 0x55e5614014a0 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc82985740 10 client.15510 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc82985740 10 client.15510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc82985740 20 client.15510 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc82985740 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e56141e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc82985740 4 client.15510 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e561374850 con 0x55e56141bf20 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.610+0000 7fcc7db47640 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e56141e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.611+0000 7fcc61ffb640 10 client.15510 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.611+0000 7fcc7db47640 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e56141e350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63bc3886a556d8ed server_cookie=cb0783ba8897e428 in_seq=0 out_seq=0 2024-09-19T06:42:29.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.611+0000 7fcc61ffb640 1 -- 172.21.15.37:0/108761079 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e561374850 con 0x55e56141bf20 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.611+0000 7fcc61ffb640 10 client.15510 handle_command_reply: tid=0 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.611+0000 7fcc82985740 1 client.15510 shutdown 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 msgr2=0x55e56140bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e56140bf40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409ba10 tx=0x7fcc74097e10 comp rx=0 tx=0).stop 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 shutdown_connections 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e56141e350 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc61ffb640 0 client.15510 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561409730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e5614091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 --2- 172.21.15.37:0/108761079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e56140bf40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 >> 172.21.15.37:0/108761079 conn(0x55e5613fda30 msgr2=0x55e561403710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 shutdown_connections 2024-09-19T06:42:29.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 1 -- 172.21.15.37:0/108761079 wait complete. 2024-09-19T06:42:29.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.612+0000 7fcc82985740 20 client.15510 trim_cache size 0 max 16384 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 Processor -- start 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 -- start start 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e5614184e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e561423bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e561424100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781ae010 con 0x55e56140c680 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781ab8e0 con 0x55e5614014a0 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.630+0000 7fcc82985740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781a9750 con 0x55e561401dc0 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e561424100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e561424100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54496/0 (socket says 172.21.15.37:54496) 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 -- 172.21.15.37:0/3288360582 learned_addr learned my addr 172.21.15.37:0/3288360582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7e348640 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e5614184e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7db47640 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e561423bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 -- 172.21.15.37:0/3288360582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 msgr2=0x55e561423bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e561423bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 -- 172.21.15.37:0/3288360582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 msgr2=0x55e5614184e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e5614184e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 -- 172.21.15.37:0/3288360582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5613fd860 con 0x55e56140c680 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e561424100 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009ca0 tx=0x7fcc70040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef076ae5e430bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc63fff640 1 -- 172.21.15.37:0/3288360582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc70034040 con 0x55e56140c680 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc63fff640 1 -- 172.21.15.37:0/3288360582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc70003e50 con 0x55e56140c680 2024-09-19T06:42:29.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.631+0000 7fcc63fff640 1 -- 172.21.15.37:0/3288360582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc700306a0 con 0x55e56140c680 2024-09-19T06:42:29.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 -- 172.21.15.37:0/3288360582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 msgr2=0x55e561424100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e561424100 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcc70009ca0 tx=0x7fcc70040000 comp rx=0 tx=0).stop 2024-09-19T06:42:29.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 -- 172.21.15.37:0/3288360582 shutdown_connections 2024-09-19T06:42:29.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e56140c680 0x55e561424100 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e561401dc0 0x55e561423bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 --2- 172.21.15.37:0/3288360582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5614014a0 0x55e5614184e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 -- 172.21.15.37:0/3288360582 >> 172.21.15.37:0/3288360582 conn(0x55e5613fda30 msgr2=0x55e561407ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 -- 172.21.15.37:0/3288360582 shutdown_connections 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.632+0000 7fcc82985740 1 -- 172.21.15.37:0/3288360582 wait complete. 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 -- start start 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 Processor -- start 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e561414f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e561410280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561410a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781a9da0 con 0x55e5614014a0 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781ab210 con 0x55e561401dc0 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcc781aa470 con 0x55e56140c680 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc82985740 10 client.0 _ll_register_callbacks cb 0x7fcc74005130 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc7e348640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e561414f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc7e348640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e561414f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54506/0 (socket says 172.21.15.37:54506) 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc7e348640 1 -- 172.21.15.37:0/1597509741 learned_addr learned my addr 172.21.15.37:0/1597509741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc7db47640 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e561410280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.633+0000 7fcc7eb49640 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561410a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc7e348640 1 -- 172.21.15.37:0/1597509741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 msgr2=0x55e561410a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc7e348640 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561410a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc7e348640 1 -- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 msgr2=0x55e561410280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc7e348640 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e561410280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc7e348640 1 -- 172.21.15.37:0/1597509741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e561402330 con 0x55e5614014a0 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc7e348640 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e561414f50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409bc40 tx=0x7fcc74098300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6efa9e1653ee30 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc61ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc61ffb640 1 -- 172.21.15.37:0/1597509741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc7409b5c0 con 0x55e5614014a0 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc61ffb640 1 -- 172.21.15.37:0/1597509741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fcc7409b0d0 con 0x55e5614014a0 2024-09-19T06:42:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc70003e50 con 0x55e5614014a0 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc61ffb640 1 -- 172.21.15.37:0/1597509741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcc7409c180 con 0x55e5614014a0 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc82985740 10 client.15516 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.634+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e561486b90 con 0x55e5614014a0 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc61ffb640 1 -- 172.21.15.37:0/1597509741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcc70003e50 con 0x55e5614014a0 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc61ffb640 1 -- 172.21.15.37:0/1597509741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fcc74009a30 con 0x55e5614014a0 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc82985740 10 client.15516 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc82985740 10 client.15516 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc82985740 20 client.15516 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc82985740 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e5614890e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc82985740 4 client.15516 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e56136ed60 con 0x55e56141bf20 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.635+0000 7fcc7db47640 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e5614890e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc7db47640 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e5614890e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=983d58ac661e4f11 server_cookie=3d30d10b2b1611e4 in_seq=0 out_seq=0 2024-09-19T06:42:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc61ffb640 10 client.15516 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:29.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc61ffb640 1 -- 172.21.15.37:0/1597509741 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e56136ed60 con 0x55e56141bf20 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc61ffb640 10 client.15516 handle_command_reply: tid=0 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 client.15516 shutdown 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 msgr2=0x55e561414f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e561414f50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcc7409bc40 tx=0x7fcc74098300 comp rx=0 tx=0).stop 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 shutdown_connections 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e56141bf20 0x55e5614890e0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e56140c680 0x55e561410a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e561401dc0 0x55e561410280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc61ffb640 0 client.15516 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 --2- 172.21.15.37:0/1597509741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5614014a0 0x55e561414f50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 >> 172.21.15.37:0/1597509741 conn(0x55e5613fda30 msgr2=0x55e5613fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 shutdown_connections 2024-09-19T06:42:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 1 -- 172.21.15.37:0/1597509741 wait complete. 2024-09-19T06:42:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.636+0000 7fcc82985740 20 client.15516 trim_cache size 0 max 16384 2024-09-19T06:42:29.642 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:29.642 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:42:29.642 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:42:29.642 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcc640a3560 msgr2=0x7fcc640a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcc640a3560 0x7fcc640a5a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcc70005cc0 tx=0x7fcc70005c50 comp rx=0 tx=0).stop 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78100e20 msgr2=0x7fcc780fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78100e20 0x7fcc780fcc80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fcc7400d950 tx=0x7fcc7400de10 comp rx=0 tx=0).stop 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 shutdown_connections 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc78102ef0 0x7fcc780fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcc78102570 0x7fcc780fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcc640a3560 0x7fcc640a5a20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 --2- 172.21.15.37:0/4236462536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcc78100e20 0x7fcc780fcc80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 >> 172.21.15.37:0/4236462536 conn(0x7fcc780fc7d0 msgr2=0x7fcc7806a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 shutdown_connections 2024-09-19T06:42:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:29.637+0000 7fcc805ac640 1 -- 172.21.15.37:0/4236462536 wait complete. 2024-09-19T06:42:30.242 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:29 smithi037 ceph-mon[31211]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T06:42:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:29 smithi096 ceph-mon[35004]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T06:42:30.302 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:42:30.302 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:42:30.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:29 smithi151 ceph-mon[35236]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T06:42:30.599 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:30.703 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 11.2 2024-09-19T06:42:30.704 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.055+0000 7efc4e6f3640 1 -- 172.21.15.37:0/3459184724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc48106f40 msgr2=0x7efc4810eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.055+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/3459184724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc48106f40 0x7efc4810eac0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7efc40009ca0 tx=0x7efc4002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.055+0000 7efc4e6f3640 1 -- 172.21.15.37:0/3459184724 shutdown_connections 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.055+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/3459184724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc48106f40 0x7efc4810eac0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.055+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/3459184724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc481027c0 0x7efc48106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.055+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/3459184724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc48101e10 0x7efc481021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.056+0000 7efc4e6f3640 1 -- 172.21.15.37:0/3459184724 >> 172.21.15.37:0/3459184724 conn(0x7efc480fd8d0 msgr2=0x7efc480ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.056+0000 7efc4e6f3640 1 -- 172.21.15.37:0/3459184724 shutdown_connections 2024-09-19T06:42:31.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.056+0000 7efc4e6f3640 1 -- 172.21.15.37:0/3459184724 wait complete. 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.056+0000 7efc4e6f3640 1 Processor -- start 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.056+0000 7efc4e6f3640 1 -- start start 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc4e6f3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc48101e10 0x7efc48198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc4e6f3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc481027c0 0x7efc481988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc4e6f3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc48106f40 0x7efc4819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc4e6f3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efc48111fa0 con 0x7efc481027c0 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc4e6f3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efc48111e20 con 0x7efc48101e10 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc4e6f3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efc48112120 con 0x7efc48106f40 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc481027c0 0x7efc481988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc481027c0 0x7efc481988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54534/0 (socket says 172.21.15.37:54534) 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 -- 172.21.15.37:0/1858248776 learned_addr learned my addr 172.21.15.37:0/1858248776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:31.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc4cc90640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc48106f40 0x7efc4819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc47fff640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc48101e10 0x7efc48198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 -- 172.21.15.37:0/1858248776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc48106f40 msgr2=0x7efc4819cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc48106f40 0x7efc4819cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 -- 172.21.15.37:0/1858248776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc48101e10 msgr2=0x7efc48198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc48101e10 0x7efc48198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.057+0000 7efc477fe640 1 -- 172.21.15.37:0/1858248776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc4819d320 con 0x7efc481027c0 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.058+0000 7efc477fe640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc481027c0 0x7efc481988b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d770 tx=0x7efc3c00db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.058+0000 7efc457fa640 1 -- 172.21.15.37:0/1858248776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c017070 con 0x7efc481027c0 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.058+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc4819d610 con 0x7efc481027c0 2024-09-19T06:42:31.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.058+0000 7efc457fa640 1 -- 172.21.15.37:0/1858248776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc3c009820 con 0x7efc481027c0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.058+0000 7efc457fa640 1 -- 172.21.15.37:0/1858248776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c004cd0 con 0x7efc481027c0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.058+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc481a4ef0 con 0x7efc481027c0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.059+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc481021f0 con 0x7efc481027c0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.060+0000 7efc457fa640 1 -- 172.21.15.37:0/1858248776 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7efc3c0099d0 con 0x7efc481027c0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.061+0000 7efc457fa640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efc300a3560 0x7efc300a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.061+0000 7efc47fff640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efc300a3560 0x7efc300a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.061+0000 7efc457fa640 1 -- 172.21.15.37:0/1858248776 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7efc3c0cf910 con 0x7efc481027c0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.062+0000 7efc47fff640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efc300a3560 0x7efc300a5a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc3800abe0 tx=0x7efc3800a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:31.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.069+0000 7efc457fa640 1 -- 172.21.15.37:0/1858248776 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7efc3c09c0b0 con 0x7efc481027c0 2024-09-19T06:42:31.169 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:31.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.323+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc481996d0 con 0x7efc481027c0 2024-09-19T06:42:31.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.324+0000 7efc457fa640 1 -- 172.21.15.37:0/1858248776 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7efc481996d0 con 0x7efc481027c0 2024-09-19T06:42:31.325 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:31.325 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:42:31.325 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efc300a3560 msgr2=0x7efc300a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efc300a3560 0x7efc300a5a20 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efc3800abe0 tx=0x7efc3800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc481027c0 msgr2=0x7efc481988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc481027c0 0x7efc481988b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00d770 tx=0x7efc3c00db70 comp rx=0 tx=0).stop 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 shutdown_connections 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efc48106f40 0x7efc4819cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efc300a3560 0x7efc300a5a20 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efc481027c0 0x7efc481988b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 --2- 172.21.15.37:0/1858248776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efc48101e10 0x7efc48198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:31.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 >> 172.21.15.37:0/1858248776 conn(0x7efc480fd8d0 msgr2=0x7efc480fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:31.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.326+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 shutdown_connections 2024-09-19T06:42:31.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:31.327+0000 7efc4e6f3640 1 -- 172.21.15.37:0/1858248776 wait complete. 2024-09-19T06:42:32.066 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:42:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:31 smithi096 ceph-mon[35004]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:31 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1858248776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:32.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:31 smithi037 ceph-mon[31211]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:32.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:31 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1858248776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.278+0000 7f1f41f35640 1 -- 172.21.15.37:0/4282452642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c108290 msgr2=0x7f1f3c10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.278+0000 7f1f41f35640 1 --2- 172.21.15.37:0/4282452642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c108290 0x7f1f3c10ec30 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f1f34009cd0 tx=0x7f1f3402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 -- 172.21.15.37:0/4282452642 shutdown_connections 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 --2- 172.21.15.37:0/4282452642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c108290 0x7f1f3c10ec30 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 --2- 172.21.15.37:0/4282452642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f3c103740 0x7f1f3c107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 --2- 172.21.15.37:0/4282452642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c102d90 0x7f1f3c103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 -- 172.21.15.37:0/4282452642 >> 172.21.15.37:0/4282452642 conn(0x7f1f3c0fc7d0 msgr2=0x7f1f3c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 -- 172.21.15.37:0/4282452642 shutdown_connections 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 -- 172.21.15.37:0/4282452642 wait complete. 2024-09-19T06:42:32.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 Processor -- start 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.279+0000 7f1f41f35640 1 -- start start 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f41f35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c102d90 0x7f1f3c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f41f35640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c103740 0x7f1f3c198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f41f35640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f3c108290 0x7f1f3c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f41f35640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c112090 con 0x7f1f3c102d90 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f41f35640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c111f10 con 0x7f1f3c103740 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f41f35640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1f3c112210 con 0x7f1f3c108290 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c102d90 0x7f1f3c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c102d90 0x7f1f3c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54552/0 (socket says 172.21.15.37:54552) 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 -- 172.21.15.37:0/2836739988 learned_addr learned my addr 172.21.15.37:0/2836739988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3affd640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c103740 0x7f1f3c198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:32.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3bfff640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f3c108290 0x7f1f3c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 -- 172.21.15.37:0/2836739988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f3c108290 msgr2=0x7f1f3c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f3c108290 0x7f1f3c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 -- 172.21.15.37:0/2836739988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c103740 msgr2=0x7f1f3c198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c103740 0x7f1f3c198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3b7fe640 1 -- 172.21.15.37:0/2836739988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f3c19d4d0 con 0x7f1f3c102d90 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.280+0000 7f1f3affd640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c103740 0x7f1f3c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.281+0000 7f1f3b7fe640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c102d90 0x7f1f3c198520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e9f0 tx=0x7f1f2c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.281+0000 7f1f38ff9640 1 -- 172.21.15.37:0/2836739988 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c00cd80 con 0x7f1f3c102d90 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.281+0000 7f1f38ff9640 1 -- 172.21.15.37:0/2836739988 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1f2c010910 con 0x7f1f3c102d90 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.281+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f3c19d7c0 con 0x7f1f3c102d90 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.281+0000 7f1f38ff9640 1 -- 172.21.15.37:0/2836739988 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f2c0188f0 con 0x7f1f3c102d90 2024-09-19T06:42:32.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.281+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f3c1a50a0 con 0x7f1f3c102d90 2024-09-19T06:42:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.282+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f3c103170 con 0x7f1f3c102d90 2024-09-19T06:42:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.284+0000 7f1f38ff9640 1 -- 172.21.15.37:0/2836739988 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f1f2c010ac0 con 0x7f1f3c102d90 2024-09-19T06:42:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.284+0000 7f1f38ff9640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1f240a3560 0x7f1f240a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.285+0000 7f1f3affd640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1f240a3560 0x7f1f240a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:32.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.285+0000 7f1f38ff9640 1 -- 172.21.15.37:0/2836739988 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f1f2c0d0830 con 0x7f1f3c102d90 2024-09-19T06:42:32.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.285+0000 7f1f3affd640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1f240a3560 0x7f1f240a5a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0feda0 tx=0x7f1f3000b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:32.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.288+0000 7f1f38ff9640 1 -- 172.21.15.37:0/2836739988 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1f2c09cfd0 con 0x7f1f3c102d90 2024-09-19T06:42:32.361 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:32.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:31 smithi151 ceph-mon[35236]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:32.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:31 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1858248776' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.526+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f3c10a2a0 con 0x7f1f3c102d90 2024-09-19T06:42:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.526+0000 7f1f38ff9640 1 -- 172.21.15.37:0/2836739988 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7f1f2c09c8d0 con 0x7f1f3c102d90 2024-09-19T06:42:32.528 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:32.528 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:42:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:42:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.528+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1f240a3560 msgr2=0x7f1f240a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.528+0000 7f1f41f35640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1f240a3560 0x7f1f240a5a20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c0feda0 tx=0x7f1f3000b390 comp rx=0 tx=0).stop 2024-09-19T06:42:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.528+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c102d90 msgr2=0x7f1f3c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c102d90 0x7f1f3c198520 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c00e9f0 tx=0x7f1f2c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 shutdown_connections 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1f3c108290 0x7f1f3c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1f3c103740 0x7f1f3c198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1f240a3560 0x7f1f240a5a20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 --2- 172.21.15.37:0/2836739988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1f3c102d90 0x7f1f3c198520 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 >> 172.21.15.37:0/2836739988 conn(0x7f1f3c0fc7d0 msgr2=0x7f1f3c106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 shutdown_connections 2024-09-19T06:42:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:32.529+0000 7f1f41f35640 1 -- 172.21.15.37:0/2836739988 wait complete. 2024-09-19T06:42:33.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:32 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2836739988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:33.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:32 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2836739988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:33.312 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0' 2024-09-19T06:42:33.313 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 131.2 --await-for 45.0 2024-09-19T06:42:33.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:32 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2836739988' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:33.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.498+0000 7f132e995640 1 -- 172.21.15.37:0/4123047857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 msgr2=0x7f1328107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.498+0000 7f132e995640 1 --2- 172.21.15.37:0/4123047857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f1328107b90 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f131c009930 tx=0x7f131c02a420 comp rx=0 tx=0).stop 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.498+0000 7f132e995640 1 -- 172.21.15.37:0/4123047857 shutdown_connections 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.498+0000 7f132e995640 1 --2- 172.21.15.37:0/4123047857 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13281082a0 0x7f132810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.498+0000 7f132e995640 1 --2- 172.21.15.37:0/4123047857 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f1328107b90 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.498+0000 7f132e995640 1 --2- 172.21.15.37:0/4123047857 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1328102d30 0x7f1328103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.498+0000 7f132e995640 1 -- 172.21.15.37:0/4123047857 >> 172.21.15.37:0/4123047857 conn(0x7f13280fc7d0 msgr2=0x7f13280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.499+0000 7f132e995640 1 -- 172.21.15.37:0/4123047857 shutdown_connections 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.499+0000 7f132e995640 1 -- 172.21.15.37:0/4123047857 wait complete. 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.499+0000 7f132e995640 1 Processor -- start 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132e995640 1 -- start start 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132e995640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1328102d30 0x7f13281a0510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132e995640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f13281a0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132e995640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13281082a0 0x7f132819a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132e995640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13281120a0 con 0x7f13281036e0 2024-09-19T06:42:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132e995640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1328111f20 con 0x7f1328102d30 2024-09-19T06:42:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132e995640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1328112220 con 0x7f13281082a0 2024-09-19T06:42:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f13281a0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f13281a0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54566/0 (socket says 172.21.15.37:54566) 2024-09-19T06:42:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f13277fe640 1 -- 172.21.15.37:0/2463615398 learned_addr learned my addr 172.21.15.37:0/2463615398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f1327fff640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1328102d30 0x7f13281a0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.500+0000 7f132cf32640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13281082a0 0x7f132819a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13277fe640 1 -- 172.21.15.37:0/2463615398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13281082a0 msgr2=0x7f132819a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13277fe640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13281082a0 0x7f132819a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13277fe640 1 -- 172.21.15.37:0/2463615398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1328102d30 msgr2=0x7f13281a0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13277fe640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1328102d30 0x7f13281a0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13277fe640 1 -- 172.21.15.37:0/2463615398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f132819aea0 con 0x7f13281036e0 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f1327fff640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1328102d30 0x7f13281a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13277fe640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f13281a0a50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f131c005940 tx=0x7f131c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13257fa640 1 -- 172.21.15.37:0/2463615398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c03f030 con 0x7f13281036e0 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13257fa640 1 -- 172.21.15.37:0/2463615398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c0044c0 con 0x7f13281036e0 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f132819b130 con 0x7f13281036e0 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f13257fa640 1 -- 172.21.15.37:0/2463615398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c035070 con 0x7f13281036e0 2024-09-19T06:42:33.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.501+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13281a7330 con 0x7f13281036e0 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 Processor -- start 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 -- start start 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f13257fa640 1 -- 172.21.15.37:0/2463615398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f131c042030 con 0x7f13281036e0 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29bd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29bda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29ca760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55caa29232f0 con 0x55caa29beb40 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55caa28cacf0 con 0x55caa29bd5d0 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f1330d6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55caa291f2b0 con 0x55caa29c8370 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f132cf32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29bd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f132cf32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29bd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54568/0 (socket says 172.21.15.37:54568) 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.505+0000 7f132cf32640 1 -- 172.21.15.37:0/902210454 learned_addr learned my addr 172.21.15.37:0/902210454 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:33.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f132cf32640 1 -- 172.21.15.37:0/902210454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 msgr2=0x55caa29ca760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f132cf32640 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29ca760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f132cf32640 1 -- 172.21.15.37:0/902210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 msgr2=0x55caa29bda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f1327fff640 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29bda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f132cf32640 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29bda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f132cf32640 1 -- 172.21.15.37:0/902210454 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caa28af3a0 con 0x55caa29beb40 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f1327fff640 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f13257fa640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13100a3560 0x7f13100a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f132cf32640 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29bd090 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1320009f90 tx=0x7f1320010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc9c1a5da6a4c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f13257fa640 1 -- 172.21.15.37:0/2463615398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f131c0f0000 con 0x7f13281036e0 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f130e7fc640 1 -- 172.21.15.37:0/902210454 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320011700 con 0x55caa29beb40 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f130e7fc640 1 -- 172.21.15.37:0/902210454 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1320011d80 con 0x55caa29beb40 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f130e7fc640 1 -- 172.21.15.37:0/902210454 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1320019e30 con 0x55caa29beb40 2024-09-19T06:42:33.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.506+0000 7f1327fff640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13100a3560 0x7f13100a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 -- 172.21.15.37:0/902210454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 msgr2=0x55caa29bd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29bd090 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f1320009f90 tx=0x7f1320010860 comp rx=0 tx=0).stop 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1327fff640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13100a3560 0x7f13100a5a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55caa29baac0 tx=0x7f1318005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 -- 172.21.15.37:0/902210454 shutdown_connections 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29ca760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29bda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 --2- 172.21.15.37:0/902210454 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29bd090 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 -- 172.21.15.37:0/902210454 >> 172.21.15.37:0/902210454 conn(0x55caa2986310 msgr2=0x55caa29b9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 -- 172.21.15.37:0/902210454 shutdown_connections 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.507+0000 7f1330d6e740 1 -- 172.21.15.37:0/902210454 wait complete. 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.508+0000 7f1330d6e740 1 -- start start 2024-09-19T06:42:33.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.508+0000 7f1330d6e740 1 Processor -- start 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.508+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.508+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.508+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1330d6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55caa29cb070 con 0x55caa29beb40 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1330d6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55caa28de060 con 0x55caa29bd5d0 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1330d6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55caa29cb1f0 con 0x55caa29c8370 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:51502/0 (socket says 172.21.15.37:51502) 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 -- 172.21.15.37:0/3174244524 learned_addr learned my addr 172.21.15.37:0/3174244524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1330d6e740 10 client.0 _ll_register_callbacks cb 0x55caa26318c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 -- 172.21.15.37:0/3174244524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 msgr2=0x55caa29c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 -- 172.21.15.37:0/3174244524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 msgr2=0x55caa29c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 -- 172.21.15.37:0/3174244524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caa26943c0 con 0x55caa29bd5d0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f1327fff640 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c7df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1318011b70 tx=0x7f13180046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=471840ccf62cf1c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f13077fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.509+0000 7f13077fe640 1 -- 172.21.15.37:0/3174244524 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131801fae0 con 0x55caa29bd5d0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.510+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1320011d80 con 0x55caa29bd5d0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.510+0000 7f13077fe640 1 -- 172.21.15.37:0/3174244524 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f131801fc80 con 0x55caa29bd5d0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.510+0000 7f13077fe640 1 -- 172.21.15.37:0/3174244524 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1318032460 con 0x55caa29bd5d0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.510+0000 7f1330d6e740 10 client.25160 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.510+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55caa29d6bc0 con 0x55caa29bd5d0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.510+0000 7f13077fe640 1 -- 172.21.15.37:0/3174244524 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1320011d80 con 0x55caa29bd5d0 2024-09-19T06:42:33.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f13077fe640 1 -- 172.21.15.37:0/3174244524 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f1318029460 con 0x55caa29bd5d0 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f1330d6e740 10 client.25160 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f1330d6e740 10 client.25160 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f1330d6e740 20 client.25160 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f1330d6e740 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa29da450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f1330d6e740 4 client.25160 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caa2930860 con 0x55caa29d8020 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.511+0000 7f13277fe640 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa29da450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.512+0000 7f13277fe640 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa29da450 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=678838ed22836df3 server_cookie=545269be0bf02803 in_seq=0 out_seq=0 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.512+0000 7f13077fe640 10 client.25160 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.512+0000 7f13077fe640 1 -- 172.21.15.37:0/3174244524 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55caa2930860 con 0x55caa29d8020 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.512+0000 7f13077fe640 10 client.25160 handle_command_reply: tid=0 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.512+0000 7f1330d6e740 1 client.25160 shutdown 2024-09-19T06:42:33.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 msgr2=0x55caa29c7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c7df0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1318011b70 tx=0x7f13180046c0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 shutdown_connections 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa29da450 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f13077fe640 0 client.25160 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 --2- 172.21.15.37:0/3174244524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c7df0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 >> 172.21.15.37:0/3174244524 conn(0x55caa2986310 msgr2=0x55caa29bf930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 shutdown_connections 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.513+0000 7f1330d6e740 1 -- 172.21.15.37:0/3174244524 wait complete. 2024-09-19T06:42:33.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.514+0000 7f1330d6e740 20 client.25160 trim_cache size 0 max 16384 2024-09-19T06:42:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 Processor -- start 2024-09-19T06:42:33.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 -- start start 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29d4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13281b24b0 con 0x55caa29beb40 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13281afd80 con 0x55caa29bd5d0 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1330d6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13281adbd0 con 0x55caa29c8370 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29d4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29d4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54580/0 (socket says 172.21.15.37:54580) 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f13277fe640 1 -- 172.21.15.37:0/34411104 learned_addr learned my addr 172.21.15.37:0/34411104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f1327fff640 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.530+0000 7f132cf32640 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f13277fe640 1 -- 172.21.15.37:0/34411104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 msgr2=0x55caa29c0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f13277fe640 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f13277fe640 1 -- 172.21.15.37:0/34411104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 msgr2=0x55caa29c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f13277fe640 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f13277fe640 1 -- 172.21.15.37:0/34411104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caa29b94f0 con 0x55caa29beb40 2024-09-19T06:42:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f132cf32640 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f13277fe640 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29d4310 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f131c0bc0e0 tx=0x7f131c0dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c130c5e2cd5f71a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f130dffb640 1 -- 172.21.15.37:0/34411104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c0d4020 con 0x55caa29beb40 2024-09-19T06:42:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f130dffb640 1 -- 172.21.15.37:0/34411104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f131c0ba1f0 con 0x55caa29beb40 2024-09-19T06:42:33.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.531+0000 7f130dffb640 1 -- 172.21.15.37:0/34411104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131c049040 con 0x55caa29beb40 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 -- 172.21.15.37:0/34411104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 msgr2=0x55caa29d4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29d4310 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f131c0bc0e0 tx=0x7f131c0dc040 comp rx=0 tx=0).stop 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 -- 172.21.15.37:0/34411104 shutdown_connections 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29c0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29d4310 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 --2- 172.21.15.37:0/34411104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 -- 172.21.15.37:0/34411104 >> 172.21.15.37:0/34411104 conn(0x55caa2986310 msgr2=0x55caa29c4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 -- 172.21.15.37:0/34411104 shutdown_connections 2024-09-19T06:42:33.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.532+0000 7f1330d6e740 1 -- 172.21.15.37:0/34411104 wait complete. 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.533+0000 7f1330d6e740 1 -- start start 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.533+0000 7f1330d6e740 1 Processor -- start 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.533+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29d0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1330d6e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29cc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1330d6e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f13281ae240 con 0x55caa29beb40 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1330d6e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13281af6b0 con 0x55caa29bd5d0 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1330d6e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f13281ae910 con 0x55caa29c8370 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1330d6e740 10 client.0 _ll_register_callbacks cb 0x7f1318009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f13277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29cbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54594/0 (socket says 172.21.15.37:54594) 2024-09-19T06:42:33.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1327fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29d0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f13277fe640 1 -- 172.21.15.37:0/1126481655 learned_addr learned my addr 172.21.15.37:0/1126481655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f132cf32640 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29cc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1327fff640 1 -- 172.21.15.37:0/1126481655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 msgr2=0x55caa29cc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1327fff640 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29cc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1327fff640 1 -- 172.21.15.37:0/1126481655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 msgr2=0x55caa29cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1327fff640 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f1327fff640 1 -- 172.21.15.37:0/1126481655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caa29cc720 con 0x55caa29bd5d0 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.534+0000 7f13277fe640 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f1327fff640 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29d0940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13180298d0 tx=0x7f1318029cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ef38a2bf2233c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f13077fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f13077fe640 1 -- 172.21.15.37:0/1126481655 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1318005b90 con 0x55caa29bd5d0 2024-09-19T06:42:33.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f13077fe640 1 -- 172.21.15.37:0/1126481655 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f131801f8e0 con 0x55caa29bd5d0 2024-09-19T06:42:33.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f131c0ba1f0 con 0x55caa29bd5d0 2024-09-19T06:42:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f13077fe640 1 -- 172.21.15.37:0/1126481655 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f131801fae0 con 0x55caa29bd5d0 2024-09-19T06:42:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f1330d6e740 10 client.25166 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55caa2a4b260 con 0x55caa29bd5d0 2024-09-19T06:42:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.535+0000 7f13077fe640 1 -- 172.21.15.37:0/1126481655 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1318005520 con 0x55caa29bd5d0 2024-09-19T06:42:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f13077fe640 1 -- 172.21.15.37:0/1126481655 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f131803a030 con 0x55caa29bd5d0 2024-09-19T06:42:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f1330d6e740 10 client.25166 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:33.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f1330d6e740 10 client.25166 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:42:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f1330d6e740 20 client.25166 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f1330d6e740 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa2a4d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f1330d6e740 4 client.25166 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:42:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f131c0ba1f0 con 0x55caa29d8020 2024-09-19T06:42:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.536+0000 7f13277fe640 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa2a4d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f13077fe640 10 client.25166 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:33.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f13277fe640 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa2a4d9c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b595747af7d16401 server_cookie=a61e7c191e5361d0 in_seq=0 out_seq=0 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f13077fe640 1 -- 172.21.15.37:0/1126481655 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f131c0ba1f0 con 0x55caa29d8020 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f13077fe640 10 client.25166 handle_command_reply: tid=0 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 client.25166 shutdown 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 msgr2=0x55caa29d0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29d0940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f13180298d0 tx=0x7f1318029cf0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 shutdown_connections 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55caa29d8020 0x55caa2a4d9c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55caa29c8370 0x55caa29cc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55caa29beb40 0x55caa29cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f13077fe640 0 client.25166 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:33.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 --2- 172.21.15.37:0/1126481655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55caa29bd5d0 0x55caa29d0940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 >> 172.21.15.37:0/1126481655 conn(0x55caa2986310 msgr2=0x55caa29c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.537+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 shutdown_connections 2024-09-19T06:42:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.538+0000 7f1330d6e740 1 -- 172.21.15.37:0/1126481655 wait complete. 2024-09-19T06:42:33.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.538+0000 7f1330d6e740 20 client.25166 trim_cache size 0 max 16384 2024-09-19T06:42:33.543 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13100a3560 msgr2=0x7f13100a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13100a3560 0x7f13100a5a20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x55caa29baac0 tx=0x7f1318005f90 comp rx=0 tx=0).stop 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 msgr2=0x7f13281a0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f13281a0a50 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f131c005940 tx=0x7f131c0040c0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 shutdown_connections 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13281082a0 0x7f132819a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f13100a3560 0x7f13100a5a20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f13281036e0 0x7f13281a0a50 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 --2- 172.21.15.37:0/2463615398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1328102d30 0x7f13281a0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:33.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 >> 172.21.15.37:0/2463615398 conn(0x7f13280fc7d0 msgr2=0x7f132810d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:33.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.539+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 shutdown_connections 2024-09-19T06:42:33.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:33.540+0000 7f132e995640 1 -- 172.21.15.37:0/2463615398 wait complete. 2024-09-19T06:42:34.176 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:42:34.230 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:34.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:33 smithi096 ceph-mon[35004]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T06:42:34.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:33 smithi037 ceph-mon[31211]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T06:42:34.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:33 smithi151 ceph-mon[35236]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 511 B/s wr, 8 op/s 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.691+0000 7f60f9c2a640 1 -- 172.21.15.37:0/3748520700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f4102d30 msgr2=0x7f60f4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.691+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/3748520700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f4102d30 0x7f60f4103110 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f60e4009930 tx=0x7f60e402a420 comp rx=0 tx=0).stop 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.691+0000 7f60f9c2a640 1 -- 172.21.15.37:0/3748520700 shutdown_connections 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.691+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/3748520700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60f41082a0 0x7f60f410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.691+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/3748520700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60f41036e0 0x7f60f4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.691+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/3748520700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f4102d30 0x7f60f4103110 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.692+0000 7f60f9c2a640 1 -- 172.21.15.37:0/3748520700 >> 172.21.15.37:0/3748520700 conn(0x7f60f40fc7d0 msgr2=0x7f60f40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.692+0000 7f60f9c2a640 1 -- 172.21.15.37:0/3748520700 shutdown_connections 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.692+0000 7f60f9c2a640 1 -- 172.21.15.37:0/3748520700 wait complete. 2024-09-19T06:42:34.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.692+0000 7f60f9c2a640 1 Processor -- start 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.692+0000 7f60f9c2a640 1 -- start start 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f9c2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60f4102d30 0x7f60f4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f9c2a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60f41036e0 0x7f60f4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f9c2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f41082a0 0x7f60f419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f9c2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4112100 con 0x7f60f41082a0 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f9c2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4111f80 con 0x7f60f4102d30 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f9c2a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60f4112280 con 0x7f60f41036e0 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f41082a0 0x7f60f419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f41082a0 0x7f60f419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54608/0 (socket says 172.21.15.37:54608) 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 -- 172.21.15.37:0/298546386 learned_addr learned my addr 172.21.15.37:0/298546386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f37fe640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60f4102d30 0x7f60f4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60f41036e0 0x7f60f4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 -- 172.21.15.37:0/298546386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60f41036e0 msgr2=0x7f60f4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60f41036e0 0x7f60f4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 -- 172.21.15.37:0/298546386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60f4102d30 msgr2=0x7f60f4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60f4102d30 0x7f60f4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 -- 172.21.15.37:0/298546386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f419d5b0 con 0x7f60f41082a0 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60f41036e0 0x7f60f4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f37fe640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60f4102d30 0x7f60f4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.693+0000 7f60f3fff640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f41082a0 0x7f60f419ced0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b970 tx=0x7f60ec00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:34.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.694+0000 7f60f0ff9640 1 -- 172.21.15.37:0/298546386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec019070 con 0x7f60f41082a0 2024-09-19T06:42:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.694+0000 7f60f0ff9640 1 -- 172.21.15.37:0/298546386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60ec00cb40 con 0x7f60f41082a0 2024-09-19T06:42:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.694+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f419d8a0 con 0x7f60f41082a0 2024-09-19T06:42:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.694+0000 7f60f0ff9640 1 -- 172.21.15.37:0/298546386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec007650 con 0x7f60f41082a0 2024-09-19T06:42:34.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.694+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f41a5180 con 0x7f60f41082a0 2024-09-19T06:42:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.697+0000 7f60fc003740 1 Processor -- start 2024-09-19T06:42:34.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.697+0000 7f60f0ff9640 1 -- 172.21.15.37:0/298546386 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f60ec01c070 con 0x7f60f41082a0 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.697+0000 7f60fc003740 1 -- start start 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60fc003740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd0f760 0x55670dd0fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60fc003740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd10080 0x55670dd1a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60fc003740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd1cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60fc003740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55670dc752a0 con 0x55670dd0f760 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60fc003740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55670dc1cb10 con 0x55670dd10080 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60fc003740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55670dc71260 con 0x55670dd1a940 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd0f760 0x55670dd0fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd0f760 0x55670dd0fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54622/0 (socket says 172.21.15.37:54622) 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 -- 172.21.15.37:0/4070185731 learned_addr learned my addr 172.21.15.37:0/4070185731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:34.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd1cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f0ff9640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f60dc0a35e0 0x7f60dc0a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 -- 172.21.15.37:0/4070185731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 msgr2=0x55670dd1cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd1cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 -- 172.21.15.37:0/4070185731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd10080 msgr2=0x55670dd1a400 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd10080 0x55670dd1a400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 -- 172.21.15.37:0/4070185731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55670d9e6b90 con 0x55670dd0f760 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f0ff9640 1 -- 172.21.15.37:0/298546386 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f60ec0cf510 con 0x7f60f41082a0 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd0f760 0x55670dd0fb40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005bc0 tx=0x7f60e800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee5445ba042f311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.698+0000 7f60f3fff640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f60dc0a35e0 0x7f60dc0a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.699+0000 7f60d9ffb640 1 -- 172.21.15.37:0/4070185731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8019030 con 0x55670dd0f760 2024-09-19T06:42:34.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.699+0000 7f60d9ffb640 1 -- 172.21.15.37:0/4070185731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e80119a0 con 0x55670dd0f760 2024-09-19T06:42:34.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.699+0000 7f60d9ffb640 1 -- 172.21.15.37:0/4070185731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800f9d0 con 0x55670dd0f760 2024-09-19T06:42:34.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.699+0000 7f60fc003740 1 -- 172.21.15.37:0/4070185731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd0f760 msgr2=0x55670dd0fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.699+0000 7f60fc003740 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd0f760 0x55670dd0fb40 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f60e8005bc0 tx=0x7f60e800b6d0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.699+0000 7f60f3fff640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f60dc0a35e0 0x7f60dc0a5aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ccb90 tx=0x7f60ec014d00 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:34.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 -- 172.21.15.37:0/4070185731 shutdown_connections 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd1cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd10080 0x55670dd1a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 --2- 172.21.15.37:0/4070185731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd0f760 0x55670dd0fb40 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 -- 172.21.15.37:0/4070185731 >> 172.21.15.37:0/4070185731 conn(0x55670dd0bcf0 msgr2=0x55670dd0c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 -- 172.21.15.37:0/4070185731 shutdown_connections 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 -- 172.21.15.37:0/4070185731 wait complete. 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 -- start start 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.700+0000 7f60fc003740 1 Processor -- start 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60fc003740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd177d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60fc003740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd17d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60fc003740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd18250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60fc003740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55670dd1d510 con 0x55670dd10080 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60fc003740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55670dbfd6e0 con 0x55670dd0f760 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60fc003740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55670dd1d690 con 0x55670dd1a940 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60fc003740 10 client.0 _ll_register_callbacks cb 0x55670dc21a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd17d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd17d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54628/0 (socket says 172.21.15.37:54628) 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 -- 172.21.15.37:0/289963411 learned_addr learned my addr 172.21.15.37:0/289963411 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd177d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd18250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 -- 172.21.15.37:0/289963411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 msgr2=0x55670dd18250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd18250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 -- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 msgr2=0x55670dd177d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:34.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd177d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 -- 172.21.15.37:0/289963411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55670dd269b0 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd18250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60f37fe640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd17d10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f60e4005ab0 tx=0x7f60e40317e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1f747b05b2e897 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.701+0000 7f60d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60d2ffd640 1 -- 172.21.15.37:0/289963411 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e403f070 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60d2ffd640 1 -- 172.21.15.37:0/289963411 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f60e4004250 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e80119a0 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60d2ffd640 1 -- 172.21.15.37:0/289963411 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f60e4035070 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 10 client.15558 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55670dd29320 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60d2ffd640 1 -- 172.21.15.37:0/289963411 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e80119a0 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60d2ffd640 1 -- 172.21.15.37:0/289963411 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f60e4004b10 con 0x55670dd10080 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 10 client.15558 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:34.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 10 client.15558 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 20 client.15558 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd2cb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 4 client.15558 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.702+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55670dc829d0 con 0x55670dd2a730 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.703+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd2cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.703+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd2cb20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cda1e26347c32f1 server_cookie=6e384b4eaad0a97e in_seq=0 out_seq=0 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.703+0000 7f60d2ffd640 10 client.15558 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60d2ffd640 1 -- 172.21.15.37:0/289963411 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55670dc829d0 con 0x55670dd2a730 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60d2ffd640 10 client.15558 handle_command_reply: tid=0 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60fc003740 1 client.15558 shutdown 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 msgr2=0x55670dd17d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60fc003740 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd17d10 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f60e4005ab0 tx=0x7f60e40317e0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 shutdown_connections 2024-09-19T06:42:34.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60fc003740 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd2cb20 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60fc003740 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd18250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60d2ffd640 0 client.15558 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.704+0000 7f60fc003740 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd17d10 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.705+0000 7f60fc003740 1 --2- 172.21.15.37:0/289963411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd177d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.705+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 >> 172.21.15.37:0/289963411 conn(0x55670dd0bcf0 msgr2=0x55670dd0ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.705+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 shutdown_connections 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.705+0000 7f60fc003740 1 -- 172.21.15.37:0/289963411 wait complete. 2024-09-19T06:42:34.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.705+0000 7f60fc003740 20 client.15558 trim_cache size 0 max 16384 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.722+0000 7f60fc003740 1 Processor -- start 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.722+0000 7f60fc003740 1 -- start start 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60fc003740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd121c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60fc003740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd323f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60fc003740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd32930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60fc003740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41b0270 con 0x55670dd10080 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60fc003740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41adb40 con 0x55670dd0f760 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60fc003740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41ab990 con 0x55670dd1a940 2024-09-19T06:42:34.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd323f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd323f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54640/0 (socket says 172.21.15.37:54640) 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 -- 172.21.15.37:0/2179591393 learned_addr learned my addr 172.21.15.37:0/2179591393 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd121c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd32930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 -- 172.21.15.37:0/2179591393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 msgr2=0x55670dd32930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd32930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 -- 172.21.15.37:0/2179591393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 msgr2=0x55670dd121c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd121c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 -- 172.21.15.37:0/2179591393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55670dd2ce90 con 0x55670dd10080 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd32930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:34.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.723+0000 7f60f37fe640 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd323f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f60e4001eb0 tx=0x7f60e402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca6a95590d767af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60d97fa640 1 -- 172.21.15.37:0/2179591393 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e403f070 con 0x55670dd10080 2024-09-19T06:42:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60d97fa640 1 -- 172.21.15.37:0/2179591393 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e4003910 con 0x55670dd10080 2024-09-19T06:42:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60d97fa640 1 -- 172.21.15.37:0/2179591393 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e4003df0 con 0x55670dd10080 2024-09-19T06:42:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60fc003740 1 -- 172.21.15.37:0/2179591393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 msgr2=0x55670dd323f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60fc003740 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd323f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f60e4001eb0 tx=0x7f60e402a420 comp rx=0 tx=0).stop 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60fc003740 1 -- 172.21.15.37:0/2179591393 shutdown_connections 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60fc003740 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd1a940 0x55670dd32930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60fc003740 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd10080 0x55670dd323f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.724+0000 7f60fc003740 1 --2- 172.21.15.37:0/2179591393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd121c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.725+0000 7f60fc003740 1 -- 172.21.15.37:0/2179591393 >> 172.21.15.37:0/2179591393 conn(0x55670dd0bcf0 msgr2=0x55670dd19850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.725+0000 7f60fc003740 1 -- 172.21.15.37:0/2179591393 shutdown_connections 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.725+0000 7f60fc003740 1 -- 172.21.15.37:0/2179591393 wait complete. 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.725+0000 7f60fc003740 1 -- start start 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.725+0000 7f60fc003740 1 Processor -- start 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.725+0000 7f60fc003740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd1e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.725+0000 7f60fc003740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd10080 0x55670dd1ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60fc003740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd1a940 0x55670dd1f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60fc003740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41ac000 con 0x55670dd1a940 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60fc003740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41ad470 con 0x55670dd0f760 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60fc003740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f60f41ac6d0 con 0x55670dd10080 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60fc003740 10 client.0 _ll_register_callbacks cb 0x7f60e4005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd1a940 0x55670dd1f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd1a940 0x55670dd1f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54646/0 (socket says 172.21.15.37:54646) 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 -- 172.21.15.37:0/873125978 learned_addr learned my addr 172.21.15.37:0/873125978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd1e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f37fe640 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd10080 0x55670dd1ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 -- 172.21.15.37:0/873125978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd10080 msgr2=0x55670dd1ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd10080 0x55670dd1ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 -- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 msgr2=0x55670dd1e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:34.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd1e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 -- 172.21.15.37:0/873125978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55670dd94660 con 0x55670dd1a940 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60f2ffd640 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd1a940 0x55670dd1f370 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f60e800b7f0 tx=0x7f60e8021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d1ca37f51f61c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.726+0000 7f60d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60d2ffd640 1 -- 172.21.15.37:0/873125978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e8032070 con 0x55670dd1a940 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60d2ffd640 1 -- 172.21.15.37:0/873125978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f60e8004030 con 0x55670dd1a940 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60e4003910 con 0x55670dd1a940 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60d2ffd640 1 -- 172.21.15.37:0/873125978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f60e8019040 con 0x55670dd1a940 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60fc003740 10 client.15564 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:34.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55670dd95940 con 0x55670dd1a940 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60d2ffd640 1 -- 172.21.15.37:0/873125978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f60e4003910 con 0x55670dd1a940 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.727+0000 7f60d2ffd640 1 -- 172.21.15.37:0/873125978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f60e8003780 con 0x55670dd1a940 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.728+0000 7f60fc003740 10 client.15564 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.728+0000 7f60fc003740 10 client.15564 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.728+0000 7f60fc003740 20 client.15564 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.728+0000 7f60fc003740 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd97d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.728+0000 7f60fc003740 4 client.15564 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}] 2024-09-19T06:42:34.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.728+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 131.2, "await_for": 45.0}) -- 0x55670dc7cd20 con 0x55670dd2a730 2024-09-19T06:42:34.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.728+0000 7f60f37fe640 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd97d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:34.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.729+0000 7f60f37fe640 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd97d50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af82087345ec4d1b server_cookie=4a81a575e1268922 in_seq=0 out_seq=0 2024-09-19T06:42:34.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.729+0000 7f60d2ffd640 10 client.15564 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:34.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.740+0000 7f60d2ffd640 1 -- 172.21.15.37:0/873125978 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x55670dc7cd20 con 0x55670dd2a730 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.740+0000 7f60d2ffd640 10 client.15564 handle_command_reply: tid=0 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.740+0000 7f60fc003740 1 client.15564 shutdown 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd1a940 msgr2=0x55670dd1f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd1a940 0x55670dd1f370 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f60e800b7f0 tx=0x7f60e8021040 comp rx=0 tx=0).stop 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 shutdown_connections 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55670dd2a730 0x55670dd97d50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55670dd1a940 0x55670dd1f370 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55670dd10080 0x55670dd1ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60d2ffd640 0 client.15564 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 --2- 172.21.15.37:0/873125978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55670dd0f760 0x55670dd1e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 >> 172.21.15.37:0/873125978 conn(0x55670dd0bcf0 msgr2=0x55670dd0a650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 shutdown_connections 2024-09-19T06:42:34.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 1 -- 172.21.15.37:0/873125978 wait complete. 2024-09-19T06:42:34.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.741+0000 7f60fc003740 20 client.15564 trim_cache size 0 max 16384 2024-09-19T06:42:34.743 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 25, 2024-09-19T06:42:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:42:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 2, 2024-09-19T06:42:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "e13bea5b": { 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 2, 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 131.2, 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:42:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:34.745 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f60dc0a35e0 msgr2=0x7f60dc0a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f60dc0a35e0 0x7f60dc0a5aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f60ec0ccb90 tx=0x7f60ec014d00 comp rx=0 tx=0).stop 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f41082a0 msgr2=0x7f60f419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f41082a0 0x7f60f419ced0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00b970 tx=0x7f60ec00be30 comp rx=0 tx=0).stop 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 shutdown_connections 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f60dc0a35e0 0x7f60dc0a5aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f60f41082a0 0x7f60f419ced0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f60f41036e0 0x7f60f4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 --2- 172.21.15.37:0/298546386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f60f4102d30 0x7f60f4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 >> 172.21.15.37:0/298546386 conn(0x7f60f40fc7d0 msgr2=0x7f60f4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:34.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 shutdown_connections 2024-09-19T06:42:34.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:34.742+0000 7f60f9c2a640 1 -- 172.21.15.37:0/298546386 wait complete. 2024-09-19T06:42:34.959 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:35.472 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e13bea5b', quiesce took 4.8 seconds. Will release after: 6.3999999999999995 2024-09-19T06:42:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 -- 172.21.15.37:0/849266711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 msgr2=0x7f3e6410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 --2- 172.21.15.37:0/849266711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e6410d950 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009cd0 tx=0x7f3e5c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 -- 172.21.15.37:0/849266711 shutdown_connections 2024-09-19T06:42:36.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 --2- 172.21.15.37:0/849266711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e6410d950 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 --2- 172.21.15.37:0/849266711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e64100a80 0x7f3e64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 --2- 172.21.15.37:0/849266711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e64106a80 0x7f3e64106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 -- 172.21.15.37:0/849266711 >> 172.21.15.37:0/849266711 conn(0x7f3e640fc7a0 msgr2=0x7f3e640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 -- 172.21.15.37:0/849266711 shutdown_connections 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.991+0000 7f3e69709640 1 -- 172.21.15.37:0/849266711 wait complete. 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.992+0000 7f3e69709640 1 Processor -- start 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.992+0000 7f3e69709640 1 -- start start 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e69709640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e64100a80 0x7f3e640fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e69709640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e640fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e69709640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e64106a80 0x7f3e640fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e69709640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e64110e80 con 0x7f3e64101420 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e69709640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e64110d00 con 0x7f3e64100a80 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e69709640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e64111000 con 0x7f3e64106a80 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e640fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e640fd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54666/0 (socket says 172.21.15.37:54666) 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 -- 172.21.15.37:0/2135078219 learned_addr learned my addr 172.21.15.37:0/2135078219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e637fe640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e64106a80 0x7f3e640fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e64100a80 0x7f3e640fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 -- 172.21.15.37:0/2135078219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e64106a80 msgr2=0x7f3e640fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e64106a80 0x7f3e640fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 -- 172.21.15.37:0/2135078219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e64100a80 msgr2=0x7f3e640fcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e64100a80 0x7f3e640fcde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 -- 172.21.15.37:0/2135078219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e641a68e0 con 0x7f3e64101420 2024-09-19T06:42:36.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e637fe640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e64106a80 0x7f3e640fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e64100a80 0x7f3e640fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e627fc640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e640fd320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e9f0 tx=0x7f3e5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.993+0000 7f3e4bfff640 1 -- 172.21.15.37:0/2135078219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5800cd80 con 0x7f3e64101420 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.994+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e641a6bd0 con 0x7f3e64101420 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.994+0000 7f3e4bfff640 1 -- 172.21.15.37:0/2135078219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e58010910 con 0x7f3e64101420 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.994+0000 7f3e4bfff640 1 -- 172.21.15.37:0/2135078219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e580188f0 con 0x7f3e64101420 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.994+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e641a7160 con 0x7f3e64101420 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.996+0000 7f3e4bfff640 1 -- 172.21.15.37:0/2135078219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f3e58010ac0 con 0x7f3e64101420 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.996+0000 7f3e6bae2740 1 Processor -- start 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.996+0000 7f3e6bae2740 1 -- start start 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ceab40 0x5607e9ce9090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9ce9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cf6760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5607e9c4f2f0 con 0x5607e9cf4370 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5607e9bf6cf0 con 0x5607e9ceab40 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5607e9c4b2b0 con 0x5607e9ce95d0 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cf6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9ce9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e637fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ceab40 0x5607e9ce9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9ce9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:39538/0 (socket says 172.21.15.37:39538) 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e62ffd640 1 -- 172.21.15.37:0/159506882 learned_addr learned my addr 172.21.15.37:0/159506882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e637fe640 1 -- 172.21.15.37:0/159506882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 msgr2=0x5607e9ce9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e637fe640 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9ce9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e637fe640 1 -- 172.21.15.37:0/159506882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 msgr2=0x5607e9cf6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e637fe640 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cf6760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e4bfff640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3e4c0a3560 0x7f3e4c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.997+0000 7f3e637fe640 1 -- 172.21.15.37:0/159506882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e9bdb3a0 con 0x5607e9ceab40 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.998+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3e4c0a3560 0x7f3e4c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.998+0000 7f3e4bfff640 1 -- 172.21.15.37:0/2135078219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f3e580d04c0 con 0x7f3e64101420 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.998+0000 7f3e627fc640 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cf6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.998+0000 7f3e637fe640 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ceab40 0x5607e9ce9090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009ca0 tx=0x7f3e5c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba5e211e8a2afaac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:35.998+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3e4c0a3560 0x7f3e4c0a5a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5607e9ce6ac0 tx=0x7f3e54005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.001+0000 7f3e497fa640 1 -- 172.21.15.37:0/159506882 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c0318c0 con 0x5607e9ceab40 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.001+0000 7f3e497fa640 1 -- 172.21.15.37:0/159506882 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5c036040 con 0x5607e9ceab40 2024-09-19T06:42:36.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.001+0000 7f3e497fa640 1 -- 172.21.15.37:0/159506882 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c004500 con 0x5607e9ceab40 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 -- 172.21.15.37:0/159506882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ceab40 msgr2=0x5607e9ce9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ceab40 0x5607e9ce9090 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c009ca0 tx=0x7f3e5c009a20 comp rx=0 tx=0).stop 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 -- 172.21.15.37:0/159506882 shutdown_connections 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cf6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9ce9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/159506882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ceab40 0x5607e9ce9090 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 -- 172.21.15.37:0/159506882 >> 172.21.15.37:0/159506882 conn(0x5607e9cb2310 msgr2=0x5607e9ce5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 -- 172.21.15.37:0/159506882 shutdown_connections 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.002+0000 7f3e6bae2740 1 -- 172.21.15.37:0/159506882 wait complete. 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 -- start start 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 Processor -- start 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ce95d0 0x5607e9cf3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9cf1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9cf15d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5607e9cf7010 con 0x5607e9ce95d0 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5607e9c0a060 con 0x5607e9cf4370 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5607e9cf7190 con 0x5607e9ceab40 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e6bae2740 10 client.0 _ll_register_callbacks cb 0x5607e995d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ce95d0 0x5607e9cf3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e62ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ce95d0 0x5607e9cf3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54694/0 (socket says 172.21.15.37:54694) 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e62ffd640 1 -- 172.21.15.37:0/307231817 learned_addr learned my addr 172.21.15.37:0/307231817 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e637fe640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9cf15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.003+0000 7f3e627fc640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9cf1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e62ffd640 1 -- 172.21.15.37:0/307231817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 msgr2=0x5607e9cf1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9cf1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e62ffd640 1 -- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 msgr2=0x5607e9cf15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9cf15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e62ffd640 1 -- 172.21.15.37:0/307231817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e99c03c0 con 0x5607e9ce95d0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e637fe640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9cf15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ce95d0 0x5607e9cf3d50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e5401fa00 tx=0x7f3e54004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c79acee7eaa674d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e46ff5640 1 -- 172.21.15.37:0/307231817 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e54004020 con 0x5607e9ce95d0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e46ff5640 1 -- 172.21.15.37:0/307231817 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f3e540041c0 con 0x5607e9ce95d0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5c036040 con 0x5607e9ce95d0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e46ff5640 1 -- 172.21.15.37:0/307231817 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e54004400 con 0x5607e9ce95d0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e6bae2740 10 client.15576 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607e9d02af0 con 0x5607e9ce95d0 2024-09-19T06:42:36.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.004+0000 7f3e46ff5640 1 -- 172.21.15.37:0/307231817 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5c036040 con 0x5607e9ce95d0 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e46ff5640 1 -- 172.21.15.37:0/307231817 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3e540039b0 con 0x5607e9ce95d0 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e6bae2740 10 client.15576 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e6bae2740 10 client.15576 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e6bae2740 20 client.15576 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d03fb0 0x5607e9d063a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e6bae2740 4 client.15576 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5607e9c5c860 con 0x5607e9d03fb0 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.005+0000 7f3e627fc640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d03fb0 0x5607e9d063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.006+0000 7f3e627fc640 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d03fb0 0x5607e9d063a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3576079a5a02fc4c server_cookie=eaf0fa07399d9e26 in_seq=0 out_seq=0 2024-09-19T06:42:36.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.006+0000 7f3e46ff5640 10 client.15576 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:36.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.006+0000 7f3e46ff5640 1 -- 172.21.15.37:0/307231817 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5607e9c5c860 con 0x5607e9d03fb0 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.006+0000 7f3e46ff5640 10 client.15576 handle_command_reply: tid=0 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.006+0000 7f3e6bae2740 1 client.15576 shutdown 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ce95d0 msgr2=0x5607e9cf3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ce95d0 0x5607e9cf3d50 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f3e5401fa00 tx=0x7f3e54004730 comp rx=0 tx=0).stop 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 shutdown_connections 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d03fb0 0x5607e9d063a0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9cf15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9cf1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/307231817 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ce95d0 0x5607e9cf3d50 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 >> 172.21.15.37:0/307231817 conn(0x5607e9cb2310 msgr2=0x5607e9ceb960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e46ff5640 0 client.15576 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:36.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 shutdown_connections 2024-09-19T06:42:36.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 1 -- 172.21.15.37:0/307231817 wait complete. 2024-09-19T06:42:36.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.007+0000 7f3e6bae2740 20 client.15576 trim_cache size 0 max 16384 2024-09-19T06:42:36.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.033+0000 7f3e6bae2740 1 Processor -- start 2024-09-19T06:42:36.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e6bae2740 1 -- start start 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ce95d0 0x5607e9cf08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9ce4c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cec120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641b2520 con 0x5607e9cf4370 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641afdf0 con 0x5607e9ce95d0 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641adc40 con 0x5607e9ceab40 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cec120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cec120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54702/0 (socket says 172.21.15.37:54702) 2024-09-19T06:42:36.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 -- 172.21.15.37:0/2684314399 learned_addr learned my addr 172.21.15.37:0/2684314399 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e627fc640 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9ce4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ce95d0 0x5607e9cf08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 -- 172.21.15.37:0/2684314399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 msgr2=0x5607e9ce4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9ce4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 -- 172.21.15.37:0/2684314399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ce95d0 msgr2=0x5607e9cf08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ce95d0 0x5607e9cf08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.034+0000 7f3e637fe640 1 -- 172.21.15.37:0/2684314399 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5607e9ceffd0 con 0x5607e9cf4370 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.035+0000 7f3e637fe640 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cec120 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02ae00 tx=0x7f3e5c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373482b354105e50 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.035+0000 7f3e48ff9640 1 -- 172.21.15.37:0/2684314399 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c031530 con 0x5607e9cf4370 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.035+0000 7f3e48ff9640 1 -- 172.21.15.37:0/2684314399 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5c036070 con 0x5607e9cf4370 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.035+0000 7f3e48ff9640 1 -- 172.21.15.37:0/2684314399 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c004860 con 0x5607e9cf4370 2024-09-19T06:42:36.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 -- 172.21.15.37:0/2684314399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 msgr2=0x5607e9cec120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cec120 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c02ae00 tx=0x7f3e5c03d040 comp rx=0 tx=0).stop 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 -- 172.21.15.37:0/2684314399 shutdown_connections 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9cf4370 0x5607e9cec120 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ceab40 0x5607e9ce4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/2684314399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9ce95d0 0x5607e9cf08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 -- 172.21.15.37:0/2684314399 >> 172.21.15.37:0/2684314399 conn(0x5607e9cb2310 msgr2=0x5607e9ce4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 -- 172.21.15.37:0/2684314399 shutdown_connections 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 -- 172.21.15.37:0/2684314399 wait complete. 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 -- start start 2024-09-19T06:42:36.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.036+0000 7f3e6bae2740 1 Processor -- start 2024-09-19T06:42:36.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9cfc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ceab40 0x5607e9cfcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e6bae2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9ced9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641ae2b0 con 0x5607e9ceab40 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ceab40 0x5607e9cfcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ceab40 0x5607e9cfcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54704/0 (socket says 172.21.15.37:54704) 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e6bae2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641af720 con 0x5607e9cf4370 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 -- 172.21.15.37:0/3904020776 learned_addr learned my addr 172.21.15.37:0/3904020776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3e641ae980 con 0x5607e9ce95d0 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e6bae2740 10 client.0 _ll_register_callbacks cb 0x7f3e5401f890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e637fe640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9ced9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9cfc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 -- 172.21.15.37:0/3904020776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 msgr2=0x5607e9cfc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9cfc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 -- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 msgr2=0x5607e9ced9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9ced9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e627fc640 1 -- 172.21.15.37:0/3904020776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5607e9cee1e0 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.037+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9cfc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e627fc640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ceab40 0x5607e9cfcc20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e580989c0 tx=0x7f3e580d0440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8334e91a8096434a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e46ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e46ff5640 1 -- 172.21.15.37:0/3904020776 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5809c560 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e46ff5640 1 -- 172.21.15.37:0/3904020776 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f3e58098bd0 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e5c036070 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e46ff5640 1 -- 172.21.15.37:0/3904020776 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e580a0910 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5607e9d6e7c0 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e46ff5640 1 -- 172.21.15.37:0/3904020776 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e5c036070 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e46ff5640 1 -- 172.21.15.37:0/3904020776 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f3e5809b9b0 con 0x5607e9ceab40 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:36.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 10 client.15582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 20 client.15582 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d6eec0 0x5607e9d712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 4 client.15582 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.038+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5607e9c56d70 con 0x5607e9d6eec0 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d6eec0 0x5607e9d712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e46ff5640 10 client.15582 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e62ffd640 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d6eec0 0x5607e9d712e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5428c4e9308dc4ec server_cookie=eeb9b7e38acc5945 in_seq=0 out_seq=0 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e46ff5640 1 -- 172.21.15.37:0/3904020776 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5607e9c56d70 con 0x5607e9d6eec0 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e46ff5640 10 client.15582 handle_command_reply: tid=0 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 client.15582 shutdown 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ceab40 msgr2=0x5607e9cfcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ceab40 0x5607e9cfcc20 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f3e580989c0 tx=0x7f3e580d0440 comp rx=0 tx=0).stop 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 shutdown_connections 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5607e9d6eec0 0x5607e9d712e0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5607e9cf4370 0x5607e9ced9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5607e9ceab40 0x5607e9cfcc20 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e6bae2740 1 --2- 172.21.15.37:0/3904020776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5607e9ce95d0 0x5607e9cfc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.039+0000 7f3e46ff5640 0 client.15582 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 >> 172.21.15.37:0/3904020776 conn(0x5607e9cb2310 msgr2=0x5607e9ce4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 shutdown_connections 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e6bae2740 1 -- 172.21.15.37:0/3904020776 wait complete. 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e6bae2740 20 client.15582 trim_cache size 0 max 16384 2024-09-19T06:42:36.045 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:42:36.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3e4c0a3560 msgr2=0x7f3e4c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3e4c0a3560 0x7f3e4c0a5a20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x5607e9ce6ac0 tx=0x7f3e54005f90 comp rx=0 tx=0).stop 2024-09-19T06:42:36.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 msgr2=0x7f3e640fd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:36.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e640fd320 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f3e5800e9f0 tx=0x7f3e5800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 shutdown_connections 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3e64106a80 0x7f3e640fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3e4c0a3560 0x7f3e4c0a5a20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3e64101420 0x7f3e640fd320 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 --2- 172.21.15.37:0/2135078219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3e64100a80 0x7f3e640fcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.040+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 >> 172.21.15.37:0/2135078219 conn(0x7f3e640fc7a0 msgr2=0x7f3e6406aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.041+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 shutdown_connections 2024-09-19T06:42:36.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:36.041+0000 7f3e69709640 1 -- 172.21.15.37:0/2135078219 wait complete. 2024-09-19T06:42:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:35 smithi096 ceph-mon[35004]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:36.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:35 smithi037 ceph-mon[31211]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:35 smithi151 ceph-mon[35236]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:36.747 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:42:37.008 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:37.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.476+0000 7f68b8d89640 1 -- 172.21.15.37:0/439564320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41036e0 msgr2=0x7f68b4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.476+0000 7f68b8d89640 1 --2- 172.21.15.37:0/439564320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41036e0 0x7f68b4107b90 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f689c009930 tx=0x7f689c02a420 comp rx=0 tx=0).stop 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.476+0000 7f68b8d89640 1 -- 172.21.15.37:0/439564320 shutdown_connections 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.476+0000 7f68b8d89640 1 --2- 172.21.15.37:0/439564320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b41082a0 0x7f68b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.476+0000 7f68b8d89640 1 --2- 172.21.15.37:0/439564320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41036e0 0x7f68b4107b90 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.476+0000 7f68b8d89640 1 --2- 172.21.15.37:0/439564320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b4102d30 0x7f68b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.476+0000 7f68b8d89640 1 -- 172.21.15.37:0/439564320 >> 172.21.15.37:0/439564320 conn(0x7f68b40fc7d0 msgr2=0x7f68b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.477+0000 7f68b8d89640 1 -- 172.21.15.37:0/439564320 shutdown_connections 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.477+0000 7f68b8d89640 1 -- 172.21.15.37:0/439564320 wait complete. 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 Processor -- start 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 -- start start 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b4102d30 0x7f68b4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b41036e0 0x7f68b4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41082a0 0x7f68b4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41120b0 con 0x7f68b41082a0 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4111f30 con 0x7f68b41036e0 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b8d89640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68b4112230 con 0x7f68b4102d30 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41082a0 0x7f68b4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41082a0 0x7f68b4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54728/0 (socket says 172.21.15.37:54728) 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 -- 172.21.15.37:0/166122767 learned_addr learned my addr 172.21.15.37:0/166122767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b259c640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b4102d30 0x7f68b4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68abfff640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b41036e0 0x7f68b4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 -- 172.21.15.37:0/166122767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b4102d30 msgr2=0x7f68b4100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b4102d30 0x7f68b4100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 -- 172.21.15.37:0/166122767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b41036e0 msgr2=0x7f68b4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b41036e0 0x7f68b4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b2d9d640 1 -- 172.21.15.37:0/166122767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68b41a6a70 con 0x7f68b41082a0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68b259c640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b4102d30 0x7f68b4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.478+0000 7f68abfff640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b41036e0 0x7f68b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.479+0000 7f68b2d9d640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41082a0 0x7f68b4101410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eb10 tx=0x7f68ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.479+0000 7f68ab7fe640 1 -- 172.21.15.37:0/166122767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac004b20 con 0x7f68b41082a0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.479+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68b41a6d60 con 0x7f68b41082a0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.479+0000 7f68ab7fe640 1 -- 172.21.15.37:0/166122767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68ac004cc0 con 0x7f68b41082a0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.479+0000 7f68ab7fe640 1 -- 172.21.15.37:0/166122767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68ac010760 con 0x7f68b41082a0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.479+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68b41a7220 con 0x7f68b41082a0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68ab7fe640 1 -- 172.21.15.37:0/166122767 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f68ac005020 con 0x7f68b41082a0 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 Processor -- start 2024-09-19T06:42:37.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 -- start start 2024-09-19T06:42:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0da090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0daa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0e5370 0x55e89c0e7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e89c0402f0 con 0x55e89c0e5370 2024-09-19T06:42:37.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e89bfe7cf0 con 0x55e89c0da5d0 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.482+0000 7f68bb162740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e89c03c2b0 con 0x55e89c0dbb40 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68ab7fe640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68a00a3560 0x7f68a00a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0e5370 0x55e89c0e7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0e5370 0x55e89c0e7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54732/0 (socket says 172.21.15.37:54732) 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68abfff640 1 -- 172.21.15.37:0/3987315868 learned_addr learned my addr 172.21.15.37:0/3987315868 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b259c640 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0daa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68abfff640 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0da090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68ab7fe640 1 -- 172.21.15.37:0/166122767 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f68ac0d0670 con 0x7f68b41082a0 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b2d9d640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68a00a3560 0x7f68a00a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b259c640 1 -- 172.21.15.37:0/3987315868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 msgr2=0x55e89c0da090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b259c640 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0da090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b259c640 1 -- 172.21.15.37:0/3987315868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0e5370 msgr2=0x55e89c0e7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b259c640 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0e5370 0x55e89c0e7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b259c640 1 -- 172.21.15.37:0/3987315868 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e89bfcc3a0 con 0x55e89c0da5d0 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.483+0000 7f68b259c640 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0daa10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009c30 tx=0x7f68a40117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d582f67bbc05c3a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.484+0000 7f68b2d9d640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68a00a3560 0x7f68a00a5a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0cdbe0 tx=0x7f68ac00efd0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.484+0000 7f68a8ff9640 1 -- 172.21.15.37:0/3987315868 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4005020 con 0x55e89c0da5d0 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.484+0000 7f68a8ff9640 1 -- 172.21.15.37:0/3987315868 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a40047a0 con 0x55e89c0da5d0 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.484+0000 7f68a8ff9640 1 -- 172.21.15.37:0/3987315868 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a40126e0 con 0x55e89c0da5d0 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.484+0000 7f68bb162740 1 -- 172.21.15.37:0/3987315868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 msgr2=0x55e89c0daa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.484+0000 7f68bb162740 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0daa10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009c30 tx=0x7f68a40117b0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- 172.21.15.37:0/3987315868 shutdown_connections 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0e5370 0x55e89c0e7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0daa10 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 --2- 172.21.15.37:0/3987315868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0da090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- 172.21.15.37:0/3987315868 >> 172.21.15.37:0/3987315868 conn(0x55e89c0a3310 msgr2=0x55e89c0d6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- 172.21.15.37:0/3987315868 shutdown_connections 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- 172.21.15.37:0/3987315868 wait complete. 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- start start 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 Processor -- start 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0dbb40 0x55e89c0e1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 0x55e89c0e24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e89c0e8070 con 0x55e89c0dbb40 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e89bffb060 con 0x55e89c0da5d0 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.485+0000 7f68bb162740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e89c0e81f0 con 0x55e89c0e5370 2024-09-19T06:42:37.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68bb162740 10 client.0 _ll_register_callbacks cb 0x55e89bd4e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0dbb40 0x55e89c0e1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0dbb40 0x55e89c0e1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54748/0 (socket says 172.21.15.37:54748) 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 -- 172.21.15.37:0/94110089 learned_addr learned my addr 172.21.15.37:0/94110089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68abfff640 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68abfff640 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 0x55e89c0e24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 -- 172.21.15.37:0/94110089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 msgr2=0x55e89c0e24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 0x55e89c0e24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 -- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 msgr2=0x55e89c0e4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0e4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 -- 172.21.15.37:0/94110089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e89bdb13c0 con 0x55e89c0dbb40 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f68b259c640 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0dbb40 0x55e89c0e1f90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f68a4004e10 tx=0x7f68a4004720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=796c9954abdc965e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f6899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:37.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.486+0000 7f6899ffb640 1 -- 172.21.15.37:0/94110089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68a4028020 con 0x55e89c0dbb40 2024-09-19T06:42:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f6899ffb640 1 -- 172.21.15.37:0/94110089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f68a4004fc0 con 0x55e89c0dbb40 2024-09-19T06:42:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68a40047a0 con 0x55e89c0dbb40 2024-09-19T06:42:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f6899ffb640 1 -- 172.21.15.37:0/94110089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68a40051e0 con 0x55e89c0dbb40 2024-09-19T06:42:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 10 client.15594 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e89c0f3a80 con 0x55e89c0dbb40 2024-09-19T06:42:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f6899ffb640 1 -- 172.21.15.37:0/94110089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f68a40047a0 con 0x55e89c0dbb40 2024-09-19T06:42:37.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f6899ffb640 1 -- 172.21.15.37:0/94110089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f68a4024460 con 0x55e89c0dbb40 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 10 client.15594 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 20 client.15594 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c0f5260 0x55e89c0f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 4 client.15594 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.487+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e89c04d860 con 0x55e89c0f5260 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.488+0000 7f68abfff640 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c0f5260 0x55e89c0f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.488+0000 7f6899ffb640 10 client.15594 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.488+0000 7f68abfff640 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c0f5260 0x55e89c0f76a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=994d44175a301e16 server_cookie=842608c5297dab0a in_seq=0 out_seq=0 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f6899ffb640 1 -- 172.21.15.37:0/94110089 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e89c04d860 con 0x55e89c0f5260 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f6899ffb640 10 client.15594 handle_command_reply: tid=0 2024-09-19T06:42:37.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 client.15594 shutdown 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0dbb40 msgr2=0x55e89c0e1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0dbb40 0x55e89c0e1f90 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f68a4004e10 tx=0x7f68a4004720 comp rx=0 tx=0).stop 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 shutdown_connections 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c0f5260 0x55e89c0f76a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 0x55e89c0e24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0dbb40 0x55e89c0e1f90 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 --2- 172.21.15.37:0/94110089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0da5d0 0x55e89c0e4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 >> 172.21.15.37:0/94110089 conn(0x55e89c0a3310 msgr2=0x55e89c0dc960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f6899ffb640 0 client.15594 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 shutdown_connections 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.489+0000 7f68bb162740 1 -- 172.21.15.37:0/94110089 wait complete. 2024-09-19T06:42:37.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.490+0000 7f68bb162740 20 client.15594 trim_cache size 0 max 16384 2024-09-19T06:42:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 Processor -- start 2024-09-19T06:42:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 -- start start 2024-09-19T06:42:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0d6e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0f1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0e5370 0x55e89c0fcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41b2330 con 0x55e89c0da5d0 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41afc00 con 0x55e89c0e5370 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68bb162740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ada50 con 0x55e89c0dbb40 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0d6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0d6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54754/0 (socket says 172.21.15.37:54754) 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68abfff640 1 -- 172.21.15.37:0/1260607956 learned_addr learned my addr 172.21.15.37:0/1260607956 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68b259c640 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0f1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68abfff640 1 -- 172.21.15.37:0/1260607956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 msgr2=0x55e89c0f1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68abfff640 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0f1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68abfff640 1 -- 172.21.15.37:0/1260607956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0e5370 msgr2=0x55e89c0fcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.516+0000 7f68abfff640 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0e5370 0x55e89c0fcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.517+0000 7f68abfff640 1 -- 172.21.15.37:0/1260607956 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e89c0d7960 con 0x55e89c0da5d0 2024-09-19T06:42:37.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.517+0000 7f68abfff640 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0d6e10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f689c009900 tx=0x7f689c040040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8f4f9eeb2e319e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.517+0000 7f689bfff640 1 -- 172.21.15.37:0/1260607956 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c035040 con 0x55e89c0da5d0 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.517+0000 7f689bfff640 1 -- 172.21.15.37:0/1260607956 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689c0035f0 con 0x55e89c0da5d0 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.517+0000 7f689bfff640 1 -- 172.21.15.37:0/1260607956 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c03d6b0 con 0x55e89c0da5d0 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 -- 172.21.15.37:0/1260607956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 msgr2=0x55e89c0d6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0d6e10 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f689c009900 tx=0x7f689c040040 comp rx=0 tx=0).stop 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 -- 172.21.15.37:0/1260607956 shutdown_connections 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0e5370 0x55e89c0fcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0dbb40 0x55e89c0f1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 --2- 172.21.15.37:0/1260607956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0d6e10 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 -- 172.21.15.37:0/1260607956 >> 172.21.15.37:0/1260607956 conn(0x55e89c0a3310 msgr2=0x55e89c0d6700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 -- 172.21.15.37:0/1260607956 shutdown_connections 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 -- 172.21.15.37:0/1260607956 wait complete. 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.518+0000 7f68bb162740 1 -- start start 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 1 Processor -- start 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0ed480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0dbb40 0x55e89c0ed9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 0x55e89c0de200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ae0c0 con 0x55e89c0da5d0 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41af530 con 0x55e89c0dbb40 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f68b41ae790 con 0x55e89c0e5370 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68bb162740 10 client.0 _ll_register_callbacks cb 0x7f68ac0093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0ed480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0ed480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54762/0 (socket says 172.21.15.37:54762) 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 -- 172.21.15.37:0/2574723485 learned_addr learned my addr 172.21.15.37:0/2574723485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68b259c640 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0dbb40 0x55e89c0ed9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 -- 172.21.15.37:0/2574723485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 msgr2=0x55e89c0de200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 0x55e89c0de200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 -- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0dbb40 msgr2=0x55e89c0ed9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0dbb40 0x55e89c0ed9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.519+0000 7f68abfff640 1 -- 172.21.15.37:0/2574723485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e89c0dd120 con 0x55e89c0da5d0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f68abfff640 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0ed480 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f689c000c00 tx=0x7f689c009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc2a037a23f0233c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f6899ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f6899ffb640 1 -- 172.21.15.37:0/2574723485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f689c035030 con 0x55e89c0da5d0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f689c0035f0 con 0x55e89c0da5d0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f6899ffb640 1 -- 172.21.15.37:0/2574723485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f689c0039a0 con 0x55e89c0da5d0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f6899ffb640 1 -- 172.21.15.37:0/2574723485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f689c032040 con 0x55e89c0da5d0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f68bb162740 10 client.15600 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e89c15f290 con 0x55e89c0da5d0 2024-09-19T06:42:37.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.520+0000 7f6899ffb640 1 -- 172.21.15.37:0/2574723485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f689c0035f0 con 0x55e89c0da5d0 2024-09-19T06:42:37.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f6899ffb640 1 -- 172.21.15.37:0/2574723485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f689c03a040 con 0x55e89c0da5d0 2024-09-19T06:42:37.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68bb162740 10 client.15600 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:37.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68bb162740 10 client.15600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:37.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68bb162740 20 client.15600 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:37.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68bb162740 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c15f8f0 0x55e89c161d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68bb162740 4 client.15600 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e89c047d70 con 0x55e89c15f8f0 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68b259c640 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c15f8f0 0x55e89c161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f6899ffb640 10 client.15600 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.521+0000 7f68b259c640 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c15f8f0 0x55e89c161d10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c0f918413f9496d server_cookie=425fd1fdb828b675 in_seq=0 out_seq=0 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f6899ffb640 1 -- 172.21.15.37:0/2574723485 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e89c047d70 con 0x55e89c15f8f0 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f6899ffb640 10 client.15600 handle_command_reply: tid=0 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 client.15600 shutdown 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 msgr2=0x55e89c0ed480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0ed480 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f689c000c00 tx=0x7f689c009660 comp rx=0 tx=0).stop 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 shutdown_connections 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e89c15f8f0 0x55e89c161d10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e89c0e5370 0x55e89c0de200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e89c0dbb40 0x55e89c0ed9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 --2- 172.21.15.37:0/2574723485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e89c0da5d0 0x55e89c0ed480 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 >> 172.21.15.37:0/2574723485 conn(0x55e89c0a3310 msgr2=0x55e89c0e0860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f6899ffb640 0 client.15600 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 shutdown_connections 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 1 -- 172.21.15.37:0/2574723485 wait complete. 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.522+0000 7f68bb162740 20 client.15600 trim_cache size 0 max 16384 2024-09-19T06:42:37.527 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:42:37.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68a00a3560 msgr2=0x7f68a00a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68a00a3560 0x7f68a00a5a20 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f68ac0cdbe0 tx=0x7f68ac00efd0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41082a0 msgr2=0x7f68b4101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41082a0 0x7f68b4101410 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f68ac00eb10 tx=0x7f68ac00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 shutdown_connections 2024-09-19T06:42:37.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f68a00a3560 0x7f68a00a5a20 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f68b41082a0 0x7f68b4101410 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f68b41036e0 0x7f68b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 --2- 172.21.15.37:0/166122767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f68b4102d30 0x7f68b4100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 >> 172.21.15.37:0/166122767 conn(0x7f68b40fc7d0 msgr2=0x7f68b4106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 shutdown_connections 2024-09-19T06:42:37.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:37.523+0000 7f68b8d89640 1 -- 172.21.15.37:0/166122767 wait complete. 2024-09-19T06:42:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 -- 172.21.15.37:0/379247250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381009d0 msgr2=0x7f0538100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 --2- 172.21.15.37:0/379247250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381009d0 0x7f0538100e30 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0528009930 tx=0x7f052802a420 comp rx=0 tx=0).stop 2024-09-19T06:42:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 -- 172.21.15.37:0/379247250 shutdown_connections 2024-09-19T06:42:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 --2- 172.21.15.37:0/379247250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0538101370 0x7f053810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 --2- 172.21.15.37:0/379247250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381009d0 0x7f0538100e30 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 --2- 172.21.15.37:0/379247250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05381069e0 0x7f0538106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 -- 172.21.15.37:0/379247250 >> 172.21.15.37:0/379247250 conn(0x7f05380fc6b0 msgr2=0x7f05380fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 -- 172.21.15.37:0/379247250 shutdown_connections 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.612+0000 7f053ff86640 1 -- 172.21.15.37:0/379247250 wait complete. 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 Processor -- start 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 -- start start 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05381009d0 0x7f0538198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0538101370 0x7f05381988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381069e0 0x7f053819cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0538110e20 con 0x7f05381069e0 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0538110ca0 con 0x7f0538101370 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053ff86640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0538110fa0 con 0x7f05381009d0 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381069e0 0x7f053819cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381069e0 0x7f053819cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54792/0 (socket says 172.21.15.37:54792) 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 -- 172.21.15.37:0/2460790908 learned_addr learned my addr 172.21.15.37:0/2460790908 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053d521640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0538101370 0x7f05381988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053dd22640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05381009d0 0x7f0538198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 -- 172.21.15.37:0/2460790908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05381009d0 msgr2=0x7f0538198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05381009d0 0x7f0538198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 -- 172.21.15.37:0/2460790908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0538101370 msgr2=0x7f05381988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0538101370 0x7f05381988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.613+0000 7f053e523640 1 -- 172.21.15.37:0/2460790908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053819d370 con 0x7f05381069e0 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.614+0000 7f053e523640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381069e0 0x7f053819cc40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f053000eb10 tx=0x7f053000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.614+0000 7f052effd640 1 -- 172.21.15.37:0/2460790908 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530004b20 con 0x7f05381069e0 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.614+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053819d660 con 0x7f05381069e0 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.614+0000 7f052effd640 1 -- 172.21.15.37:0/2460790908 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0530004cc0 con 0x7f05381069e0 2024-09-19T06:42:37.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.614+0000 7f052effd640 1 -- 172.21.15.37:0/2460790908 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0530010760 con 0x7f05381069e0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.614+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05381a4ef0 con 0x7f05381069e0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.616+0000 7f052effd640 1 -- 172.21.15.37:0/2460790908 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f0530005020 con 0x7f05381069e0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 Processor -- start 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 -- start start 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370399d0 0x564637039db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463703a2f0 0x564637048b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564636f9e2f0 con 0x5646370490c0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564636f46270 con 0x5646370399d0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f0542dfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564636f9a2b0 con 0x56463703a2f0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f053d521640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f053dd22640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463703a2f0 0x564637048b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f052effd640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05240a3610 0x7f05240a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f053d521640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54800/0 (socket says 172.21.15.37:54800) 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f053d521640 1 -- 172.21.15.37:0/941886882 learned_addr learned my addr 172.21.15.37:0/941886882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.617+0000 7f053d521640 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370399d0 0x564637039db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053e523640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05240a3610 0x7f05240a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f052effd640 1 -- 172.21.15.37:0/2460790908 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f05300d0630 con 0x7f05381069e0 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053dd22640 1 -- 172.21.15.37:0/941886882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370399d0 msgr2=0x564637039db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053dd22640 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370399d0 0x564637039db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053dd22640 1 -- 172.21.15.37:0/941886882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 msgr2=0x56463704b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053dd22640 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053dd22640 1 -- 172.21.15.37:0/941886882 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564636f45340 con 0x56463703a2f0 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053d521640 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053dd22640 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463703a2f0 0x564637048b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0534009af0 tx=0x7f05340113e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bbcea60788b4a280 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f053e523640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05240a3610 0x7f05240a5ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05300cdba0 tx=0x7f053000efd0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f0523fff640 1 -- 172.21.15.37:0/941886882 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534011e10 con 0x56463703a2f0 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f0523fff640 1 -- 172.21.15.37:0/941886882 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0534012930 con 0x56463703a2f0 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.618+0000 7f0523fff640 1 -- 172.21.15.37:0/941886882 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053400fa30 con 0x56463703a2f0 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 -- 172.21.15.37:0/941886882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463703a2f0 msgr2=0x564637048b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463703a2f0 0x564637048b80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0534009af0 tx=0x7f05340113e0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 -- 172.21.15.37:0/941886882 shutdown_connections 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56463703a2f0 0x564637048b80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 --2- 172.21.15.37:0/941886882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370399d0 0x564637039db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 -- 172.21.15.37:0/941886882 >> 172.21.15.37:0/941886882 conn(0x564637036060 msgr2=0x564637036440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 -- 172.21.15.37:0/941886882 shutdown_connections 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.619+0000 7f0542dfc740 1 -- 172.21.15.37:0/941886882 wait complete. 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.620+0000 7f0542dfc740 1 -- start start 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.620+0000 7f0542dfc740 1 Processor -- start 2024-09-19T06:42:37.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.620+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x564637045ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637046a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f0542dfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56463704bcf0 con 0x56463703a2f0 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f0542dfc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564636f27290 con 0x5646370490c0 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f0542dfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56463704be70 con 0x5646370399d0 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f0542dfc740 10 client.0 _ll_register_callbacks cb 0x564636f58140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54818/0 (socket says 172.21.15.37:54818) 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 -- 172.21.15.37:0/4040057533 learned_addr learned my addr 172.21.15.37:0/4040057533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053d521640 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x564637045ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053d521640 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637046a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 -- 172.21.15.37:0/4040057533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 msgr2=0x564637045ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x564637045ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 -- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 msgr2=0x564637046a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637046a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 -- 172.21.15.37:0/4040057533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564636cb1210 con 0x56463703a2f0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f053dd22640 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046530 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f053400f630 tx=0x7f0534005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa78cd223813e21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f05217fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f05217fa640 1 -- 172.21.15.37:0/4040057533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534028070 con 0x56463703a2f0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f05217fa640 1 -- 172.21.15.37:0/4040057533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0534004040 con 0x56463703a2f0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.621+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0534012930 con 0x56463703a2f0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f05217fa640 1 -- 172.21.15.37:0/4040057533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0534034050 con 0x56463703a2f0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 10 client.15612 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564636d7d620 con 0x56463703a2f0 2024-09-19T06:42:37.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f05217fa640 1 -- 172.21.15.37:0/4040057533 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0534012930 con 0x56463703a2f0 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f05217fa640 1 -- 172.21.15.37:0/4040057533 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f05340035f0 con 0x56463703a2f0 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 10 client.15612 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 10 client.15612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 20 client.15612 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564637058fe0 0x56463705b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 4 client.15612 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564636faba20 con 0x564637058fe0 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.622+0000 7f053d521640 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564637058fe0 0x56463705b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.623+0000 7f053d521640 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564637058fe0 0x56463705b3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7960afd39015d39 server_cookie=a70a77a0a4406e6d in_seq=0 out_seq=0 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.623+0000 7f05217fa640 10 client.15612 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.623+0000 7f05217fa640 1 -- 172.21.15.37:0/4040057533 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564636faba20 con 0x564637058fe0 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.623+0000 7f05217fa640 10 client.15612 handle_command_reply: tid=0 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.623+0000 7f0542dfc740 1 client.15612 shutdown 2024-09-19T06:42:37.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 msgr2=0x564637046530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046530 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f053400f630 tx=0x7f0534005760 comp rx=0 tx=0).stop 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 shutdown_connections 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564637058fe0 0x56463705b3d0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637046a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046530 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f05217fa640 0 client.15612 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 --2- 172.21.15.37:0/4040057533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x564637045ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 >> 172.21.15.37:0/4040057533 conn(0x564637036060 msgr2=0x56463703bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 shutdown_connections 2024-09-19T06:42:37.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 1 -- 172.21.15.37:0/4040057533 wait complete. 2024-09-19T06:42:37.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.624+0000 7f0542dfc740 20 client.15612 trim_cache size 0 max 16384 2024-09-19T06:42:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.638+0000 7f0542dfc740 1 Processor -- start 2024-09-19T06:42:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.638+0000 7f0542dfc740 1 -- start start 2024-09-19T06:42:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.638+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x5646370466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.638+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.638+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637060a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f0542dfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05381adf90 con 0x56463703a2f0 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f0542dfc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05381ab860 con 0x5646370490c0 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f0542dfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05381a96d0 con 0x5646370399d0 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54828/0 (socket says 172.21.15.37:54828) 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 -- 172.21.15.37:0/3001592540 learned_addr learned my addr 172.21.15.37:0/3001592540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053d521640 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x5646370466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 -- 172.21.15.37:0/3001592540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 msgr2=0x5646370466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053d521640 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637060a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x5646370466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 -- 172.21.15.37:0/3001592540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 msgr2=0x564637060a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637060a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 -- 172.21.15.37:0/3001592540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564637040d50 con 0x56463703a2f0 2024-09-19T06:42:37.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053d521640 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637060a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f053dd22640 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f05340095b0 tx=0x7f0534005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d29acc141a8ec9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f05237fe640 1 -- 172.21.15.37:0/3001592540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f053403d070 con 0x56463703a2f0 2024-09-19T06:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f05237fe640 1 -- 172.21.15.37:0/3001592540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05340118e0 con 0x56463703a2f0 2024-09-19T06:42:37.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.639+0000 7f05237fe640 1 -- 172.21.15.37:0/3001592540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0534028070 con 0x56463703a2f0 2024-09-19T06:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 -- 172.21.15.37:0/3001592540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 msgr2=0x564637046c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f05340095b0 tx=0x7f0534005040 comp rx=0 tx=0).stop 2024-09-19T06:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 -- 172.21.15.37:0/3001592540 shutdown_connections 2024-09-19T06:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646370490c0 0x564637060a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56463703a2f0 0x564637046c00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 --2- 172.21.15.37:0/3001592540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x5646370466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 -- 172.21.15.37:0/3001592540 >> 172.21.15.37:0/3001592540 conn(0x564637036060 msgr2=0x564637045fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 -- 172.21.15.37:0/3001592540 shutdown_connections 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.640+0000 7f0542dfc740 1 -- 172.21.15.37:0/3001592540 wait complete. 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 -- start start 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 Processor -- start 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x56463704cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56463703a2f0 0x56463704d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05381a9d20 con 0x5646370490c0 2024-09-19T06:42:37.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05381ab190 con 0x56463703a2f0 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05381aa3f0 con 0x5646370399d0 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f0542dfc740 10 client.0 _ll_register_callbacks cb 0x7f0534005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f053d521640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f053d521640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54840/0 (socket says 172.21.15.37:54840) 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f053d521640 1 -- 172.21.15.37:0/1517054712 learned_addr learned my addr 172.21.15.37:0/1517054712 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f053d521640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x56463704cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.641+0000 7f053dd22640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56463703a2f0 0x56463704d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f053d521640 1 -- 172.21.15.37:0/1517054712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 msgr2=0x56463704cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f053d521640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x56463704cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f053d521640 1 -- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56463703a2f0 msgr2=0x56463704d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f053d521640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56463703a2f0 0x56463704d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f053d521640 1 -- 172.21.15.37:0/1517054712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564637035830 con 0x5646370490c0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f053dd22640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56463703a2f0 0x56463704d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f053d521640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704d980 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0528005940 tx=0x7f052803f040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e214445b38da0d57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f05217fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f05217fa640 1 -- 172.21.15.37:0/1517054712 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0528035040 con 0x5646370490c0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f05217fa640 1 -- 172.21.15.37:0/1517054712 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0528004450 con 0x5646370490c0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05340118e0 con 0x5646370490c0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f05217fa640 1 -- 172.21.15.37:0/1517054712 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f05280035f0 con 0x5646370490c0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f0542dfc740 10 client.15618 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5646370c3ba0 con 0x5646370490c0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.642+0000 7f05217fa640 1 -- 172.21.15.37:0/1517054712 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05340118e0 con 0x5646370490c0 2024-09-19T06:42:37.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f05217fa640 1 -- 172.21.15.37:0/1517054712 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f052803d7f0 con 0x5646370490c0 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f0542dfc740 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f0542dfc740 10 client.15618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f0542dfc740 20 client.15618 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f0542dfc740 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5646370c42a0 0x5646370c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f0542dfc740 4 client.15618 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564636fa5d70 con 0x5646370c42a0 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.643+0000 7f053dd22640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5646370c42a0 0x5646370c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.644+0000 7f05217fa640 10 client.15618 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.644+0000 7f053dd22640 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5646370c42a0 0x5646370c66c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c816f1a994664b4 server_cookie=1510fcb44839c0f0 in_seq=0 out_seq=0 2024-09-19T06:42:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.650+0000 7f05217fa640 1 -- 172.21.15.37:0/1517054712 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564636fa5d70 con 0x5646370c42a0 2024-09-19T06:42:37.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.650+0000 7f05217fa640 10 client.15618 handle_command_reply: tid=0 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.650+0000 7f0542dfc740 1 client.15618 shutdown 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.650+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 msgr2=0x56463704d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.650+0000 7f0542dfc740 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704d980 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0528005940 tx=0x7f052803f040 comp rx=0 tx=0).stop 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 shutdown_connections 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5646370c42a0 0x5646370c66c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5646370490c0 0x56463704d980 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56463703a2f0 0x56463704d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f05217fa640 0 client.15618 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 --2- 172.21.15.37:0/1517054712 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5646370399d0 0x56463704cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 >> 172.21.15.37:0/1517054712 conn(0x564637036060 msgr2=0x56463703bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 shutdown_connections 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 1 -- 172.21.15.37:0/1517054712 wait complete. 2024-09-19T06:42:37.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f0542dfc740 20 client.15618 trim_cache size 0 max 16384 2024-09-19T06:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:42:37.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05240a3610 msgr2=0x7f05240a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05240a3610 0x7f05240a5ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05300cdba0 tx=0x7f053000efd0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381069e0 msgr2=0x7f053819cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381069e0 0x7f053819cc40 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f053000eb10 tx=0x7f053000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 shutdown_connections 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05240a3610 0x7f05240a5ad0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05381069e0 0x7f053819cc40 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0538101370 0x7f05381988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 --2- 172.21.15.37:0/2460790908 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05381009d0 0x7f0538198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 >> 172.21.15.37:0/2460790908 conn(0x7f05380fc6b0 msgr2=0x7f0538104740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.651+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 shutdown_connections 2024-09-19T06:42:37.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:37.652+0000 7f053ff86640 1 -- 172.21.15.37:0/2460790908 wait complete. 2024-09-19T06:42:38.196 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:38 smithi037 ceph-mon[31211]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:38 smithi096 ceph-mon[35004]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:38.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:38 smithi151 ceph-mon[35236]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-19T06:42:39.230 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:42:39.230 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:42:39.499 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:39.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 -- 172.21.15.37:0/4240550874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc100e20 msgr2=0x7fb0dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 --2- 172.21.15.37:0/4240550874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc100e20 0x7fb0dc101200 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009900 tx=0x7fb0d802a480 comp rx=0 tx=0).stop 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 -- 172.21.15.37:0/4240550874 shutdown_connections 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 --2- 172.21.15.37:0/4240550874 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0dc102ef0 0x7fb0dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 --2- 172.21.15.37:0/4240550874 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0dc102570 0x7fb0dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 --2- 172.21.15.37:0/4240550874 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc100e20 0x7fb0dc101200 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 -- 172.21.15.37:0/4240550874 >> 172.21.15.37:0/4240550874 conn(0x7fb0dc0fc7d0 msgr2=0x7fb0dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 -- 172.21.15.37:0/4240550874 shutdown_connections 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.981+0000 7fb0e446b640 1 -- 172.21.15.37:0/4240550874 wait complete. 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 Processor -- start 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 -- start start 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0dc100e20 0x7fb0dc1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc102570 0x7fb0dc198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0dc102ef0 0x7fb0dc19cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc110f50 con 0x7fb0dc102570 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc110dd0 con 0x7fb0dc102ef0 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e446b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1110d0 con 0x7fb0dc100e20 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e1a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc102570 0x7fb0dc198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e1a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc102570 0x7fb0dc198a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35854/0 (socket says 172.21.15.37:35854) 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2360342051 learned_addr learned my addr 172.21.15.37:0/2360342051 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e2207640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0dc100e20 0x7fb0dc1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.982+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0dc102ef0 0x7fb0dc19cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2360342051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0dc100e20 msgr2=0x7fb0dc1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0dc100e20 0x7fb0dc1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2360342051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0dc102ef0 msgr2=0x7fb0dc19cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:39.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0dc102ef0 0x7fb0dc19cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2360342051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0dc19d470 con 0x7fb0dc102570 2024-09-19T06:42:39.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc102570 0x7fb0dc198a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000d950 tx=0x7fb0d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:39.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0cf7fe640 1 -- 172.21.15.37:0/2360342051 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d0017030 con 0x7fb0dc102570 2024-09-19T06:42:39.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0cf7fe640 1 -- 172.21.15.37:0/2360342051 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d0004020 con 0x7fb0dc102570 2024-09-19T06:42:39.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0dc19d760 con 0x7fb0dc102570 2024-09-19T06:42:39.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0cf7fe640 1 -- 172.21.15.37:0/2360342051 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00047a0 con 0x7fb0dc102570 2024-09-19T06:42:39.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.983+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0dc1a5040 con 0x7fb0dc102570 2024-09-19T06:42:39.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.985+0000 7fb0cf7fe640 1 -- 172.21.15.37:0/2360342051 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fb0d00041d0 con 0x7fb0dc102570 2024-09-19T06:42:39.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.986+0000 7fb0cf7fe640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0c80a3560 0x7fb0c80a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.986+0000 7fb0cf7fe640 1 -- 172.21.15.37:0/2360342051 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb0d00cfa60 con 0x7fb0dc102570 2024-09-19T06:42:39.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.986+0000 7fb0e2207640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0c80a3560 0x7fb0c80a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.986+0000 7fb0e6844740 1 Processor -- start 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e6844740 1 -- start start 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e2207640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0c80a3560 0x7fb0c80a5a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009230 tx=0x7fb0d80058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd5af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd65800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd68130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e6844740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562e6dcc0280 con 0x562e6dd5b480 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e6844740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e6dc67b10 con 0x562e6dd65d40 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e6844740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e6dcbc240 con 0x562e6dd5ab60 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e1a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd65800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e1a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd65800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35868/0 (socket says 172.21.15.37:35868) 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e1a06640 1 -- 172.21.15.37:0/931498807 learned_addr learned my addr 172.21.15.37:0/931498807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:39.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd68130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.987+0000 7fb0e2207640 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd5af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0e1a06640 1 -- 172.21.15.37:0/931498807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 msgr2=0x562e6dd5af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd5af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0e1a06640 1 -- 172.21.15.37:0/931498807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 msgr2=0x562e6dd68130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd68130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0e1a06640 1 -- 172.21.15.37:0/931498807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6dc741f0 con 0x562e6dd5b480 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd68130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd65800 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0097f60 tx=0x7fb0d009c200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb06f1c09dd3936 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0ccdea640 1 -- 172.21.15.37:0/931498807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00a4770 con 0x562e6dd5b480 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0ccdea640 1 -- 172.21.15.37:0/931498807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d0099e00 con 0x562e6dd5b480 2024-09-19T06:42:39.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.988+0000 7fb0ccdea640 1 -- 172.21.15.37:0/931498807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d009b1b0 con 0x562e6dd5b480 2024-09-19T06:42:39.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 -- 172.21.15.37:0/931498807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 msgr2=0x562e6dd65800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:39.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd65800 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb0d0097f60 tx=0x7fb0d009c200 comp rx=0 tx=0).stop 2024-09-19T06:42:39.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 -- 172.21.15.37:0/931498807 shutdown_connections 2024-09-19T06:42:39.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd68130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd65800 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 --2- 172.21.15.37:0/931498807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd5af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 -- 172.21.15.37:0/931498807 >> 172.21.15.37:0/931498807 conn(0x562e6dd570f0 msgr2=0x562e6dd574d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 -- 172.21.15.37:0/931498807 shutdown_connections 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.989+0000 7fb0e6844740 1 -- 172.21.15.37:0/931498807 wait complete. 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 -- start start 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 Processor -- start 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd62cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd5b480 0x562e6dd631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd65d40 0x562e6dd63730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562e6dd68910 con 0x562e6dd5ab60 2024-09-19T06:42:39.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e6dc7af10 con 0x562e6dd5b480 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e6dd68a90 con 0x562e6dd65d40 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd62cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e6844740 10 client.0 _ll_register_callbacks cb 0x562e6dc43a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd62cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35872/0 (socket says 172.21.15.37:35872) 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2207640 1 -- 172.21.15.37:0/3075022111 learned_addr learned my addr 172.21.15.37:0/3075022111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd65d40 0x562e6dd63730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd5b480 0x562e6dd631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2a08640 1 -- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd5b480 msgr2=0x562e6dd631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd5b480 0x562e6dd631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2a08640 1 -- 172.21.15.37:0/3075022111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 msgr2=0x562e6dd62cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd62cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2a08640 1 -- 172.21.15.37:0/3075022111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6da316a0 con 0x562e6dd65d40 2024-09-19T06:42:39.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.990+0000 7fb0e2207640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd62cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd65d40 0x562e6dd63730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40126d0 tx=0x7fb0d4012b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21f27997e4505269 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/3075022111 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4003d80 con 0x562e6dd65d40 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d0099e00 con 0x562e6dd65d40 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/3075022111 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb0d4004b20 con 0x562e6dd65d40 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/3075022111 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0d4004d20 con 0x562e6dd65d40 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0e6844740 10 client.25162 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e6dd74650 con 0x562e6dd65d40 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.991+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/3075022111 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d0099e00 con 0x562e6dd65d40 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/3075022111 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0d4019030 con 0x562e6dd65d40 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e6844740 10 client.25162 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e6844740 10 client.25162 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:39.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e6844740 20 client.25162 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dd75a60 0x562e6dd77e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e6844740 4 client.25162 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e6dccd7f0 con 0x562e6dd75a60 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e2207640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dd75a60 0x562e6dd77e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0c5ffb640 10 client.25162 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.992+0000 7fb0e2207640 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dd75a60 0x562e6dd77e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73fb50d121020ea0 server_cookie=1a7dc9b00691b9b3 in_seq=0 out_seq=0 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/3075022111 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e6dccd7f0 con 0x562e6dd75a60 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0c5ffb640 10 client.25162 handle_command_reply: tid=0 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 client.25162 shutdown 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd65d40 msgr2=0x562e6dd63730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd65d40 0x562e6dd63730 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb0d40126d0 tx=0x7fb0d4012b90 comp rx=0 tx=0).stop 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 shutdown_connections 2024-09-19T06:42:39.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dd75a60 0x562e6dd77e50 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd65d40 0x562e6dd63730 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd5b480 0x562e6dd631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3075022111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd62cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0c5ffb640 0 client.25162 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 >> 172.21.15.37:0/3075022111 conn(0x562e6dd570f0 msgr2=0x562e6dd56270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 shutdown_connections 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.993+0000 7fb0e6844740 1 -- 172.21.15.37:0/3075022111 wait complete. 2024-09-19T06:42:40.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:39.994+0000 7fb0e6844740 20 client.25162 trim_cache size 0 max 16384 2024-09-19T06:42:40.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 Processor -- start 2024-09-19T06:42:40.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 -- start start 2024-09-19T06:42:40.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd5d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:40.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5b480 0x562e6dd7d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:40.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd7dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:40.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1b0230 con 0x562e6dd5ab60 2024-09-19T06:42:40.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1adb00 con 0x562e6dd65d40 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e6844740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1ab950 con 0x562e6dd5b480 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e2207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd5d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e2207640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd5d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35880/0 (socket says 172.21.15.37:35880) 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.028+0000 7fb0e2207640 1 -- 172.21.15.37:0/3494702555 learned_addr learned my addr 172.21.15.37:0/3494702555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5b480 0x562e6dd7d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2207640 1 -- 172.21.15.37:0/3494702555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5b480 msgr2=0x562e6dd7d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2207640 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5b480 0x562e6dd7d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd7dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2207640 1 -- 172.21.15.37:0/3494702555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 msgr2=0x562e6dd7dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2207640 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd7dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2207640 1 -- 172.21.15.37:0/3494702555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e6dd58130 con 0x562e6dd5ab60 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd7dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0e2207640 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd5d5c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005940 tx=0x7fb0d803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521f84515dd34ee5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:40.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0c7fff640 1 -- 172.21.15.37:0/3494702555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d8035050 con 0x562e6dd5ab60 2024-09-19T06:42:40.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0c7fff640 1 -- 172.21.15.37:0/3494702555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d8004750 con 0x562e6dd5ab60 2024-09-19T06:42:40.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.029+0000 7fb0c7fff640 1 -- 172.21.15.37:0/3494702555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80318c0 con 0x562e6dd5ab60 2024-09-19T06:42:40.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 -- 172.21.15.37:0/3494702555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 msgr2=0x562e6dd5d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:40.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd5d5c0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8005940 tx=0x7fb0d803e000 comp rx=0 tx=0).stop 2024-09-19T06:42:40.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 -- 172.21.15.37:0/3494702555 shutdown_connections 2024-09-19T06:42:40.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dd7dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5b480 0x562e6dd7d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 --2- 172.21.15.37:0/3494702555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5ab60 0x562e6dd5d5c0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 -- 172.21.15.37:0/3494702555 >> 172.21.15.37:0/3494702555 conn(0x562e6dd570f0 msgr2=0x562e6dd64c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 -- 172.21.15.37:0/3494702555 shutdown_connections 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 -- 172.21.15.37:0/3494702555 wait complete. 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.030+0000 7fb0e6844740 1 -- start start 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 1 Processor -- start 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd69dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd6a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dddf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1abfc0 con 0x562e6dd5b480 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1ad430 con 0x562e6dd65d40 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0dc1ac690 con 0x562e6dd5ab60 2024-09-19T06:42:40.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd6a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e6844740 10 client.0 _ll_register_callbacks cb 0x7fb0d40099a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd6a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35886/0 (socket says 172.21.15.37:35886) 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2749473929 learned_addr learned my addr 172.21.15.37:0/2749473929 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e2a08640 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dddf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e2207640 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd69dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2749473929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 msgr2=0x562e6dd69dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd69dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 msgr2=0x562e6dddf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dddf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.031+0000 7fb0e1a06640 1 -- 172.21.15.37:0/2749473929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e6dd6abf0 con 0x562e6dd5b480 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e1a06640 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd6a310 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00cd060 tx=0x7fb0d000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908cb7be4ab5f862 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:40.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/2749473929 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0d00b8030 con 0x562e6dd5b480 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/2749473929 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb0d0084e00 con 0x562e6dd5b480 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0d8004750 con 0x562e6dd5b480 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/2749473929 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb0d00a4770 con 0x562e6dd5b480 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 10 client.15636 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e6dde0c70 con 0x562e6dd5b480 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/2749473929 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb0d00ba2c0 con 0x562e6dd5b480 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/2749473929 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fb0d009af50 con 0x562e6dd5b480 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 10 client.15636 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 10 client.15636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 20 client.15636 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dde0fa0 0x562e6dde33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 4 client.15636 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:42:40.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.032+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7fb0d8004750 con 0x562e6dde0fa0 2024-09-19T06:42:40.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.033+0000 7fb0e2207640 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dde0fa0 0x562e6dde33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:40.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.033+0000 7fb0c5ffb640 10 client.15636 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:40.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.033+0000 7fb0e2207640 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dde0fa0 0x562e6dde33c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f5d3ff7e8b2202a server_cookie=6952677a1cc70b8b in_seq=0 out_seq=0 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0c5ffb640 1 -- 172.21.15.37:0/2749473929 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7fb0d8004750 con 0x562e6dde0fa0 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0c5ffb640 10 client.15636 handle_command_reply: tid=0 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 client.15636 shutdown 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 msgr2=0x562e6dd6a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd6a310 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb0d00cd060 tx=0x7fb0d000b530 comp rx=0 tx=0).stop 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 shutdown_connections 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562e6dde0fa0 0x562e6dde33c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e6dd65d40 0x562e6dddf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e6dd5b480 0x562e6dd6a310 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 --2- 172.21.15.37:0/2749473929 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e6dd5ab60 0x562e6dd69dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 >> 172.21.15.37:0/2749473929 conn(0x562e6dd570f0 msgr2=0x562e6dd55a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0c5ffb640 0 client.15636 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:40.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 shutdown_connections 2024-09-19T06:42:40.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 1 -- 172.21.15.37:0/2749473929 wait complete. 2024-09-19T06:42:40.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.035+0000 7fb0e6844740 20 client.15636 trim_cache size 0 max 16384 2024-09-19T06:42:40.040 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:42:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "4ee4335d-9b18-4257-8df4-3971290ccc02", 2024-09-19T06:42:40.040 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:42:40.040 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0c80a3560 msgr2=0x7fb0c80a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0c80a3560 0x7fb0c80a5a20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009230 tx=0x7fb0d80058d0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc102570 msgr2=0x7fb0dc198a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc102570 0x7fb0dc198a00 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fb0d000d950 tx=0x7fb0d000de10 comp rx=0 tx=0).stop 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 shutdown_connections 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0dc102ef0 0x7fb0dc19cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0c80a3560 0x7fb0c80a5a20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0dc102570 0x7fb0dc198a00 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 --2- 172.21.15.37:0/2360342051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0dc100e20 0x7fb0dc1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:40.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 >> 172.21.15.37:0/2360342051 conn(0x7fb0dc0fc7d0 msgr2=0x7fb0dc06a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:40.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 shutdown_connections 2024-09-19T06:42:40.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:40.036+0000 7fb0e446b640 1 -- 172.21.15.37:0/2360342051 wait complete. 2024-09-19T06:42:40.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:40 smithi096 ceph-mon[35004]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T06:42:40.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:40 smithi037 ceph-mon[31211]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T06:42:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:40 smithi151 ceph-mon[35236]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-09-19T06:42:40.849 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:42:41.200 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:41 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:42:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:41 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:41 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:42:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:41 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:41.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:41 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:42:41.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:41 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:41.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:41 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:42:41.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:41 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:41.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:41 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:42:41.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:41 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:41.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:41 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:42:41.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:41 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 -- 172.21.15.37:0/3925128372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9040102570 msgr2=0x7f90401029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 --2- 172.21.15.37:0/3925128372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9040102570 0x7f90401029b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f9034009900 tx=0x7f903402a480 comp rx=0 tx=0).stop 2024-09-19T06:42:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 -- 172.21.15.37:0/3925128372 shutdown_connections 2024-09-19T06:42:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 --2- 172.21.15.37:0/3925128372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9040102ef0 0x7f904010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 --2- 172.21.15.37:0/3925128372 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9040102570 0x7f90401029b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 --2- 172.21.15.37:0/3925128372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9040100e20 0x7f9040101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 -- 172.21.15.37:0/3925128372 >> 172.21.15.37:0/3925128372 conn(0x7f90400fc7d0 msgr2=0x7f90400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 -- 172.21.15.37:0/3925128372 shutdown_connections 2024-09-19T06:42:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.720+0000 7f90461f7640 1 -- 172.21.15.37:0/3925128372 wait complete. 2024-09-19T06:42:41.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 Processor -- start 2024-09-19T06:42:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 -- start start 2024-09-19T06:42:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9040100e20 0x7f904010bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9040102ef0 0x7f904010c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f904010c580 0x7f90401a6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9040110f80 con 0x7f904010c580 2024-09-19T06:42:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9040110e00 con 0x7f9040102ef0 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f90461f7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9040111100 con 0x7f9040100e20 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f903effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9040102ef0 0x7f904010c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f903effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9040102ef0 0x7f904010c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34886/0 (socket says 172.21.15.37:34886) 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.721+0000 7f903effd640 1 -- 172.21.15.37:0/4244345464 learned_addr learned my addr 172.21.15.37:0/4244345464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903f7fe640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9040100e20 0x7f904010bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903ffff640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f904010c580 0x7f90401a6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903ffff640 1 -- 172.21.15.37:0/4244345464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9040100e20 msgr2=0x7f904010bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903ffff640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9040100e20 0x7f904010bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903ffff640 1 -- 172.21.15.37:0/4244345464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9040102ef0 msgr2=0x7f904010c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903ffff640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9040102ef0 0x7f904010c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903ffff640 1 -- 172.21.15.37:0/4244345464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90401a6ee0 con 0x7f904010c580 2024-09-19T06:42:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903f7fe640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9040100e20 0x7f904010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903ffff640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f904010c580 0x7f90401a6810 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f903800cab0 tx=0x7f903800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.722+0000 7f903cff9640 1 -- 172.21.15.37:0/4244345464 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903800ae20 con 0x7f904010c580 2024-09-19T06:42:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.723+0000 7f903cff9640 1 -- 172.21.15.37:0/4244345464 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f903800f930 con 0x7f904010c580 2024-09-19T06:42:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.723+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90401a71d0 con 0x7f904010c580 2024-09-19T06:42:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.723+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90401a7690 con 0x7f904010c580 2024-09-19T06:42:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.723+0000 7f903cff9640 1 -- 172.21.15.37:0/4244345464 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9038018b10 con 0x7f904010c580 2024-09-19T06:42:41.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.726+0000 7f903cff9640 1 -- 172.21.15.37:0/4244345464 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f903800f450 con 0x7f904010c580 2024-09-19T06:42:41.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.728+0000 7f90485d0740 1 Processor -- start 2024-09-19T06:42:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.728+0000 7f903cff9640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90280a35e0 0x7f90280a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.728+0000 7f90485d0740 1 -- start start 2024-09-19T06:42:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.728+0000 7f903effd640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90280a35e0 0x7f90280a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.728+0000 7f90485d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138647cb0 0x563138648090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.728+0000 7f90485d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x563138652950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.728+0000 7f90485d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x563138655280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f90485d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5631385a52e0 con 0x5631386485d0 2024-09-19T06:42:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f903effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x563138652950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f903effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x563138652950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35904/0 (socket says 172.21.15.37:35904) 2024-09-19T06:42:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f903effd640 1 -- 172.21.15.37:0/1368538651 learned_addr learned my addr 172.21.15.37:0/1368538651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f903f7fe640 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138647cb0 0x563138648090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f903ffff640 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x563138655280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f90485d0740 1 -- 172.21.15.37:0/1368538651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56313854cce0 con 0x563138647cb0 2024-09-19T06:42:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f90485d0740 1 -- 172.21.15.37:0/1368538651 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5631385a12a0 con 0x563138652e90 2024-09-19T06:42:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f903cff9640 1 -- 172.21.15.37:0/4244345464 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f90380d0400 con 0x7f904010c580 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.729+0000 7f903effd640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90280a35e0 0x7f90280a5aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9034000c00 tx=0x7f9034004120 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f903effd640 1 -- 172.21.15.37:0/1368538651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 msgr2=0x563138655280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f903effd640 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x563138655280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f903effd640 1 -- 172.21.15.37:0/1368538651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138647cb0 msgr2=0x563138648090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f903effd640 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138647cb0 0x563138648090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f903effd640 1 -- 172.21.15.37:0/1368538651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563138531440 con 0x5631386485d0 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f903effd640 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x563138652950 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f90340096c0 tx=0x7f9034046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=926559e12e4c7fbf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f9025ffb640 1 -- 172.21.15.37:0/1368538651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034035050 con 0x5631386485d0 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f9025ffb640 1 -- 172.21.15.37:0/1368538651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9034003d40 con 0x5631386485d0 2024-09-19T06:42:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.730+0000 7f9025ffb640 1 -- 172.21.15.37:0/1368538651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903403c9a0 con 0x5631386485d0 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 -- 172.21.15.37:0/1368538651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 msgr2=0x563138652950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x563138652950 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f90340096c0 tx=0x7f9034046000 comp rx=0 tx=0).stop 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 -- 172.21.15.37:0/1368538651 shutdown_connections 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x563138655280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x563138652950 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 --2- 172.21.15.37:0/1368538651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138647cb0 0x563138648090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 -- 172.21.15.37:0/1368538651 >> 172.21.15.37:0/1368538651 conn(0x563138644240 msgr2=0x563138644620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 -- 172.21.15.37:0/1368538651 shutdown_connections 2024-09-19T06:42:41.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.731+0000 7f90485d0740 1 -- 172.21.15.37:0/1368538651 wait complete. 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.732+0000 7f90485d0740 1 -- start start 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.732+0000 7f90485d0740 1 Processor -- start 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.732+0000 7f90485d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138647cb0 0x5631386526b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f90485d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138647cb0 0x5631386526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f90485d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313864fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f90485d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563138655ab0 con 0x563138647cb0 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f903effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f903effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34906/0 (socket says 172.21.15.37:34906) 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f903effd640 1 -- 172.21.15.37:0/3799237687 learned_addr learned my addr 172.21.15.37:0/3799237687 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f90485d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563138560050 con 0x5631386485d0 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563138655c30 con 0x563138652e90 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138647cb0 0x5631386526b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35910/0 (socket says 172.21.15.37:35910) 2024-09-19T06:42:41.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f90485d0740 10 client.0 _ll_register_callbacks cb 0x5631385a0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:41.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.733+0000 7f903ffff640 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313864fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f903f7fe640 1 -- 172.21.15.37:0/3799237687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 msgr2=0x56313864fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f903f7fe640 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313864fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f903f7fe640 1 -- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 msgr2=0x56313864f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f903f7fe640 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f903f7fe640 1 -- 172.21.15.37:0/3799237687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631383163c0 con 0x563138647cb0 2024-09-19T06:42:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f903effd640 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:41.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f903f7fe640 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138647cb0 0x5631386526b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9030011740 tx=0x7f9030011c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5bb10034178b15 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.734+0000 7f901effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.735+0000 7f901effd640 1 -- 172.21.15.37:0/3799237687 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030010ba0 con 0x563138647cb0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.735+0000 7f901effd640 1 -- 172.21.15.37:0/3799237687 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f9030010d40 con 0x563138647cb0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.735+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9034003d40 con 0x563138647cb0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.735+0000 7f901effd640 1 -- 172.21.15.37:0/3799237687 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9030004020 con 0x563138647cb0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.735+0000 7f90485d0740 10 client.15648 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.735+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5631386613d0 con 0x563138647cb0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f901effd640 1 -- 172.21.15.37:0/3799237687 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9034003d40 con 0x563138647cb0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f901effd640 1 -- 172.21.15.37:0/3799237687 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9030004910 con 0x563138647cb0 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f90485d0740 10 client.15648 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f90485d0740 10 client.15648 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f90485d0740 20 client.15648 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f90485d0740 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563138662950 0x563138664de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f90485d0740 4 client.15648 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:41.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.736+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631385b2850 con 0x563138662950 2024-09-19T06:42:41.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.737+0000 7f903effd640 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563138662950 0x563138664de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.737+0000 7f903effd640 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563138662950 0x563138664de0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a080c658cabb50e8 server_cookie=ddf5ce28f1900704 in_seq=0 out_seq=0 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.737+0000 7f901effd640 10 client.15648 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f901effd640 1 -- 172.21.15.37:0/3799237687 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5631385b2850 con 0x563138662950 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f901effd640 10 client.15648 handle_command_reply: tid=0 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 client.15648 shutdown 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138647cb0 msgr2=0x5631386526b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138647cb0 0x5631386526b0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9030011740 tx=0x7f9030011c00 comp rx=0 tx=0).stop 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 shutdown_connections 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563138662950 0x563138664de0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313864fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 --2- 172.21.15.37:0/3799237687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138647cb0 0x5631386526b0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 >> 172.21.15.37:0/3799237687 conn(0x563138644240 msgr2=0x56313863ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.738+0000 7f901effd640 0 client.15648 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.739+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 shutdown_connections 2024-09-19T06:42:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.739+0000 7f90485d0740 1 -- 172.21.15.37:0/3799237687 wait complete. 2024-09-19T06:42:41.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.739+0000 7f90485d0740 20 client.15648 trim_cache size 0 max 16384 2024-09-19T06:42:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 Processor -- start 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 -- start start 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138644240 0x56313864b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313866a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f902c0110c0 con 0x563138644240 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f902c00e990 con 0x5631386485d0 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.756+0000 7f90485d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f902c00c7e0 con 0x563138652e90 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138644240 0x56313864b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313866a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313866a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:38192/0 (socket says 172.21.15.37:38192) 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34912/0 (socket says 172.21.15.37:34912) 2024-09-19T06:42:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138644240 0x56313864b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35912/0 (socket says 172.21.15.37:35912) 2024-09-19T06:42:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903ffff640 1 -- 172.21.15.37:0/2774356777 learned_addr learned my addr 172.21.15.37:0/2774356777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 -- 172.21.15.37:0/2774356777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 msgr2=0x56313866a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 --2- 172.21.15.37:0/2774356777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313866a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 -- 172.21.15.37:0/2774356777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138644240 msgr2=0x56313864b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 --2- 172.21.15.37:0/2774356777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138644240 0x56313864b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 -- 172.21.15.37:0/2774356777 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563138644070 con 0x5631386485d0 2024-09-19T06:42:41.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f903effd640 1 --2- 172.21.15.37:0/2774356777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864e940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x563138665690 tx=0x7f903402a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7724871021e939e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:41.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f90257fa640 1 -- 172.21.15.37:0/2774356777 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034035040 con 0x5631386485d0 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f90257fa640 1 -- 172.21.15.37:0/2774356777 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9034003d40 con 0x5631386485d0 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.757+0000 7f90257fa640 1 -- 172.21.15.37:0/2774356777 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9034004e10 con 0x5631386485d0 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.758+0000 7f90485d0740 1 -- 172.21.15.37:0/2774356777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 msgr2=0x56313864e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.758+0000 7f90485d0740 1 --2- 172.21.15.37:0/2774356777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864e940 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x563138665690 tx=0x7f903402a480 comp rx=0 tx=0).stop 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.758+0000 7f90485d0740 1 -- 172.21.15.37:0/2774356777 shutdown_connections 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.758+0000 7f90485d0740 1 --2- 172.21.15.37:0/2774356777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138652e90 0x56313866a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.758+0000 7f90485d0740 1 --2- 172.21.15.37:0/2774356777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5631386485d0 0x56313864e940 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.758+0000 7f90485d0740 1 --2- 172.21.15.37:0/2774356777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563138644240 0x56313864b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.758+0000 7f90485d0740 1 -- 172.21.15.37:0/2774356777 >> 172.21.15.37:0/2774356777 conn(0x563138650000 msgr2=0x56313864e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 -- 172.21.15.37:0/2774356777 shutdown_connections 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 -- 172.21.15.37:0/2774356777 wait complete. 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 -- start start 2024-09-19T06:42:41.761 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 Processor -- start 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138644240 0x56313865b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x56313865b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138652e90 0x56313864c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f902c00ce50 con 0x5631386485d0 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f902c00e2c0 con 0x563138652e90 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.759+0000 7f90485d0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f902c00d520 con 0x563138644240 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f90485d0740 10 client.0 _ll_register_callbacks cb 0x7f903000bbd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x56313865b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x56313865b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35916/0 (socket says 172.21.15.37:35916) 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903effd640 1 -- 172.21.15.37:0/171415784 learned_addr learned my addr 172.21.15.37:0/171415784 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903f7fe640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138644240 0x56313865b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903ffff640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138652e90 0x56313864c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903f7fe640 1 -- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138652e90 msgr2=0x56313864c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903f7fe640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138652e90 0x56313864c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903f7fe640 1 -- 172.21.15.37:0/171415784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 msgr2=0x56313865b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903f7fe640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x56313865b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903f7fe640 1 -- 172.21.15.37:0/171415784 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563138649230 con 0x563138644240 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903ffff640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138652e90 0x56313864c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.760+0000 7f903effd640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x56313865b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:41.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f903f7fe640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138644240 0x56313865b180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9030004d80 tx=0x7f9030005c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e623564e9b77532 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f901effd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f901effd640 1 -- 172.21.15.37:0/171415784 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030028040 con 0x563138644240 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f901effd640 1 -- 172.21.15.37:0/171415784 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f9030010db0 con 0x563138644240 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9034003d40 con 0x563138644240 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f901effd640 1 -- 172.21.15.37:0/171415784 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9030004660 con 0x563138644240 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f90485d0740 10 client.25183 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.761+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56313865a8b0 con 0x563138644240 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f901effd640 1 -- 172.21.15.37:0/171415784 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9030011e50 con 0x563138644240 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f901effd640 1 -- 172.21.15.37:0/171415784 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f90300035f0 con 0x563138644240 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f90485d0740 10 client.25183 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f90485d0740 10 client.25183 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f90485d0740 20 client.25183 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:41.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f90485d0740 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5631386cd800 0x5631386cfc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f90485d0740 4 client.25183 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f9034003d40 con 0x5631386cd800 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.762+0000 7f903effd640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5631386cd800 0x5631386cfc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.763+0000 7f903effd640 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5631386cd800 0x5631386cfc20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0ec1e8437e1c4f2 server_cookie=1244613445824b11 in_seq=0 out_seq=0 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.763+0000 7f901effd640 10 client.25183 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.763+0000 7f901effd640 1 -- 172.21.15.37:0/171415784 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f9034003d40 con 0x5631386cd800 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.763+0000 7f901effd640 10 client.25183 handle_command_reply: tid=0 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.763+0000 7f90485d0740 1 client.25183 shutdown 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.763+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138644240 msgr2=0x56313865b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.763+0000 7f90485d0740 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138644240 0x56313865b180 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9030004d80 tx=0x7f9030005c40 comp rx=0 tx=0).stop 2024-09-19T06:42:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 shutdown_connections 2024-09-19T06:42:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5631386cd800 0x5631386cfc20 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563138652e90 0x56313864c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5631386485d0 0x56313865b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 --2- 172.21.15.37:0/171415784 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563138644240 0x56313865b180 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 >> 172.21.15.37:0/171415784 conn(0x563138650000 msgr2=0x56313863a350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f901effd640 0 client.25183 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 shutdown_connections 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 1 -- 172.21.15.37:0/171415784 wait complete. 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.764+0000 7f90485d0740 20 client.25183 trim_cache size 0 max 16384 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:42:41.768 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90280a35e0 msgr2=0x7f90280a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90280a35e0 0x7f90280a5aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9034000c00 tx=0x7f9034004120 comp rx=0 tx=0).stop 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f904010c580 msgr2=0x7f90401a6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f904010c580 0x7f90401a6810 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f903800cab0 tx=0x7f903800cf70 comp rx=0 tx=0).stop 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 shutdown_connections 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90280a35e0 0x7f90280a5aa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f904010c580 0x7f90401a6810 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9040102ef0 0x7f904010c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 --2- 172.21.15.37:0/4244345464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9040100e20 0x7f904010bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:41.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 >> 172.21.15.37:0/4244345464 conn(0x7f90400fc7d0 msgr2=0x7f9040103bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:41.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 shutdown_connections 2024-09-19T06:42:41.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:41.765+0000 7f90461f7640 1 -- 172.21.15.37:0/4244345464 wait complete. 2024-09-19T06:42:41.873 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:42:42.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:42 smithi096 ceph-mon[35004]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T06:42:42.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:42 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:42:42.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:42 smithi037 ceph-mon[31211]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T06:42:42.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:42 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:42:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:42 smithi151 ceph-mon[35236]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 5 op/s 2024-09-19T06:42:42.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:42 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:42:42.579 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:42:42.580 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:42:42.725 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:43.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.215+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2557464765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4106a80 msgr2=0x7fc2e4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.215+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2557464765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4106a80 0x7fc2e4106e60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009930 tx=0x7fc2d802a420 comp rx=0 tx=0).stop 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.215+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2557464765 shutdown_connections 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.215+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2557464765 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e4101420 0x7fc2e410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.215+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2557464765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e4100a80 0x7fc2e4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.215+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2557464765 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4106a80 0x7fc2e4106e60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.215+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2557464765 >> 172.21.15.37:0/2557464765 conn(0x7fc2e40fc7a0 msgr2=0x7fc2e40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.216+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2557464765 shutdown_connections 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.216+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2557464765 wait complete. 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.216+0000 7fc2eb24b640 1 Processor -- start 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.216+0000 7fc2eb24b640 1 -- start start 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2eb24b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e4100a80 0x7fc2e40fce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2eb24b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4101420 0x7fc2e40fd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:43.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2eb24b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e4106a80 0x7fc2e40fd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2eb24b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e4110e40 con 0x7fc2e4101420 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2eb24b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e4110cc0 con 0x7fc2e4106a80 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2eb24b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2e4110fc0 con 0x7fc2e4100a80 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4101420 0x7fc2e40fd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4101420 0x7fc2e40fd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35936/0 (socket says 172.21.15.37:35936) 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 -- 172.21.15.37:0/2682334407 learned_addr learned my addr 172.21.15.37:0/2682334407 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e8fe7640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e4100a80 0x7fc2e40fce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e97e8640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e4106a80 0x7fc2e40fd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 -- 172.21.15.37:0/2682334407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e4100a80 msgr2=0x7fc2e40fce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e4100a80 0x7fc2e40fce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 -- 172.21.15.37:0/2682334407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e4106a80 msgr2=0x7fc2e40fd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e4106a80 0x7fc2e40fd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.217+0000 7fc2e3fff640 1 -- 172.21.15.37:0/2682334407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2e41a6910 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.218+0000 7fc2e3fff640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4101420 0x7fc2e40fd3b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000d950 tx=0x7fc2d000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.218+0000 7fc2e1ffb640 1 -- 172.21.15.37:0/2682334407 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d0017070 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.218+0000 7fc2e1ffb640 1 -- 172.21.15.37:0/2682334407 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2d0009c30 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.218+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2e41a6c00 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.218+0000 7fc2e1ffb640 1 -- 172.21.15.37:0/2682334407 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2d000f6b0 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.218+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2e41a7140 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.219+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2e4100ee0 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.221+0000 7fc2e1ffb640 1 -- 172.21.15.37:0/2682334407 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fc2d00109d0 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.221+0000 7fc2e1ffb640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc2d40a3560 0x7fc2d40a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.221+0000 7fc2e1ffb640 1 -- 172.21.15.37:0/2682334407 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fc2d00cfc40 con 0x7fc2e4101420 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.222+0000 7fc2e8fe7640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc2d40a3560 0x7fc2d40a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.222+0000 7fc2e8fe7640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc2d40a3560 0x7fc2d40a5a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009900 tx=0x7fc2d80058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:43.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.226+0000 7fc2e1ffb640 1 -- 172.21.15.37:0/2682334407 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc2d0009de0 con 0x7fc2e4101420 2024-09-19T06:42:43.278 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.517+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc2e4109a70 con 0x7fc2e4101420 2024-09-19T06:42:43.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.518+0000 7fc2e1ffb640 1 -- 172.21.15.37:0/2682334407 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7fc2d009c3e0 con 0x7fc2e4101420 2024-09-19T06:42:43.519 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:43.519 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:42:43.519 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc2d40a3560 msgr2=0x7fc2d40a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc2d40a3560 0x7fc2d40a5a20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8009900 tx=0x7fc2d80058d0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4101420 msgr2=0x7fc2e40fd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4101420 0x7fc2e40fd3b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fc2d000d950 tx=0x7fc2d000de10 comp rx=0 tx=0).stop 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 shutdown_connections 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2e4106a80 0x7fc2e40fd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc2d40a3560 0x7fc2d40a5a20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2e4101420 0x7fc2e40fd3b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 --2- 172.21.15.37:0/2682334407 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2e4100a80 0x7fc2e40fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 >> 172.21.15.37:0/2682334407 conn(0x7fc2e40fc7a0 msgr2=0x7fc2e406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 shutdown_connections 2024-09-19T06:42:43.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:43.520+0000 7fc2eb24b640 1 -- 172.21.15.37:0/2682334407 wait complete. 2024-09-19T06:42:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/4152063766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 msgr2=0x7fa3c41108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/4152063766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c41108a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc009ca0 tx=0x7fa3bc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/4152063766 shutdown_connections 2024-09-19T06:42:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/4152063766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c41108a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/4152063766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3c4074f50 0x7fa3c410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/4152063766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c40745a0 0x7fa3c4074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/4152063766 >> 172.21.15.37:0/4152063766 conn(0x7fa3c40fd8d0 msgr2=0x7fa3c40ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/4152063766 shutdown_connections 2024-09-19T06:42:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/4152063766 wait complete. 2024-09-19T06:42:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.003+0000 7fa3cb1fc640 1 Processor -- start 2024-09-19T06:42:44.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3cb1fc640 1 -- start start 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3cb1fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c40745a0 0x7fa3c419a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3cb1fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3c4074f50 0x7fa3c419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3cb1fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c419ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3cb1fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41141e0 con 0x7fa3c410e470 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3cb1fc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4114060 con 0x7fa3c40745a0 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3cb1fc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4114360 con 0x7fa3c4074f50 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c419ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c419ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35966/0 (socket says 172.21.15.37:35966) 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 -- 172.21.15.37:0/748770041 learned_addr learned my addr 172.21.15.37:0/748770041 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c3fff640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3c4074f50 0x7fa3c419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:44.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 -- 172.21.15.37:0/748770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3c4074f50 msgr2=0x7fa3c419a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3c4074f50 0x7fa3c419a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 -- 172.21.15.37:0/748770041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c40745a0 msgr2=0x7fa3c419a400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c40745a0 0x7fa3c419a400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c9799640 1 -- 172.21.15.37:0/748770041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419f3b0 con 0x7fa3c410e470 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.004+0000 7fa3c3fff640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3c4074f50 0x7fa3c419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.005+0000 7fa3c9799640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c419ecd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc031f60 tx=0x7fa3bc0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.005+0000 7fa3c1ffb640 1 -- 172.21.15.37:0/748770041 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc02fb60 con 0x7fa3c410e470 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.005+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419f640 con 0x7fa3c410e470 2024-09-19T06:42:44.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.005+0000 7fa3c1ffb640 1 -- 172.21.15.37:0/748770041 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3bc02fd00 con 0x7fa3c410e470 2024-09-19T06:42:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.005+0000 7fa3c1ffb640 1 -- 172.21.15.37:0/748770041 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0468c0 con 0x7fa3c410e470 2024-09-19T06:42:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.005+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a6f80 con 0x7fa3c410e470 2024-09-19T06:42:44.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.007+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c419b760 con 0x7fa3c410e470 2024-09-19T06:42:44.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.008+0000 7fa3c1ffb640 1 -- 172.21.15.37:0/748770041 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fa3bc046a60 con 0x7fa3c410e470 2024-09-19T06:42:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.009+0000 7fa3c1ffb640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3b40a35e0 0x7fa3b40a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.009+0000 7fa3c8f98640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3b40a35e0 0x7fa3b40a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:44.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.009+0000 7fa3c1ffb640 1 -- 172.21.15.37:0/748770041 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fa3bc0f18b0 con 0x7fa3c410e470 2024-09-19T06:42:44.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.010+0000 7fa3c8f98640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3b40a35e0 0x7fa3b40a5aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005fd0 tx=0x7fa3b8005c70 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:44.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.017+0000 7fa3c1ffb640 1 -- 172.21.15.37:0/748770041 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa3bc0be160 con 0x7fa3c410e470 2024-09-19T06:42:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:44 smithi096 ceph-mon[35004]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:42:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:44 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2682334407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:44.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:44 smithi037 ceph-mon[31211]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:42:44.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:44 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2682334407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:44.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.275+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3c41a7380 con 0x7fa3c410e470 2024-09-19T06:42:44.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.276+0000 7fa3c1ffb640 1 -- 172.21.15.37:0/748770041 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7fa3bc0bda60 con 0x7fa3c410e470 2024-09-19T06:42:44.277 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:42:44.277 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:42:44.277 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:42:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3b40a35e0 msgr2=0x7fa3b40a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3b40a35e0 0x7fa3b40a5aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005fd0 tx=0x7fa3b8005c70 comp rx=0 tx=0).stop 2024-09-19T06:42:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 msgr2=0x7fa3c419ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:44.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c419ecd0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc031f60 tx=0x7fa3bc0095b0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 shutdown_connections 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3b40a35e0 0x7fa3b40a5aa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3c410e470 0x7fa3c419ecd0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3c4074f50 0x7fa3c419a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 --2- 172.21.15.37:0/748770041 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3c40745a0 0x7fa3c419a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 >> 172.21.15.37:0/748770041 conn(0x7fa3c40fd8d0 msgr2=0x7fa3c410fdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 shutdown_connections 2024-09-19T06:42:44.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:44.279+0000 7fa3cb1fc640 1 -- 172.21.15.37:0/748770041 wait complete. 2024-09-19T06:42:44.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:44 smithi151 ceph-mon[35236]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:42:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:44 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2682334407' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:44.619 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id e13bea5b --release --await' 2024-09-19T06:42:44.620 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id e13bea5b --release --await 2024-09-19T06:42:45.247 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:45 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/748770041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:45.283 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:42:45.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:45 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/748770041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:45.407 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:45.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:45 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/748770041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:42:45.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3858192726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 msgr2=0x7f4f081029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:45.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3858192726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 0x7f4f081029b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009890 tx=0x7f4ef002a3f0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3858192726 shutdown_connections 2024-09-19T06:42:45.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3858192726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08102ef0 0x7f4f0810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3858192726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 0x7f4f081029b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3858192726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f08100e20 0x7f4f08101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3858192726 >> 172.21.15.37:0/3858192726 conn(0x7f4f080fc7d0 msgr2=0x7f4f080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:45.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3858192726 shutdown_connections 2024-09-19T06:42:45.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.960+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3858192726 wait complete. 2024-09-19T06:42:45.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.961+0000 7f4f0eea3640 1 Processor -- start 2024-09-19T06:42:45.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.961+0000 7f4f0eea3640 1 -- start start 2024-09-19T06:42:45.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.961+0000 7f4f0eea3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08100e20 0x7f4f08197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.961+0000 7f4f0eea3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 0x7f4f081979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.961+0000 7f4f0eea3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f08102ef0 0x7f4f0819bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0eea3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08110f80 con 0x7f4f08100e20 2024-09-19T06:42:45.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0eea3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08110e00 con 0x7f4f08102570 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0eea3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08111100 con 0x7f4f08102ef0 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08100e20 0x7f4f08197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08100e20 0x7f4f08197460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35990/0 (socket says 172.21.15.37:35990) 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 -- 172.21.15.37:0/3190481503 learned_addr learned my addr 172.21.15.37:0/3190481503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f07fff640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 0x7f4f081979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0d440640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f08102ef0 0x7f4f0819bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 -- 172.21.15.37:0/3190481503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f08102ef0 msgr2=0x7f4f0819bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f08102ef0 0x7f4f0819bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 -- 172.21.15.37:0/3190481503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 msgr2=0x7f4f081979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 0x7f4f081979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 -- 172.21.15.37:0/3190481503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0819c410 con 0x7f4f08100e20 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f07fff640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 0x7f4f081979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.962+0000 7f4f0cc3f640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08100e20 0x7f4f08197460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8010cc0 tx=0x7f4ef800c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.963+0000 7f4f05ffb640 1 -- 172.21.15.37:0/3190481503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef801d070 con 0x7f4f08100e20 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.963+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0819c700 con 0x7f4f08100e20 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.963+0000 7f4f05ffb640 1 -- 172.21.15.37:0/3190481503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef80188f0 con 0x7f4f08100e20 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.963+0000 7f4f05ffb640 1 -- 172.21.15.37:0/3190481503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef80178f0 con 0x7f4f08100e20 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.963+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f08069f70 con 0x7f4f08100e20 2024-09-19T06:42:45.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.966+0000 7f4f1127c740 1 Processor -- start 2024-09-19T06:42:45.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.966+0000 7f4f1127c740 1 -- start start 2024-09-19T06:42:45.967 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:45.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.966+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875963300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.966+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 0x55787597ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.966+0000 7f4f0d440640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875963300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.966+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x557875985a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f1127c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5578758c8620 con 0x557875962f20 2024-09-19T06:42:45.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f1127c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55787586fe50 con 0x557875983660 2024-09-19T06:42:45.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f1127c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5578758c45e0 con 0x557875963840 2024-09-19T06:42:45.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f0d440640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875963300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35998/0 (socket says 172.21.15.37:35998) 2024-09-19T06:42:45.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f0d440640 1 -- 172.21.15.37:0/2720386468 learned_addr learned my addr 172.21.15.37:0/2720386468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f07fff640 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x557875985a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f0d440640 1 -- 172.21.15.37:0/2720386468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 msgr2=0x55787597ed10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f0d440640 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 0x55787597ed10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f0d440640 1 -- 172.21.15.37:0/2720386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 msgr2=0x557875985a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f0d440640 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x557875985a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.967+0000 7f4f0d440640 1 -- 172.21.15.37:0/2720386468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557875860d40 con 0x557875962f20 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.968+0000 7f4f07fff640 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x557875985a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.968+0000 7f4f05ffb640 1 -- 172.21.15.37:0/3190481503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f4ef8020070 con 0x7f4f08100e20 2024-09-19T06:42:45.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.968+0000 7f4f0d440640 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875963300 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009c40 tx=0x7f4f00011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c14cbd6a278b68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:45.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.968+0000 7f4efeffd640 1 -- 172.21.15.37:0/2720386468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00004230 con 0x557875962f20 2024-09-19T06:42:45.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.969+0000 7f4efeffd640 1 -- 172.21.15.37:0/2720386468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f000043d0 con 0x557875962f20 2024-09-19T06:42:45.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.969+0000 7f4efeffd640 1 -- 172.21.15.37:0/2720386468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f000198a0 con 0x557875962f20 2024-09-19T06:42:45.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.969+0000 7f4f05ffb640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4ef40a3660 0x7f4ef40a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.969+0000 7f4f05ffb640 1 -- 172.21.15.37:0/3190481503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f4ef80cf540 con 0x7f4f08100e20 2024-09-19T06:42:45.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.969+0000 7f4f0cc3f640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4ef40a3660 0x7f4ef40a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.969+0000 7f4f1127c740 1 -- 172.21.15.37:0/2720386468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 msgr2=0x557875963300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.969+0000 7f4f1127c740 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875963300 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4f00009c40 tx=0x7f4f00011810 comp rx=0 tx=0).stop 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 -- 172.21.15.37:0/2720386468 shutdown_connections 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x557875985a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 0x55787597ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 --2- 172.21.15.37:0/2720386468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875963300 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 -- 172.21.15.37:0/2720386468 >> 172.21.15.37:0/2720386468 conn(0x55787595f5f0 msgr2=0x55787595f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 -- 172.21.15.37:0/2720386468 shutdown_connections 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 -- 172.21.15.37:0/2720386468 wait complete. 2024-09-19T06:42:45.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f0cc3f640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4ef40a3660 0x7f4ef40a5b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80ccab0 tx=0x7f4ef800cf60 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:45.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 -- start start 2024-09-19T06:42:45.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.970+0000 7f4f1127c740 1 Processor -- start 2024-09-19T06:42:45.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.971+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875982fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.971+0000 7f4f0d440640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875982fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.971+0000 7f4f0d440640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875982fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36002/0 (socket says 172.21.15.37:36002) 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.971+0000 7f4f0d440640 1 -- 172.21.15.37:0/245922353 learned_addr learned my addr 172.21.15.37:0/245922353 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.971+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 0x55787597c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x55787597c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557875986230 con 0x557875962f20 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557875850f10 con 0x557875983660 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5578759863b0 con 0x557875963840 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f1127c740 10 client.0 _ll_register_callbacks cb 0x557875874db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f07fff640 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 0x55787597c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f0d440640 1 -- 172.21.15.37:0/245922353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 msgr2=0x55787597c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:45.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f0d440640 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 0x55787597c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f0d440640 1 -- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 msgr2=0x55787597c7a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:42:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f0d440640 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x55787597c7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.972+0000 7f4f0d440640 1 -- 172.21.15.37:0/245922353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578755d7580 con 0x557875962f20 2024-09-19T06:42:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4f0d440640 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875982fb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dfd7fe7640ff4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4ee7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:45.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4ee7fff640 1 -- 172.21.15.37:0/245922353 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f00028070 con 0x557875962f20 2024-09-19T06:42:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4ee7fff640 1 -- 172.21.15.37:0/245922353 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f4f00005040 con 0x557875962f20 2024-09-19T06:42:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f000043d0 con 0x557875962f20 2024-09-19T06:42:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4ee7fff640 1 -- 172.21.15.37:0/245922353 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f00034050 con 0x557875962f20 2024-09-19T06:42:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4f1127c740 10 client.15678 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.973+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5578756a1960 con 0x557875962f20 2024-09-19T06:42:45.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4ee7fff640 1 -- 172.21.15.37:0/245922353 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f00009890 con 0x557875962f20 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4ee7fff640 1 -- 172.21.15.37:0/245922353 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4f000195a0 con 0x557875962f20 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4f1127c740 10 client.15678 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4f1127c740 10 client.15678 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4f1127c740 20 client.15678 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875997690 0x557875999a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4f1127c740 4 client.15678 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.974+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4f000043d0 con 0x557875997690 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.976+0000 7f4f07fff640 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875997690 0x557875999a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.976+0000 7f4f07fff640 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875997690 0x557875999a80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aac94ac15f735ccf server_cookie=17f6814211667d4f in_seq=0 out_seq=0 2024-09-19T06:42:45.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.976+0000 7f4ee7fff640 10 client.15678 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4ee7fff640 1 -- 172.21.15.37:0/245922353 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4f000043d0 con 0x557875997690 2024-09-19T06:42:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4ee7fff640 10 client.15678 handle_command_reply: tid=0 2024-09-19T06:42:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 client.15678 shutdown 2024-09-19T06:42:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 msgr2=0x557875982fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:45.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875982fb0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f00006010 tx=0x7f4f000045e0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 shutdown_connections 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875997690 0x557875999a80 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x55787597c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875963840 0x55787597c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.977+0000 7f4f1127c740 1 --2- 172.21.15.37:0/245922353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875962f20 0x557875982fb0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.978+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 >> 172.21.15.37:0/245922353 conn(0x55787595f5f0 msgr2=0x557875965350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.978+0000 7f4ee7fff640 0 client.15678 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.978+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 shutdown_connections 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.978+0000 7f4f1127c740 1 -- 172.21.15.37:0/245922353 wait complete. 2024-09-19T06:42:45.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:45.979+0000 7f4f1127c740 20 client.15678 trim_cache size 0 max 16384 2024-09-19T06:42:46.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.014+0000 7f4f1127c740 1 Processor -- start 2024-09-19T06:42:46.015 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.014+0000 7f4f1127c740 1 -- start start 2024-09-19T06:42:46.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.014+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875962f20 0x55787595ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.014+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x55787599f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.014+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x55787599f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x55787599f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x55787599f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36010/0 (socket says 172.21.15.37:36010) 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 -- 172.21.15.37:0/467018595 learned_addr learned my addr 172.21.15.37:0/467018595 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f0d440640 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x55787599f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f1127c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f081a5880 con 0x557875963840 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f1127c740 1 -- 172.21.15.37:0/467018595 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08073580 con 0x557875983660 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f1127c740 1 -- 172.21.15.37:0/467018595 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f080713d0 con 0x557875962f20 2024-09-19T06:42:46.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f0d440640 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875962f20 0x55787595ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 -- 172.21.15.37:0/467018595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875962f20 msgr2=0x55787595ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875962f20 0x55787595ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 -- 172.21.15.37:0/467018595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 msgr2=0x55787599f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x55787599f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f07fff640 1 -- 172.21.15.37:0/467018595 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55787595dbb0 con 0x557875963840 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.015+0000 7f4f0d440640 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875962f20 0x55787595ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.016+0000 7f4f07fff640 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x55787599f0b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009890 tx=0x7f4ef0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f57921fff432b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.016+0000 7f4efe7fc640 1 -- 172.21.15.37:0/467018595 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0037040 con 0x557875963840 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.016+0000 7f4efe7fc640 1 -- 172.21.15.37:0/467018595 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ef0032cd0 con 0x557875963840 2024-09-19T06:42:46.018 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.016+0000 7f4efe7fc640 1 -- 172.21.15.37:0/467018595 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0003bf0 con 0x557875963840 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 -- 172.21.15.37:0/467018595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 msgr2=0x55787599f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x55787599f0b0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0009890 tx=0x7f4ef0005f30 comp rx=0 tx=0).stop 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 -- 172.21.15.37:0/467018595 shutdown_connections 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875983660 0x55787599f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x55787599f0b0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 --2- 172.21.15.37:0/467018595 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875962f20 0x55787595ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 -- 172.21.15.37:0/467018595 >> 172.21.15.37:0/467018595 conn(0x55787595f5f0 msgr2=0x55787597afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 -- 172.21.15.37:0/467018595 shutdown_connections 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 -- 172.21.15.37:0/467018595 wait complete. 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 -- start start 2024-09-19T06:42:46.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.017+0000 7f4f1127c740 1 Processor -- start 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875962f20 0x557875993ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x557875994410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f1127c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875983660 0x557875990ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f1127c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08071a40 con 0x557875963840 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x557875994410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x557875994410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36026/0 (socket says 172.21.15.37:36026) 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 -- 172.21.15.37:0/264521778 learned_addr learned my addr 172.21.15.37:0/264521778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08072eb0 con 0x557875962f20 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4f08072110 con 0x557875983660 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f0d440640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875962f20 0x557875993ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f0d440640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875983660 0x557875990ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 -- 172.21.15.37:0/264521778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875983660 msgr2=0x557875990ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875983660 0x557875990ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 -- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875962f20 msgr2=0x557875993ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875962f20 0x557875993ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f1127c740 10 client.0 _ll_register_callbacks cb 0x7f4f00003820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.018+0000 7f4f07fff640 1 -- 172.21.15.37:0/264521778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5578759913e0 con 0x557875963840 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef0032cd0 con 0x557875963840 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4f0d440640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875962f20 0x557875993ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4f07fff640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x557875994410 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0000c00 tx=0x7f4ef0032f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe4806925c37d5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4ee7fff640 10 client.15684 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4ee7fff640 1 -- 172.21.15.37:0/264521778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0037040 con 0x557875963840 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4ee7fff640 1 -- 172.21.15.37:0/264521778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f4ef00329f0 con 0x557875963840 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4ee7fff640 1 -- 172.21.15.37:0/264521778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4ef00040c0 con 0x557875963840 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4f1127c740 10 client.15684 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:46.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.019+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=25}) -- 0x557875a026c0 con 0x557875963840 2024-09-19T06:42:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4ee7fff640 1 -- 172.21.15.37:0/264521778 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f4ef0059030 con 0x557875963840 2024-09-19T06:42:46.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4f1127c740 10 client.15684 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4f1127c740 10 client.15684 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4f1127c740 20 client.15684 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4f1127c740 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875a02c50 0x557875a05070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4f1127c740 4 client.15684 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e13bea5b", "release": true, "await": true}] 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e13bea5b", "release": true, "await": true}) -- 0x5578758d5b90 con 0x557875a02c50 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.020+0000 7f4f0d440640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875a02c50 0x557875a05070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.021+0000 7f4ee7fff640 10 client.15684 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.021+0000 7f4f0d440640 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875a02c50 0x557875a05070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64cbb661e1384c51 server_cookie=977103149b40c84f in_seq=0 out_seq=0 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4ee7fff640 1 -- 172.21.15.37:0/264521778 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5578758d5b90 con 0x557875a02c50 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4ee7fff640 10 client.15684 handle_command_reply: tid=0 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 client.15684 shutdown 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 msgr2=0x557875994410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x557875994410 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0000c00 tx=0x7f4ef0032f00 comp rx=0 tx=0).stop 2024-09-19T06:42:46.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 shutdown_connections 2024-09-19T06:42:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557875a02c50 0x557875a05070 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4ee7fff640 0 client.15684 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557875983660 0x557875990ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557875963840 0x557875994410 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.022+0000 7f4f1127c740 1 --2- 172.21.15.37:0/264521778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557875962f20 0x557875993ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.023+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 >> 172.21.15.37:0/264521778 conn(0x55787595f5f0 msgr2=0x557875981a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.023+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 shutdown_connections 2024-09-19T06:42:46.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.023+0000 7f4f1127c740 1 -- 172.21.15.37:0/264521778 wait complete. 2024-09-19T06:42:46.026 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:46.026 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 25, 2024-09-19T06:42:46.026 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:42:46.027 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 4, 2024-09-19T06:42:46.027 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:42:46.027 INFO:teuthology.orchestra.run.smithi037.stdout: "e13bea5b": { 2024-09-19T06:42:46.027 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 4, 2024-09-19T06:42:46.027 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:42:46.027 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:42:46.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:42:46.028 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:42:46.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:42:46.028 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:42:46.028 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 131.2, 2024-09-19T06:42:46.028 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:46.029 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:46.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:46.030 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:46.030 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:46.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.023+0000 7f4f1127c740 20 client.15684 trim_cache size 0 max 16384 2024-09-19T06:42:46.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4ef40a3660 msgr2=0x7f4ef40a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4ef40a3660 0x7f4ef40a5b20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4ef80ccab0 tx=0x7f4ef800cf60 comp rx=0 tx=0).stop 2024-09-19T06:42:46.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08100e20 msgr2=0x7f4f08197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08100e20 0x7f4f08197460 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f4ef8010cc0 tx=0x7f4ef800c640 comp rx=0 tx=0).stop 2024-09-19T06:42:46.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 shutdown_connections 2024-09-19T06:42:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4f08102ef0 0x7f4f0819bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4f08102570 0x7f4f081979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4ef40a3660 0x7f4ef40a5b20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 --2- 172.21.15.37:0/3190481503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4f08100e20 0x7f4f08197460 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.024+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 >> 172.21.15.37:0/3190481503 conn(0x7f4f080fc7d0 msgr2=0x7f4f081048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.025+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 shutdown_connections 2024-09-19T06:42:46.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.025+0000 7f4f0eea3640 1 -- 172.21.15.37:0/3190481503 wait complete. 2024-09-19T06:42:46.094 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:46 smithi037 ceph-mon[31211]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:42:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:46 smithi151 ceph-mon[35236]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:42:46.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:46 smithi096 ceph-mon[35004]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:42:46.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.726+0000 7fa4a26bc640 1 -- 172.21.15.37:0/2144325213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1082a0 msgr2=0x7fa49c10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.726+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/2144325213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1082a0 0x7fa49c10ec40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fa494009cd0 tx=0x7fa49402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.727+0000 7fa4a26bc640 1 -- 172.21.15.37:0/2144325213 shutdown_connections 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.727+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/2144325213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1082a0 0x7fa49c10ec40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.727+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/2144325213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49c1036e0 0x7fa49c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.727+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/2144325213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa49c102d30 0x7fa49c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.727+0000 7fa4a26bc640 1 -- 172.21.15.37:0/2144325213 >> 172.21.15.37:0/2144325213 conn(0x7fa49c0fc7d0 msgr2=0x7fa49c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.727+0000 7fa4a26bc640 1 -- 172.21.15.37:0/2144325213 shutdown_connections 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.727+0000 7fa4a26bc640 1 -- 172.21.15.37:0/2144325213 wait complete. 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.728+0000 7fa4a26bc640 1 Processor -- start 2024-09-19T06:42:46.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.728+0000 7fa4a26bc640 1 -- start start 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.728+0000 7fa4a26bc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49c102d30 0x7fa49c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.728+0000 7fa4a26bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1036e0 0x7fa49c19aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.728+0000 7fa4a26bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa49c1082a0 0x7fa49c19ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa4a26bc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1120a0 con 0x7fa49c1036e0 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa4a26bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c111f20 con 0x7fa49c1082a0 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa4a26bc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c112220 con 0x7fa49c102d30 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1036e0 0x7fa49c19aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1036e0 0x7fa49c19aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36032/0 (socket says 172.21.15.37:36032) 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 -- 172.21.15.37:0/269978260 learned_addr learned my addr 172.21.15.37:0/269978260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa49c1082a0 0x7fa49c19ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49bfff640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49c102d30 0x7fa49c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 -- 172.21.15.37:0/269978260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49c102d30 msgr2=0x7fa49c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49c102d30 0x7fa49c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 -- 172.21.15.37:0/269978260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa49c1082a0 msgr2=0x7fa49c19ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa49c1082a0 0x7fa49c19ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 -- 172.21.15.37:0/269978260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa49c19f540 con 0x7fa49c1036e0 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49bfff640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49c102d30 0x7fa49c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.729+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1036e0 0x7fa49c19aad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d740 tx=0x7fa49000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.730+0000 7fa4997fa640 1 -- 172.21.15.37:0/269978260 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490017070 con 0x7fa49c1036e0 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.730+0000 7fa4997fa640 1 -- 172.21.15.37:0/269978260 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa49000de50 con 0x7fa49c1036e0 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.730+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa49c19f830 con 0x7fa49c1036e0 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.730+0000 7fa4997fa640 1 -- 172.21.15.37:0/269978260 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa490021060 con 0x7fa49c1036e0 2024-09-19T06:42:46.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.730+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa49c1a7110 con 0x7fa49c1036e0 2024-09-19T06:42:46.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.733+0000 7fa4997fa640 1 -- 172.21.15.37:0/269978260 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fa49001a070 con 0x7fa49c1036e0 2024-09-19T06:42:46.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.733+0000 7fa4a4a95740 1 Processor -- start 2024-09-19T06:42:46.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.733+0000 7fa4a4a95740 1 -- start start 2024-09-19T06:42:46.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723467090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577234675d0 0x557723467a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 0x55772347cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5577233cd2f0 con 0x557723468b40 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557723374cf0 con 0x5577234675d0 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577233c92b0 con 0x55772347ab80 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4997fa640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4840a3560 0x7fa4840a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a0c59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723467090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a0c59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723467090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36046/0 (socket says 172.21.15.37:36046) 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a0c59640 1 -- 172.21.15.37:0/294443155 learned_addr learned my addr 172.21.15.37:0/294443155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4997fa640 1 -- 172.21.15.37:0/269978260 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fa4900cfd60 con 0x7fa49c1036e0 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a0c59640 1 -- 172.21.15.37:0/294443155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 msgr2=0x55772347cf70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:42:46.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577234675d0 0x557723467a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 0x55772347cf70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a0c59640 1 -- 172.21.15.37:0/294443155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577234675d0 msgr2=0x557723467a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.734+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577234675d0 0x557723467a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.735+0000 7fa4a0c59640 1 -- 172.21.15.37:0/294443155 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577233593a0 con 0x557723468b40 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.735+0000 7fa49bfff640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4840a3560 0x7fa4840a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.735+0000 7fa49bfff640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4840a3560 0x7fa4840a5a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0058c0 tx=0x7fa48c005850 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.735+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723467090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa494009870 tx=0x7fa4940314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f627b58879632196 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.736+0000 7fa4827fc640 1 -- 172.21.15.37:0/294443155 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494048070 con 0x557723468b40 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.736+0000 7fa4827fc640 1 -- 172.21.15.37:0/294443155 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494036070 con 0x557723468b40 2024-09-19T06:42:46.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.736+0000 7fa4827fc640 1 -- 172.21.15.37:0/294443155 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494004500 con 0x557723468b40 2024-09-19T06:42:46.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.736+0000 7fa4a4a95740 1 -- 172.21.15.37:0/294443155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 msgr2=0x557723467090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.736+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723467090 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa494009870 tx=0x7fa4940314e0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 -- 172.21.15.37:0/294443155 shutdown_connections 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 0x55772347cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577234675d0 0x557723467a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/294443155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723467090 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 -- 172.21.15.37:0/294443155 >> 172.21.15.37:0/294443155 conn(0x557723430310 msgr2=0x557723463c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 -- 172.21.15.37:0/294443155 shutdown_connections 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 -- 172.21.15.37:0/294443155 wait complete. 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 -- start start 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.737+0000 7fa4a4a95740 1 Processor -- start 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577234675d0 0x55772347a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x557723477860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 0x557723477ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55772347d820 con 0x5577234675d0 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557723388060 con 0x557723468b40 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55772347d9a0 con 0x55772347ab80 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a4a95740 10 client.0 _ll_register_callbacks cb 0x5577230db8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577234675d0 0x55772347a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577234675d0 0x55772347a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36048/0 (socket says 172.21.15.37:36048) 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 -- 172.21.15.37:0/278889891 learned_addr learned my addr 172.21.15.37:0/278889891 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 0x557723477ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x557723477860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 -- 172.21.15.37:0/278889891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 msgr2=0x557723477ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 0x557723477ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 -- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 msgr2=0x557723477860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x557723477860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.738+0000 7fa49bfff640 1 -- 172.21.15.37:0/278889891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55772313e3c0 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.739+0000 7fa49bfff640 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577234675d0 0x55772347a5b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0116c0 tx=0x7fa48c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861fea0a19ff1a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.739+0000 7fa47b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.739+0000 7fa47b7fe640 1 -- 172.21.15.37:0/278889891 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa48c022070 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.739+0000 7fa47b7fe640 1 -- 172.21.15.37:0/278889891 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fa48c01e860 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.739+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa494036070 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa47b7fe640 1 -- 172.21.15.37:0/278889891 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa48c01ea60 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa4a4a95740 10 client.15696 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557723489360 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa47b7fe640 1 -- 172.21.15.37:0/278889891 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494036070 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa47b7fe640 1 -- 172.21.15.37:0/278889891 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa48c0103d0 con 0x5577234675d0 2024-09-19T06:42:46.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa4a4a95740 10 client.15696 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa4a4a95740 10 client.15696 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa4a4a95740 20 client.15696 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.740+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x55772348cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.741+0000 7fa4a4a95740 4 client.15696 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.741+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577233da860 con 0x55772348a820 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.741+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x55772348cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.741+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x55772348cc50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a65c5cbeda2b0266 server_cookie=e17aed8c951b4989 in_seq=0 out_seq=0 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.741+0000 7fa47b7fe640 10 client.15696 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.742+0000 7fa47b7fe640 1 -- 172.21.15.37:0/278889891 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577233da860 con 0x55772348a820 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.742+0000 7fa47b7fe640 10 client.15696 handle_command_reply: tid=0 2024-09-19T06:42:46.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.742+0000 7fa4a4a95740 1 client.15696 shutdown 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.742+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577234675d0 msgr2=0x55772347a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.742+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577234675d0 0x55772347a5b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0116c0 tx=0x7fa48c0046c0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 shutdown_connections 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x55772348cc50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa47b7fe640 0 client.15696 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772347ab80 0x557723477ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x557723477860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/278889891 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577234675d0 0x55772347a5b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 >> 172.21.15.37:0/278889891 conn(0x557723430310 msgr2=0x557723469960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 shutdown_connections 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 1 -- 172.21.15.37:0/278889891 wait complete. 2024-09-19T06:42:46.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.743+0000 7fa4a4a95740 20 client.15696 trim_cache size 0 max 16384 2024-09-19T06:42:46.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 Processor -- start 2024-09-19T06:42:46.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 -- start start 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x557723492520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723492a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772347ab80 0x557723474ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1ae100 con 0x557723468b40 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1ab9d0 con 0x55772347ab80 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1a9840 con 0x5577234675d0 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa49b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723492a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa49b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723492a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36058/0 (socket says 172.21.15.37:36058) 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa49b7fe640 1 -- 172.21.15.37:0/2936722954 learned_addr learned my addr 172.21.15.37:0/2936722954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.769+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772347ab80 0x557723474ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa49bfff640 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x557723492520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa4a0c59640 1 -- 172.21.15.37:0/2936722954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 msgr2=0x557723492520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x557723492520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa4a0c59640 1 -- 172.21.15.37:0/2936722954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 msgr2=0x557723492a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723492a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa4a0c59640 1 -- 172.21.15.37:0/2936722954 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557723475db0 con 0x55772347ab80 2024-09-19T06:42:46.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723492a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:46.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772347ab80 0x557723474ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa494009870 tx=0x7fa494005bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfd429a0c4a4e19f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.770+0000 7fa481ffb640 1 -- 172.21.15.37:0/2936722954 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494048040 con 0x55772347ab80 2024-09-19T06:42:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.771+0000 7fa481ffb640 1 -- 172.21.15.37:0/2936722954 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494036040 con 0x55772347ab80 2024-09-19T06:42:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.771+0000 7fa481ffb640 1 -- 172.21.15.37:0/2936722954 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494004440 con 0x55772347ab80 2024-09-19T06:42:46.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.771+0000 7fa4a4a95740 1 -- 172.21.15.37:0/2936722954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772347ab80 msgr2=0x557723474ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.771+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772347ab80 0x557723474ac0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa494009870 tx=0x7fa494005bf0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 -- 172.21.15.37:0/2936722954 shutdown_connections 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772347ab80 0x557723474ac0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557723468b40 0x557723492a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/2936722954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x557723492520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 -- 172.21.15.37:0/2936722954 >> 172.21.15.37:0/2936722954 conn(0x557723430310 msgr2=0x557723476950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 -- 172.21.15.37:0/2936722954 shutdown_connections 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 -- 172.21.15.37:0/2936722954 wait complete. 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 -- start start 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.772+0000 7fa4a4a95740 1 Processor -- start 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x55772347eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x55772347f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a4a95740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772347ab80 0x55772347f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a4a95740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1a9e90 con 0x55772347ab80 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a0c59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772347ab80 0x55772347f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a0c59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772347ab80 0x55772347f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36066/0 (socket says 172.21.15.37:36066) 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a0c59640 1 -- 172.21.15.37:0/1166215058 learned_addr learned my addr 172.21.15.37:0/1166215058 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1ab300 con 0x557723468b40 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa49b7fe640 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x55772347f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa49bfff640 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x55772347eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa49c1aa560 con 0x5577234675d0 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.773+0000 7fa4a4a95740 10 client.0 _ll_register_callbacks cb 0x7fa48c015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa4a0c59640 1 -- 172.21.15.37:0/1166215058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 msgr2=0x55772347eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x55772347eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa4a0c59640 1 -- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 msgr2=0x55772347f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:46.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x55772347f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa4a0c59640 1 -- 172.21.15.37:0/1166215058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577234f46f0 con 0x55772347ab80 2024-09-19T06:42:46.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa4a0c59640 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772347ab80 0x55772347f5d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa494031800 tx=0x7fa494009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1e4b4ca14c96f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:46.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa47b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:46.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa47b7fe640 1 -- 172.21.15.37:0/1166215058 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa494048070 con 0x55772347ab80 2024-09-19T06:42:46.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa47b7fe640 1 -- 172.21.15.37:0/1166215058 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fa49402e040 con 0x55772347ab80 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.774+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa494036040 con 0x55772347ab80 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.775+0000 7fa47b7fe640 1 -- 172.21.15.37:0/1166215058 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa494031ad0 con 0x55772347ab80 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.775+0000 7fa4a4a95740 10 client.15702 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.775+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577234f5950 con 0x55772347ab80 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.775+0000 7fa47b7fe640 1 -- 172.21.15.37:0/1166215058 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa494036040 con 0x55772347ab80 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.775+0000 7fa47b7fe640 1 -- 172.21.15.37:0/1166215058 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa49404b030 con 0x55772347ab80 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa4a4a95740 10 client.15702 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa4a4a95740 10 client.15702 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa4a4a95740 20 client.15702 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x5577234f7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa4a4a95740 4 client.15702 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577233d4d70 con 0x55772348a820 2024-09-19T06:42:46.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa49bfff640 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x5577234f7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa49bfff640 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x5577234f7b30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3c03e0fd554182a3 server_cookie=49f4fcbebe27dd06 in_seq=0 out_seq=0 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.776+0000 7fa47b7fe640 10 client.15702 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa47b7fe640 1 -- 172.21.15.37:0/1166215058 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577233d4d70 con 0x55772348a820 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa47b7fe640 10 client.15702 handle_command_reply: tid=0 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa4a4a95740 1 client.15702 shutdown 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772347ab80 msgr2=0x55772347f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772347ab80 0x55772347f5d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fa494031800 tx=0x7fa494009900 comp rx=0 tx=0).stop 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 shutdown_connections 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55772348a820 0x5577234f7b30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772347ab80 0x55772347f5d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa47b7fe640 0 client.15702 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.777+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557723468b40 0x55772347f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.778+0000 7fa4a4a95740 1 --2- 172.21.15.37:0/1166215058 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577234675d0 0x55772347eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.778+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 >> 172.21.15.37:0/1166215058 conn(0x557723430310 msgr2=0x557723469110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.778+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 shutdown_connections 2024-09-19T06:42:46.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.778+0000 7fa4a4a95740 1 -- 172.21.15.37:0/1166215058 wait complete. 2024-09-19T06:42:46.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.778+0000 7fa4a4a95740 20 client.15702 trim_cache size 0 max 16384 2024-09-19T06:42:46.784 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:42:46.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4840a3560 msgr2=0x7fa4840a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4840a3560 0x7fa4840a5a20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa48c0058c0 tx=0x7fa48c005850 comp rx=0 tx=0).stop 2024-09-19T06:42:46.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1036e0 msgr2=0x7fa49c19aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:46.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1036e0 0x7fa49c19aad0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fa49000d740 tx=0x7fa49000dc00 comp rx=0 tx=0).stop 2024-09-19T06:42:46.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 shutdown_connections 2024-09-19T06:42:46.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa49c1082a0 0x7fa49c19ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4840a3560 0x7fa4840a5a20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa49c1036e0 0x7fa49c19aad0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 --2- 172.21.15.37:0/269978260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa49c102d30 0x7fa49c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:46.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 >> 172.21.15.37:0/269978260 conn(0x7fa49c0fc7d0 msgr2=0x7fa49c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:46.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 shutdown_connections 2024-09-19T06:42:46.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:46.779+0000 7fa4a26bc640 1 -- 172.21.15.37:0/269978260 wait complete. 2024-09-19T06:42:47.462 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:42:47.855 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e13bea5b', total seconds elapsed: 17.2 2024-09-19T06:42:47.914 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:48.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:48 smithi037 ceph-mon[31211]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:42:48.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:48 smithi151 ceph-mon[35236]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 -- 172.21.15.37:0/874944002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 msgr2=0x7f67a8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 --2- 172.21.15.37:0/874944002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a8103110 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009930 tx=0x7f67a402a420 comp rx=0 tx=0).stop 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 -- 172.21.15.37:0/874944002 shutdown_connections 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 --2- 172.21.15.37:0/874944002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a81082a0 0x7f67a810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 --2- 172.21.15.37:0/874944002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a81036e0 0x7f67a8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 --2- 172.21.15.37:0/874944002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a8103110 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 -- 172.21.15.37:0/874944002 >> 172.21.15.37:0/874944002 conn(0x7f67a80fc7d0 msgr2=0x7f67a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 -- 172.21.15.37:0/874944002 shutdown_connections 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.424+0000 7f67af69b640 1 -- 172.21.15.37:0/874944002 wait complete. 2024-09-19T06:42:48.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.425+0000 7f67af69b640 1 Processor -- start 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.425+0000 7f67af69b640 1 -- start start 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67af69b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67af69b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a81036e0 0x7f67a8198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67af69b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a81082a0 0x7f67a819ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67af69b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81120a0 con 0x7f67a8102d30 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67af69b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8111f20 con 0x7f67a81082a0 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67af69b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67a8112220 con 0x7f67a81036e0 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67ad437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67ad437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36090/0 (socket says 172.21.15.37:36090) 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67ad437640 1 -- 172.21.15.37:0/1623057815 learned_addr learned my addr 172.21.15.37:0/1623057815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67acc36640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a81036e0 0x7f67a8198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67adc38640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a81082a0 0x7f67a819ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67ad437640 1 -- 172.21.15.37:0/1623057815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a81036e0 msgr2=0x7f67a8198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67ad437640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a81036e0 0x7f67a8198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.426+0000 7f67ad437640 1 -- 172.21.15.37:0/1623057815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a81082a0 msgr2=0x7f67a819ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f67ad437640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a81082a0 0x7f67a819ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f67ad437640 1 -- 172.21.15.37:0/1623057815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67a819d570 con 0x7f67a8102d30 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f67adc38640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a81082a0 0x7f67a819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f67ad437640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a81985c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009230 tx=0x7f67a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f679e7fc640 1 -- 172.21.15.37:0/1623057815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a403f030 con 0x7f67a8102d30 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67a819d800 con 0x7f67a8102d30 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f679e7fc640 1 -- 172.21.15.37:0/1623057815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f67a4003ad0 con 0x7f67a8102d30 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f679e7fc640 1 -- 172.21.15.37:0/1623057815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4035070 con 0x7f67a8102d30 2024-09-19T06:42:48.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.427+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67a81a5140 con 0x7f67a8102d30 2024-09-19T06:42:48.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.430+0000 7f67b1a74740 1 Processor -- start 2024-09-19T06:42:48.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.430+0000 7f67b1a74740 1 -- start start 2024-09-19T06:42:48.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a99b40 0x562e06a98090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06a98a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06aadf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67b1a74740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562e069fe2f0 con 0x562e06aabb80 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67b1a74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e069a5cf0 con 0x562e06a99b40 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67b1a74740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e069fa2b0 con 0x562e06a985d0 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67acc36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06aadf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67acc36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06aadf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36104/0 (socket says 172.21.15.37:36104) 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67acc36640 1 -- 172.21.15.37:0/818698834 learned_addr learned my addr 172.21.15.37:0/818698834 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f67adc38640 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a99b40 0x562e06a98090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.431+0000 7f679e7fc640 1 -- 172.21.15.37:0/1623057815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f67a4042030 con 0x7f67a8102d30 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67acc36640 1 -- 172.21.15.37:0/818698834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 msgr2=0x562e06a98a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67acc36640 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06a98a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67acc36640 1 -- 172.21.15.37:0/818698834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a99b40 msgr2=0x562e06a98090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:48.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67acc36640 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a99b40 0x562e06a98090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67acc36640 1 -- 172.21.15.37:0/818698834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e0698a3a0 con 0x562e06aabb80 2024-09-19T06:42:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67acc36640 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06aadf70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f679800bc10 tx=0x7f67980107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3767f69639e03c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67937fe640 1 -- 172.21.15.37:0/818698834 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679801c480 con 0x562e06aabb80 2024-09-19T06:42:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67937fe640 1 -- 172.21.15.37:0/818698834 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f679801cb00 con 0x562e06aabb80 2024-09-19T06:42:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f67937fe640 1 -- 172.21.15.37:0/818698834 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798011b70 con 0x562e06aabb80 2024-09-19T06:42:48.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.432+0000 7f679e7fc640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67940a35e0 0x7f67940a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.433+0000 7f679e7fc640 1 -- 172.21.15.37:0/1623057815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f67a40f04e0 con 0x7f67a8102d30 2024-09-19T06:42:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.433+0000 7f67adc38640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67940a35e0 0x7f67940a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.433+0000 7f67b1a74740 1 -- 172.21.15.37:0/818698834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 msgr2=0x562e06aadf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.433+0000 7f67b1a74740 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06aadf70 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f679800bc10 tx=0x7f67980107a0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.433+0000 7f67adc38640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67940a35e0 0x7f67940a5aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0009780 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.433+0000 7f67b1a74740 1 -- 172.21.15.37:0/818698834 shutdown_connections 2024-09-19T06:42:48.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06aadf70 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06a98a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 --2- 172.21.15.37:0/818698834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a99b40 0x562e06a98090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 -- 172.21.15.37:0/818698834 >> 172.21.15.37:0/818698834 conn(0x562e06a61310 msgr2=0x562e06a94c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 -- 172.21.15.37:0/818698834 shutdown_connections 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 -- 172.21.15.37:0/818698834 wait complete. 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 -- start start 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.434+0000 7f67b1a74740 1 Processor -- start 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06aab630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a99b40 0x562e06aa88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67ad437640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06aab630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67ad437640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06aab630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:38396/0 (socket says 172.21.15.37:38396) 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67acc36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a99b40 0x562e06aa88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67acc36640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a99b40 0x562e06aa88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36118/0 (socket says 172.21.15.37:36118) 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67ad437640 1 -- 172.21.15.37:0/1528226881 learned_addr learned my addr 172.21.15.37:0/1528226881 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67b1a74740 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aa9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562e06aae880 con 0x562e06a99b40 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e069b9060 con 0x562e06aabb80 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.435+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562e06aaea00 con 0x562e06a985d0 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67adc38640 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aa9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67ad437640 1 -- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 msgr2=0x562e06aa9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67ad437640 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aa9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67ad437640 1 -- 172.21.15.37:0/1528226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a99b40 msgr2=0x562e06aa88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67ad437640 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a99b40 0x562e06aa88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67b1a74740 10 client.0 _ll_register_callbacks cb 0x562e0670c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67ad437640 1 -- 172.21.15.37:0/1528226881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e0676f3c0 con 0x562e06a985d0 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f679801cb00 con 0x562e06a985d0 2024-09-19T06:42:48.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67acc36640 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a99b40 0x562e06aa88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f67ad437640 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06aab630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67a40bc280 tx=0x7f67a40bc180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=815fbc01e2e36d2e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f6790ff9640 10 client.25231 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.436+0000 7f6790ff9640 1 -- 172.21.15.37:0/1528226881 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4049040 con 0x562e06a985d0 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f6790ff9640 1 -- 172.21.15.37:0/1528226881 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f67a4003c80 con 0x562e06a985d0 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f6790ff9640 1 -- 172.21.15.37:0/1528226881 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67a40c0930 con 0x562e06a985d0 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 10 client.25231 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({fsmap=25}) -- 0x562e06aae970 con 0x562e06a985d0 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f6790ff9640 1 -- 172.21.15.37:0/1528226881 <== mon.1 v2:172.21.15.151:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f67a40c4d60 con 0x562e06a985d0 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 10 client.25231 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 10 client.25231 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 20 client.25231 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06abb660 0x562e06abdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 4 client.25231 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.437+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e06a0b860 con 0x562e06abb660 2024-09-19T06:42:48.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.438+0000 7f67acc36640 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06abb660 0x562e06abdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.438+0000 7f67acc36640 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06abb660 0x562e06abdad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c31dce5ba739d194 server_cookie=938f565a80a1a890 in_seq=0 out_seq=0 2024-09-19T06:42:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.438+0000 7f6790ff9640 10 client.25231 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f6790ff9640 1 -- 172.21.15.37:0/1528226881 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562e06a0b860 con 0x562e06abb660 2024-09-19T06:42:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f6790ff9640 10 client.25231 handle_command_reply: tid=0 2024-09-19T06:42:48.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 client.25231 shutdown 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 msgr2=0x562e06aab630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06aab630 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f67a40bc280 tx=0x7f67a40bc180 comp rx=0 tx=0).stop 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 shutdown_connections 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06abb660 0x562e06abdad0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aa9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f6790ff9640 0 client.25231 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a99b40 0x562e06aa88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.439+0000 7f67b1a74740 1 --2- 172.21.15.37:0/1528226881 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a985d0 0x562e06aab630 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.440+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 >> 172.21.15.37:0/1528226881 conn(0x562e06a61310 msgr2=0x562e06a9a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:48.445 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.440+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 shutdown_connections 2024-09-19T06:42:48.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.440+0000 7f67b1a74740 1 -- 172.21.15.37:0/1528226881 wait complete. 2024-09-19T06:42:48.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.440+0000 7f67b1a74740 20 client.25231 trim_cache size 0 max 16384 2024-09-19T06:42:48.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.465+0000 7f67b1a74740 1 Processor -- start 2024-09-19T06:42:48.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.465+0000 7f67b1a74740 1 -- start start 2024-09-19T06:42:48.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a985d0 0x562e06a94d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06aa81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06ac3340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67b1a74740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81ade80 con 0x562e06aabb80 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67b1a74740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81ab750 con 0x562e06a985d0 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67b1a74740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81a95a0 con 0x562e06a99b40 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06aa81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06aa81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:38398/0 (socket says 172.21.15.37:38398) 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 -- 172.21.15.37:0/2634191911 learned_addr learned my addr 172.21.15.37:0/2634191911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67ad437640 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a985d0 0x562e06a94d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 -- 172.21.15.37:0/2634191911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a985d0 msgr2=0x562e06a94d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a985d0 0x562e06a94d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 -- 172.21.15.37:0/2634191911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 msgr2=0x562e06ac3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:48.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06ac3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.466+0000 7f67acc36640 1 -- 172.21.15.37:0/2634191911 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e06a9b120 con 0x562e06a99b40 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.467+0000 7f67acc36640 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06aa81c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67980116e0 tx=0x7f679800b5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0036067fb528c55 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.467+0000 7f6792ffd640 1 -- 172.21.15.37:0/2634191911 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f679801c480 con 0x562e06a99b40 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.467+0000 7f6792ffd640 1 -- 172.21.15.37:0/2634191911 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6798005040 con 0x562e06a99b40 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.467+0000 7f6792ffd640 1 -- 172.21.15.37:0/2634191911 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6798011b80 con 0x562e06a99b40 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 -- 172.21.15.37:0/2634191911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 msgr2=0x562e06aa81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06aa81c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67980116e0 tx=0x7f679800b5b0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 -- 172.21.15.37:0/2634191911 shutdown_connections 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06aabb80 0x562e06ac3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06aa81c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2634191911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06a985d0 0x562e06a94d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 -- 172.21.15.37:0/2634191911 >> 172.21.15.37:0/2634191911 conn(0x562e06a61310 msgr2=0x562e06a93d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:48.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 -- 172.21.15.37:0/2634191911 shutdown_connections 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.468+0000 7f67b1a74740 1 -- 172.21.15.37:0/2634191911 wait complete. 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.469+0000 7f67b1a74740 1 -- start start 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.469+0000 7f67b1a74740 1 Processor -- start 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.469+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a985d0 0x562e06ab4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.469+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06ab4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.469+0000 7f67b1a74740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aaf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.469+0000 7f67ad437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a985d0 0x562e06ab4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.469+0000 7f67ad437640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a985d0 0x562e06ab4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36136/0 (socket says 172.21.15.37:36136) 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67ad437640 1 -- 172.21.15.37:0/2841221256 learned_addr learned my addr 172.21.15.37:0/2841221256 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67acc36640 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06ab4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81a9c10 con 0x562e06a985d0 2024-09-19T06:42:48.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81ab080 con 0x562e06aabb80 2024-09-19T06:42:48.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67adc38640 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aaf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f67a81aa2e0 con 0x562e06a99b40 2024-09-19T06:42:48.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67b1a74740 10 client.0 _ll_register_callbacks cb 0x7f67a40b8820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67ad437640 1 -- 172.21.15.37:0/2841221256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 msgr2=0x562e06ab4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67ad437640 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06ab4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67ad437640 1 -- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 msgr2=0x562e06aaf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.470+0000 7f67ad437640 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aaf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f67ad437640 1 -- 172.21.15.37:0/2841221256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e06aafdd0 con 0x562e06a985d0 2024-09-19T06:42:48.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f67ad437640 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a985d0 0x562e06ab4020 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f67a40c0af0 tx=0x7f67a402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e58ba698d24dad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f6790ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f6790ff9640 1 -- 172.21.15.37:0/2841221256 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a4049040 con 0x562e06a985d0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f6790ff9640 1 -- 172.21.15.37:0/2841221256 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f67a40f0070 con 0x562e06a985d0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6798005040 con 0x562e06a985d0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f6790ff9640 1 -- 172.21.15.37:0/2841221256 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f67a40c51f0 con 0x562e06a985d0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f67b1a74740 10 client.15717 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.471+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562e06b25f30 con 0x562e06a985d0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f6790ff9640 1 -- 172.21.15.37:0/2841221256 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6798005040 con 0x562e06a985d0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f6790ff9640 1 -- 172.21.15.37:0/2841221256 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f67a40bb9d0 con 0x562e06a985d0 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f67b1a74740 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f67b1a74740 10 client.15717 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f67b1a74740 20 client.15717 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:48.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06b26470 0x562e06b28860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f67b1a74740 4 client.15717 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562e06a05d70 con 0x562e06b26470 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.472+0000 7f67acc36640 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06b26470 0x562e06b28860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.473+0000 7f67acc36640 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06b26470 0x562e06b28860 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=25ae34c3095dc09 server_cookie=f389ca4de8607f6 in_seq=0 out_seq=0 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.473+0000 7f6790ff9640 10 client.15717 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.473+0000 7f6790ff9640 1 -- 172.21.15.37:0/2841221256 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562e06a05d70 con 0x562e06b26470 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.473+0000 7f6790ff9640 10 client.15717 handle_command_reply: tid=0 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.473+0000 7f67b1a74740 1 client.15717 shutdown 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a985d0 msgr2=0x562e06ab4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a985d0 0x562e06ab4020 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f67a40c0af0 tx=0x7f67a402a420 comp rx=0 tx=0).stop 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 shutdown_connections 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x562e06b26470 0x562e06b28860 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e06aabb80 0x562e06aaf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f6790ff9640 0 client.15717 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562e06a99b40 0x562e06ab4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 --2- 172.21.15.37:0/2841221256 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562e06a985d0 0x562e06ab4020 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 >> 172.21.15.37:0/2841221256 conn(0x562e06a61310 msgr2=0x562e06a934e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 shutdown_connections 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 1 -- 172.21.15.37:0/2841221256 wait complete. 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.474+0000 7f67b1a74740 20 client.15717 trim_cache size 0 max 16384 2024-09-19T06:42:48.481 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67940a35e0 msgr2=0x7f67940a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67940a35e0 0x7f67940a5aa0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67a0005fd0 tx=0x7f67a0009780 comp rx=0 tx=0).stop 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 msgr2=0x7f67a81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a81985c0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f67a4009230 tx=0x7f67a4004270 comp rx=0 tx=0).stop 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 shutdown_connections 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f67a81082a0 0x7f67a819ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f67a81036e0 0x7f67a8198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67940a35e0 0x7f67940a5aa0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 --2- 172.21.15.37:0/1623057815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f67a8102d30 0x7f67a81985c0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:48.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 >> 172.21.15.37:0/1623057815 conn(0x7f67a80fc7d0 msgr2=0x7f67a8105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:48.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 shutdown_connections 2024-09-19T06:42:48.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:48.475+0000 7f67af69b640 1 -- 172.21.15.37:0/1623057815 wait complete. 2024-09-19T06:42:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:48 smithi096 ceph-mon[35004]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-09-19T06:42:49.249 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:42:49.529 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:50.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 -- 172.21.15.37:0/594705808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45001027a0 msgr2=0x7f45001067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 --2- 172.21.15.37:0/594705808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45001027a0 0x7f45001067f0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f44f4009900 tx=0x7f44f402a410 comp rx=0 tx=0).stop 2024-09-19T06:42:50.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 -- 172.21.15.37:0/594705808 shutdown_connections 2024-09-19T06:42:50.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 --2- 172.21.15.37:0/594705808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4500106f20 0x7f450010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 --2- 172.21.15.37:0/594705808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45001027a0 0x7f45001067f0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 --2- 172.21.15.37:0/594705808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 0x7f45001021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 -- 172.21.15.37:0/594705808 >> 172.21.15.37:0/594705808 conn(0x7f45000fd8f0 msgr2=0x7f45000ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 -- 172.21.15.37:0/594705808 shutdown_connections 2024-09-19T06:42:50.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.054+0000 7f45062cd640 1 -- 172.21.15.37:0/594705808 wait complete. 2024-09-19T06:42:50.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.056+0000 7f45062cd640 1 Processor -- start 2024-09-19T06:42:50.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.056+0000 7f45062cd640 1 -- start start 2024-09-19T06:42:50.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.056+0000 7f45062cd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 0x7f45001983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.056+0000 7f45062cd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45001027a0 0x7f4500198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.056+0000 7f45062cd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4500106f20 0x7f450019ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f45062cd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4500111f80 con 0x7f45001027a0 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f45062cd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4500111e00 con 0x7f4500101df0 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f45062cd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4500112100 con 0x7f4500106f20 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 0x7f45001983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 0x7f45001983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44902/0 (socket says 172.21.15.37:44902) 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 -- 172.21.15.37:0/4004304683 learned_addr learned my addr 172.21.15.37:0/4004304683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f450486a640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4500106f20 0x7f450019ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 -- 172.21.15.37:0/4004304683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4500106f20 msgr2=0x7f450019ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4500106f20 0x7f450019ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 -- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45001027a0 msgr2=0x7f4500198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45001027a0 0x7f4500198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.057+0000 7f44fffff640 1 -- 172.21.15.37:0/4004304683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f450019d390 con 0x7f4500101df0 2024-09-19T06:42:50.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.058+0000 7f44fffff640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 0x7f45001983e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44f000d950 tx=0x7f44f000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.058+0000 7f44fd7fa640 1 -- 172.21.15.37:0/4004304683 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0017070 con 0x7f4500101df0 2024-09-19T06:42:50.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.058+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450019d680 con 0x7f4500101df0 2024-09-19T06:42:50.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.058+0000 7f44fd7fa640 1 -- 172.21.15.37:0/4004304683 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f0009c30 con 0x7f4500101df0 2024-09-19T06:42:50.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.058+0000 7f44fd7fa640 1 -- 172.21.15.37:0/4004304683 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f000f6b0 con 0x7f4500101df0 2024-09-19T06:42:50.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.058+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45001a4f60 con 0x7f4500101df0 2024-09-19T06:42:50.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.061+0000 7f44fd7fa640 1 -- 172.21.15.37:0/4004304683 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f44f00109d0 con 0x7f4500101df0 2024-09-19T06:42:50.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 Processor -- start 2024-09-19T06:42:50.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 -- start start 2024-09-19T06:42:50.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14231c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d14230560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230b80 0x561d14245f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d141962f0 con 0x561d14230120 2024-09-19T06:42:50.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561d1413dcf0 con 0x561d14231840 2024-09-19T06:42:50.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f45086a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561d141922b0 con 0x561d14230b80 2024-09-19T06:42:50.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f44fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d14230560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f44fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d14230560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47188/0 (socket says 172.21.15.37:47188) 2024-09-19T06:42:50.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f44fffff640 1 -- 172.21.15.37:0/122146103 learned_addr learned my addr 172.21.15.37:0/122146103 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.065 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.062+0000 7f450486a640 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14231c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230b80 0x561d14245f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44fffff640 1 -- 172.21.15.37:0/122146103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230b80 msgr2=0x561d14245f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44fffff640 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230b80 0x561d14245f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44fffff640 1 -- 172.21.15.37:0/122146103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 msgr2=0x561d14231c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44fffff640 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14231c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44fffff640 1 -- 172.21.15.37:0/122146103 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d141223a0 con 0x561d14230120 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44fffff640 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d14230560 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f44f00cd060 tx=0x7f44f00d8620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89992edd296bb48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.063+0000 7f44fd7fa640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44e80a3660 0x7f44e80a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.064+0000 7f44e67fc640 1 -- 172.21.15.37:0/122146103 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f00dc470 con 0x561d14230120 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.064+0000 7f450486a640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44e80a3660 0x7f44e80a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.064+0000 7f44e67fc640 1 -- 172.21.15.37:0/122146103 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f00e1070 con 0x561d14230120 2024-09-19T06:42:50.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.064+0000 7f44e67fc640 1 -- 172.21.15.37:0/122146103 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f00db740 con 0x561d14230120 2024-09-19T06:42:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.064+0000 7f44fd7fa640 1 -- 172.21.15.37:0/4004304683 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f44f00cfbe0 con 0x7f4500101df0 2024-09-19T06:42:50.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.064+0000 7f45086a6740 1 -- 172.21.15.37:0/122146103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 msgr2=0x561d14230560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.064+0000 7f45086a6740 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d14230560 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f44f00cd060 tx=0x7f44f00d8620 comp rx=0 tx=0).stop 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f450486a640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44e80a3660 0x7f44e80a5b20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f44f8007f00 tx=0x7f44f8007520 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f45086a6740 1 -- 172.21.15.37:0/122146103 shutdown_connections 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f45086a6740 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230b80 0x561d14245f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f45086a6740 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d14230560 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f45086a6740 1 --2- 172.21.15.37:0/122146103 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14231c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f45086a6740 1 -- 172.21.15.37:0/122146103 >> 172.21.15.37:0/122146103 conn(0x561d141f9310 msgr2=0x561d1422cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f45086a6740 1 -- 172.21.15.37:0/122146103 shutdown_connections 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.065+0000 7f45086a6740 1 -- 172.21.15.37:0/122146103 wait complete. 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.066+0000 7f45086a6740 1 -- start start 2024-09-19T06:42:50.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.066+0000 7f45086a6740 1 Processor -- start 2024-09-19T06:42:50.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.066+0000 7f45086a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230120 0x561d14245d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.066+0000 7f45086a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230b80 0x561d14242fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.066+0000 7f45086a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14243500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.066+0000 7f45086a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561d14246970 con 0x561d14230b80 2024-09-19T06:42:50.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f45086a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561d14151060 con 0x561d14231840 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f45086a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561d14246af0 con 0x561d14230120 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f45086a6740 10 client.0 _ll_register_callbacks cb 0x561d13ea48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230b80 0x561d14242fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230b80 0x561d14242fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47204/0 (socket says 172.21.15.37:47204) 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 -- 172.21.15.37:0/3627773183 learned_addr learned my addr 172.21.15.37:0/3627773183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44fffff640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230120 0x561d14245d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f450486a640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14243500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 -- 172.21.15.37:0/3627773183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230120 msgr2=0x561d14245d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230120 0x561d14245d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 -- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 msgr2=0x561d14243500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14243500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 -- 172.21.15.37:0/3627773183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d13f073c0 con 0x561d14230b80 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44fffff640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230120 0x561d14245d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:50.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230b80 0x561d14242fc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f44f4004fd0 tx=0x7f44f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6559355472155821 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.067+0000 7f44df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.068+0000 7f44df7fe640 1 -- 172.21.15.37:0/3627773183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f4041070 con 0x561d14230b80 2024-09-19T06:42:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.068+0000 7f44df7fe640 1 -- 172.21.15.37:0/3627773183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f44f4005200 con 0x561d14230b80 2024-09-19T06:42:50.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.068+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44f00e1070 con 0x561d14230b80 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.068+0000 7f44df7fe640 1 -- 172.21.15.37:0/3627773183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44f4005400 con 0x561d14230b80 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.068+0000 7f45086a6740 10 client.15726 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.068+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d142522d0 con 0x561d14230b80 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.068+0000 7f44df7fe640 1 -- 172.21.15.37:0/3627773183 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f00e1070 con 0x561d14230b80 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f44df7fe640 1 -- 172.21.15.37:0/3627773183 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44f40464f0 con 0x561d14230b80 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f45086a6740 10 client.15726 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f45086a6740 10 client.15726 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f45086a6740 20 client.15726 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f45086a6740 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d14253a80 0x561d14255f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f45086a6740 4 client.15726 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d141a3860 con 0x561d14253a80 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.069+0000 7f44fffff640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d14253a80 0x561d14255f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.070+0000 7f44fffff640 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d14253a80 0x561d14255f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb0132f02b79edc server_cookie=7575d1ba3e62d024 in_seq=0 out_seq=0 2024-09-19T06:42:50.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.070+0000 7f44df7fe640 10 client.15726 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.070+0000 7f44df7fe640 1 -- 172.21.15.37:0/3627773183 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561d141a3860 con 0x561d14253a80 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.070+0000 7f44df7fe640 10 client.15726 handle_command_reply: tid=0 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 client.15726 shutdown 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230b80 msgr2=0x561d14242fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230b80 0x561d14242fc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f44f4004fd0 tx=0x7f44f40046c0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 shutdown_connections 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d14253a80 0x561d14255f20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d14243500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230b80 0x561d14242fc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 --2- 172.21.15.37:0/3627773183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14230120 0x561d14245d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 >> 172.21.15.37:0/3627773183 conn(0x561d141f9310 msgr2=0x561d1422bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f44df7fe640 0 client.15726 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 shutdown_connections 2024-09-19T06:42:50.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 1 -- 172.21.15.37:0/3627773183 wait complete. 2024-09-19T06:42:50.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.071+0000 7f45086a6740 20 client.15726 trim_cache size 0 max 16384 2024-09-19T06:42:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.097+0000 7f45086a6740 1 Processor -- start 2024-09-19T06:42:50.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f45086a6740 1 -- start start 2024-09-19T06:42:50.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f45086a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1422ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f45086a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14231840 0x561d14232c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f45086a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d1424fc40 0x561d14234300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f45086a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45001ae050 con 0x561d14230120 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f45086a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45001ab920 con 0x561d1424fc40 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f45086a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45001a9770 con 0x561d14231840 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f44fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1422ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f44fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1422ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47218/0 (socket says 172.21.15.37:47218) 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f44fffff640 1 -- 172.21.15.37:0/3177325955 learned_addr learned my addr 172.21.15.37:0/3177325955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.098+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14231840 0x561d14232c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f450486a640 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d1424fc40 0x561d14234300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44fffff640 1 -- 172.21.15.37:0/3177325955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14231840 msgr2=0x561d14232c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44fffff640 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14231840 0x561d14232c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44fffff640 1 -- 172.21.15.37:0/3177325955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d1424fc40 msgr2=0x561d14234300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44fffff640 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d1424fc40 0x561d14234300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44fffff640 1 -- 172.21.15.37:0/3177325955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d14243830 con 0x561d14230120 2024-09-19T06:42:50.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44fffff640 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1422ce10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f44f00980e0 tx=0x7f44f00cff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fe8178cfd56dcf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44e5ffb640 1 -- 172.21.15.37:0/3177325955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0020040 con 0x561d14230120 2024-09-19T06:42:50.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44e5ffb640 1 -- 172.21.15.37:0/3177325955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f00e1040 con 0x561d14230120 2024-09-19T06:42:50.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.099+0000 7f44e5ffb640 1 -- 172.21.15.37:0/3177325955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f009b7c0 con 0x561d14230120 2024-09-19T06:42:50.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.100+0000 7f45086a6740 1 -- 172.21.15.37:0/3177325955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 msgr2=0x561d1422ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.100+0000 7f45086a6740 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1422ce10 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f44f00980e0 tx=0x7f44f00cff90 comp rx=0 tx=0).stop 2024-09-19T06:42:50.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.100+0000 7f45086a6740 1 -- 172.21.15.37:0/3177325955 shutdown_connections 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.100+0000 7f45086a6740 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d1424fc40 0x561d14234300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.100+0000 7f45086a6740 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d14231840 0x561d14232c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.100+0000 7f45086a6740 1 --2- 172.21.15.37:0/3177325955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1422ce10 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.100+0000 7f45086a6740 1 -- 172.21.15.37:0/3177325955 >> 172.21.15.37:0/3177325955 conn(0x561d141f9310 msgr2=0x561d1422c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.101+0000 7f45086a6740 1 -- 172.21.15.37:0/3177325955 shutdown_connections 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.101+0000 7f45086a6740 1 -- 172.21.15.37:0/3177325955 wait complete. 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.101+0000 7f45086a6740 1 -- start start 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.101+0000 7f45086a6740 1 Processor -- start 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f45086a6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1424c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f45086a6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d1425b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f45086a6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d1424fc40 0x561d1425b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f45086a6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f45001a9de0 con 0x561d14230120 2024-09-19T06:42:50.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1424c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f45086a6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f45001ab250 con 0x561d14231840 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f45086a6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f45001aa4b0 con 0x561d1424fc40 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44fffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1424c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47228/0 (socket says 172.21.15.37:47228) 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44fffff640 1 -- 172.21.15.37:0/584681089 learned_addr learned my addr 172.21.15.37:0/584681089 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f45086a6740 10 client.0 _ll_register_callbacks cb 0x7f44f4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d1425b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f450486a640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d1424fc40 0x561d1425b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44fffff640 1 -- 172.21.15.37:0/584681089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d1424fc40 msgr2=0x561d1425b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44fffff640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d1424fc40 0x561d1425b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44fffff640 1 -- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 msgr2=0x561d1425b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.102+0000 7f44fffff640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d1425b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.103+0000 7f44fffff640 1 -- 172.21.15.37:0/584681089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d1425bbb0 con 0x561d14230120 2024-09-19T06:42:50.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.103+0000 7f450486a640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d1424fc40 0x561d1425b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.103+0000 7f44fffff640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1424c310 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f44f00d3c20 tx=0x7f44f0005ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0783797075ee59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.103+0000 7f44df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.103+0000 7f44df7fe640 1 -- 172.21.15.37:0/584681089 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0020070 con 0x561d14230120 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.103+0000 7f44df7fe640 1 -- 172.21.15.37:0/584681089 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f44f00982f0 con 0x561d14230120 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.103+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44f00e1040 con 0x561d14230120 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f44df7fe640 1 -- 172.21.15.37:0/584681089 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44f009b5e0 con 0x561d14230120 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 10 client.15732 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561d1424acb0 con 0x561d14230120 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f44df7fe640 1 -- 172.21.15.37:0/584681089 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44f00e1040 con 0x561d14230120 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f44df7fe640 1 -- 172.21.15.37:0/584681089 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f44f009bee0 con 0x561d14230120 2024-09-19T06:42:50.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 10 client.15732 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 10 client.15732 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 20 client.15732 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d142be8a0 0x561d142c0c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 4 client.15732 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.104+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d1419dd70 con 0x561d142be8a0 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.105+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d142be8a0 0x561d142c0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.105+0000 7f44df7fe640 10 client.15732 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.105+0000 7f44ff7fe640 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d142be8a0 0x561d142c0c90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63a011fbb5b2fe5 server_cookie=b683d3e1378a6c17 in_seq=0 out_seq=0 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.105+0000 7f44df7fe640 1 -- 172.21.15.37:0/584681089 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d1419dd70 con 0x561d142be8a0 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.105+0000 7f44df7fe640 10 client.15732 handle_command_reply: tid=0 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.105+0000 7f45086a6740 1 client.15732 shutdown 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 msgr2=0x561d1424c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1424c310 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f44f00d3c20 tx=0x7f44f0005ab0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 shutdown_connections 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561d142be8a0 0x561d142c0c90 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f44df7fe640 0 client.15732 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561d1424fc40 0x561d1425b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d14231840 0x561d1425b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 --2- 172.21.15.37:0/584681089 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561d14230120 0x561d1424c310 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 >> 172.21.15.37:0/584681089 conn(0x561d141f9310 msgr2=0x561d142366a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 shutdown_connections 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 1 -- 172.21.15.37:0/584681089 wait complete. 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.106+0000 7f45086a6740 20 client.15732 trim_cache size 0 max 16384 2024-09-19T06:42:50.111 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:42:50.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44e80a3660 msgr2=0x7f44e80a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44e80a3660 0x7f44e80a5b20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f44f8007f00 tx=0x7f44f8007520 comp rx=0 tx=0).stop 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 msgr2=0x7f45001983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 0x7f45001983e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44f000d950 tx=0x7f44f000de10 comp rx=0 tx=0).stop 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 shutdown_connections 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4500106f20 0x7f450019ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44e80a3660 0x7f44e80a5b20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f45001027a0 0x7f4500198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.107+0000 7f45062cd640 1 --2- 172.21.15.37:0/4004304683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4500101df0 0x7f45001983e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.108+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 >> 172.21.15.37:0/4004304683 conn(0x7f45000fd8f0 msgr2=0x7f45000fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.108+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 shutdown_connections 2024-09-19T06:42:50.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:50.108+0000 7f45062cd640 1 -- 172.21.15.37:0/4004304683 wait complete. 2024-09-19T06:42:50.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:50 smithi037 ceph-mon[31211]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:42:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:50 smithi151 ceph-mon[35236]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:42:50.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:50 smithi096 ceph-mon[35004]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.787+0000 7fdafbca6640 1 -- 172.21.15.37:0/1829201935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf4101350 msgr2=0x7fdaf410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.787+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1829201935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf4101350 0x7fdaf410d880 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdaec009cd0 tx=0x7fdaec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.788+0000 7fdafbca6640 1 -- 172.21.15.37:0/1829201935 shutdown_connections 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.788+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1829201935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf4101350 0x7fdaf410d880 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.788+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1829201935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf41009b0 0x7fdaf4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.788+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1829201935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf41069c0 0x7fdaf4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.788+0000 7fdafbca6640 1 -- 172.21.15.37:0/1829201935 >> 172.21.15.37:0/1829201935 conn(0x7fdaf4077990 msgr2=0x7fdaf40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.788+0000 7fdafbca6640 1 -- 172.21.15.37:0/1829201935 shutdown_connections 2024-09-19T06:42:50.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.788+0000 7fdafbca6640 1 -- 172.21.15.37:0/1829201935 wait complete. 2024-09-19T06:42:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.789+0000 7fdafbca6640 1 Processor -- start 2024-09-19T06:42:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.789+0000 7fdafbca6640 1 -- start start 2024-09-19T06:42:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.789+0000 7fdafbca6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf41009b0 0x7fdaf4197220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafbca6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf4101350 0x7fdaf4197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafbca6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf41069c0 0x7fdaf419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafbca6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4110e50 con 0x7fdaf41069c0 2024-09-19T06:42:50.792 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafbca6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4110cd0 con 0x7fdaf41009b0 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafbca6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf4110fd0 con 0x7fdaf4101350 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafa243640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf41069c0 0x7fdaf419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdaf9241640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf4101350 0x7fdaf4197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafa243640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf41069c0 0x7fdaf419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47246/0 (socket says 172.21.15.37:47246) 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafa243640 1 -- 172.21.15.37:0/1883297464 learned_addr learned my addr 172.21.15.37:0/1883297464 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf41009b0 0x7fdaf4197220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdaf9241640 1 -- 172.21.15.37:0/1883297464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf41009b0 msgr2=0x7fdaf4197220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdaf9241640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf41009b0 0x7fdaf4197220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdaf9241640 1 -- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf41069c0 msgr2=0x7fdaf419bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdaf9241640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf41069c0 0x7fdaf419bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdaf9241640 1 -- 172.21.15.37:0/1883297464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaf419c210 con 0x7fdaf4101350 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.790+0000 7fdafa243640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf41069c0 0x7fdaf419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:50.793 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.791+0000 7fdaf9241640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf4101350 0x7fdaf4197760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009f10 tx=0x7fdae400ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.791+0000 7fdaeaffd640 1 -- 172.21.15.37:0/1883297464 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae40159e0 con 0x7fdaf4101350 2024-09-19T06:42:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.791+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaf419c500 con 0x7fdaf4101350 2024-09-19T06:42:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.791+0000 7fdaeaffd640 1 -- 172.21.15.37:0/1883297464 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdae400cd90 con 0x7fdaf4101350 2024-09-19T06:42:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.791+0000 7fdaeaffd640 1 -- 172.21.15.37:0/1883297464 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdae4010a90 con 0x7fdaf4101350 2024-09-19T06:42:50.794 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.791+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaf40698a0 con 0x7fdaf4101350 2024-09-19T06:42:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.795+0000 7fdafea3a740 1 Processor -- start 2024-09-19T06:42:50.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.795+0000 7fdaeaffd640 1 -- 172.21.15.37:0/1883297464 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fdae4010c30 con 0x7fdaf4101350 2024-09-19T06:42:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.796+0000 7fdafea3a740 1 -- start start 2024-09-19T06:42:50.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.796+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99148ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.796+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d9914a0f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdaf9a42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d9914a0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdaeaffd640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdae00a3690 0x7fdae00a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdafa243640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99148ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdafa243640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99148ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:35550/0 (socket says 172.21.15.37:35550) 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdaf9a42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d9914a0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47262/0 (socket says 172.21.15.37:47262) 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdafa243640 1 -- 172.21.15.37:0/616067418 learned_addr learned my addr 172.21.15.37:0/616067418 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdaeaffd640 1 -- 172.21.15.37:0/1883297464 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fdae40d4560 con 0x7fdaf4101350 2024-09-19T06:42:50.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdafea3a740 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 0x55d9914a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d9913f22f0 con 0x55d99148e330 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d99139a270 con 0x55d9914a14c0 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.797+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d9913ee2b0 con 0x55d99148da10 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdaf9241640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdae00a3690 0x7fdae00a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdaf9a42640 1 -- 172.21.15.37:0/616067418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 msgr2=0x55d99148ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99148ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdaf9241640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdae00a3690 0x7fdae00a5b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdae40d1ad0 tx=0x7fdae4010430 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdaf9a42640 1 -- 172.21.15.37:0/616067418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 msgr2=0x55d9914a38b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 0x55d9914a38b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdaf9a42640 1 -- 172.21.15.37:0/616067418 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d991399340 con 0x55d99148e330 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.798+0000 7fdafa243640 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99148ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:50.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.799+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d9914a0f80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009c00 tx=0x7fdaf0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966480f6670dd090 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.799+0000 7fdadffff640 1 -- 172.21.15.37:0/616067418 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0004050 con 0x55d99148e330 2024-09-19T06:42:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.799+0000 7fdadffff640 1 -- 172.21.15.37:0/616067418 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaf0005500 con 0x55d99148e330 2024-09-19T06:42:50.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.799+0000 7fdadffff640 1 -- 172.21.15.37:0/616067418 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0004e00 con 0x55d99148e330 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.800+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 msgr2=0x55d9914a0f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.800+0000 7fdafea3a740 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d9914a0f80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009c00 tx=0x7fdaf0011810 comp rx=0 tx=0).stop 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 shutdown_connections 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 0x55d9914a38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d9914a0f80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 --2- 172.21.15.37:0/616067418 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99148ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 >> 172.21.15.37:0/616067418 conn(0x55d99148a060 msgr2=0x55d99148a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 shutdown_connections 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 -- 172.21.15.37:0/616067418 wait complete. 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 -- start start 2024-09-19T06:42:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.801+0000 7fdafea3a740 1 Processor -- start 2024-09-19T06:42:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99149e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d99149e930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 0x55d99149ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafea3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d9914a4090 con 0x55d99148e330 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafea3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d99137b290 con 0x55d9914a14c0 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafea3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d9914a4210 con 0x55d99148da10 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafea3a740 10 client.0 _ll_register_callbacks cb 0x55d9913ac140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdaf9a42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d99149e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdaf9a42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d99149e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47276/0 (socket says 172.21.15.37:47276) 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdaf9a42640 1 -- 172.21.15.37:0/2300608500 learned_addr learned my addr 172.21.15.37:0/2300608500 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.802+0000 7fdafa243640 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99149e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdaf9a42640 1 -- 172.21.15.37:0/2300608500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 msgr2=0x55d99149e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99149e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdaf9a42640 1 -- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 msgr2=0x55d99149ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdafa243640 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 0x55d99149ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 0x55d99149ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdaf9a42640 1 -- 172.21.15.37:0/2300608500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d991105210 con 0x55d99148e330 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d99149e930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009fe0 tx=0x7fdaf0005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9323b1464c0ab45a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.803+0000 7fdadd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaf0005500 con 0x55d99148e330 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdadd7fa640 1 -- 172.21.15.37:0/2300608500 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0029070 con 0x55d99148e330 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdadd7fa640 1 -- 172.21.15.37:0/2300608500 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fdaf0003a80 con 0x55d99148e330 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdadd7fa640 1 -- 172.21.15.37:0/2300608500 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdaf0035050 con 0x55d99148e330 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdafea3a740 10 client.15744 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55d9911d1620 con 0x55d99148e330 2024-09-19T06:42:50.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdadd7fa640 1 -- 172.21.15.37:0/2300608500 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdaf0003cc0 con 0x55d99148e330 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdafea3a740 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdafea3a740 10 client.15744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.804+0000 7fdafea3a740 20 client.15744 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.805+0000 7fdafea3a740 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d9914b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.805+0000 7fdafea3a740 4 client.15744 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.805+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9913ffa20 con 0x55d9914b1200 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.805+0000 7fdafa243640 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d9914b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.805+0000 7fdadd7fa640 10 client.15744 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.805+0000 7fdafa243640 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d9914b35f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c924ae5969d5912f server_cookie=1b1415651ac17d45 in_seq=0 out_seq=0 2024-09-19T06:42:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.806+0000 7fdadd7fa640 1 -- 172.21.15.37:0/2300608500 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d9913ffa20 con 0x55d9914b1200 2024-09-19T06:42:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.806+0000 7fdadd7fa640 10 client.15744 handle_command_reply: tid=0 2024-09-19T06:42:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.806+0000 7fdafea3a740 1 client.15744 shutdown 2024-09-19T06:42:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.807+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 msgr2=0x55d99149e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.807+0000 7fdafea3a740 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d99149e930 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fdaf0009fe0 tx=0x7fdaf0005780 comp rx=0 tx=0).stop 2024-09-19T06:42:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 shutdown_connections 2024-09-19T06:42:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d9914b35f0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdadd7fa640 0 client.15744 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d9914a14c0 0x55d99149ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148e330 0x55d99149e930 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 --2- 172.21.15.37:0/2300608500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d99148da10 0x55d99149e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 >> 172.21.15.37:0/2300608500 conn(0x55d99148a060 msgr2=0x55d99148fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 shutdown_connections 2024-09-19T06:42:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.811+0000 7fdafea3a740 1 -- 172.21.15.37:0/2300608500 wait complete. 2024-09-19T06:42:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.812+0000 7fdafea3a740 20 client.15744 trim_cache size 0 max 16384 2024-09-19T06:42:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 Processor -- start 2024-09-19T06:42:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 -- start start 2024-09-19T06:42:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914b8c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 0x55d9914b9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ade70 con 0x55d99148da10 2024-09-19T06:42:50.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ab740 con 0x55d99148e330 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafea3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41a9590 con 0x55d9914a14c0 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafa243640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafa243640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914890b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47280/0 (socket says 172.21.15.37:47280) 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.841+0000 7fdafa243640 1 -- 172.21.15.37:0/3431453055 learned_addr learned my addr 172.21.15.37:0/3431453055 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914b8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdafa243640 1 -- 172.21.15.37:0/3431453055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 msgr2=0x55d9914b9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdafa243640 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 0x55d9914b9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdafa243640 1 -- 172.21.15.37:0/3431453055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 msgr2=0x55d9914b8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdafa243640 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914b8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdafa243640 1 -- 172.21.15.37:0/3431453055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d991499150 con 0x55d99148da10 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdafa243640 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914890b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000f80 tx=0x7fdaec02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39667e7c8a5b03aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdadf7fe640 1 -- 172.21.15.37:0/3431453055 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec031c90 con 0x55d99148da10 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdadf7fe640 1 -- 172.21.15.37:0/3431453055 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec036040 con 0x55d99148da10 2024-09-19T06:42:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.842+0000 7fdadf7fe640 1 -- 172.21.15.37:0/3431453055 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec0046b0 con 0x55d99148da10 2024-09-19T06:42:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 -- 172.21.15.37:0/3431453055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 msgr2=0x55d9914890b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914890b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fdaec000f80 tx=0x7fdaec02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 -- 172.21.15.37:0/3431453055 shutdown_connections 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 0x55d9914b9190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914b8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 --2- 172.21.15.37:0/3431453055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914890b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 -- 172.21.15.37:0/3431453055 >> 172.21.15.37:0/3431453055 conn(0x55d99148a060 msgr2=0x55d9914889a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 -- 172.21.15.37:0/3431453055 shutdown_connections 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.843+0000 7fdafea3a740 1 -- 172.21.15.37:0/3431453055 wait complete. 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.844+0000 7fdafea3a740 1 -- start start 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.844+0000 7fdafea3a740 1 Processor -- start 2024-09-19T06:42:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914a5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914a5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafea3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 0x55d99151ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafea3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41a9c00 con 0x55d99148da10 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914a5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafea3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41ab070 con 0x55d99148e330 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafea3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdaf41aa2d0 con 0x55d9914a14c0 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914a5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47294/0 (socket says 172.21.15.37:47294) 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 -- 172.21.15.37:0/1474836242 learned_addr learned my addr 172.21.15.37:0/1474836242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafea3a740 10 client.0 _ll_register_callbacks cb 0x7fdaf0009a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914a5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 -- 172.21.15.37:0/1474836242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 msgr2=0x55d99151ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 0x55d99151ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 -- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 msgr2=0x55d9914a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914a5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdafa243640 1 -- 172.21.15.37:0/1474836242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d99148e8c0 con 0x55d99148da10 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.845+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdafa243640 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914a5520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdaec02aea0 tx=0x7fdaec031d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d4c2f9ef4658742 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdadd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdadd7fa640 1 -- 172.21.15.37:0/1474836242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaec004740 con 0x55d99148da10 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdadd7fa640 1 -- 172.21.15.37:0/1474836242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fdaec0305a0 con 0x55d99148da10 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaec036040 con 0x55d99148da10 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdadd7fa640 1 -- 172.21.15.37:0/1474836242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdaec004b40 con 0x55d99148da10 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdafea3a740 10 client.15750 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d99151c060 con 0x55d99148da10 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdadd7fa640 1 -- 172.21.15.37:0/1474836242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdaec036040 con 0x55d99148da10 2024-09-19T06:42:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.846+0000 7fdadd7fa640 1 -- 172.21.15.37:0/1474836242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fdaec030070 con 0x55d99148da10 2024-09-19T06:42:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdafea3a740 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdafea3a740 10 client.15750 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdafea3a740 20 client.15750 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdafea3a740 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d99151e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdafea3a740 4 client.15750 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:42:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d9913f9d70 con 0x55d9914b1200 2024-09-19T06:42:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d99151e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdadd7fa640 10 client.15750 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.847+0000 7fdaf9a42640 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d99151e640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8cf35bdf4688dae server_cookie=52c696c78c9107ac in_seq=0 out_seq=0 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdadd7fa640 1 -- 172.21.15.37:0/1474836242 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d9913f9d70 con 0x55d9914b1200 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdadd7fa640 10 client.15750 handle_command_reply: tid=0 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 client.15750 shutdown 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 msgr2=0x55d9914a5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914a5520 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fdaec02aea0 tx=0x7fdaec031d20 comp rx=0 tx=0).stop 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 shutdown_connections 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d9914b1200 0x55d99151e640 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d9914a14c0 0x55d99151ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdadd7fa640 0 client.15750 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d99148e330 0x55d9914a5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 --2- 172.21.15.37:0/1474836242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d99148da10 0x55d9914a5520 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 >> 172.21.15.37:0/1474836242 conn(0x55d99148a060 msgr2=0x55d9914891c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 shutdown_connections 2024-09-19T06:42:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.851+0000 7fdafea3a740 1 -- 172.21.15.37:0/1474836242 wait complete. 2024-09-19T06:42:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafea3a740 20 client.15750 trim_cache size 0 max 16384 2024-09-19T06:42:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:42:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:42:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:42:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:42:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdae00a3690 msgr2=0x7fdae00a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdae00a3690 0x7fdae00a5b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdae40d1ad0 tx=0x7fdae4010430 comp rx=0 tx=0).stop 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf4101350 msgr2=0x7fdaf4197760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf4101350 0x7fdaf4197760 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009f10 tx=0x7fdae400ed10 comp rx=0 tx=0).stop 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 shutdown_connections 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdae00a3690 0x7fdae00a5b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdaf41069c0 0x7fdaf419bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdaf4101350 0x7fdaf4197760 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 --2- 172.21.15.37:0/1883297464 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaf41009b0 0x7fdaf4197220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 >> 172.21.15.37:0/1883297464 conn(0x7fdaf4077990 msgr2=0x7fdaf40fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 shutdown_connections 2024-09-19T06:42:50.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:42:50.852+0000 7fdafbca6640 1 -- 172.21.15.37:0/1883297464 wait complete. 2024-09-19T06:42:51.829 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:42:51.829 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:42:52.107 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:52.232 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:52 smithi037 ceph-mon[31211]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-19T06:42:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:52 smithi151 ceph-mon[35236]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-19T06:42:52.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:52 smithi096 ceph-mon[35004]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-19T06:42:52.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.679+0000 7f99d76fc640 1 -- 172.21.15.37:0/1833248555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0100e20 msgr2=0x7f99d0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.679+0000 7f99d76fc640 1 --2- 172.21.15.37:0/1833248555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0100e20 0x7f99d0101200 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f99cc009980 tx=0x7f99cc02e8b0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.680+0000 7f99d76fc640 1 -- 172.21.15.37:0/1833248555 shutdown_connections 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.680+0000 7f99d76fc640 1 --2- 172.21.15.37:0/1833248555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0102ef0 0x7f99d010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.680+0000 7f99d76fc640 1 --2- 172.21.15.37:0/1833248555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d0102570 0x7f99d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.680+0000 7f99d76fc640 1 --2- 172.21.15.37:0/1833248555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0100e20 0x7f99d0101200 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.680+0000 7f99d76fc640 1 -- 172.21.15.37:0/1833248555 >> 172.21.15.37:0/1833248555 conn(0x7f99d00fc7d0 msgr2=0x7f99d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.681+0000 7f99d76fc640 1 -- 172.21.15.37:0/1833248555 shutdown_connections 2024-09-19T06:42:52.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.681+0000 7f99d76fc640 1 -- 172.21.15.37:0/1833248555 wait complete. 2024-09-19T06:42:52.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 Processor -- start 2024-09-19T06:42:52.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 -- start start 2024-09-19T06:42:52.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0100e20 0x7f99d01a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0102570 0x7f99d01a4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d0102ef0 0x7f99d019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99d010fe80 con 0x7f99d0102570 2024-09-19T06:42:52.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99d010fd00 con 0x7f99d0100e20 2024-09-19T06:42:52.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d76fc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99d0110000 con 0x7f99d0102ef0 2024-09-19T06:42:52.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.682+0000 7f99d4c97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0102570 0x7f99d01a4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0102570 0x7f99d01a4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47316/0 (socket says 172.21.15.37:47316) 2024-09-19T06:42:52.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 -- 172.21.15.37:0/3715735691 learned_addr learned my addr 172.21.15.37:0/3715735691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:52.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d5498640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0100e20 0x7f99d01a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 -- 172.21.15.37:0/3715735691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d0102ef0 msgr2=0x7f99d019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d0102ef0 0x7f99d019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 -- 172.21.15.37:0/3715735691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0100e20 msgr2=0x7f99d01a47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0100e20 0x7f99d01a47b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 -- 172.21.15.37:0/3715735691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d019f0a0 con 0x7f99d0102570 2024-09-19T06:42:52.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.683+0000 7f99d4c97640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0102570 0x7f99d01a4cf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f99c000a8d0 tx=0x7f99c000ad90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:52.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.684+0000 7f99c67fc640 1 -- 172.21.15.37:0/3715735691 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00079b0 con 0x7f99d0102570 2024-09-19T06:42:52.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.684+0000 7f99c67fc640 1 -- 172.21.15.37:0/3715735691 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c0005500 con 0x7f99d0102570 2024-09-19T06:42:52.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.684+0000 7f99c67fc640 1 -- 172.21.15.37:0/3715735691 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c0014410 con 0x7f99d0102570 2024-09-19T06:42:52.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.684+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d019f390 con 0x7f99d0102570 2024-09-19T06:42:52.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.684+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d01ab680 con 0x7f99d0102570 2024-09-19T06:42:52.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.688+0000 7f99c67fc640 1 -- 172.21.15.37:0/3715735691 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f99c00145b0 con 0x7f99d0102570 2024-09-19T06:42:52.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.689+0000 7f99c67fc640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99bc0a3660 0x7f99bc0a5b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.690+0000 7f99d5498640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99bc0a3660 0x7f99bc0a5b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.690+0000 7f99d9ad5740 1 Processor -- start 2024-09-19T06:42:52.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99c67fc640 1 -- 172.21.15.37:0/3715735691 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f99c00cf370 con 0x7f99d0102570 2024-09-19T06:42:52.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d5498640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99bc0a3660 0x7f99bc0a5b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f99cc006010 tx=0x7f99cc035440 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:52.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d9ad5740 1 -- start start 2024-09-19T06:42:52.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a69547f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a69552800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a69555130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a694ad280 con 0x562a69547b60 2024-09-19T06:42:52.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a69454b10 con 0x562a69552d40 2024-09-19T06:42:52.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562a694a9240 con 0x562a69548480 2024-09-19T06:42:52.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d5c99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a69555130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d5498640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a69547f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d5c99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a69555130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44994/0 (socket says 172.21.15.37:44994) 2024-09-19T06:42:52.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.691+0000 7f99d5c99640 1 -- 172.21.15.37:0/1999843373 learned_addr learned my addr 172.21.15.37:0/1999843373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:52.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d4c97640 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a69552800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d5c99640 1 -- 172.21.15.37:0/1999843373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 msgr2=0x562a69552800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a69552800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d5c99640 1 -- 172.21.15.37:0/1999843373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 msgr2=0x562a69547f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a69547f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d5c99640 1 -- 172.21.15.37:0/1999843373 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a694611f0 con 0x562a69552d40 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d5498640 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a69547f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a69555130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009de0 tx=0x7f99c800d6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f0e5be1d32da406 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99bb7fe640 1 -- 172.21.15.37:0/1999843373 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8003b70 con 0x562a69552d40 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99bb7fe640 1 -- 172.21.15.37:0/1999843373 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c8003d10 con 0x562a69552d40 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.692+0000 7f99bb7fe640 1 -- 172.21.15.37:0/1999843373 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8010970 con 0x562a69552d40 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.693+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1999843373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 msgr2=0x562a69555130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.693+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a69555130 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99c8009de0 tx=0x7f99c800d6b0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.693+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1999843373 shutdown_connections 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.693+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a69555130 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.693+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a69552800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.693+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1999843373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a69547f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.693+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1999843373 >> 172.21.15.37:0/1999843373 conn(0x562a695440f0 msgr2=0x562a695444d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.694+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1999843373 shutdown_connections 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.694+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1999843373 wait complete. 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.694+0000 7f99d9ad5740 1 -- start start 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.694+0000 7f99d9ad5740 1 Processor -- start 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6954fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a695502d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562a69555910 con 0x562a69547b60 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a69467f10 con 0x562a69552d40 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562a69555a90 con 0x562a69548480 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d9ad5740 10 client.0 _ll_register_callbacks cb 0x562a69430a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d5c99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a695502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d5498640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d5c99640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a695502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:45004/0 (socket says 172.21.15.37:45004) 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d5c99640 1 -- 172.21.15.37:0/1919079150 learned_addr learned my addr 172.21.15.37:0/1919079150 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.695+0000 7f99d4c97640 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6954fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d5c99640 1 -- 172.21.15.37:0/1919079150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 msgr2=0x562a6954fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6954fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d5c99640 1 -- 172.21.15.37:0/1919079150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 msgr2=0x562a695527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d5c99640 1 -- 172.21.15.37:0/1919079150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a6921e6a0 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d5498640 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a695502d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005f40 tx=0x7f99c8003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ac0010a5c0b0667 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99b8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1919079150 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c8004430 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c8003d10 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.696+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1919079150 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f99c8010670 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.697+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1919079150 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99c8010890 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.697+0000 7f99d9ad5740 10 client.25352 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.697+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562a695614a0 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.697+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1919079150 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c8003d10 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1919079150 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f99c8022070 con 0x562a69552d40 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99d9ad5740 10 client.25352 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:52.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99d9ad5740 10 client.25352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99d9ad5740 20 client.25352 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a69562c10 0x562a69565090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99d9ad5740 4 client.25352 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a694ba7f0 con 0x562a69562c10 2024-09-19T06:42:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.698+0000 7f99d5498640 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a69562c10 0x562a69565090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.699+0000 7f99d5498640 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a69562c10 0x562a69565090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b645b6e65cdafd8 server_cookie=b03b6b92593bd1cf in_seq=0 out_seq=0 2024-09-19T06:42:52.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.699+0000 7f99b8ff9640 10 client.25352 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.699+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1919079150 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562a694ba7f0 con 0x562a69562c10 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.699+0000 7f99b8ff9640 10 client.25352 handle_command_reply: tid=0 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.699+0000 7f99d9ad5740 1 client.25352 shutdown 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 msgr2=0x562a695502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a695502d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005f40 tx=0x7f99c8003c20 comp rx=0 tx=0).stop 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 shutdown_connections 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a69562c10 0x562a69565090 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a695502d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6954fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1919079150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 >> 172.21.15.37:0/1919079150 conn(0x562a695440f0 msgr2=0x562a69549e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99b8ff9640 0 client.25352 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 shutdown_connections 2024-09-19T06:42:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1919079150 wait complete. 2024-09-19T06:42:52.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.700+0000 7f99d9ad5740 20 client.25352 trim_cache size 0 max 16384 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.735+0000 7f99d9ad5740 1 Processor -- start 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.735+0000 7f99d9ad5740 1 -- start start 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6956a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a6956aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99b4011090 con 0x562a69547b60 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99b400e960 con 0x562a69552d40 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99b400c7b0 con 0x562a69548480 2024-09-19T06:42:52.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5498640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5498640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47340/0 (socket says 172.21.15.37:47340) 2024-09-19T06:42:52.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5498640 1 -- 172.21.15.37:0/256496893 learned_addr learned my addr 172.21.15.37:0/256496893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:52.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5c99640 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a6956aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d4c97640 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6956a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5c99640 1 -- 172.21.15.37:0/256496893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 msgr2=0x562a6956a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5c99640 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6956a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5c99640 1 -- 172.21.15.37:0/256496893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 msgr2=0x562a695527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5c99640 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5c99640 1 -- 172.21.15.37:0/256496893 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a695482b0 con 0x562a69552d40 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.736+0000 7f99d5498640 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.737+0000 7f99d5c99640 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a6956aee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005f10 tx=0x7f99c8009d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39c2d4c9656a729 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.737+0000 7f99baffd640 1 -- 172.21.15.37:0/256496893 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c801ad80 con 0x562a69552d40 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.737+0000 7f99baffd640 1 -- 172.21.15.37:0/256496893 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99c80099e0 con 0x562a69552d40 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.737+0000 7f99baffd640 1 -- 172.21.15.37:0/256496893 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c801a670 con 0x562a69552d40 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 -- 172.21.15.37:0/256496893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 msgr2=0x562a6956aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a6956aee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f99c8005f10 tx=0x7f99c8009d10 comp rx=0 tx=0).stop 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 -- 172.21.15.37:0/256496893 shutdown_connections 2024-09-19T06:42:52.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69552d40 0x562a6956aee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69548480 0x562a6956a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/256496893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69547b60 0x562a695527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 -- 172.21.15.37:0/256496893 >> 172.21.15.37:0/256496893 conn(0x562a695440f0 msgr2=0x562a6954e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 -- 172.21.15.37:0/256496893 shutdown_connections 2024-09-19T06:42:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.738+0000 7f99d9ad5740 1 -- 172.21.15.37:0/256496893 wait complete. 2024-09-19T06:42:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.739+0000 7f99d9ad5740 1 -- start start 2024-09-19T06:42:52.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.739+0000 7f99d9ad5740 1 Processor -- start 2024-09-19T06:42:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.739+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69547b60 0x562a69556c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.739+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69548480 0x562a695571c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d9ad5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69552d40 0x562a69557700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99b400ce20 con 0x562a69548480 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99b400e290 con 0x562a69547b60 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d9ad5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99b400d4f0 con 0x562a69552d40 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69548480 0x562a695571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69548480 0x562a695571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47348/0 (socket says 172.21.15.37:47348) 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 -- 172.21.15.37:0/1930191492 learned_addr learned my addr 172.21.15.37:0/1930191492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d5498640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69547b60 0x562a69556c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 -- 172.21.15.37:0/1930191492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69552d40 msgr2=0x562a69557700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69552d40 0x562a69557700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69552d40 0x562a69557700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d9ad5740 10 client.0 _ll_register_callbacks cb 0x7f99c8009c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 -- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69547b60 msgr2=0x562a69556c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69547b60 0x562a69556c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d4c97640 1 -- 172.21.15.37:0/1930191492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a695ccb70 con 0x562a69548480 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d5c99640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69552d40 0x562a69557700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.740+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99c80099e0 con 0x562a69548480 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.741+0000 7f99d4c97640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69548480 0x562a695571c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f99c009b110 tx=0x7f99c009bc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a21199b9deb3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.741+0000 7f99b8ff9640 10 client.15768 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.741+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1930191492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c00b4060 con 0x562a69548480 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.741+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1930191492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f99c0007b50 con 0x562a69548480 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.741+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1930191492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99c0007d70 con 0x562a69548480 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.741+0000 7f99d9ad5740 10 client.15768 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.741+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=25}) -- 0x562a695cddb0 con 0x562a69548480 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1930191492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f99c009a860 con 0x562a69548480 2024-09-19T06:42:52.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d9ad5740 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d9ad5740 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d9ad5740 20 client.15768 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a695ce420 0x562a695d0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d9ad5740 4 client.15768 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a694b4d00 con 0x562a695ce420 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d5498640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a695ce420 0x562a695d0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99b8ff9640 10 client.15768 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:52.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.742+0000 7f99d5498640 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a695ce420 0x562a695d0840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846dcdd9781620c1 server_cookie=6c66442faa75bbaa in_seq=0 out_seq=0 2024-09-19T06:42:52.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.743+0000 7f99b8ff9640 1 -- 172.21.15.37:0/1930191492 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a694b4d00 con 0x562a695ce420 2024-09-19T06:42:52.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.743+0000 7f99b8ff9640 10 client.15768 handle_command_reply: tid=0 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.743+0000 7f99d9ad5740 1 client.15768 shutdown 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69548480 msgr2=0x562a695571c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69548480 0x562a695571c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f99c009b110 tx=0x7f99c009bc40 comp rx=0 tx=0).stop 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 shutdown_connections 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562a695ce420 0x562a695d0840 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562a69552d40 0x562a69557700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562a69548480 0x562a695571c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 --2- 172.21.15.37:0/1930191492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a69547b60 0x562a69556c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 >> 172.21.15.37:0/1930191492 conn(0x562a695440f0 msgr2=0x562a695495b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99b8ff9640 0 client.15768 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 shutdown_connections 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.744+0000 7f99d9ad5740 1 -- 172.21.15.37:0/1930191492 wait complete. 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.745+0000 7f99d9ad5740 20 client.15768 trim_cache size 0 max 16384 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "82dc0932-7b62-4588-9361-5e2e7b457957", 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:42:52.750 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.745+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99bc0a3660 msgr2=0x7f99bc0a5b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.745+0000 7f99d76fc640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99bc0a3660 0x7f99bc0a5b20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f99cc006010 tx=0x7f99cc035440 comp rx=0 tx=0).stop 2024-09-19T06:42:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.745+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0102570 msgr2=0x7f99d01a4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.745+0000 7f99d76fc640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0102570 0x7f99d01a4cf0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f99c000a8d0 tx=0x7f99c000ad90 comp rx=0 tx=0).stop 2024-09-19T06:42:52.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 shutdown_connections 2024-09-19T06:42:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99d0102ef0 0x7f99d019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99bc0a3660 0x7f99bc0a5b20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99d0102570 0x7f99d01a4cf0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 --2- 172.21.15.37:0/3715735691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99d0100e20 0x7f99d01a47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 >> 172.21.15.37:0/3715735691 conn(0x7f99d00fc7d0 msgr2=0x7f99d00fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 shutdown_connections 2024-09-19T06:42:52.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:52.746+0000 7f99d76fc640 1 -- 172.21.15.37:0/3715735691 wait complete. 2024-09-19T06:42:53.629 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:42:54.028 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:42:54.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:54 smithi037 ceph-mon[31211]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T06:42:54.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:54 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:42:54.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:54 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:54.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:54 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:42:54.296 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:54 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:54 smithi151 ceph-mon[35236]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T06:42:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:54 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:42:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:54 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:54 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:42:54.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:54 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:54 smithi096 ceph-mon[35004]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-09-19T06:42:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:54 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:42:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:54 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:54 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:42:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:54 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:42:54.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 -- 172.21.15.37:0/3800960108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 msgr2=0x7fea08106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 --2- 172.21.15.37:0/3800960108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 0x7fea08106e50 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fea04009900 tx=0x7fea0402a410 comp rx=0 tx=0).stop 2024-09-19T06:42:54.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 -- 172.21.15.37:0/3800960108 shutdown_connections 2024-09-19T06:42:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 --2- 172.21.15.37:0/3800960108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08101410 0x7fea0810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 --2- 172.21.15.37:0/3800960108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea08100a70 0x7fea08100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 --2- 172.21.15.37:0/3800960108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 0x7fea08106e50 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 -- 172.21.15.37:0/3800960108 >> 172.21.15.37:0/3800960108 conn(0x7fea080fc7d0 msgr2=0x7fea080febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 -- 172.21.15.37:0/3800960108 shutdown_connections 2024-09-19T06:42:54.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.539+0000 7fea0fffa640 1 -- 172.21.15.37:0/3800960108 wait complete. 2024-09-19T06:42:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.540+0000 7fea0fffa640 1 Processor -- start 2024-09-19T06:42:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.540+0000 7fea0fffa640 1 -- start start 2024-09-19T06:42:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0fffa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08100a70 0x7fea08198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0fffa640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea08101410 0x7fea08198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0fffa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 0x7fea0819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0fffa640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111f50 con 0x7fea08106a70 2024-09-19T06:42:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0dd96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08100a70 0x7fea08198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0e597640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 0x7fea0819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0dd96640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08100a70 0x7fea08198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:45032/0 (socket says 172.21.15.37:45032) 2024-09-19T06:42:54.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0dd96640 1 -- 172.21.15.37:0/1646198776 learned_addr learned my addr 172.21.15.37:0/1646198776 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea08111dd0 con 0x7fea08100a70 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.541+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea081120d0 con 0x7fea08101410 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.542+0000 7fea0dd96640 1 -- 172.21.15.37:0/1646198776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea08101410 msgr2=0x7fea08198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.542+0000 7fea0dd96640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea08101410 0x7fea08198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.542+0000 7fea0dd96640 1 -- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 msgr2=0x7fea0819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.542+0000 7fea0dd96640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 0x7fea0819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.542+0000 7fea0dd96640 1 -- 172.21.15.37:0/1646198776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0819d400 con 0x7fea08100a70 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.542+0000 7fea0dd96640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08100a70 0x7fea08198450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea04006010 tx=0x7fea04004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.542+0000 7fe9feffd640 1 -- 172.21.15.37:0/1646198776 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0403f030 con 0x7fea08100a70 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.543+0000 7fe9feffd640 1 -- 172.21.15.37:0/1646198776 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea04004440 con 0x7fea08100a70 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.543+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0819d690 con 0x7fea08100a70 2024-09-19T06:42:54.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.543+0000 7fe9feffd640 1 -- 172.21.15.37:0/1646198776 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04035070 con 0x7fea08100a70 2024-09-19T06:42:54.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.543+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea081a4fd0 con 0x7fea08100a70 2024-09-19T06:42:54.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.546+0000 7fe9feffd640 1 -- 172.21.15.37:0/1646198776 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7fea04042030 con 0x7fea08100a70 2024-09-19T06:42:54.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.546+0000 7fea123d3740 1 Processor -- start 2024-09-19T06:42:54.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.547+0000 7fea123d3740 1 -- start start 2024-09-19T06:42:54.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.547+0000 7fea123d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5360 0x55a909fd5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.547+0000 7fea123d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea123d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe4a90 0x55a909fe6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea123d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a909f3b2e0 con 0x55a909fe4a90 2024-09-19T06:42:54.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea123d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a909ee2ce0 con 0x55a909fd5360 2024-09-19T06:42:54.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea123d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a909f372a0 con 0x55a909fd5c80 2024-09-19T06:42:54.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fe9feffd640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9f40a35e0 0x7fe9f40a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea0dd96640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea0dd96640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:35652/0 (socket says 172.21.15.37:35652) 2024-09-19T06:42:54.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea0dd96640 1 -- 172.21.15.37:0/3592322688 learned_addr learned my addr 172.21.15.37:0/3592322688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:54.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fea0d595640 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe4a90 0x55a909fe6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.548+0000 7fe9feffd640 1 -- 172.21.15.37:0/1646198776 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fea040f0530 con 0x7fea08100a70 2024-09-19T06:42:54.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.549+0000 7fea0d595640 1 -- 172.21.15.37:0/3592322688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 msgr2=0x55a909fe4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:54.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.549+0000 7fea0e597640 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5360 0x55a909fd5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.549+0000 7fea0d595640 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.549+0000 7fea0d595640 1 -- 172.21.15.37:0/3592322688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5360 msgr2=0x55a909fd5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:54.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.549+0000 7fea0d595640 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5360 0x55a909fd5740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.549+0000 7fea0d595640 1 -- 172.21.15.37:0/3592322688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a909ec7440 con 0x55a909fe4a90 2024-09-19T06:42:54.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.550+0000 7fea0e597640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9f40a35e0 0x7fe9f40a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.550+0000 7fea0d595640 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe4a90 0x55a909fe6e80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009b60 tx=0x7fe9f800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1dd1a826e7de964 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:54.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.550+0000 7fe9f3fff640 1 -- 172.21.15.37:0/3592322688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8009e50 con 0x55a909fe4a90 2024-09-19T06:42:54.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.550+0000 7fe9f3fff640 1 -- 172.21.15.37:0/3592322688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8009e50 con 0x55a909fe4a90 2024-09-19T06:42:54.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.551+0000 7fe9f3fff640 1 -- 172.21.15.37:0/3592322688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80047a0 con 0x55a909fe4a90 2024-09-19T06:42:54.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea0e597640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9f40a35e0 0x7fe9f40a5aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fea00005c40 tx=0x7fea00015040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:54.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea123d3740 1 -- 172.21.15.37:0/3592322688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe4a90 msgr2=0x55a909fe6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea123d3740 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe4a90 0x55a909fe6e80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8009b60 tx=0x7fe9f800e720 comp rx=0 tx=0).stop 2024-09-19T06:42:54.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea123d3740 1 -- 172.21.15.37:0/3592322688 shutdown_connections 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea123d3740 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe4a90 0x55a909fe6e80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea123d3740 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea123d3740 1 --2- 172.21.15.37:0/3592322688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5360 0x55a909fd5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.552+0000 7fea123d3740 1 -- 172.21.15.37:0/3592322688 >> 172.21.15.37:0/3592322688 conn(0x55a909fd1a30 msgr2=0x55a909fd1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.553+0000 7fea123d3740 1 -- 172.21.15.37:0/3592322688 shutdown_connections 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.553+0000 7fea123d3740 1 -- 172.21.15.37:0/3592322688 wait complete. 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.553+0000 7fea123d3740 1 -- start start 2024-09-19T06:42:54.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.554+0000 7fea123d3740 1 Processor -- start 2024-09-19T06:42:54.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.554+0000 7fea123d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd5360 0x55a909fe4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.554+0000 7fea123d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea0dd96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd5360 0x55a909fe4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea0dd96640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd5360 0x55a909fe4420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47396/0 (socket says 172.21.15.37:47396) 2024-09-19T06:42:54.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea0dd96640 1 -- 172.21.15.37:0/2979224127 learned_addr learned my addr 172.21.15.37:0/2979224127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:54.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea123d3740 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fe1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a909fe7660 con 0x55a909fd5360 2024-09-19T06:42:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a909ef6050 con 0x55a909fe4a90 2024-09-19T06:42:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a909fe77e0 con 0x55a909fd5c80 2024-09-19T06:42:54.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea123d3740 10 client.0 _ll_register_callbacks cb 0x55a909f36160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.555+0000 7fea0d595640 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fea0dd96640 1 -- 172.21.15.37:0/2979224127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 msgr2=0x55a909fe16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fea0dd96640 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fea0dd96640 1 -- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 msgr2=0x55a909fe1c50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:42:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fea0dd96640 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fe1c50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fea0dd96640 1 -- 172.21.15.37:0/2979224127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a909cac3c0 con 0x55a909fd5360 2024-09-19T06:42:54.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fea0dd96640 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd5360 0x55a909fe4420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fea040bc390 tx=0x7fea040bc460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8671f9022f605607 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:54.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fe9f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:54.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fe9f17fa640 1 -- 172.21.15.37:0/2979224127 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04049070 con 0x55a909fd5360 2024-09-19T06:42:54.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fe9f17fa640 1 -- 172.21.15.37:0/2979224127 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fea040bc8f0 con 0x55a909fd5360 2024-09-19T06:42:54.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f8009e50 con 0x55a909fd5360 2024-09-19T06:42:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.556+0000 7fe9f17fa640 1 -- 172.21.15.37:0/2979224127 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea040bbff0 con 0x55a909fd5360 2024-09-19T06:42:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.557+0000 7fea123d3740 10 client.15780 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.557+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a90a0040d0 con 0x55a909fd5360 2024-09-19T06:42:54.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.557+0000 7fe9f17fa640 1 -- 172.21.15.37:0/2979224127 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8009e50 con 0x55a909fd5360 2024-09-19T06:42:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.558+0000 7fe9f17fa640 1 -- 172.21.15.37:0/2979224127 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea040c4e70 con 0x55a909fd5360 2024-09-19T06:42:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.558+0000 7fea123d3740 10 client.15780 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.558+0000 7fea123d3740 10 client.15780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.558+0000 7fea123d3740 20 client.15780 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:54.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.558+0000 7fea123d3740 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a0054c0 0x55a90a0078f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.558+0000 7fea123d3740 4 client.15780 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.558+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a909f48850 con 0x55a90a0054c0 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.559+0000 7fea0d595640 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a0054c0 0x55a90a0078f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.560+0000 7fea0d595640 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a0054c0 0x55a90a0078f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8305db15df0ebd server_cookie=ea27a6177977cb9d in_seq=0 out_seq=0 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.560+0000 7fe9f17fa640 10 client.15780 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fe9f17fa640 1 -- 172.21.15.37:0/2979224127 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a909f48850 con 0x55a90a0054c0 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fe9f17fa640 10 client.15780 handle_command_reply: tid=0 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 client.15780 shutdown 2024-09-19T06:42:54.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd5360 msgr2=0x55a909fe4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd5360 0x55a909fe4420 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fea040bc390 tx=0x7fea040bc460 comp rx=0 tx=0).stop 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 shutdown_connections 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a0054c0 0x55a90a0078f0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fe1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909fe16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 --2- 172.21.15.37:0/2979224127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd5360 0x55a909fe4420 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fe9f17fa640 0 client.15780 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 >> 172.21.15.37:0/2979224127 conn(0x55a909fd1a30 msgr2=0x55a909fd7790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 shutdown_connections 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 1 -- 172.21.15.37:0/2979224127 wait complete. 2024-09-19T06:42:54.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.561+0000 7fea123d3740 20 client.15780 trim_cache size 0 max 16384 2024-09-19T06:42:54.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.595+0000 7fea123d3740 1 Processor -- start 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.595+0000 7fea123d3740 1 -- start start 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.595+0000 7fea123d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909ff0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.595+0000 7fea123d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fd0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea123d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd0fa0 0x55a90a00d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea123d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea081a55d0 con 0x55a909fd0fa0 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea123d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea08072120 con 0x55a909fe4a90 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea123d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea08070020 con 0x55a909fd5c80 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea0e597640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd0fa0 0x55a90a00d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea0e597640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd0fa0 0x55a90a00d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47410/0 (socket says 172.21.15.37:47410) 2024-09-19T06:42:54.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea0e597640 1 -- 172.21.15.37:0/2964095966 learned_addr learned my addr 172.21.15.37:0/2964095966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea0d595640 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fd0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.596+0000 7fea0dd96640 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909ff0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0d595640 1 -- 172.21.15.37:0/2964095966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 msgr2=0x55a909ff0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0d595640 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909ff0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0d595640 1 -- 172.21.15.37:0/2964095966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd0fa0 msgr2=0x55a90a00d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0d595640 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd0fa0 0x55a90a00d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0d595640 1 -- 172.21.15.37:0/2964095966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a909fe0680 con 0x55a909fe4a90 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0e597640 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd0fa0 0x55a90a00d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0dd96640 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909ff0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.597+0000 7fea0d595640 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fd0a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000c00 tx=0x7fe9f80056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ab0de7098f5313b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:54.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.598+0000 7fe9f37fe640 1 -- 172.21.15.37:0/2964095966 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8032070 con 0x55a909fe4a90 2024-09-19T06:42:54.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.598+0000 7fe9f37fe640 1 -- 172.21.15.37:0/2964095966 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8004520 con 0x55a909fe4a90 2024-09-19T06:42:54.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.598+0000 7fe9f37fe640 1 -- 172.21.15.37:0/2964095966 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f80047a0 con 0x55a909fe4a90 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.598+0000 7fea123d3740 1 -- 172.21.15.37:0/2964095966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 msgr2=0x55a909fd0a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.598+0000 7fea123d3740 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fd0a60 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8000c00 tx=0x7fe9f80056f0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 -- 172.21.15.37:0/2964095966 shutdown_connections 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fd0fa0 0x55a90a00d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fe4a90 0x55a909fd0a60 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 --2- 172.21.15.37:0/2964095966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd5c80 0x55a909ff0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 -- 172.21.15.37:0/2964095966 >> 172.21.15.37:0/2964095966 conn(0x55a909fd1a30 msgr2=0x55a909fd0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 -- 172.21.15.37:0/2964095966 shutdown_connections 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 -- 172.21.15.37:0/2964095966 wait complete. 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 -- start start 2024-09-19T06:42:54.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.599+0000 7fea123d3740 1 Processor -- start 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea123d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd0fa0 0x55a909fed330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea123d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5c80 0x55a909fe85c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea123d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe8b50 0x55a909fe8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea123d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fea080705e0 con 0x55a909fe8b50 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea123d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea08071a50 con 0x55a909fd5c80 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea123d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fea08070cb0 con 0x55a909fd0fa0 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea123d3740 10 client.0 _ll_register_callbacks cb 0x7fea04009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe8b50 0x55a909fe8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe8b50 0x55a909fe8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:47418/0 (socket says 172.21.15.37:47418) 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 -- 172.21.15.37:0/680200639 learned_addr learned my addr 172.21.15.37:0/680200639 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 -- 172.21.15.37:0/680200639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd0fa0 msgr2=0x55a909fed330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd0fa0 0x55a909fed330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 -- 172.21.15.37:0/680200639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5c80 msgr2=0x55a909fe85c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5c80 0x55a909fe85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.600+0000 7fea0e597640 1 -- 172.21.15.37:0/680200639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a90a06b3e0 con 0x55a909fe8b50 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fea0e597640 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe8b50 0x55a909fe8fb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fea00010ab0 tx=0x7fea00009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad207e842c24dcee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fe9f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fe9f17fa640 1 -- 172.21.15.37:0/680200639 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0001bdf0 con 0x55a909fe8b50 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9f8004520 con 0x55a909fe8b50 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fe9f17fa640 1 -- 172.21.15.37:0/680200639 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fea00004b20 con 0x55a909fe8b50 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fe9f17fa640 1 -- 172.21.15.37:0/680200639 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea00004d20 con 0x55a909fe8b50 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fea123d3740 10 client.15786 fetch_fsmap learned FSMap version 25 2024-09-19T06:42:54.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a90a06c6f0 con 0x55a909fe8b50 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.601+0000 7fe9f17fa640 1 -- 172.21.15.37:0/680200639 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe9f8004520 con 0x55a909fe8b50 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.602+0000 7fe9f17fa640 1 -- 172.21.15.37:0/680200639 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fea00027070 con 0x55a909fe8b50 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.602+0000 7fea123d3740 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.602+0000 7fea123d3740 10 client.15786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.602+0000 7fea123d3740 20 client.15786 populate_metadata read hostname 'smithi037' 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.602+0000 7fea123d3740 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a06ce40 0x55a90a06f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.602+0000 7fea123d3740 4 client.15786 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.602+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a909f42d60 con 0x55a90a06ce40 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.603+0000 7fea0dd96640 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a06ce40 0x55a90a06f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:42:54.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.604+0000 7fea0dd96640 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a06ce40 0x55a90a06f2e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acb9d7c80726762d server_cookie=da764de730136bff in_seq=0 out_seq=0 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.604+0000 7fe9f17fa640 10 client.15786 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.604+0000 7fe9f17fa640 1 -- 172.21.15.37:0/680200639 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a909f42d60 con 0x55a90a06ce40 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.604+0000 7fe9f17fa640 10 client.15786 handle_command_reply: tid=0 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.604+0000 7fea123d3740 1 client.15786 shutdown 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.604+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe8b50 msgr2=0x55a909fe8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.604+0000 7fea123d3740 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe8b50 0x55a909fe8fb0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fea00010ab0 tx=0x7fea00009500 comp rx=0 tx=0).stop 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 shutdown_connections 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a90a06ce40 0x55a90a06f2e0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a909fe8b50 0x55a909fe8fb0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a909fd5c80 0x55a909fe85c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fe9f17fa640 0 client.15786 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 --2- 172.21.15.37:0/680200639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a909fd0fa0 0x55a909fed330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 >> 172.21.15.37:0/680200639 conn(0x55a909fd1a30 msgr2=0x55a909fd6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 shutdown_connections 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 1 -- 172.21.15.37:0/680200639 wait complete. 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.605+0000 7fea123d3740 20 client.15786 trim_cache size 0 max 16384 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "82dc0932-7b62-4588-9361-5e2e7b457957": { 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:42:54.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "82dc0932-7b62-4588-9361-5e2e7b457957", 2024-09-19T06:42:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:42:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:54.610 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:42:54.610 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:42:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9f40a35e0 msgr2=0x7fe9f40a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9f40a35e0 0x7fe9f40a5aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fea00005c40 tx=0x7fea00015040 comp rx=0 tx=0).stop 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08100a70 msgr2=0x7fea08198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08100a70 0x7fea08198450 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea04006010 tx=0x7fea04004040 comp rx=0 tx=0).stop 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 shutdown_connections 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9f40a35e0 0x7fe9f40a5aa0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fea08106a70 0x7fea0819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fea08101410 0x7fea08198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 --2- 172.21.15.37:0/1646198776 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08100a70 0x7fea08198450 secure :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fea04006010 tx=0x7fea04004040 comp rx=0 tx=0).stop 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.606+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 >> 172.21.15.37:0/1646198776 conn(0x7fea080fc7d0 msgr2=0x7fea080fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.607+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 shutdown_connections 2024-09-19T06:42:54.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:42:54.607+0000 7fea0fffa640 1 -- 172.21.15.37:0/1646198776 wait complete. 2024-09-19T06:42:55.428 INFO:tasks.cephfs.filesystem:scrub status for tag:82dc0932-7b62-4588-9361-5e2e7b457957 - {'path': '/', 'tag': '82dc0932-7b62-4588-9361-5e2e7b457957', 'options': 'recursive,force'} 2024-09-19T06:42:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:56 smithi151 ceph-mon[35236]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T06:42:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:56 smithi096 ceph-mon[35004]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T06:42:56.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:56 smithi037 ceph-mon[31211]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T06:42:57.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:57 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:42:57.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:57 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:57.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:57 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:42:57.502 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:57 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:57.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:57 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:42:57.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:57 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:42:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:58 smithi096 ceph-mon[35004]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T06:42:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:42:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:58 smithi037 ceph-mon[31211]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T06:42:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:42:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:58 smithi151 ceph-mon[35236]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-19T06:42:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:42:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:42:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:42:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:42:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:00 smithi096 ceph-mon[35004]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T06:43:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:00 smithi037 ceph-mon[31211]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T06:43:00.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:00 smithi151 ceph-mon[35236]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 341 B/s wr, 9 op/s 2024-09-19T06:43:02.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:02 smithi096 ceph-mon[35004]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T06:43:02.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:02 smithi037 ceph-mon[31211]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T06:43:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:02 smithi151 ceph-mon[35236]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 -- 172.21.15.37:0/2677758110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7514074ec0 msgr2=0x7f751410de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 --2- 172.21.15.37:0/2677758110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7514074ec0 0x7f751410de50 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f7504009900 tx=0x7f750402a480 comp rx=0 tx=0).stop 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 -- 172.21.15.37:0/2677758110 shutdown_connections 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 --2- 172.21.15.37:0/2677758110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f751410e390 0x7f7514110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 --2- 172.21.15.37:0/2677758110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7514074ec0 0x7f751410de50 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 --2- 172.21.15.37:0/2677758110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75140745a0 0x7f7514074980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 -- 172.21.15.37:0/2677758110 >> 172.21.15.37:0/2677758110 conn(0x7f75140fd7d0 msgr2=0x7f75140ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 -- 172.21.15.37:0/2677758110 shutdown_connections 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.979+0000 7f751c227640 1 -- 172.21.15.37:0/2677758110 wait complete. 2024-09-19T06:43:03.983 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 Processor -- start 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 -- start start 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75140745a0 0x7f751419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f751410e390 0x7f751419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751419f010 0x7f751419f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75140745a0 0x7f751419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75140745a0 0x7f751419a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44304/0 (socket says 172.21.15.37:44304) 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 -- 172.21.15.37:0/3102022824 learned_addr learned my addr 172.21.15.37:0/3102022824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7514114080 con 0x7f75140745a0 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7514113f00 con 0x7f751419f010 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7514114200 con 0x7f751410e390 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f75197c2640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f751410e390 0x7f751419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f751a7c4640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751419f010 0x7f751419f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.984 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 -- 172.21.15.37:0/3102022824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f751410e390 msgr2=0x7f751419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f751410e390 0x7f751419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 -- 172.21.15.37:0/3102022824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751419f010 msgr2=0x7f751419f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751419f010 0x7f751419f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.980+0000 7f7519fc3640 1 -- 172.21.15.37:0/3102022824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751419fb00 con 0x7f75140745a0 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.981+0000 7f7519fc3640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75140745a0 0x7f751419a750 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f751000d950 tx=0x7f751000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.981+0000 7f750affd640 1 -- 172.21.15.37:0/3102022824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510017070 con 0x7f75140745a0 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.981+0000 7f750affd640 1 -- 172.21.15.37:0/3102022824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7510009c30 con 0x7f75140745a0 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.981+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751419fdf0 con 0x7f75140745a0 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.981+0000 7f750affd640 1 -- 172.21.15.37:0/3102022824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751000f6b0 con 0x7f75140745a0 2024-09-19T06:43:03.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.981+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75141a7600 con 0x7f75140745a0 2024-09-19T06:43:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.983+0000 7f750affd640 1 -- 172.21.15.37:0/3102022824 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f75100109d0 con 0x7f75140745a0 2024-09-19T06:43:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.984+0000 7f750affd640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75000a3690 0x7f75000a5b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.984+0000 7f751efc4740 1 Processor -- start 2024-09-19T06:43:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.984+0000 7f750affd640 1 -- 172.21.15.37:0/3102022824 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f75100cfb90 con 0x7f75140745a0 2024-09-19T06:43:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f75197c2640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75000a3690 0x7f75000a5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751efc4740 1 -- start start 2024-09-19T06:43:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751efc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faecaf0 0x55bb8faeced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751efc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faed410 0x55bb8faf7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751efc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751efc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bb8fa512f0 con 0x55bb8faf7cb0 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751efc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bb8f9f9270 con 0x55bb8faed410 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751efc4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bb8fa4d2b0 con 0x55bb8faecaf0 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f75197c2640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75000a3690 0x7f75000a5b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7504005940 tx=0x7f75040058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751a7c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751a7c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44316/0 (socket says 172.21.15.37:44316) 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f751a7c4640 1 -- 172.21.15.37:0/3465420354 learned_addr learned my addr 172.21.15.37:0/3465420354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f75197c2640 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faed410 0x55bb8faf7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.985+0000 7f7519fc3640 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faecaf0 0x55bb8faeced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f75197c2640 1 -- 172.21.15.37:0/3465420354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faecaf0 msgr2=0x55bb8faeced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f75197c2640 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faecaf0 0x55bb8faeced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f75197c2640 1 -- 172.21.15.37:0/3465420354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 msgr2=0x55bb8fafa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f75197c2640 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f75197c2640 1 -- 172.21.15.37:0/3465420354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb8f9f8340 con 0x55bb8faed410 2024-09-19T06:43:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f751a7c4640 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f75197c2640 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faed410 0x55bb8faf7770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7504008000 tx=0x7f750403d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f9edc702eba0e94 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f74fffff640 1 -- 172.21.15.37:0/3465420354 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750404d070 con 0x55bb8faed410 2024-09-19T06:43:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f74fffff640 1 -- 172.21.15.37:0/3465420354 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504005520 con 0x55bb8faed410 2024-09-19T06:43:03.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.986+0000 7f74fffff640 1 -- 172.21.15.37:0/3465420354 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7504035040 con 0x55bb8faed410 2024-09-19T06:43:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 -- 172.21.15.37:0/3465420354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faed410 msgr2=0x55bb8faf7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faed410 0x55bb8faf7770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7504008000 tx=0x7f750403d000 comp rx=0 tx=0).stop 2024-09-19T06:43:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 -- 172.21.15.37:0/3465420354 shutdown_connections 2024-09-19T06:43:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faed410 0x55bb8faf7770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 --2- 172.21.15.37:0/3465420354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faecaf0 0x55bb8faeced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 -- 172.21.15.37:0/3465420354 >> 172.21.15.37:0/3465420354 conn(0x55bb8fae9060 msgr2=0x55bb8fae9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 -- 172.21.15.37:0/3465420354 shutdown_connections 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.987+0000 7f751efc4740 1 -- 172.21.15.37:0/3465420354 wait complete. 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.988+0000 7f751efc4740 1 -- start start 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.988+0000 7f751efc4740 1 Processor -- start 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.988+0000 7f751efc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faf4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.988+0000 7f751efc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.988+0000 7f751efc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44332/0 (socket says 172.21.15.37:44332) 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f7519fc3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faf4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 -- 172.21.15.37:0/1843786814 learned_addr learned my addr 172.21.15.37:0/1843786814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bb8fafa880 con 0x55bb8faf7cb0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bb8f9da290 con 0x55bb8faecaf0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bb8fafaa00 con 0x55bb8faed410 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f75197c2640 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 -- 172.21.15.37:0/1843786814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 msgr2=0x55bb8faf50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 -- 172.21.15.37:0/1843786814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 msgr2=0x55bb8faf4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faf4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 -- 172.21.15.37:0/1843786814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb8f764210 con 0x55bb8faf7cb0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751a7c4640 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5610 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f750c012ad0 tx=0x7f750c012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5df9e8ba0c561c68 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751efc4740 10 client.0 _ll_register_callbacks cb 0x55bb8fa0b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.989+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7504005520 con 0x55bb8faf7cb0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.990+0000 7f74fd7fa640 10 client.15798 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.990+0000 7f74fd7fa640 1 -- 172.21.15.37:0/1843786814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c010e50 con 0x55bb8faf7cb0 2024-09-19T06:43:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.990+0000 7f74fd7fa640 1 -- 172.21.15.37:0/1843786814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f750c01c3f0 con 0x55bb8faf7cb0 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.990+0000 7f74fd7fa640 1 -- 172.21.15.37:0/1843786814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f750c01c630 con 0x55bb8faf7cb0 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.990+0000 7f751efc4740 10 client.15798 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.990+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55bb8f830620 con 0x55bb8faf7cb0 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.990+0000 7f74fd7fa640 1 -- 172.21.15.37:0/1843786814 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f750c00f040 con 0x55bb8faf7cb0 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f751efc4740 10 client.15798 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f751efc4740 10 client.15798 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f751efc4740 20 client.15798 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f751efc4740 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb07a70 0x55bb8fb09f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f751efc4740 4 client.15798 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb8fa5ea20 con 0x55bb8fb07a70 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f7519fc3640 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb07a70 0x55bb8fb09f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f74fd7fa640 10 client.15798 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:03.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.991+0000 7f7519fc3640 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb07a70 0x55bb8fb09f10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c683bd95cc1ba9d4 server_cookie=df0769d943531fa in_seq=0 out_seq=0 2024-09-19T06:43:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f74fd7fa640 1 -- 172.21.15.37:0/1843786814 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bb8fa5ea20 con 0x55bb8fb07a70 2024-09-19T06:43:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f74fd7fa640 10 client.15798 handle_command_reply: tid=0 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 client.15798 shutdown 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 msgr2=0x55bb8faf5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5610 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f750c012ad0 tx=0x7f750c012f90 comp rx=0 tx=0).stop 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 shutdown_connections 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb07a70 0x55bb8fb09f10 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f74fd7fa640 0 client.15798 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5610 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 --2- 172.21.15.37:0/1843786814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faf4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.992+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 >> 172.21.15.37:0/1843786814 conn(0x55bb8fae9060 msgr2=0x55bb8faeed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.993+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 shutdown_connections 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.993+0000 7f751efc4740 1 -- 172.21.15.37:0/1843786814 wait complete. 2024-09-19T06:43:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:03.993+0000 7f751efc4740 20 client.15798 trim_cache size 0 max 16384 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 Processor -- start 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 -- start start 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75141b2820 con 0x55bb8faf7cb0 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75141b00f0 con 0x55bb8faecaf0 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751efc4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75141adf40 con 0x55bb8faed410 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751a7c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751a7c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44348/0 (socket says 172.21.15.37:44348) 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.007+0000 7f751a7c4640 1 -- 172.21.15.37:0/729640346 learned_addr learned my addr 172.21.15.37:0/729640346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:04.010 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f75197c2640 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f7519fc3640 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f751a7c4640 1 -- 172.21.15.37:0/729640346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 msgr2=0x55bb8faf50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f751a7c4640 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f751a7c4640 1 -- 172.21.15.37:0/729640346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 msgr2=0x55bb8faef560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f751a7c4640 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faef560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f751a7c4640 1 -- 172.21.15.37:0/729640346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb8faec870 con 0x55bb8faf7cb0 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f751a7c4640 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5600 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f750c002230 tx=0x7f750c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a8e1c5245947f5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f74ff7fe640 1 -- 172.21.15.37:0/729640346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c010c50 con 0x55bb8faf7cb0 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f74ff7fe640 1 -- 172.21.15.37:0/729640346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504005520 con 0x55bb8faf7cb0 2024-09-19T06:43:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.008+0000 7f74ff7fe640 1 -- 172.21.15.37:0/729640346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f750c01c7d0 con 0x55bb8faf7cb0 2024-09-19T06:43:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 -- 172.21.15.37:0/729640346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 msgr2=0x55bb8faf5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5600 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f750c002230 tx=0x7f750c015000 comp rx=0 tx=0).stop 2024-09-19T06:43:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 -- 172.21.15.37:0/729640346 shutdown_connections 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faf7cb0 0x55bb8faf5600 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8faf50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 --2- 172.21.15.37:0/729640346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faecaf0 0x55bb8faef560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 -- 172.21.15.37:0/729640346 >> 172.21.15.37:0/729640346 conn(0x55bb8fae9060 msgr2=0x55bb8faf6c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 -- 172.21.15.37:0/729640346 shutdown_connections 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.009+0000 7f751efc4740 1 -- 172.21.15.37:0/729640346 wait complete. 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.010+0000 7f751efc4740 1 -- start start 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.010+0000 7f751efc4740 1 Processor -- start 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.010+0000 7f751efc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faecaf0 0x55bb8fb005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.010+0000 7f751efc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8fafb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.010+0000 7f751efc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f751efc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75141ae5b0 con 0x55bb8faecaf0 2024-09-19T06:43:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f751efc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75141afa20 con 0x55bb8faf7cb0 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f751efc4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75141aec80 con 0x55bb8faed410 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f751efc4740 10 client.0 _ll_register_callbacks cb 0x7f750c009aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faecaf0 0x55bb8fb005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faecaf0 0x55bb8fb005a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44356/0 (socket says 172.21.15.37:44356) 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 -- 172.21.15.37:0/2463917547 learned_addr learned my addr 172.21.15.37:0/2463917547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f75197c2640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8fafb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f751a7c4640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 -- 172.21.15.37:0/2463917547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 msgr2=0x55bb8fafb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8fafb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 -- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faf7cb0 msgr2=0x55bb8fafc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 -- 172.21.15.37:0/2463917547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb8fb717f0 con 0x55bb8faecaf0 2024-09-19T06:43:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f751a7c4640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f7519fc3640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faecaf0 0x55bb8fb005a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f751009bbc0 tx=0x7f7510010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e158c85912332e03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f74fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.011+0000 7f74fd7fa640 1 -- 172.21.15.37:0/2463917547 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7510020040 con 0x55bb8faecaf0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f74fd7fa640 1 -- 172.21.15.37:0/2463917547 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f751009b080 con 0x55bb8faecaf0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7504005520 con 0x55bb8faecaf0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f74fd7fa640 1 -- 172.21.15.37:0/2463917547 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75100a48a0 con 0x55bb8faecaf0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 10 client.15804 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55bb8fb72a00 con 0x55bb8faecaf0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f74fd7fa640 1 -- 172.21.15.37:0/2463917547 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7504005520 con 0x55bb8faecaf0 2024-09-19T06:43:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f74fd7fa640 1 -- 172.21.15.37:0/2463917547 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f75100ba300 con 0x55bb8faecaf0 2024-09-19T06:43:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 20 client.15804 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb73120 0x55bb8fb75510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 4 client.15804 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:43:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.012+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bb8fa58d70 con 0x55bb8fb73120 2024-09-19T06:43:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.013+0000 7f75197c2640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb73120 0x55bb8fb75510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.013+0000 7f75197c2640 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb73120 0x55bb8fb75510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=866f75386535dfa0 server_cookie=b6010d7ba636e014 in_seq=0 out_seq=0 2024-09-19T06:43:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.013+0000 7f74fd7fa640 10 client.15804 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.019+0000 7f74fd7fa640 1 -- 172.21.15.37:0/2463917547 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bb8fa58d70 con 0x55bb8fb73120 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.019+0000 7f74fd7fa640 10 client.15804 handle_command_reply: tid=0 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.019+0000 7f751efc4740 1 client.15804 shutdown 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faecaf0 msgr2=0x55bb8fb005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faecaf0 0x55bb8fb005a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f751009bbc0 tx=0x7f7510010d70 comp rx=0 tx=0).stop 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 shutdown_connections 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bb8fb73120 0x55bb8fb75510 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bb8faf7cb0 0x55bb8fafc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f74fd7fa640 0 client.15804 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:04.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bb8faed410 0x55bb8fafb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 --2- 172.21.15.37:0/2463917547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bb8faecaf0 0x55bb8fb005a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 >> 172.21.15.37:0/2463917547 conn(0x55bb8fae9060 msgr2=0x55bb8fae79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 shutdown_connections 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 1 -- 172.21.15.37:0/2463917547 wait complete. 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.020+0000 7f751efc4740 20 client.15804 trim_cache size 0 max 16384 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:43:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:43:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75000a3690 msgr2=0x7f75000a5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75000a3690 0x7f75000a5b50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7504005940 tx=0x7f75040058d0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75140745a0 msgr2=0x7f751419a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75140745a0 0x7f751419a750 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f751000d950 tx=0x7f751000de10 comp rx=0 tx=0).stop 2024-09-19T06:43:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 shutdown_connections 2024-09-19T06:43:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f751419f010 0x7f751419f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f751410e390 0x7f751419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75000a3690 0x7f75000a5b50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 --2- 172.21.15.37:0/3102022824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75140745a0 0x7f751419a750 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 >> 172.21.15.37:0/3102022824 conn(0x7f75140fd7d0 msgr2=0x7f75140fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 shutdown_connections 2024-09-19T06:43:04.025 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:04.021+0000 7f751c227640 1 -- 172.21.15.37:0/3102022824 wait complete. 2024-09-19T06:43:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:04 smithi096 ceph-mon[35004]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T06:43:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:04 smithi037 ceph-mon[31211]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T06:43:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:04 smithi151 ceph-mon[35236]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-09-19T06:43:06.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:06 smithi096 ceph-mon[35004]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-19T06:43:06.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:06 smithi037 ceph-mon[31211]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-19T06:43:06.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:06 smithi151 ceph-mon[35236]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-19T06:43:08.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:08 smithi096 ceph-mon[35004]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-19T06:43:08.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:08 smithi037 ceph-mon[31211]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-19T06:43:08.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:08 smithi151 ceph-mon[35236]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-09-19T06:43:08.656 INFO:tasks.quiescer.fs.[cephfs]:Total duration (40.7) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:43:08.656 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T06:43:08.657 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:43:08.922 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:09.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 -- 172.21.15.37:0/3083013190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9840074620 msgr2=0x7f9840074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:09.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 --2- 172.21.15.37:0/3083013190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9840074620 0x7f9840074a00 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f983c009900 tx=0x7f983c02a480 comp rx=0 tx=0).stop 2024-09-19T06:43:09.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 -- 172.21.15.37:0/3083013190 shutdown_connections 2024-09-19T06:43:09.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 --2- 172.21.15.37:0/3083013190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f984010e3c0 0x7f9840110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 --2- 172.21.15.37:0/3083013190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840074f40 0x7f984010de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 --2- 172.21.15.37:0/3083013190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9840074620 0x7f9840074a00 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 -- 172.21.15.37:0/3083013190 >> 172.21.15.37:0/3083013190 conn(0x7f98400fd8c0 msgr2=0x7f98400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 -- 172.21.15.37:0/3083013190 shutdown_connections 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.490+0000 7f9847d8a640 1 -- 172.21.15.37:0/3083013190 wait complete. 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 Processor -- start 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 -- start start 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840074620 0x7f984019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9840074f40 0x7f984019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:09.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984010e3c0 0x7f984019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9840114150 con 0x7f984010e3c0 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9840113fd0 con 0x7f9840074620 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9847d8a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98401142d0 con 0x7f9840074f40 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9846327640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984010e3c0 0x7f984019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9846327640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984010e3c0 0x7f984019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55284/0 (socket says 172.21.15.37:55284) 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9846327640 1 -- 172.21.15.37:0/2029319974 learned_addr learned my addr 172.21.15.37:0/2029319974 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9845325640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9840074f40 0x7f984019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.491+0000 7f9845b26640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840074620 0x7f984019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9846327640 1 -- 172.21.15.37:0/2029319974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9840074f40 msgr2=0x7f984019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9846327640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9840074f40 0x7f984019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9846327640 1 -- 172.21.15.37:0/2029319974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840074620 msgr2=0x7f984019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9846327640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840074620 0x7f984019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9846327640 1 -- 172.21.15.37:0/2029319974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984019f6c0 con 0x7f984010e3c0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9845325640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9840074f40 0x7f984019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9846327640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984010e3c0 0x7f984019efe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f983800eb10 tx=0x7f983800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9832ffd640 1 -- 172.21.15.37:0/2029319974 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838004b20 con 0x7f984010e3c0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9832ffd640 1 -- 172.21.15.37:0/2029319974 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9838004cc0 con 0x7f984010e3c0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984019f9b0 con 0x7f984010e3c0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9832ffd640 1 -- 172.21.15.37:0/2029319974 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838010760 con 0x7f984010e3c0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.492+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98401a7290 con 0x7f984010e3c0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.493+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9840074a00 con 0x7f984010e3c0 2024-09-19T06:43:09.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.495+0000 7f9832ffd640 1 -- 172.21.15.37:0/2029319974 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f9838005000 con 0x7f984010e3c0 2024-09-19T06:43:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.495+0000 7f9832ffd640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f982c0a3560 0x7f982c0a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.496+0000 7f9832ffd640 1 -- 172.21.15.37:0/2029319974 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f98380d0700 con 0x7f984010e3c0 2024-09-19T06:43:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.496+0000 7f9845b26640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f982c0a3560 0x7f982c0a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.496+0000 7f9845b26640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f982c0a3560 0x7f982c0a5a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f983c009230 tx=0x7f983c0058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:09.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.499+0000 7f9832ffd640 1 -- 172.21.15.37:0/2029319974 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9838054210 con 0x7f984010e3c0 2024-09-19T06:43:09.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.712+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f984019ba70 con 0x7f984010e3c0 2024-09-19T06:43:09.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.713+0000 7f9832ffd640 1 -- 172.21.15.37:0/2029319974 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7f984019ba70 con 0x7f984010e3c0 2024-09-19T06:43:09.715 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:43:09.715 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:43:09.715 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:43:09.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f982c0a3560 msgr2=0x7f982c0a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:09.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f982c0a3560 0x7f982c0a5a20 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f983c009230 tx=0x7f983c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984010e3c0 msgr2=0x7f984019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:09.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984010e3c0 0x7f984019efe0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f983800eb10 tx=0x7f983800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 shutdown_connections 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f982c0a3560 0x7f982c0a5a20 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f984010e3c0 0x7f984019efe0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9840074f40 0x7f984019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 --2- 172.21.15.37:0/2029319974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9840074620 0x7f984019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 >> 172.21.15.37:0/2029319974 conn(0x7f98400fd8c0 msgr2=0x7f98400fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 shutdown_connections 2024-09-19T06:43:09.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:09.715+0000 7f9847d8a640 1 -- 172.21.15.37:0/2029319974 wait complete. 2024-09-19T06:43:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:10 smithi037 ceph-mon[31211]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:43:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:10 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2029319974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:10.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:10 smithi096 ceph-mon[35004]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:43:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:10 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2029319974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:10.530 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T06:43:10.530 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T06:43:10.564 INFO:tasks.workunit.client.0.smithi037.stderr:Updating files: 87% (11258/12896) Updating files: 88% (11349/12896) Updating files: 89% (11478/12896) Updating files: 90% (11607/12896) Updating files: 91% (11736/12896) Updating files: 92% (11865/12896) Updating files: 93% (11994/12896) Updating files: 94% (12123/12896) Updating files: 95% (12252/12896) Updating files: 96% (12381/12896) Updating files: 97% (12510/12896) Updating files: 98% (12639/12896) Updating files: 99% (12768/12896) Updating files: 100% (12896/12896) Updating files: 100% (12896/12896), done. 2024-09-19T06:43:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:10 smithi151 ceph-mon[35236]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-09-19T06:43:10.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:10 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2029319974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:10.797 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 -- 172.21.15.37:0/2496677973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410102ef0 msgr2=0x7f741010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 --2- 172.21.15.37:0/2496677973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410102ef0 0x7f741010da40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7408009ca0 tx=0x7f740802a950 comp rx=0 tx=0).stop 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 -- 172.21.15.37:0/2496677973 shutdown_connections 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 --2- 172.21.15.37:0/2496677973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410102ef0 0x7f741010da40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 --2- 172.21.15.37:0/2496677973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7410102570 0x7f74101029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 --2- 172.21.15.37:0/2496677973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7410100e20 0x7f7410101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 -- 172.21.15.37:0/2496677973 >> 172.21.15.37:0/2496677973 conn(0x7f74100fc7d0 msgr2=0x7f74100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 -- 172.21.15.37:0/2496677973 shutdown_connections 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.307+0000 7f7416837640 1 -- 172.21.15.37:0/2496677973 wait complete. 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 Processor -- start 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 -- start start 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410100e20 0x7f74101993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7410102570 0x7f7410199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7410102ef0 0x7f7410199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7410110f80 con 0x7f7410100e20 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7410110e00 con 0x7f7410102570 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f7416837640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7410111100 con 0x7f7410102ef0 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f740ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410100e20 0x7f74101993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f740ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410100e20 0x7f74101993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55310/0 (socket says 172.21.15.37:55310) 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.308+0000 7f740ffff640 1 -- 172.21.15.37:0/345884221 learned_addr learned my addr 172.21.15.37:0/345884221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740f7fe640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7410102570 0x7f7410199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f7414dd4640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7410102ef0 0x7f7410199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740ffff640 1 -- 172.21.15.37:0/345884221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7410102ef0 msgr2=0x7f7410199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740ffff640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7410102ef0 0x7f7410199e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740ffff640 1 -- 172.21.15.37:0/345884221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7410102570 msgr2=0x7f7410199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740ffff640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7410102570 0x7f7410199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740ffff640 1 -- 172.21.15.37:0/345884221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74101a47d0 con 0x7f7410100e20 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f7414dd4640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7410102ef0 0x7f7410199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740ffff640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410100e20 0x7f74101993d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f740000c970 tx=0x7f740000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740d7fa640 1 -- 172.21.15.37:0/345884221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400007bb0 con 0x7f7410100e20 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740d7fa640 1 -- 172.21.15.37:0/345884221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7400007d50 con 0x7f7410100e20 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74101a4ac0 con 0x7f7410100e20 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f740d7fa640 1 -- 172.21.15.37:0/345884221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740000f450 con 0x7f7410100e20 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.309+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101a5000 con 0x7f7410100e20 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f740d7fa640 1 -- 172.21.15.37:0/345884221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f7400005020 con 0x7f7410100e20 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 Processor -- start 2024-09-19T06:43:11.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 -- start start 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374aaf4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaf5170 0x56374aaff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaffa30 0x56374ab01e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56374aa59160 con 0x56374aaffa30 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56374aa00720 con 0x56374aaf4850 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.312+0000 7f7418c10740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56374aa55120 con 0x56374aaf5170 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740d7fa640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f73f80a35e0 0x7f73f80a5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaffa30 0x56374ab01e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaffa30 0x56374ab01e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55326/0 (socket says 172.21.15.37:55326) 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 -- 172.21.15.37:0/2240110079 learned_addr learned my addr 172.21.15.37:0/2240110079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f7414dd4640 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374aaf4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740d7fa640 1 -- 172.21.15.37:0/345884221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f74000cfd60 con 0x7f7410100e20 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f7414dd4640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f73f80a35e0 0x7f73f80a5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740ffff640 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaf5170 0x56374aaff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 -- 172.21.15.37:0/2240110079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaf5170 msgr2=0x56374aaff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaf5170 0x56374aaff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 -- 172.21.15.37:0/2240110079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 msgr2=0x56374aaf4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374aaf4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 -- 172.21.15.37:0/2240110079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374a7caba0 con 0x56374aaffa30 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740ffff640 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaf5170 0x56374aaff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f740f7fe640 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaffa30 0x56374ab01e20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f740400ab70 tx=0x7f740400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26a6a50972a03c31 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.313+0000 7f7414dd4640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f73f80a35e0 0x7f73f80a5aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7408009a60 tx=0x7f74080095c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f73f67fc640 1 -- 172.21.15.37:0/2240110079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740401d070 con 0x56374aaffa30 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f73f67fc640 1 -- 172.21.15.37:0/2240110079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404018b20 con 0x56374aaffa30 2024-09-19T06:43:11.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f73f67fc640 1 -- 172.21.15.37:0/2240110079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7404010a50 con 0x56374aaffa30 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 -- 172.21.15.37:0/2240110079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaffa30 msgr2=0x56374ab01e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaffa30 0x56374ab01e20 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f740400ab70 tx=0x7f740400f790 comp rx=0 tx=0).stop 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 -- 172.21.15.37:0/2240110079 shutdown_connections 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaffa30 0x56374ab01e20 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaf5170 0x56374aaff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 --2- 172.21.15.37:0/2240110079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374aaf4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 -- 172.21.15.37:0/2240110079 >> 172.21.15.37:0/2240110079 conn(0x56374aaf0de0 msgr2=0x56374aaf11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 -- 172.21.15.37:0/2240110079 shutdown_connections 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.314+0000 7f7418c10740 1 -- 172.21.15.37:0/2240110079 wait complete. 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 -- start start 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 Processor -- start 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf4850 0x56374aafc890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf5170 0x56374aafcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aafd310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56374ab02600 con 0x56374aaf4850 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56374a9e1900 con 0x56374aaf5170 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56374ab02780 con 0x56374aaffa30 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f7418c10740 10 client.0 _ll_register_callbacks cb 0x56374a9df780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f740ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf4850 0x56374aafc890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f740ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf4850 0x56374aafc890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55328/0 (socket says 172.21.15.37:55328) 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.315+0000 7f740ffff640 1 -- 172.21.15.37:0/3090407461 learned_addr learned my addr 172.21.15.37:0/3090407461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f7414dd4640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aafd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740f7fe640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf5170 0x56374aafcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740ffff640 1 -- 172.21.15.37:0/3090407461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 msgr2=0x56374aafd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740ffff640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aafd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740ffff640 1 -- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf5170 msgr2=0x56374aafcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740ffff640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf5170 0x56374aafcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740ffff640 1 -- 172.21.15.37:0/3090407461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374aa21c50 con 0x56374aaf4850 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f7414dd4640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aafd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740f7fe640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf5170 0x56374aafcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f740ffff640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf4850 0x56374aafc890 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74000cd360 tx=0x7f740009c5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35098a403681b1d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f73ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f73ef7fe640 1 -- 172.21.15.37:0/3090407461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740001d040 con 0x56374aaf4850 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f73ef7fe640 1 -- 172.21.15.37:0/3090407461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f7400005310 con 0x56374aaf4850 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7404018b20 con 0x56374aaf4850 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f73ef7fe640 1 -- 172.21.15.37:0/3090407461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74000a4a70 con 0x56374aaf4850 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f7418c10740 10 client.15822 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56374ab0e1a0 con 0x56374aaf4850 2024-09-19T06:43:11.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.316+0000 7f73ef7fe640 1 -- 172.21.15.37:0/3090407461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404018b20 con 0x56374aaf4850 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f73ef7fe640 1 -- 172.21.15.37:0/3090407461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f74000cd580 con 0x56374aaf4850 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f7418c10740 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f7418c10740 10 client.15822 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f7418c10740 20 client.15822 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f7418c10740 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab0f650 0x56374ab11ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f7418c10740 4 client.15822 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56374aa66890 con 0x56374ab0f650 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.317+0000 7f740f7fe640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab0f650 0x56374ab11ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.318+0000 7f73ef7fe640 10 client.15822 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.318+0000 7f740f7fe640 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab0f650 0x56374ab11ac0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dae61f8d08e729 server_cookie=fba09073e658f1c6 in_seq=0 out_seq=0 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.318+0000 7f73ef7fe640 1 -- 172.21.15.37:0/3090407461 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56374aa66890 con 0x56374ab0f650 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.318+0000 7f73ef7fe640 10 client.15822 handle_command_reply: tid=0 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.318+0000 7f7418c10740 1 client.15822 shutdown 2024-09-19T06:43:11.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.318+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf4850 msgr2=0x56374aafc890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.318+0000 7f7418c10740 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf4850 0x56374aafc890 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f74000cd360 tx=0x7f740009c5c0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 shutdown_connections 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab0f650 0x56374ab11ac0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aafd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf5170 0x56374aafcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 --2- 172.21.15.37:0/3090407461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf4850 0x56374aafc890 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f73ef7fe640 0 client.15822 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 >> 172.21.15.37:0/3090407461 conn(0x56374aaf0de0 msgr2=0x56374aaf6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 shutdown_connections 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 1 -- 172.21.15.37:0/3090407461 wait complete. 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.319+0000 7f7418c10740 20 client.15822 trim_cache size 0 max 16384 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 Processor -- start 2024-09-19T06:43:11.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 -- start start 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab17390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab178d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aaf9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74101b01c0 con 0x56374aaf5170 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ada90 con 0x56374aaf4850 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7418c10740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ab8e0 con 0x56374aaffa30 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f740f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab178d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f740f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab178d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55332/0 (socket says 172.21.15.37:55332) 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f740f7fe640 1 -- 172.21.15.37:0/1601068108 learned_addr learned my addr 172.21.15.37:0/1601068108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f740ffff640 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab17390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.336+0000 7f7414dd4640 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aaf9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f740f7fe640 1 -- 172.21.15.37:0/1601068108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 msgr2=0x56374aaf9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f740f7fe640 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aaf9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f740f7fe640 1 -- 172.21.15.37:0/1601068108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 msgr2=0x56374ab17390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f740f7fe640 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab17390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f740f7fe640 1 -- 172.21.15.37:0/1601068108 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56374aafb840 con 0x56374aaf5170 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f740f7fe640 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab178d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f74040106c0 tx=0x7f740400a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88ab9a1451751d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f73f5ffb640 1 -- 172.21.15.37:0/1601068108 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740401d040 con 0x56374aaf5170 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f73f5ffb640 1 -- 172.21.15.37:0/1601068108 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404005040 con 0x56374aaf5170 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.337+0000 7f73f5ffb640 1 -- 172.21.15.37:0/1601068108 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74040109c0 con 0x56374aaf5170 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 -- 172.21.15.37:0/1601068108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 msgr2=0x56374ab178d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab178d0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f74040106c0 tx=0x7f740400a700 comp rx=0 tx=0).stop 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 -- 172.21.15.37:0/1601068108 shutdown_connections 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374aaf9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab178d0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 --2- 172.21.15.37:0/1601068108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab17390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 -- 172.21.15.37:0/1601068108 >> 172.21.15.37:0/1601068108 conn(0x56374aaf0de0 msgr2=0x56374aaf6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 -- 172.21.15.37:0/1601068108 shutdown_connections 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 -- 172.21.15.37:0/1601068108 wait complete. 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 -- start start 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.338+0000 7f7418c10740 1 Processor -- start 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7418c10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab039c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7418c10740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab03f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7418c10740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374ab04440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7418c10740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74101abf50 con 0x56374aaf5170 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7418c10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ad3c0 con 0x56374aaf4850 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7418c10740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74101ac620 con 0x56374aaffa30 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7418c10740 10 client.0 _ll_register_callbacks cb 0x7f7400004920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab03f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab03f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55344/0 (socket says 172.21.15.37:55344) 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 -- 172.21.15.37:0/574349710 learned_addr learned my addr 172.21.15.37:0/574349710 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740ffff640 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f7414dd4640 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374ab04440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 -- 172.21.15.37:0/574349710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 msgr2=0x56374ab04440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374ab04440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 -- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 msgr2=0x56374ab039c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab039c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.339+0000 7f740f7fe640 1 -- 172.21.15.37:0/574349710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56374ab7da10 con 0x56374aaf5170 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f740f7fe640 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab03f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7404005940 tx=0x7f74040105e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73c240b2dd440bba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f73ef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f73ef7fe640 1 -- 172.21.15.37:0/574349710 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740401d040 con 0x56374aaf5170 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f73ef7fe640 1 -- 172.21.15.37:0/574349710 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f740400fc90 con 0x56374aaf5170 2024-09-19T06:43:11.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7404005040 con 0x56374aaf5170 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f73ef7fe640 1 -- 172.21.15.37:0/574349710 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f74040051f0 con 0x56374aaf5170 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f7418c10740 10 client.15828 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56374ab7ecd0 con 0x56374aaf5170 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f73ef7fe640 1 -- 172.21.15.37:0/574349710 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7404005040 con 0x56374aaf5170 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f73ef7fe640 1 -- 172.21.15.37:0/574349710 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f740402b070 con 0x56374aaf5170 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f7418c10740 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f7418c10740 10 client.15828 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.340+0000 7f7418c10740 20 client.15828 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.341+0000 7f7418c10740 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab7f2b0 0x56374ab816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.341+0000 7f7418c10740 4 client.15828 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.341+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x56374aa60be0 con 0x56374ab7f2b0 2024-09-19T06:43:11.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.341+0000 7f740ffff640 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab7f2b0 0x56374ab816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:11.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.341+0000 7f73ef7fe640 10 client.15828 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.341+0000 7f740ffff640 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab7f2b0 0x56374ab816a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6738feac00cd53d8 server_cookie=12818b2da3abdef in_seq=0 out_seq=0 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f73ef7fe640 1 -- 172.21.15.37:0/574349710 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x56374aa60be0 con 0x56374ab7f2b0 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f73ef7fe640 10 client.15828 handle_command_reply: tid=0 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 client.15828 shutdown 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 msgr2=0x56374ab03f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab03f00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7404005940 tx=0x7f74040105e0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 shutdown_connections 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56374ab7f2b0 0x56374ab816a0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56374aaffa30 0x56374ab04440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56374aaf5170 0x56374ab03f00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 --2- 172.21.15.37:0/574349710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56374aaf4850 0x56374ab039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 >> 172.21.15.37:0/574349710 conn(0x56374aaf0de0 msgr2=0x56374aafc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.348+0000 7f73ef7fe640 0 client.15828 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 shutdown_connections 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7418c10740 1 -- 172.21.15.37:0/574349710 wait complete. 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7418c10740 20 client.15828 trim_cache size 0 max 16384 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 25, 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:43:11.360 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 6, 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "6fb810d1": { 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 6, 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 140.3, 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:43:11.361 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:11.362 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f73f80a35e0 msgr2=0x7f73f80a5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f73f80a35e0 0x7f73f80a5aa0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7408009a60 tx=0x7f74080095c0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410100e20 msgr2=0x7f74101993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410100e20 0x7f74101993d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f740000c970 tx=0x7f740000ce30 comp rx=0 tx=0).stop 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 shutdown_connections 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7410102ef0 0x7f7410199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7410102570 0x7f7410199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f73f80a35e0 0x7f73f80a5aa0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 --2- 172.21.15.37:0/345884221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7410100e20 0x7f74101993d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.349+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 >> 172.21.15.37:0/345884221 conn(0x7f74100fc7d0 msgr2=0x7f74101068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.350+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 shutdown_connections 2024-09-19T06:43:11.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:11.350+0000 7f7416837640 1 -- 172.21.15.37:0/345884221 wait complete. 2024-09-19T06:43:11.401 INFO:tasks.workunit.client.0.smithi037.stderr:Note: switching to '6fad5dd73c93545e52f57f681e140a9c3529c11e'. 2024-09-19T06:43:11.401 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-09-19T06:43:11.401 INFO:tasks.workunit.client.0.smithi037.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr:state without impacting any branches by switching back to a branch. 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr: git switch -c 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr:Or undo this operation with: 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr: git switch - 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-09-19T06:43:11.402 INFO:tasks.workunit.client.0.smithi037.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-19T06:43:11.403 INFO:tasks.workunit.client.0.smithi037.stderr: 2024-09-19T06:43:11.403 INFO:tasks.workunit.client.0.smithi037.stderr:HEAD is now at 6fad5dd73c9 Merge pull request #59812 from neha-ojha/wip-rtd-squid 2024-09-19T06:43:11.407 DEBUG:teuthology.orchestra.run.smithi037:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-19T06:43:11.490 INFO:tasks.workunit.client.0.smithi037.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-19T06:43:11.492 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:43:11.492 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-19T06:43:12.230 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6fb810d1', quiesce took 3.6 seconds. Will release after: 16.75 2024-09-19T06:43:12.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:12 smithi096 ceph-mon[35004]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T06:43:12.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:12 smithi037 ceph-mon[31211]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T06:43:12.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:12 smithi151 ceph-mon[35236]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-09-19T06:43:12.658 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-19T06:43:12.745 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-19T06:43:12.788 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-19T06:43:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:43:12.790 INFO:tasks.workunit.client.0.smithi037.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-19T06:43:12.832 INFO:tasks.workunit.client.0.smithi037.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-19T06:43:12.835 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:43:12.835 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-19T06:43:12.852 INFO:tasks.workunit:Running workunits matching suites/dbench.sh on client.0... 2024-09-19T06:43:12.853 INFO:tasks.workunit:Running workunit suites/dbench.sh... 2024-09-19T06:43:12.854 DEBUG:teuthology.orchestra.run.smithi037:workunit test suites/dbench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh 2024-09-19T06:43:14.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:14 smithi096 ceph-mon[35004]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T06:43:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:14 smithi037 ceph-mon[31211]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T06:43:14.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:14 smithi151 ceph-mon[35236]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-09-19T06:43:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:16 smithi096 ceph-mon[35004]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T06:43:16.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:16 smithi037 ceph-mon[31211]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T06:43:16.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:16 smithi151 ceph-mon[35236]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 -- 172.21.15.37:0/1482755687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc102cc0 msgr2=0x7f2ccc1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 --2- 172.21.15.37:0/1482755687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc102cc0 0x7f2ccc1030a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009900 tx=0x7f2cbc02a480 comp rx=0 tx=0).stop 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 -- 172.21.15.37:0/1482755687 shutdown_connections 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 --2- 172.21.15.37:0/1482755687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc10c110 0x7f2ccc10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 --2- 172.21.15.37:0/1482755687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc1035e0 0x7f2ccc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 --2- 172.21.15.37:0/1482755687 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc102cc0 0x7f2ccc1030a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 -- 172.21.15.37:0/1482755687 >> 172.21.15.37:0/1482755687 conn(0x7f2ccc0fc6b0 msgr2=0x7f2ccc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 -- 172.21.15.37:0/1482755687 shutdown_connections 2024-09-19T06:43:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.121+0000 7f2cd0959640 1 -- 172.21.15.37:0/1482755687 wait complete. 2024-09-19T06:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 Processor -- start 2024-09-19T06:43:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 -- start start 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc102cc0 0x7f2ccc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1035e0 0x7f2ccc1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc10c110 0x7f2ccc19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc111f40 con 0x7f2ccc1035e0 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc111dc0 con 0x7f2ccc102cc0 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cd0959640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1120c0 con 0x7f2ccc10c110 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1035e0 0x7f2ccc1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1035e0 0x7f2ccc1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55362/0 (socket says 172.21.15.37:55362) 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3268193872 learned_addr learned my addr 172.21.15.37:0/3268193872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2cca59c640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc102cc0 0x7f2ccc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.122+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc10c110 0x7f2ccc19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3268193872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc10c110 msgr2=0x7f2ccc19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc10c110 0x7f2ccc19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3268193872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc102cc0 msgr2=0x7f2ccc198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc102cc0 0x7f2ccc198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3268193872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc19d360 con 0x7f2ccc1035e0 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1035e0 0x7f2ccc1988b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000d750 tx=0x7f2cc000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cbb7fe640 1 -- 172.21.15.37:0/3268193872 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0017070 con 0x7f2ccc1035e0 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cbb7fe640 1 -- 172.21.15.37:0/3268193872 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0005020 con 0x7f2ccc1035e0 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc19d650 con 0x7f2ccc1035e0 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cbb7fe640 1 -- 172.21.15.37:0/3268193872 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0004d00 con 0x7f2ccc1035e0 2024-09-19T06:43:17.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.123+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1a4ee0 con 0x7f2ccc1035e0 2024-09-19T06:43:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cbb7fe640 1 -- 172.21.15.37:0/3268193872 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 25) ==== 138639+0+0 (secure 0 0 0) 0x7f2cc0009a30 con 0x7f2ccc1035e0 2024-09-19T06:43:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 Processor -- start 2024-09-19T06:43:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 -- start start 2024-09-19T06:43:17.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a10ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a11410 0x555851a1b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a1e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5558519752f0 con 0x555851a1bcb0 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55585191d270 con 0x555851a10af0 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558519712b0 con 0x555851a11410 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a1e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a1e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55366/0 (socket says 172.21.15.37:55366) 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/2890591508 learned_addr learned my addr 172.21.15.37:0/2890591508 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.126+0000 7f2cca59c640 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a11410 0x555851a1b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a10ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cbb7fe640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cb40a3610 0x7f2cb40a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/2890591508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a11410 msgr2=0x555851a1b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a11410 0x555851a1b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/2890591508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 msgr2=0x555851a10ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a10ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/2890591508 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55585191c340 con 0x555851a1bcb0 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cca59c640 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a11410 0x555851a1b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a10ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cb40a3610 0x7f2cb40a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cbb7fe640 1 -- 172.21.15.37:0/3268193872 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f2cc00cf950 con 0x7f2ccc1035e0 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a1e0a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00cce00 tx=0x7f2cc0022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eb35857264977a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cb8ff9640 1 -- 172.21.15.37:0/2890591508 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc009b5e0 con 0x555851a1bcb0 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cb8ff9640 1 -- 172.21.15.37:0/2890591508 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc001e120 con 0x555851a1bcb0 2024-09-19T06:43:17.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.127+0000 7f2cb8ff9640 1 -- 172.21.15.37:0/2890591508 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc009b400 con 0x555851a1bcb0 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 -- 172.21.15.37:0/2890591508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 msgr2=0x555851a1e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a1e0a0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2cc00cce00 tx=0x7f2cc0022000 comp rx=0 tx=0).stop 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cb40a3610 0x7f2cb40a5ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4005fd0 tx=0x7f2cc4004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 -- 172.21.15.37:0/2890591508 shutdown_connections 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a1e0a0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a11410 0x555851a1b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/2890591508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a10ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 -- 172.21.15.37:0/2890591508 >> 172.21.15.37:0/2890591508 conn(0x555851a0d060 msgr2=0x555851a0d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 -- 172.21.15.37:0/2890591508 shutdown_connections 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.128+0000 7f2cd37d3740 1 -- 172.21.15.37:0/2890591508 wait complete. 2024-09-19T06:43:17.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 -- start start 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 Processor -- start 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a10af0 0x555851a18b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a19070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555851a1e820 con 0x555851a10af0 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558518fe290 con 0x555851a11410 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555851a1e9a0 con 0x555851a1bcb0 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a10af0 0x555851a18b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cd37d3740 10 client.0 _ll_register_callbacks cb 0x55585192f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cca59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a10af0 0x555851a18b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55374/0 (socket says 172.21.15.37:55374) 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cca59c640 1 -- 172.21.15.37:0/483949017 learned_addr learned my addr 172.21.15.37:0/483949017 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.129+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a19070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cca59c640 1 -- 172.21.15.37:0/483949017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 msgr2=0x555851a195b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cca59c640 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a195b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cca59c640 1 -- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 msgr2=0x555851a19070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cca59c640 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a19070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cca59c640 1 -- 172.21.15.37:0/483949017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555851688210 con 0x555851a10af0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cca59c640 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a10af0 0x555851a18b30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009230 tx=0x7f2cbc005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36cbf53fb5756002 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/483949017 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc035040 con 0x555851a10af0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/483949017 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f2cbc0055c0 con 0x555851a10af0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc001e120 con 0x555851a10af0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/483949017 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cbc0046c0 con 0x555851a10af0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cd37d3740 10 client.15840 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555851754620 con 0x555851a10af0 2024-09-19T06:43:17.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/483949017 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc001e120 con 0x555851a10af0 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.130+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/483949017 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2cbc003c70 con 0x555851a10af0 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cd37d3740 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cd37d3740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cd37d3740 20 client.15840 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a2bb00 0x555851a2def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cd37d3740 4 client.15840 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555851982a20 con 0x555851a2bb00 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a2bb00 0x555851a2def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a2bb00 0x555851a2def0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c881153603528be server_cookie=3469ef22dbfec22e in_seq=0 out_seq=0 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.131+0000 7f2cb1ffb640 10 client.15840 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/483949017 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555851982a20 con 0x555851a2bb00 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cb1ffb640 10 client.15840 handle_command_reply: tid=0 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 client.15840 shutdown 2024-09-19T06:43:17.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a10af0 msgr2=0x555851a18b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a10af0 0x555851a18b30 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc009230 tx=0x7f2cbc005040 comp rx=0 tx=0).stop 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 shutdown_connections 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a2bb00 0x555851a2def0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a195b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a19070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cb1ffb640 0 client.15840 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/483949017 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a10af0 0x555851a18b30 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 >> 172.21.15.37:0/483949017 conn(0x555851a0d060 msgr2=0x555851a12dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 shutdown_connections 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.132+0000 7f2cd37d3740 1 -- 172.21.15.37:0/483949017 wait complete. 2024-09-19T06:43:17.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.133+0000 7f2cd37d3740 20 client.15840 trim_cache size 0 max 16384 2024-09-19T06:43:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.147+0000 7f2cd37d3740 1 Processor -- start 2024-09-19T06:43:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.147+0000 7f2cd37d3740 1 -- start start 2024-09-19T06:43:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a10af0 0x555851a193e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a1aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.150 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1b0150 con 0x555851a1bcb0 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ada20 con 0x555851a11410 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ab870 con 0x555851a10af0 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a337b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55376/0 (socket says 172.21.15.37:55376) 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 -- 172.21.15.37:0/649952426 learned_addr learned my addr 172.21.15.37:0/649952426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a1aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 -- 172.21.15.37:0/649952426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a10af0 msgr2=0x555851a193e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a10af0 0x555851a193e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 -- 172.21.15.37:0/649952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 msgr2=0x555851a1aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a1aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 -- 172.21.15.37:0/649952426 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555851a2e260 con 0x555851a1bcb0 2024-09-19T06:43:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a1aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a337b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4013420 tx=0x7f2cc4013f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aa0a5648712eb93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.148+0000 7f2cb3fff640 1 -- 172.21.15.37:0/649952426 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc401d580 con 0x555851a1bcb0 2024-09-19T06:43:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cb3fff640 1 -- 172.21.15.37:0/649952426 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc401dc00 con 0x555851a1bcb0 2024-09-19T06:43:17.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cb3fff640 1 -- 172.21.15.37:0/649952426 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4012d70 con 0x555851a1bcb0 2024-09-19T06:43:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 -- 172.21.15.37:0/649952426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 msgr2=0x555851a337b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a337b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4013420 tx=0x7f2cc4013f90 comp rx=0 tx=0).stop 2024-09-19T06:43:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 -- 172.21.15.37:0/649952426 shutdown_connections 2024-09-19T06:43:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a1bcb0 0x555851a337b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a11410 0x555851a1aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/649952426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a10af0 0x555851a193e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 -- 172.21.15.37:0/649952426 >> 172.21.15.37:0/649952426 conn(0x555851a0d060 msgr2=0x555851a1a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 -- 172.21.15.37:0/649952426 shutdown_connections 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 -- 172.21.15.37:0/649952426 wait complete. 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.149+0000 7f2cd37d3740 1 -- start start 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 1 Processor -- start 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a24140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a11410 0x555851a1f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a1f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1abee0 con 0x555851a11410 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ad350 con 0x555851a10af0 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ac5b0 con 0x555851a1bcb0 2024-09-19T06:43:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cd37d3740 10 client.0 _ll_register_callbacks cb 0x7f2cbc009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a11410 0x555851a1f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a11410 0x555851a1f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55380/0 (socket says 172.21.15.37:55380) 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3705572294 learned_addr learned my addr 172.21.15.37:0/3705572294 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cca59c640 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a24140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2ccad9d640 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a1f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3705572294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 msgr2=0x555851a1f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a1f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 msgr2=0x555851a24140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a24140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.150+0000 7f2cc9d9b640 1 -- 172.21.15.37:0/3705572294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555851a1ffb0 con 0x555851a11410 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cc9d9b640 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a11410 0x555851a1f4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2cc009ba30 tx=0x7f2cc00cce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=143da6e2e638bd08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cb1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:17.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/3705572294 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc009b400 con 0x555851a11410 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/3705572294 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f2cc0099b40 con 0x555851a11410 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc401dc00 con 0x555851a11410 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/3705572294 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2cc009b5e0 con 0x555851a11410 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 10 client.15846 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555851a96520 con 0x555851a11410 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/3705572294 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc401dc00 con 0x555851a11410 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/3705572294 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f2cc0005560 con 0x555851a11410 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 10 client.15846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 20 client.15846 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a96860 0x555851a98c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 4 client.15846 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:43:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55585197cd70 con 0x555851a96860 2024-09-19T06:43:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.151+0000 7f2cca59c640 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a96860 0x555851a98c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.152+0000 7f2cca59c640 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a96860 0x555851a98c50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5b6819fcb19d45e server_cookie=34ce8f18f6efe1b1 in_seq=0 out_seq=0 2024-09-19T06:43:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.152+0000 7f2cb1ffb640 10 client.15846 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.157+0000 7f2cb1ffb640 1 -- 172.21.15.37:0/3705572294 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55585197cd70 con 0x555851a96860 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.157+0000 7f2cb1ffb640 10 client.15846 handle_command_reply: tid=0 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 client.15846 shutdown 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a11410 msgr2=0x555851a1f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a11410 0x555851a1f4b0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2cc009ba30 tx=0x7f2cc00cce50 comp rx=0 tx=0).stop 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 shutdown_connections 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555851a96860 0x555851a98c50 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555851a1bcb0 0x555851a1f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555851a11410 0x555851a1f4b0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 --2- 172.21.15.37:0/3705572294 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555851a10af0 0x555851a24140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cb1ffb640 0 client.15846 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 >> 172.21.15.37:0/3705572294 conn(0x555851a0d060 msgr2=0x555851a0c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 shutdown_connections 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 1 -- 172.21.15.37:0/3705572294 wait complete. 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.158+0000 7f2cd37d3740 20 client.15846 trim_cache size 0 max 16384 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:43:17.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:43:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cb40a3610 msgr2=0x7f2cb40a5ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cb40a3610 0x7f2cb40a5ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4005fd0 tx=0x7f2cc4004a40 comp rx=0 tx=0).stop 2024-09-19T06:43:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1035e0 msgr2=0x7f2ccc1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1035e0 0x7f2ccc1988b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2cc000d750 tx=0x7f2cc000dc10 comp rx=0 tx=0).stop 2024-09-19T06:43:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 shutdown_connections 2024-09-19T06:43:17.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc10c110 0x7f2ccc19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cb40a3610 0x7f2cb40a5ad0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1035e0 0x7f2ccc1988b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 --2- 172.21.15.37:0/3268193872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc102cc0 0x7f2ccc198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 >> 172.21.15.37:0/3268193872 conn(0x7f2ccc0fc6b0 msgr2=0x7f2ccc0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 shutdown_connections 2024-09-19T06:43:17.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:17.159+0000 7f2cd0959640 1 -- 172.21.15.37:0/3268193872 wait complete. 2024-09-19T06:43:18.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:18 smithi096 ceph-mon[35004]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T06:43:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:18 smithi037 ceph-mon[31211]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T06:43:18.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:18 smithi151 ceph-mon[35236]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-19T06:43:20.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:20 smithi037 ceph-mon[31211]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-19T06:43:20.516 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:20 smithi037 ceph-mon[31211]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T06:43:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:20 smithi096 ceph-mon[35004]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-19T06:43:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:20 smithi096 ceph-mon[35004]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T06:43:20.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:20 smithi151 ceph-mon[35236]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-09-19T06:43:20.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:20 smithi151 ceph-mon[35236]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-09-19T06:43:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:22 smithi096 ceph-mon[35004]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:22 smithi037 ceph-mon[31211]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:22 smithi151 ceph-mon[35236]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:24.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:24 smithi096 ceph-mon[35004]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:43:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:24 smithi037 ceph-mon[31211]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:43:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:24 smithi151 ceph-mon[35236]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:43:25.429 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:43:25.703 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:26.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.174+0000 7fc07aac1640 1 -- 172.21.15.37:0/1752268944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074101de0 msgr2=0x7fc0741021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.174+0000 7fc07aac1640 1 --2- 172.21.15.37:0/1752268944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074101de0 0x7fc0741021c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc068009900 tx=0x7fc06802a410 comp rx=0 tx=0).stop 2024-09-19T06:43:26.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- 172.21.15.37:0/1752268944 shutdown_connections 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 --2- 172.21.15.37:0/1752268944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074106f10 0x7fc07410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 --2- 172.21.15.37:0/1752268944 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc074102790 0x7fc0741067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 --2- 172.21.15.37:0/1752268944 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074101de0 0x7fc0741021c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- 172.21.15.37:0/1752268944 >> 172.21.15.37:0/1752268944 conn(0x7fc0740fd8c0 msgr2=0x7fc0740ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- 172.21.15.37:0/1752268944 shutdown_connections 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- 172.21.15.37:0/1752268944 wait complete. 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 Processor -- start 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- start start 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074101de0 0x7fc0741983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc074102790 0x7fc074198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074106f10 0x7fc07419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc074111f70 con 0x7fc074106f10 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc074111df0 con 0x7fc074101de0 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.175+0000 7fc07aac1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741120f0 con 0x7fc074102790 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074106f10 0x7fc07419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074106f10 0x7fc07419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44682/0 (socket says 172.21.15.37:44682) 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 -- 172.21.15.37:0/960821584 learned_addr learned my addr 172.21.15.37:0/960821584 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc073fff640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc074102790 0x7fc074198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07885d640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074101de0 0x7fc0741983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 -- 172.21.15.37:0/960821584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc074102790 msgr2=0x7fc074198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc074102790 0x7fc074198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 -- 172.21.15.37:0/960821584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074101de0 msgr2=0x7fc0741983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074101de0 0x7fc0741983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 -- 172.21.15.37:0/960821584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07419d380 con 0x7fc074106f10 2024-09-19T06:43:26.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07885d640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074101de0 0x7fc0741983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07905e640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074106f10 0x7fc07419cca0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00cce0 tx=0x7fc06c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc071ffb640 1 -- 172.21.15.37:0/960821584 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00f450 con 0x7fc074106f10 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07419d670 con 0x7fc074106f10 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc071ffb640 1 -- 172.21.15.37:0/960821584 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c00fad0 con 0x7fc074106f10 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.176+0000 7fc071ffb640 1 -- 172.21.15.37:0/960821584 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c018c10 con 0x7fc074106f10 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.177+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0741a4f50 con 0x7fc074106f10 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc071ffb640 1 -- 172.21.15.37:0/960821584 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc06c00f5f0 con 0x7fc074106f10 2024-09-19T06:43:26.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 Processor -- start 2024-09-19T06:43:26.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 -- start start 2024-09-19T06:43:26.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc071ffb640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc0640a33b0 0x7fc0640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4dd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d4ec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d4eee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55951d4432e0 con 0x55951d4eca90 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55951d3eace0 con 0x55951d4dd380 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55951d43f2a0 con 0x55951d4ddca0 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc073fff640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc0640a33b0 0x7fc0640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc071ffb640 1 -- 172.21.15.37:0/960821584 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fc06c0d04d0 con 0x7fc074106f10 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07905e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d4eee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.179+0000 7fc07905e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d4eee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44684/0 (socket says 172.21.15.37:44684) 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07905e640 1 -- 172.21.15.37:0/2896585287 learned_addr learned my addr 172.21.15.37:0/2896585287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07885d640 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4dd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07905e640 1 -- 172.21.15.37:0/2896585287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 msgr2=0x55951d4ec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07905e640 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d4ec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07905e640 1 -- 172.21.15.37:0/2896585287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 msgr2=0x55951d4dd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07905e640 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4dd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07905e640 1 -- 172.21.15.37:0/2896585287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951d3cf440 con 0x55951d4eca90 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc073fff640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc0640a33b0 0x7fc0640a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0740fdce0 tx=0x7fc06000aec0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc07905e640 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d4eee80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc06c098bb0 tx=0x7fc06c09c0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78be481b45e8ae8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc05effd640 1 -- 172.21.15.37:0/2896585287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c09c750 con 0x55951d4eca90 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc05effd640 1 -- 172.21.15.37:0/2896585287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c0a0b00 con 0x55951d4eca90 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.180+0000 7fc05effd640 1 -- 172.21.15.37:0/2896585287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c00fc80 con 0x55951d4eca90 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2896585287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 msgr2=0x55951d4eee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d4eee80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc06c098bb0 tx=0x7fc06c09c0b0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2896585287 shutdown_connections 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d4eee80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d4ec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2896585287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4dd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2896585287 >> 172.21.15.37:0/2896585287 conn(0x55951d4d9a30 msgr2=0x55951d4d9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2896585287 shutdown_connections 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2896585287 wait complete. 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 -- start start 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.181+0000 7fc07ce9a740 1 Processor -- start 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4ec400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4ddca0 0x55951d4e9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4eca90 0x55951d4e9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55951d4ef660 con 0x55951d4ddca0 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55951d3fe050 con 0x55951d4dd380 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55951d4ef7e0 con 0x55951d4eca90 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07ce9a740 10 client.0 _ll_register_callbacks cb 0x55951d43e160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:26.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4ddca0 0x55951d4e9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4ddca0 0x55951d4e9700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44692/0 (socket says 172.21.15.37:44692) 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 -- 172.21.15.37:0/2334576498 learned_addr learned my addr 172.21.15.37:0/2334576498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07885d640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4ec400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07905e640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4eca90 0x55951d4e9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 -- 172.21.15.37:0/2334576498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4eca90 msgr2=0x55951d4e9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4eca90 0x55951d4e9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 -- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 msgr2=0x55951d4ec400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4ec400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc073fff640 1 -- 172.21.15.37:0/2334576498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951d1b43c0 con 0x55951d4ddca0 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07905e640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4eca90 0x55951d4e9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.182+0000 7fc07885d640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc073fff640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4ddca0 0x55951d4e9700 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc0600128d0 tx=0x7fc0600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=800a5c8450589909 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:26.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc053fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc053fff640 1 -- 172.21.15.37:0/2334576498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc060010db0 con 0x55951d4ddca0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc06c0a0b00 con 0x55951d4ddca0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc053fff640 1 -- 172.21.15.37:0/2334576498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fc06001f480 con 0x55951d4ddca0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc053fff640 1 -- 172.21.15.37:0/2334576498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc06002d050 con 0x55951d4ddca0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 10 client.15858 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55951d4fb110 con 0x55951d4ddca0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc053fff640 1 -- 172.21.15.37:0/2334576498 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c0a0b00 con 0x55951d4ddca0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc053fff640 1 -- 172.21.15.37:0/2334576498 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc06001e6e0 con 0x55951d4ddca0 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 20 client.15858 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d4fc5d0 0x55951d4fea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 4 client.15858 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.183+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55951d450850 con 0x55951d4fc5d0 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.184+0000 7fc07885d640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d4fc5d0 0x55951d4fea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.184+0000 7fc07885d640 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d4fc5d0 0x55951d4fea40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c18d56e34665073 server_cookie=3eb976fb89801c7c in_seq=0 out_seq=0 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.184+0000 7fc053fff640 10 client.15858 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc053fff640 1 -- 172.21.15.37:0/2334576498 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55951d450850 con 0x55951d4fc5d0 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc053fff640 10 client.15858 handle_command_reply: tid=0 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 client.15858 shutdown 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4ddca0 msgr2=0x55951d4e9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4ddca0 0x55951d4e9700 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fc0600128d0 tx=0x7fc0600046c0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 shutdown_connections 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d4fc5d0 0x55951d4fea40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4eca90 0x55951d4e9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4ddca0 0x55951d4e9700 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/2334576498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4ec400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc053fff640 0 client.15858 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 >> 172.21.15.37:0/2334576498 conn(0x55951d4d9a30 msgr2=0x55951d4df790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 shutdown_connections 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 1 -- 172.21.15.37:0/2334576498 wait complete. 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.185+0000 7fc07ce9a740 20 client.15858 trim_cache size 0 max 16384 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.202+0000 7fc07ce9a740 1 Processor -- start 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.202+0000 7fc07ce9a740 1 -- start start 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d504310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d504850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741adfa0 con 0x55951d4eca90 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741ab870 con 0x55951d4dd380 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741a96c0 con 0x55951d4ddca0 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d504850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d504850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44706/0 (socket says 172.21.15.37:44706) 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 -- 172.21.15.37:0/1440972457 learned_addr learned my addr 172.21.15.37:0/1440972457 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07885d640 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 -- 172.21.15.37:0/1440972457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 msgr2=0x55951d504310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d504310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 -- 172.21.15.37:0/1440972457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 msgr2=0x55951d4f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 -- 172.21.15.37:0/1440972457 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55951d4e86b0 con 0x55951d4eca90 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07885d640 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:26.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc07905e640 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d504850 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc06c09c6e0 tx=0x7fc06c09c1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b135fa7d58cb63be server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc05e7fc640 1 -- 172.21.15.37:0/1440972457 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c09c750 con 0x55951d4eca90 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc05e7fc640 1 -- 172.21.15.37:0/1440972457 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c00f8e0 con 0x55951d4eca90 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.203+0000 7fc05e7fc640 1 -- 172.21.15.37:0/1440972457 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06c0a4f30 con 0x55951d4eca90 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1440972457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 msgr2=0x55951d504850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d504850 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fc06c09c6e0 tx=0x7fc06c09c1e0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1440972457 shutdown_connections 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4eca90 0x55951d504850 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d504310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1440972457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4dd380 0x55951d4f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1440972457 >> 172.21.15.37:0/1440972457 conn(0x55951d4d9a30 msgr2=0x55951d4e4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1440972457 shutdown_connections 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1440972457 wait complete. 2024-09-19T06:43:26.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 -- start start 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.204+0000 7fc07ce9a740 1 Processor -- start 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4dd380 0x55951d4f5350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d4f0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07ce9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4eca90 0x55951d4f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741a9d30 con 0x55951d4dd380 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741ab1a0 con 0x55951d4eca90 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07ce9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc0741aa400 con 0x55951d4ddca0 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07ce9a740 10 client.0 _ll_register_callbacks cb 0x7fc060016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4dd380 0x55951d4f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4dd380 0x55951d4f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44720/0 (socket says 172.21.15.37:44720) 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 -- 172.21.15.37:0/1911364262 learned_addr learned my addr 172.21.15.37:0/1911364262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:26.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 -- 172.21.15.37:0/1911364262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 msgr2=0x55951d4f0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d4f0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 -- 172.21.15.37:0/1911364262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4eca90 msgr2=0x55951d4f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4eca90 0x55951d4f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 -- 172.21.15.37:0/1911364262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55951d5662e0 con 0x55951d4dd380 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc07885d640 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4dd380 0x55951d4f5350 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc068009230 tx=0x7fc06802ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20fd8ee5496b2255 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.205+0000 7fc053fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc053fff640 1 -- 172.21.15.37:0/1911364262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068040030 con 0x55951d4dd380 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc053fff640 1 -- 172.21.15.37:0/1911364262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fc068005040 con 0x55951d4dd380 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc06c00f8e0 con 0x55951d4dd380 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc053fff640 1 -- 172.21.15.37:0/1911364262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc068050050 con 0x55951d4dd380 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 10 client.15864 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:26.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55951d567400 con 0x55951d4dd380 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc053fff640 1 -- 172.21.15.37:0/1911364262 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc06c00f8e0 con 0x55951d4dd380 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc053fff640 1 -- 172.21.15.37:0/1911364262 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fc06804b030 con 0x55951d4dd380 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 20 client.15864 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d567ba0 0x55951d56a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 4 client.15864 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55951d44ad60 con 0x55951d567ba0 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.206+0000 7fc073fff640 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d567ba0 0x55951d56a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc073fff640 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d567ba0 0x55951d56a040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=148f79f85ce3c213 server_cookie=a567f38981ae791a in_seq=0 out_seq=0 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc053fff640 10 client.15864 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc053fff640 1 -- 172.21.15.37:0/1911364262 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55951d44ad60 con 0x55951d567ba0 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc053fff640 10 client.15864 handle_command_reply: tid=0 2024-09-19T06:43:26.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 client.15864 shutdown 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4dd380 msgr2=0x55951d4f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4dd380 0x55951d4f5350 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc068009230 tx=0x7fc06802ab20 comp rx=0 tx=0).stop 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 shutdown_connections 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55951d567ba0 0x55951d56a040 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55951d4eca90 0x55951d4f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55951d4ddca0 0x55951d4f0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc053fff640 0 client.15864 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 --2- 172.21.15.37:0/1911364262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55951d4dd380 0x55951d4f5350 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 >> 172.21.15.37:0/1911364262 conn(0x55951d4d9a30 msgr2=0x55951d4eaf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:26.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 shutdown_connections 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.207+0000 7fc07ce9a740 1 -- 172.21.15.37:0/1911364262 wait complete. 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07ce9a740 20 client.15864 trim_cache size 0 max 16384 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc0640a33b0 msgr2=0x7fc0640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc0640a33b0 0x7fc0640a5870 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc0740fdce0 tx=0x7fc06000aec0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074106f10 msgr2=0x7fc07419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074106f10 0x7fc07419cca0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc06c00cce0 tx=0x7fc06c00a620 comp rx=0 tx=0).stop 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 shutdown_connections 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc0640a33b0 0x7fc0640a5870 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc074106f10 0x7fc07419cca0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc074102790 0x7fc074198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 --2- 172.21.15.37:0/960821584 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc074101de0 0x7fc0741983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 >> 172.21.15.37:0/960821584 conn(0x7fc0740fd8c0 msgr2=0x7fc0740febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 shutdown_connections 2024-09-19T06:43:26.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:26.208+0000 7fc07aac1640 1 -- 172.21.15.37:0/960821584 wait complete. 2024-09-19T06:43:26.510 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:26 smithi037 ceph-mon[31211]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:26.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:26 smithi096 ceph-mon[35004]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:26.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:26 smithi151 ceph-mon[35236]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:27.001 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:43:27.002 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:43:27.261 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 -- 172.21.15.37:0/3254414226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 msgr2=0x7f61581008e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 --2- 172.21.15.37:0/3254414226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f61581008e0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f6148009900 tx=0x7f614802a410 comp rx=0 tx=0).stop 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 -- 172.21.15.37:0/3254414226 shutdown_connections 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 --2- 172.21.15.37:0/3254414226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158109110 0x7f6158110c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 --2- 172.21.15.37:0/3254414226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6158100eb0 0x7f61581089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 --2- 172.21.15.37:0/3254414226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f61581008e0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 -- 172.21.15.37:0/3254414226 >> 172.21.15.37:0/3254414226 conn(0x7f61580fc5c0 msgr2=0x7f61580fe9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 -- 172.21.15.37:0/3254414226 shutdown_connections 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 -- 172.21.15.37:0/3254414226 wait complete. 2024-09-19T06:43:27.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 Processor -- start 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.759+0000 7f615fddf640 1 -- start start 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615fddf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f615819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615fddf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100eb0 0x7f615819ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615fddf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6158109110 0x7f615819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615fddf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6158114150 con 0x7f6158100500 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615fddf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6158113fd0 con 0x7f6158100eb0 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615fddf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61581142d0 con 0x7f6158109110 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f615819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f615819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44744/0 (socket says 172.21.15.37:44744) 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 -- 172.21.15.37:0/1302468785 learned_addr learned my addr 172.21.15.37:0/1302468785 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 -- 172.21.15.37:0/1302468785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6158109110 msgr2=0x7f615819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6158109110 0x7f615819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 -- 172.21.15.37:0/1302468785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100eb0 msgr2=0x7f615819ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100eb0 0x7f615819ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 -- 172.21.15.37:0/1302468785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f615819f680 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.760+0000 7f615db7b640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f615819a6d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6148006010 tx=0x7f61480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.761+0000 7f6156ffd640 1 -- 172.21.15.37:0/1302468785 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614803f030 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.761+0000 7f6156ffd640 1 -- 172.21.15.37:0/1302468785 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6148003ce0 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.761+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f615819f910 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.761+0000 7f6156ffd640 1 -- 172.21.15.37:0/1302468785 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6148035070 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.761+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61581a7250 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.762+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61581008e0 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.763+0000 7f6156ffd640 1 -- 172.21.15.37:0/1302468785 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6148042030 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.764+0000 7f6156ffd640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61400a3430 0x7f61400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.764+0000 7f615d37a640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61400a3430 0x7f61400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.764+0000 7f6156ffd640 1 -- 172.21.15.37:0/1302468785 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f61480f0360 con 0x7f6158100500 2024-09-19T06:43:27.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.765+0000 7f615d37a640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61400a3430 0x7f61400a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f614c005fd0 tx=0x7f614c005850 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:27.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.769+0000 7f6156ffd640 1 -- 172.21.15.37:0/1302468785 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f61480bce00 con 0x7f6158100500 2024-09-19T06:43:27.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.981+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f615819ba30 con 0x7f6158100500 2024-09-19T06:43:27.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.982+0000 7f6156ffd640 1 -- 172.21.15.37:0/1302468785 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7f61480bc700 con 0x7f6158100500 2024-09-19T06:43:27.984 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:43:27.985 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:43:27.985 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61400a3430 msgr2=0x7f61400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61400a3430 0x7f61400a58f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f614c005fd0 tx=0x7f614c005850 comp rx=0 tx=0).stop 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 msgr2=0x7f615819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f615819a6d0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f6148006010 tx=0x7f61480042f0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 shutdown_connections 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6158109110 0x7f615819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6158100eb0 0x7f615819ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61400a3430 0x7f61400a58f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 --2- 172.21.15.37:0/1302468785 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6158100500 0x7f615819a6d0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 >> 172.21.15.37:0/1302468785 conn(0x7f61580fc5c0 msgr2=0x7f6158101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:27.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 shutdown_connections 2024-09-19T06:43:27.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:27.985+0000 7f615fddf640 1 -- 172.21.15.37:0/1302468785 wait complete. 2024-09-19T06:43:28.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:28 smithi096 ceph-mon[35004]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:28.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:28 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1302468785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:28.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:28 smithi037 ceph-mon[31211]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:28.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:28 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1302468785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:28.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:28 smithi151 ceph-mon[35236]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.9 KiB/s wr, 7 op/s 2024-09-19T06:43:28.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:28 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1302468785' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:28.677 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:43:28.936 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:28.981 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:43:29.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 -- 172.21.15.37:0/427340432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 msgr2=0x7f99a010d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/427340432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a010d950 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9998009cd0 tx=0x7f999802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 -- 172.21.15.37:0/427340432 shutdown_connections 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/427340432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a010d950 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/427340432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a0100a80 0x7f99a0100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/427340432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a0106a80 0x7f99a0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 -- 172.21.15.37:0/427340432 >> 172.21.15.37:0/427340432 conn(0x7f99a00fc7a0 msgr2=0x7f99a00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 -- 172.21.15.37:0/427340432 shutdown_connections 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.401+0000 7f99a7cc2640 1 -- 172.21.15.37:0/427340432 wait complete. 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 Processor -- start 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 -- start start 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a0100a80 0x7f99a01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a0106a80 0x7f99a019cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0110e80 con 0x7f99a0101420 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0110d00 con 0x7f99a0100a80 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a7cc2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99a0111000 con 0x7f99a0106a80 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a0198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44770/0 (socket says 172.21.15.37:44770) 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 -- 172.21.15.37:0/638047563 learned_addr learned my addr 172.21.15.37:0/638047563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a0100a80 0x7f99a01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a625f640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a0106a80 0x7f99a019cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 -- 172.21.15.37:0/638047563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a0106a80 msgr2=0x7f99a019cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a0106a80 0x7f99a019cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 -- 172.21.15.37:0/638047563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a0100a80 msgr2=0x7f99a01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a0100a80 0x7f99a01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a525d640 1 -- 172.21.15.37:0/638047563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99a019d280 con 0x7f99a0101420 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a0100a80 0x7f99a01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.402+0000 7f99a625f640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a0106a80 0x7f99a019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.403+0000 7f99a525d640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a0198810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f999000c920 tx=0x7f999000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.403+0000 7f9996ffd640 1 -- 172.21.15.37:0/638047563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990007c70 con 0x7f99a0101420 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.403+0000 7f9996ffd640 1 -- 172.21.15.37:0/638047563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9990007e10 con 0x7f99a0101420 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.403+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99a019d570 con 0x7f99a0101420 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.403+0000 7f9996ffd640 1 -- 172.21.15.37:0/638047563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9990018890 con 0x7f99a0101420 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.403+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99a01a4e50 con 0x7f99a0101420 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.405+0000 7f9996ffd640 1 -- 172.21.15.37:0/638047563 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9990018a70 con 0x7f99a0101420 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.405+0000 7f99aa09b740 1 Processor -- start 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99aa09b740 1 -- start start 2024-09-19T06:43:29.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a211090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a211a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a222b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99aa09b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56432a1772f0 con 0x56432a220780 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99aa09b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56432a11ecf0 con 0x56432a212b40 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99aa09b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56432a1732b0 con 0x56432a2115d0 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a222b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a222b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44780/0 (socket says 172.21.15.37:44780) 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 -- 172.21.15.37:0/4268704050 learned_addr learned my addr 172.21.15.37:0/4268704050 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a211a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a625f640 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a211090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f9996ffd640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f998c0a3430 0x7f998c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 -- 172.21.15.37:0/4268704050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 msgr2=0x56432a211a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a211a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 -- 172.21.15.37:0/4268704050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 msgr2=0x56432a211090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a211090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a525d640 1 -- 172.21.15.37:0/4268704050 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432a1033a0 con 0x56432a220780 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.406+0000 7f99a625f640 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a211090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.407+0000 7f99a625f640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f998c0a3430 0x7f998c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.407+0000 7f9996ffd640 1 -- 172.21.15.37:0/638047563 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f99900cfe80 con 0x7f99a0101420 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.407+0000 7f99a525d640 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a222b70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f99900cd570 tx=0x7f9990020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32025a71a931262 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.407+0000 7f998bfff640 1 -- 172.21.15.37:0/4268704050 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999009bf30 con 0x56432a220780 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.407+0000 7f998bfff640 1 -- 172.21.15.37:0/4268704050 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999009c860 con 0x56432a220780 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.407+0000 7f998bfff640 1 -- 172.21.15.37:0/4268704050 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99900d36e0 con 0x56432a220780 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 -- 172.21.15.37:0/4268704050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 msgr2=0x56432a222b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a222b70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f99900cd570 tx=0x7f9990020000 comp rx=0 tx=0).stop 2024-09-19T06:43:29.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99a625f640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f998c0a3430 0x7f998c0a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9998006010 tx=0x7f99980095c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 -- 172.21.15.37:0/4268704050 shutdown_connections 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a222b70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a211a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 --2- 172.21.15.37:0/4268704050 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a211090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 -- 172.21.15.37:0/4268704050 >> 172.21.15.37:0/4268704050 conn(0x56432a1da310 msgr2=0x56432a20dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 -- 172.21.15.37:0/4268704050 shutdown_connections 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 -- 172.21.15.37:0/4268704050 wait complete. 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 -- start start 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.408+0000 7f99aa09b740 1 Processor -- start 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a220160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a21d410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a21d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99aa09b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56432a223420 con 0x56432a220780 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99aa09b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56432a132060 con 0x56432a212b40 2024-09-19T06:43:29.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99aa09b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56432a2235a0 con 0x56432a2115d0 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99aa09b740 10 client.0 _ll_register_callbacks cb 0x564329e858c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a21d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a21d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44792/0 (socket says 172.21.15.37:44792) 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 -- 172.21.15.37:0/1597372822 learned_addr learned my addr 172.21.15.37:0/1597372822 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a525d640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a21d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a220160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 -- 172.21.15.37:0/1597372822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 msgr2=0x56432a220160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a220160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 -- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 msgr2=0x56432a21d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a21d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 -- 172.21.15.37:0/1597372822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564329ee83c0 con 0x56432a220780 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a525d640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a21d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:29.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99a625f640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a21d9e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9998003c20 tx=0x7f999802a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db5d1fe0903e89e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.409+0000 7f99897fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99897fa640 1 -- 172.21.15.37:0/1597372822 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9998004450 con 0x56432a220780 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999009c860 con 0x56432a220780 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99897fa640 1 -- 172.21.15.37:0/1597372822 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f99980045f0 con 0x56432a220780 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99897fa640 1 -- 172.21.15.37:0/1597372822 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f99980047f0 con 0x56432a220780 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56432a22eea0 con 0x56432a220780 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99897fa640 1 -- 172.21.15.37:0/1597372822 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999009c860 con 0x56432a220780 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99897fa640 1 -- 172.21.15.37:0/1597372822 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f9998030070 con 0x56432a220780 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 10 client.15882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 20 client.15882 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:29.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a2302b0 0x56432a2326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 4 client.15882 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56432a184860 con 0x56432a2302b0 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.410+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a2302b0 0x56432a2326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.411+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a2302b0 0x56432a2326a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e660cf45220ff764 server_cookie=59555554fd568bcd in_seq=0 out_seq=0 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.411+0000 7f99897fa640 10 client.15882 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99897fa640 1 -- 172.21.15.37:0/1597372822 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56432a184860 con 0x56432a2302b0 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99897fa640 10 client.15882 handle_command_reply: tid=0 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 client.15882 shutdown 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 msgr2=0x56432a21d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a21d9e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9998003c20 tx=0x7f999802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 shutdown_connections 2024-09-19T06:43:29.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a2302b0 0x56432a2326a0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a220780 0x56432a21d9e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a212b40 0x56432a21d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99897fa640 0 client.15882 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 --2- 172.21.15.37:0/1597372822 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a2115d0 0x56432a220160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 >> 172.21.15.37:0/1597372822 conn(0x56432a1da310 msgr2=0x56432a213960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 shutdown_connections 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 1 -- 172.21.15.37:0/1597372822 wait complete. 2024-09-19T06:43:29.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.412+0000 7f99aa09b740 20 client.15882 trim_cache size 0 max 16384 2024-09-19T06:43:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 Processor -- start 2024-09-19T06:43:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 -- start start 2024-09-19T06:43:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a2115d0 0x56432a22c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a212b40 0x56432a21f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a220780 0x56432a237f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99a01addb0 con 0x56432a212b40 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99a01ab680 con 0x56432a2115d0 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99aa09b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99a01a94d0 con 0x56432a220780 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99a525d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a212b40 0x56432a21f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99a525d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a212b40 0x56432a21f380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53816/0 (socket says 172.21.15.37:53816) 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99a525d640 1 -- 172.21.15.37:0/117838549 learned_addr learned my addr 172.21.15.37:0/117838549 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.426+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a2115d0 0x56432a22c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f99a625f640 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a220780 0x56432a237f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f99a525d640 1 -- 172.21.15.37:0/117838549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a220780 msgr2=0x56432a237f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f99a525d640 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a220780 0x56432a237f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f99a525d640 1 -- 172.21.15.37:0/117838549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a2115d0 msgr2=0x56432a22c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f99a525d640 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a2115d0 0x56432a22c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f99a525d640 1 -- 172.21.15.37:0/117838549 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56432a21b960 con 0x56432a212b40 2024-09-19T06:43:29.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f99a525d640 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a212b40 0x56432a21f380 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f99900cdbd0 tx=0x7f99900d0180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bb6f434e1da76df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f998b7fe640 1 -- 172.21.15.37:0/117838549 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999001dd40 con 0x56432a212b40 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f998b7fe640 1 -- 172.21.15.37:0/117838549 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999009c860 con 0x56432a212b40 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.427+0000 7f998b7fe640 1 -- 172.21.15.37:0/117838549 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999009bac0 con 0x56432a212b40 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 -- 172.21.15.37:0/117838549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a212b40 msgr2=0x56432a21f380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a212b40 0x56432a21f380 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f99900cdbd0 tx=0x7f99900d0180 comp rx=0 tx=0).stop 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 -- 172.21.15.37:0/117838549 shutdown_connections 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a220780 0x56432a237f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a212b40 0x56432a21f380 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 --2- 172.21.15.37:0/117838549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a2115d0 0x56432a22c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 -- 172.21.15.37:0/117838549 >> 172.21.15.37:0/117838549 conn(0x56432a1da310 msgr2=0x56432a21ec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 -- 172.21.15.37:0/117838549 shutdown_connections 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 -- 172.21.15.37:0/117838549 wait complete. 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.428+0000 7f99aa09b740 1 -- start start 2024-09-19T06:43:29.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 1 Processor -- start 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a2115d0 0x56432a228f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a212b40 0x56432a224260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a220780 0x56432a2247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99a01a9b40 con 0x56432a2115d0 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99a01aafb0 con 0x56432a220780 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99a01aa210 con 0x56432a212b40 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99a5a5e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a2115d0 0x56432a228f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99a5a5e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a2115d0 0x56432a228f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53830/0 (socket says 172.21.15.37:53830) 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99a5a5e640 1 -- 172.21.15.37:0/2733595438 learned_addr learned my addr 172.21.15.37:0/2733595438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99aa09b740 10 client.0 _ll_register_callbacks cb 0x7f9998009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99a525d640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a212b40 0x56432a224260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.429+0000 7f99a625f640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a220780 0x56432a2247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.437 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99a5a5e640 1 -- 172.21.15.37:0/2733595438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a212b40 msgr2=0x56432a224260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a212b40 0x56432a224260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99a5a5e640 1 -- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a220780 msgr2=0x56432a2247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a220780 0x56432a2247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99a5a5e640 1 -- 172.21.15.37:0/2733595438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56432a224f80 con 0x56432a2115d0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99a625f640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a220780 0x56432a2247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99a5a5e640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a2115d0 0x56432a228f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f999c000c00 tx=0x7f999c00f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=389b2a1ec368c753 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99897fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99897fa640 1 -- 172.21.15.37:0/2733595438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f999c01c9a0 con 0x56432a2115d0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99897fa640 1 -- 172.21.15.37:0/2733595438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f999c01cb40 con 0x56432a2115d0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f999009c860 con 0x56432a2115d0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99897fa640 1 -- 172.21.15.37:0/2733595438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f999c025450 con 0x56432a2115d0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99aa09b740 10 client.15888 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.430+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56432a29af30 con 0x56432a2115d0 2024-09-19T06:43:29.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99897fa640 1 -- 172.21.15.37:0/2733595438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f999009c860 con 0x56432a2115d0 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99897fa640 1 -- 172.21.15.37:0/2733595438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f999c02e6e0 con 0x56432a2115d0 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99aa09b740 10 client.15888 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99aa09b740 10 client.15888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99aa09b740 20 client.15888 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99aa09b740 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a29b5b0 0x56432a29d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99aa09b740 4 client.15888 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56432a17ed70 con 0x56432a29b5b0 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.431+0000 7f99a525d640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a29b5b0 0x56432a29d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99a525d640 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a29b5b0 0x56432a29d9a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eef3f99b6b4480b1 server_cookie=335cfb616d8446ab in_seq=0 out_seq=0 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99897fa640 10 client.15888 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99897fa640 1 -- 172.21.15.37:0/2733595438 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56432a17ed70 con 0x56432a29b5b0 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99897fa640 10 client.15888 handle_command_reply: tid=0 2024-09-19T06:43:29.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 client.15888 shutdown 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a2115d0 msgr2=0x56432a228f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a2115d0 0x56432a228f30 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f999c000c00 tx=0x7f999c00f8b0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 shutdown_connections 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56432a29b5b0 0x56432a29d9a0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56432a220780 0x56432a2247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56432a212b40 0x56432a224260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 --2- 172.21.15.37:0/2733595438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56432a2115d0 0x56432a228f30 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99897fa640 0 client.15888 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 >> 172.21.15.37:0/2733595438 conn(0x56432a1da310 msgr2=0x56432a21f490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 shutdown_connections 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 1 -- 172.21.15.37:0/2733595438 wait complete. 2024-09-19T06:43:29.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.432+0000 7f99aa09b740 20 client.15888 trim_cache size 0 max 16384 2024-09-19T06:43:29.441 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:43:29.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f998c0a3430 msgr2=0x7f998c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f998c0a3430 0x7f998c0a58f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9998006010 tx=0x7f99980095c0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 msgr2=0x7f99a0198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a0198810 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f999000c920 tx=0x7f999000cde0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 shutdown_connections 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99a0106a80 0x7f99a019cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f998c0a3430 0x7f998c0a58f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99a0101420 0x7f99a0198810 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 --2- 172.21.15.37:0/638047563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99a0100a80 0x7f99a01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 >> 172.21.15.37:0/638047563 conn(0x7f99a00fc7a0 msgr2=0x7f99a00fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.433+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 shutdown_connections 2024-09-19T06:43:29.442 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:29.434+0000 7f99a7cc2640 1 -- 172.21.15.37:0/638047563 wait complete. 2024-09-19T06:43:30.092 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:30.107 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:43:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.250+0000 7f5ed3242640 1 -- 172.21.15.37:0/2280046852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc103640 msgr2=0x7f5ecc10bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.250+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2280046852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc103640 0x7f5ecc10bb20 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009900 tx=0x7f5eb402a480 comp rx=0 tx=0).stop 2024-09-19T06:43:30.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- 172.21.15.37:0/2280046852 shutdown_connections 2024-09-19T06:43:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2280046852 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ecc10c0f0 0x7f5ecc10e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2280046852 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc103640 0x7f5ecc10bb20 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2280046852 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ecc102d20 0x7f5ecc103100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- 172.21.15.37:0/2280046852 >> 172.21.15.37:0/2280046852 conn(0x7f5ecc077990 msgr2=0x7f5ecc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- 172.21.15.37:0/2280046852 shutdown_connections 2024-09-19T06:43:30.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- 172.21.15.37:0/2280046852 wait complete. 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 Processor -- start 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- start start 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ecc102d20 0x7f5ecc198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ecc103640 0x7f5ecc198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc10c0f0 0x7f5ecc19cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111ec0 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc111d40 con 0x7f5ecc102d20 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.251+0000 7f5ed3242640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc112040 con 0x7f5ecc103640 2024-09-19T06:43:30.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc10c0f0 0x7f5ecc19cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc10c0f0 0x7f5ecc19cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53852/0 (socket says 172.21.15.37:53852) 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 -- 172.21.15.37:0/2010031777 learned_addr learned my addr 172.21.15.37:0/2010031777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ecc102d20 0x7f5ecc198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ecc103640 0x7f5ecc198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 -- 172.21.15.37:0/2010031777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ecc103640 msgr2=0x7f5ecc198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ecc103640 0x7f5ecc198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 -- 172.21.15.37:0/2010031777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ecc102d20 msgr2=0x7f5ecc198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ecc102d20 0x7f5ecc198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 -- 172.21.15.37:0/2010031777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ecc19d240 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed17df640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc10c0f0 0x7f5ecc19cb10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400cce0 tx=0x7f5ec400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ec9ffb640 1 -- 172.21.15.37:0/2010031777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec400f450 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ecc19d530 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ec9ffb640 1 -- 172.21.15.37:0/2010031777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec400fad0 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.252+0000 7f5ec9ffb640 1 -- 172.21.15.37:0/2010031777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4018b60 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.253+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ecc1a4dc0 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.255+0000 7f5ed5fd1740 1 Processor -- start 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.255+0000 7f5ec9ffb640 1 -- 172.21.15.37:0/2010031777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5ec400fc80 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.255+0000 7f5ed5fd1740 1 -- start start 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd890ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd89b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd89e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd7f52f0 con 0x55b3bd890af0 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd79d270 con 0x55b3bd89bcb0 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd7f12b0 con 0x55b3bd891410 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd890ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd890ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53866/0 (socket says 172.21.15.37:53866) 2024-09-19T06:43:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 -- 172.21.15.37:0/637582242 learned_addr learned my addr 172.21.15.37:0/637582242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd89b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd89e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 -- 172.21.15.37:0/637582242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 msgr2=0x55b3bd89b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd89b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 -- 172.21.15.37:0/637582242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 msgr2=0x55b3bd89e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd89e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 -- 172.21.15.37:0/637582242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bd79c340 con 0x55b3bd890af0 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ec9ffb640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5eb80a3460 0x7f5eb80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd890ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5eb402a9a0 tx=0x7f5eb4005d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=236b9dbb4ed2f0af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.256+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5eb80a3460 0x7f5eb80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.257+0000 7f5ec9ffb640 1 -- 172.21.15.37:0/2010031777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f5ec40d0140 con 0x7f5ecc10c0f0 2024-09-19T06:43:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.257+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5eb80a3460 0x7f5eb80a5920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55b3bd88dec0 tx=0x7f5ebc00a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.258+0000 7f5ec2ffd640 1 -- 172.21.15.37:0/637582242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4044020 con 0x55b3bd890af0 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.258+0000 7f5ec2ffd640 1 -- 172.21.15.37:0/637582242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb40035f0 con 0x55b3bd890af0 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.258+0000 7f5ec2ffd640 1 -- 172.21.15.37:0/637582242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4035040 con 0x55b3bd890af0 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/637582242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 msgr2=0x55b3bd890ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd890ed0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f5eb402a9a0 tx=0x7f5eb4005d90 comp rx=0 tx=0).stop 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/637582242 shutdown_connections 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd89e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd89b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/637582242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd890ed0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/637582242 >> 172.21.15.37:0/637582242 conn(0x55b3bd88d060 msgr2=0x55b3bd88d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/637582242 shutdown_connections 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.259+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/637582242 wait complete. 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 -- start start 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 Processor -- start 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd890af0 0x55b3bd898b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd891410 0x55b3bd899090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8995d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd89e880 con 0x55b3bd891410 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd77e290 con 0x55b3bd890af0 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b3bd89ea00 con 0x55b3bd89bcb0 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed5fd1740 10 client.0 _ll_register_callbacks cb 0x55b3bd7af140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd891410 0x55b3bd899090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ecbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd891410 0x55b3bd899090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53870/0 (socket says 172.21.15.37:53870) 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ecbfff640 1 -- 172.21.15.37:0/3466500402 learned_addr learned my addr 172.21.15.37:0/3466500402 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed0fde640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd890af0 0x55b3bd898b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.260+0000 7f5ed17df640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed0fde640 1 -- 172.21.15.37:0/3466500402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd89bcb0 msgr2=0x55b3bd8995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed0fde640 1 -- 172.21.15.37:0/3466500402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd891410 msgr2=0x55b3bd899090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd891410 0x55b3bd899090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed0fde640 1 -- 172.21.15.37:0/3466500402 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3bd508210 con 0x55b3bd890af0 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd891410 0x55b3bd899090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed17df640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd890af0 0x55b3bd898b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc01cc30 tx=0x7f5ebc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e608338cb0fe03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5eabfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5eabfff640 1 -- 172.21.15.37:0/3466500402 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc00f8a0 con 0x55b3bd890af0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5eb40035f0 con 0x55b3bd890af0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.261+0000 7f5eabfff640 1 -- 172.21.15.37:0/3466500402 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f5ebc016030 con 0x55b3bd890af0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5eabfff640 1 -- 172.21.15.37:0/3466500402 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ebc032050 con 0x55b3bd890af0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 10 client.25442 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3bd5d4620 con 0x55b3bd890af0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5eabfff640 1 -- 172.21.15.37:0/3466500402 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5eb40035f0 con 0x55b3bd890af0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5eabfff640 1 -- 172.21.15.37:0/3466500402 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ebc02d030 con 0x55b3bd890af0 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 10 client.25442 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 10 client.25442 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:30.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 20 client.25442 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd8abd40 0x55b3bd8ae170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 4 client.25442 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b3bd802a20 con 0x55b3bd8abd40 2024-09-19T06:43:30.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.262+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd8abd40 0x55b3bd8ae170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.263+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd8abd40 0x55b3bd8ae170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a8ec2f23b08dbae server_cookie=33e255955138f63c in_seq=0 out_seq=0 2024-09-19T06:43:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.263+0000 7f5eabfff640 10 client.25442 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.263+0000 7f5eabfff640 1 -- 172.21.15.37:0/3466500402 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b3bd802a20 con 0x55b3bd8abd40 2024-09-19T06:43:30.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.263+0000 7f5eabfff640 10 client.25442 handle_command_reply: tid=0 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 client.25442 shutdown 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd890af0 msgr2=0x55b3bd898b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd890af0 0x55b3bd898b50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc01cc30 tx=0x7f5ebc004730 comp rx=0 tx=0).stop 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 shutdown_connections 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd8abd40 0x55b3bd8ae170 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5eabfff640 0 client.25442 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd891410 0x55b3bd899090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3466500402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd890af0 0x55b3bd898b50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 >> 172.21.15.37:0/3466500402 conn(0x55b3bd88d060 msgr2=0x55b3bd892dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 shutdown_connections 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3466500402 wait complete. 2024-09-19T06:43:30.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.264+0000 7f5ed5fd1740 20 client.25442 trim_cache size 0 max 16384 2024-09-19T06:43:30.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 Processor -- start 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 -- start start 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd890af0 0x55b3bd89aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd891410 0x55b3bd899400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8b3a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1aff30 con 0x55b3bd89bcb0 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1ad800 con 0x55b3bd891410 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1ab650 con 0x55b3bd890af0 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8b3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8b3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53872/0 (socket says 172.21.15.37:53872) 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 -- 172.21.15.37:0/3931535025 learned_addr learned my addr 172.21.15.37:0/3931535025 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd891410 0x55b3bd899400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd890af0 0x55b3bd89aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 -- 172.21.15.37:0/3931535025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd890af0 msgr2=0x55b3bd89aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd890af0 0x55b3bd89aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 -- 172.21.15.37:0/3931535025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd891410 msgr2=0x55b3bd899400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd891410 0x55b3bd899400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.280+0000 7f5ed17df640 1 -- 172.21.15.37:0/3931535025 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b3bd8ae7f0 con 0x55b3bd89bcb0 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.281+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd890af0 0x55b3bd89aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.281+0000 7f5ed17df640 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8b3a80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4098df0 tx=0x7f5ec409c080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8b38f29465eedf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.281+0000 7f5ec27fc640 1 -- 172.21.15.37:0/3931535025 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40b4070 con 0x55b3bd89bcb0 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.281+0000 7f5ec27fc640 1 -- 172.21.15.37:0/3931535025 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec40a07d0 con 0x55b3bd89bcb0 2024-09-19T06:43:30.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.281+0000 7f5ec27fc640 1 -- 172.21.15.37:0/3931535025 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec409bad0 con 0x55b3bd89bcb0 2024-09-19T06:43:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3931535025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd89bcb0 msgr2=0x55b3bd8b3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8b3a80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4098df0 tx=0x7f5ec409c080 comp rx=0 tx=0).stop 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3931535025 shutdown_connections 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd89bcb0 0x55b3bd8b3a80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd891410 0x55b3bd899400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/3931535025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd890af0 0x55b3bd89aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3931535025 >> 172.21.15.37:0/3931535025 conn(0x55b3bd88d060 msgr2=0x55b3bd89a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3931535025 shutdown_connections 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/3931535025 wait complete. 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 -- start start 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.282+0000 7f5ed5fd1740 1 Processor -- start 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd8a4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd8a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd895750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1abcc0 con 0x55b3bd890af0 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1ad130 con 0x55b3bd89bcb0 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5ecc1ac390 con 0x55b3bd891410 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 10 client.0 _ll_register_callbacks cb 0x7f5ebc00a6e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd8a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd8a4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53878/0 (socket says 172.21.15.37:53878) 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 -- 172.21.15.37:0/988125821 learned_addr learned my addr 172.21.15.37:0/988125821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd8a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed17df640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd895750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 -- 172.21.15.37:0/988125821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 msgr2=0x55b3bd8a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd8a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 -- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 msgr2=0x55b3bd895750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:30.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd895750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 -- 172.21.15.37:0/988125821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b3bd88dcf0 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd8a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed0fde640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd8a4070 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005b70 tx=0x7f5ebc00aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c701b4d8e60cae59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5eabfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5eabfff640 1 -- 172.21.15.37:0/988125821 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc03a070 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5eabfff640 1 -- 172.21.15.37:0/988125821 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f5ebc00fb40 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.283+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ec40a07d0 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5eabfff640 1 -- 172.21.15.37:0/988125821 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5ebc045050 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 10 client.15906 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b3bd8aa870 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5eabfff640 1 -- 172.21.15.37:0/988125821 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ec40a07d0 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5eabfff640 1 -- 172.21.15.37:0/988125821 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f5ebc02d070 con 0x55b3bd890af0 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 10 client.15906 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:30.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 20 client.15906 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd916a00 0x55b3bd918e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 4 client.15906 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:43:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b3bd7fcd70 con 0x55b3bd916a00 2024-09-19T06:43:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.284+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd916a00 0x55b3bd918e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.285+0000 7f5eabfff640 10 client.15906 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:30.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.285+0000 7f5ecbfff640 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd916a00 0x55b3bd918e20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d294db2b0b37eb8 server_cookie=92255e655e1d9b4b in_seq=0 out_seq=0 2024-09-19T06:43:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.288+0000 7f5eabfff640 1 -- 172.21.15.37:0/988125821 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b3bd7fcd70 con 0x55b3bd916a00 2024-09-19T06:43:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.288+0000 7f5eabfff640 10 client.15906 handle_command_reply: tid=0 2024-09-19T06:43:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.288+0000 7f5ed5fd1740 1 client.15906 shutdown 2024-09-19T06:43:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 msgr2=0x55b3bd8a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd8a4070 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc005b70 tx=0x7f5ebc00aa60 comp rx=0 tx=0).stop 2024-09-19T06:43:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 shutdown_connections 2024-09-19T06:43:30.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b3bd916a00 0x55b3bd918e20 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b3bd89bcb0 0x55b3bd895750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5eabfff640 0 client.15906 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b3bd891410 0x55b3bd8a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 --2- 172.21.15.37:0/988125821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b3bd890af0 0x55b3bd8a4070 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 >> 172.21.15.37:0/988125821 conn(0x55b3bd88d060 msgr2=0x55b3bd88c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 shutdown_connections 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 1 -- 172.21.15.37:0/988125821 wait complete. 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.289+0000 7f5ed5fd1740 20 client.15906 trim_cache size 0 max 16384 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:43:30.294 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:43:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:43:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5eb80a3460 msgr2=0x7f5eb80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5eb80a3460 0x7f5eb80a5920 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55b3bd88dec0 tx=0x7f5ebc00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc10c0f0 msgr2=0x7f5ecc19cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc10c0f0 0x7f5ecc19cb10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400cce0 tx=0x7f5ec400a620 comp rx=0 tx=0).stop 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 shutdown_connections 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5eb80a3460 0x7f5eb80a5920 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5ecc10c0f0 0x7f5ecc19cb10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5ecc103640 0x7f5ecc198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 --2- 172.21.15.37:0/2010031777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5ecc102d20 0x7f5ecc198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 >> 172.21.15.37:0/2010031777 conn(0x7f5ecc077990 msgr2=0x7f5ecc0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 shutdown_connections 2024-09-19T06:43:30.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:30.290+0000 7f5ed3242640 1 -- 172.21.15.37:0/2010031777 wait complete. 2024-09-19T06:43:30.383 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:30 smithi037 ceph-mon[31211]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:43:30.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:30 smithi096 ceph-mon[35004]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:43:30.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:30 smithi151 ceph-mon[35236]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.9 KiB/s wr, 9 op/s 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 -- 172.21.15.37:0/1743690774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc10c180 msgr2=0x7fcecc10e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 --2- 172.21.15.37:0/1743690774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc10c180 0x7fcecc10e5f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009ca0 tx=0x7fcec402a950 comp rx=0 tx=0).stop 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 -- 172.21.15.37:0/1743690774 shutdown_connections 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 --2- 172.21.15.37:0/1743690774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc10c180 0x7fcecc10e5f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 --2- 172.21.15.37:0/1743690774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc1036c0 0x7fcecc10bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 --2- 172.21.15.37:0/1743690774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcecc102da0 0x7fcecc103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 -- 172.21.15.37:0/1743690774 >> 172.21.15.37:0/1743690774 conn(0x7fcecc0fc780 msgr2=0x7fcecc0feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 -- 172.21.15.37:0/1743690774 shutdown_connections 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 -- 172.21.15.37:0/1743690774 wait complete. 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.602+0000 7fced39db640 1 Processor -- start 2024-09-19T06:43:30.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced39db640 1 -- start start 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced39db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc102da0 0x7fcecc100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced39db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc1036c0 0x7fcecc100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced39db640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcecc10c180 0x7fcecc101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced39db640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc111f50 con 0x7fcecc1036c0 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced39db640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc111dd0 con 0x7fcecc102da0 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced39db640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcecc1120d0 con 0x7fcecc10c180 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced1777640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc102da0 0x7fcecc100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced1777640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc102da0 0x7fcecc100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53180/0 (socket says 172.21.15.37:53180) 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced0f76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc1036c0 0x7fcecc100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced1777640 1 -- 172.21.15.37:0/1072738160 learned_addr learned my addr 172.21.15.37:0/1072738160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced1f78640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcecc10c180 0x7fcecc101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced0f76640 1 -- 172.21.15.37:0/1072738160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcecc10c180 msgr2=0x7fcecc101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced0f76640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcecc10c180 0x7fcecc101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced0f76640 1 -- 172.21.15.37:0/1072738160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc102da0 msgr2=0x7fcecc100a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced0f76640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc102da0 0x7fcecc100a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced0f76640 1 -- 172.21.15.37:0/1072738160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcecc1aad40 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.603+0000 7fced1777640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc102da0 0x7fcecc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.604+0000 7fced0f76640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc1036c0 0x7fcecc100f40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c970 tx=0x7fcebc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.604+0000 7fcec27fc640 1 -- 172.21.15.37:0/1072738160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc007bb0 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.604+0000 7fcec27fc640 1 -- 172.21.15.37:0/1072738160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcebc007d50 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.604+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcecc1aaf70 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.604+0000 7fcec27fc640 1 -- 172.21.15.37:0/1072738160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcebc018820 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.604+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcecc1ab420 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.605+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcecc06aff0 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.606+0000 7fcec27fc640 1 -- 172.21.15.37:0/1072738160 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcebc0189c0 con 0x7fcecc1036c0 2024-09-19T06:43:30.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.607+0000 7fcec27fc640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fceb80a3430 0x7fceb80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.607+0000 7fced1777640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fceb80a3430 0x7fceb80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.607+0000 7fcec27fc640 1 -- 172.21.15.37:0/1072738160 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fcebc0cfd10 con 0x7fcecc1036c0 2024-09-19T06:43:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.607+0000 7fced1777640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fceb80a3430 0x7fceb80a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcec80058c0 tx=0x7fcec800a380 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:30.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.611+0000 7fcec27fc640 1 -- 172.21.15.37:0/1072738160 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcebc0d5020 con 0x7fcecc1036c0 2024-09-19T06:43:30.658 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.833+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcecc103180 con 0x7fcecc1036c0 2024-09-19T06:43:30.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.834+0000 7fcec27fc640 1 -- 172.21.15.37:0/1072738160 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7fcebc014030 con 0x7fcecc1036c0 2024-09-19T06:43:30.835 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:43:30.835 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:43:30.835 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fceb80a3430 msgr2=0x7fceb80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fceb80a3430 0x7fceb80a58f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcec80058c0 tx=0x7fcec800a380 comp rx=0 tx=0).stop 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc1036c0 msgr2=0x7fcecc100f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc1036c0 0x7fcecc100f40 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00c970 tx=0x7fcebc00ce30 comp rx=0 tx=0).stop 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 shutdown_connections 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcecc10c180 0x7fcecc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fceb80a3430 0x7fceb80a58f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcecc1036c0 0x7fcecc100f40 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 --2- 172.21.15.37:0/1072738160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcecc102da0 0x7fcecc100a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 >> 172.21.15.37:0/1072738160 conn(0x7fcecc0fc780 msgr2=0x7fcecc0fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 shutdown_connections 2024-09-19T06:43:30.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:30.836+0000 7fced39db640 1 -- 172.21.15.37:0/1072738160 wait complete. 2024-09-19T06:43:31.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:31 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1072738160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:31.532 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:31 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1072738160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:31.576 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 6fb810d1 --release --await' 2024-09-19T06:43:31.576 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 6fb810d1 --release --await 2024-09-19T06:43:31.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:31 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1072738160' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:31.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.747+0000 7fd958615640 1 -- 172.21.15.37:0/1059259858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102ef0 msgr2=0x7fd95010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.747+0000 7fd958615640 1 --2- 172.21.15.37:0/1059259858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102ef0 0x7fd95010da40 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd948009ca0 tx=0x7fd94802a950 comp rx=0 tx=0).stop 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 -- 172.21.15.37:0/1059259858 shutdown_connections 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 --2- 172.21.15.37:0/1059259858 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102ef0 0x7fd95010da40 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 --2- 172.21.15.37:0/1059259858 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd950102570 0x7fd9501029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 --2- 172.21.15.37:0/1059259858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd950100e20 0x7fd950101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 -- 172.21.15.37:0/1059259858 >> 172.21.15.37:0/1059259858 conn(0x7fd9500fc7d0 msgr2=0x7fd9500febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 -- 172.21.15.37:0/1059259858 shutdown_connections 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 -- 172.21.15.37:0/1059259858 wait complete. 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 Processor -- start 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 -- start start 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd950100e20 0x7fd9501993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102570 0x7fd950199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.748+0000 7fd958615640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd950102ef0 0x7fd950199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd958615640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd950110f80 con 0x7fd950102570 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd958615640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd950110e00 con 0x7fd950100e20 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd958615640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd950111100 con 0x7fd950102ef0 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102570 0x7fd950199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102570 0x7fd950199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53908/0 (socket says 172.21.15.37:53908) 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 -- 172.21.15.37:0/3300145128 learned_addr learned my addr 172.21.15.37:0/3300145128 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd9563b1640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd950100e20 0x7fd9501993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd956bb2640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd950102ef0 0x7fd950199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 -- 172.21.15.37:0/3300145128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd950102ef0 msgr2=0x7fd950199e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd950102ef0 0x7fd950199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 -- 172.21.15.37:0/3300145128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd950100e20 msgr2=0x7fd9501993d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd950100e20 0x7fd9501993d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 -- 172.21.15.37:0/3300145128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9501a47d0 con 0x7fd950102570 2024-09-19T06:43:31.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd955bb0640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102570 0x7fd950199910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd94000d950 tx=0x7fd94000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd9477fe640 1 -- 172.21.15.37:0/3300145128 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940017070 con 0x7fd950102570 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd9477fe640 1 -- 172.21.15.37:0/3300145128 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd940009c30 con 0x7fd950102570 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd9477fe640 1 -- 172.21.15.37:0/3300145128 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94000f6b0 con 0x7fd950102570 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9501a4ac0 con 0x7fd950102570 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.749+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9501a5050 con 0x7fd950102570 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 Processor -- start 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 -- start start 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd9477fe640 1 -- 172.21.15.37:0/3300145128 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd9400109d0 con 0x7fd950102570 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ebb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ebb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ec6340 0x559839ec8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559839e212f0 con 0x559839ebcb10 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559839dc8cf0 con 0x559839ebb5a0 2024-09-19T06:43:31.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559839e1d2b0 con 0x559839ec6340 2024-09-19T06:43:31.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd956bb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ebb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd956bb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ebb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53914/0 (socket says 172.21.15.37:53914) 2024-09-19T06:43:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.752+0000 7fd956bb2640 1 -- 172.21.15.37:0/2965175293 learned_addr learned my addr 172.21.15.37:0/2965175293 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ec6340 0x559839ec8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.762 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ebb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd956bb2640 1 -- 172.21.15.37:0/2965175293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ec6340 msgr2=0x559839ec8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ec6340 0x559839ec8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd956bb2640 1 -- 172.21.15.37:0/2965175293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 msgr2=0x559839ebb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ebb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd956bb2640 1 -- 172.21.15.37:0/2965175293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559839dad3a0 con 0x559839ebcb10 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ebb060 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd94802af40 tx=0x7fd948009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907c67706ce6d7c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd944ff9640 1 -- 172.21.15.37:0/2965175293 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948034050 con 0x559839ebcb10 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd9477fe640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd93c0a3430 0x7fd93c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd944ff9640 1 -- 172.21.15.37:0/2965175293 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd94802fb30 con 0x559839ebcb10 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.753+0000 7fd944ff9640 1 -- 172.21.15.37:0/2965175293 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd948004440 con 0x559839ebcb10 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd9563b1640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd93c0a3430 0x7fd93c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd9477fe640 1 -- 172.21.15.37:0/3300145128 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fd9400cf830 con 0x7fd950102570 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2965175293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 msgr2=0x559839ebb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd9563b1640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd93c0a3430 0x7fd93c0a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x559839eb8a90 tx=0x7fd94c00b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ebb060 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fd94802af40 tx=0x7fd948009800 comp rx=0 tx=0).stop 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2965175293 shutdown_connections 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ec6340 0x559839ec8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ebb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2965175293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ebb060 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2965175293 >> 172.21.15.37:0/2965175293 conn(0x559839e84310 msgr2=0x559839eb7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.754+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2965175293 shutdown_connections 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.755+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2965175293 wait complete. 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.755+0000 7fd95a9ee740 1 -- start start 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.755+0000 7fd95a9ee740 1 Processor -- start 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ec5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebcb10 0x559839ec2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ec35d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559839ecd3f0 con 0x559839ec6340 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559839ddc060 con 0x559839ebcb10 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559839ecd570 con 0x559839ebb5a0 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd95a9ee740 10 client.0 _ll_register_callbacks cb 0x559839b2f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ec35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ec35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53926/0 (socket says 172.21.15.37:53926) 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 -- 172.21.15.37:0/2883597670 learned_addr learned my addr 172.21.15.37:0/2883597670 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebcb10 0x559839ec2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ec5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 -- 172.21.15.37:0/2883597670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 msgr2=0x559839ec5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ec5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 -- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebcb10 msgr2=0x559839ec2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebcb10 0x559839ec2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.756+0000 7fd956bb2640 1 -- 172.21.15.37:0/2883597670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559839b923c0 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ec35d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd948009620 tx=0x7fd948003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfebd86c69ef8b0a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd939ffb640 1 -- 172.21.15.37:0/2883597670 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd94802f930 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd939ffb640 1 -- 172.21.15.37:0/2883597670 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fd9480043d0 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd94802fb30 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd939ffb640 1 -- 172.21.15.37:0/2883597670 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd948059050 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd95a9ee740 10 client.15924 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559839ecd3f0 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd939ffb640 1 -- 172.21.15.37:0/2883597670 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd94802fb30 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.757+0000 7fd939ffb640 1 -- 172.21.15.37:0/2883597670 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd948041070 con 0x559839ec6340 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd95a9ee740 10 client.15924 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:31.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd95a9ee740 10 client.15924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd95a9ee740 20 client.15924 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839edc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd95a9ee740 4 client.15924 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559839e2e860 con 0x559839eda010 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839edc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839edc400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c4a6e0843d23cc5c server_cookie=9e5a0c8d5a767760 in_seq=0 out_seq=0 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.758+0000 7fd939ffb640 10 client.15924 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd939ffb640 1 -- 172.21.15.37:0/2883597670 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559839e2e860 con 0x559839eda010 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd939ffb640 10 client.15924 handle_command_reply: tid=0 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd95a9ee740 1 client.15924 shutdown 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 msgr2=0x559839ec35d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ec35d0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7fd948009620 tx=0x7fd948003c20 comp rx=0 tx=0).stop 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 shutdown_connections 2024-09-19T06:43:31.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839edc400 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd939ffb640 0 client.15924 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ec35d0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.759+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebcb10 0x559839ec2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.760+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2883597670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ec5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.760+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 >> 172.21.15.37:0/2883597670 conn(0x559839e84310 msgr2=0x559839ebd900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.760+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 shutdown_connections 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.760+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2883597670 wait complete. 2024-09-19T06:43:31.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.760+0000 7fd95a9ee740 20 client.15924 trim_cache size 0 max 16384 2024-09-19T06:43:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.782+0000 7fd95a9ee740 1 Processor -- start 2024-09-19T06:43:31.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.782+0000 7fd95a9ee740 1 -- start start 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ed6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebcb10 0x559839ee1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ee2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501adf40 con 0x559839ec6340 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501ab810 con 0x559839ebb5a0 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501a9660 con 0x559839ebcb10 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebcb10 0x559839ee1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebcb10 0x559839ee1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:53354/0 (socket says 172.21.15.37:53354) 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 -- 172.21.15.37:0/2024481346 learned_addr learned my addr 172.21.15.37:0/2024481346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ee2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ed6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 -- 172.21.15.37:0/2024481346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 msgr2=0x559839ed6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ed6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 -- 172.21.15.37:0/2024481346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 msgr2=0x559839ee2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ee2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd955bb0640 1 -- 172.21.15.37:0/2024481346 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559839ec1530 con 0x559839ebcb10 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.783+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ed6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.784+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebcb10 0x559839ee1b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd94009b810 tx=0x7fd940000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89dc4da1156a2c7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.784+0000 7fd93bfff640 1 -- 172.21.15.37:0/2024481346 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940020040 con 0x559839ebcb10 2024-09-19T06:43:31.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.784+0000 7fd93bfff640 1 -- 172.21.15.37:0/2024481346 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd94009fec0 con 0x559839ebcb10 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.784+0000 7fd93bfff640 1 -- 172.21.15.37:0/2024481346 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9400860b0 con 0x559839ebcb10 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2024481346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebcb10 msgr2=0x559839ee1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebcb10 0x559839ee1b20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd94009b810 tx=0x7fd940000bc0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2024481346 shutdown_connections 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ec6340 0x559839ee2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebcb10 0x559839ee1b20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2024481346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ebb5a0 0x559839ed6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2024481346 >> 172.21.15.37:0/2024481346 conn(0x559839e84310 msgr2=0x559839ec5060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2024481346 shutdown_connections 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2024481346 wait complete. 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.785+0000 7fd95a9ee740 1 -- start start 2024-09-19T06:43:31.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 1 Processor -- start 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ed2890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ed2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ec6340 0x559839ec9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501a9cd0 con 0x559839ebcb10 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501ab140 con 0x559839ec6340 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9501aa3a0 con 0x559839ebb5a0 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd95a9ee740 10 client.0 _ll_register_callbacks cb 0x7fd948005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:31.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ed2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ed2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53930/0 (socket says 172.21.15.37:53930) 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 -- 172.21.15.37:0/2797286840 learned_addr learned my addr 172.21.15.37:0/2797286840 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ed2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd956bb2640 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ec6340 0x559839ec9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 -- 172.21.15.37:0/2797286840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 msgr2=0x559839ed2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ed2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 -- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ec6340 msgr2=0x559839ec9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ec6340 0x559839ec9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.786+0000 7fd955bb0640 1 -- 172.21.15.37:0/2797286840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559839ec9fb0 con 0x559839ebcb10 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd955bb0640 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ed2dd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd94009baa0 tx=0x7fd940010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0d04c14535d6a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd939ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd939ffb640 1 -- 172.21.15.37:0/2797286840 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd940020070 con 0x559839ebcb10 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd939ffb640 1 -- 172.21.15.37:0/2797286840 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fd9400860b0 con 0x559839ebcb10 2024-09-19T06:43:31.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd94009fec0 con 0x559839ebcb10 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd939ffb640 1 -- 172.21.15.37:0/2797286840 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd9400d5140 con 0x559839ebcb10 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 10 client.15930 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559839ed02e0 con 0x559839ebcb10 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd939ffb640 1 -- 172.21.15.37:0/2797286840 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9400b4060 con 0x559839ebcb10 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd939ffb640 1 -- 172.21.15.37:0/2797286840 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fd9400cfce0 con 0x559839ebcb10 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 10 client.15930 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 10 client.15930 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 20 client.15930 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839f4b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 4 client.15930 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.787+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fd94009fec0 con 0x559839eda010 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.788+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839f4b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:31.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.788+0000 7fd9563b1640 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839f4b500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8886a71dde730042 server_cookie=8e13d6e5b8b5ec1e in_seq=0 out_seq=0 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.788+0000 7fd939ffb640 10 client.15930 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.788+0000 7fd939ffb640 1 -- 172.21.15.37:0/2797286840 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fd94009fec0 con 0x559839eda010 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.788+0000 7fd939ffb640 10 client.15930 handle_command_reply: tid=0 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.788+0000 7fd95a9ee740 1 client.15930 shutdown 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 msgr2=0x559839ed2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ed2dd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd94009baa0 tx=0x7fd940010d70 comp rx=0 tx=0).stop 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 shutdown_connections 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559839eda010 0x559839f4b500 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559839ec6340 0x559839ec9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559839ebcb10 0x559839ed2dd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 --2- 172.21.15.37:0/2797286840 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559839ebb5a0 0x559839ed2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 >> 172.21.15.37:0/2797286840 conn(0x559839e84310 msgr2=0x559839ec1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd939ffb640 0 client.15930 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:43:31.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 shutdown_connections 2024-09-19T06:43:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 1 -- 172.21.15.37:0/2797286840 wait complete. 2024-09-19T06:43:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.789+0000 7fd95a9ee740 20 client.15930 trim_cache size 0 max 16384 2024-09-19T06:43:31.794 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:43:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd93c0a3430 msgr2=0x7fd93c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd93c0a3430 0x7fd93c0a58f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x559839eb8a90 tx=0x7fd94c00b390 comp rx=0 tx=0).stop 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102570 msgr2=0x7fd950199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102570 0x7fd950199910 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fd94000d950 tx=0x7fd94000de10 comp rx=0 tx=0).stop 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 shutdown_connections 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd950102ef0 0x7fd950199e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd93c0a3430 0x7fd93c0a58f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd950102570 0x7fd950199910 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 --2- 172.21.15.37:0/3300145128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd950100e20 0x7fd9501993d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 >> 172.21.15.37:0/3300145128 conn(0x7fd9500fc7d0 msgr2=0x7fd9501068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 shutdown_connections 2024-09-19T06:43:31.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:31.790+0000 7fd958615640 1 -- 172.21.15.37:0/3300145128 wait complete. 2024-09-19T06:43:32.453 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:32 smithi037 ceph-mon[31211]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T06:43:32.481 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:43:32.513 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:32.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:32 smithi096 ceph-mon[35004]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T06:43:32.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:32 smithi151 ceph-mon[35236]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-09-19T06:43:32.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 -- 172.21.15.37:0/1208391700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c102570 msgr2=0x7f789c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:32.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 --2- 172.21.15.37:0/1208391700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c102570 0x7f789c1029b0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7890009900 tx=0x7f789002a480 comp rx=0 tx=0).stop 2024-09-19T06:43:32.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 -- 172.21.15.37:0/1208391700 shutdown_connections 2024-09-19T06:43:32.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 --2- 172.21.15.37:0/1208391700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f789c102ef0 0x7f789c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 --2- 172.21.15.37:0/1208391700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c102570 0x7f789c1029b0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 --2- 172.21.15.37:0/1208391700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c100e20 0x7f789c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 -- 172.21.15.37:0/1208391700 >> 172.21.15.37:0/1208391700 conn(0x7f789c0fc7d0 msgr2=0x7f789c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:32.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 -- 172.21.15.37:0/1208391700 shutdown_connections 2024-09-19T06:43:32.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.982+0000 7f78a0d62640 1 -- 172.21.15.37:0/1208391700 wait complete. 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 Processor -- start 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 -- start start 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c100e20 0x7f789c1a47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f789c102570 0x7f789c1a4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c102ef0 0x7f789c19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f789c110f50 con 0x7f789c100e20 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f789c110dd0 con 0x7f789c102ef0 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.983+0000 7f78a0d62640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1110d0 con 0x7f789c102570 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c100e20 0x7f789c1a47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c100e20 0x7f789c1a47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53946/0 (socket says 172.21.15.37:53946) 2024-09-19T06:43:32.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 -- 172.21.15.37:0/2706367808 learned_addr learned my addr 172.21.15.37:0/2706367808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f7899d9b640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f789c102570 0x7f789c1a4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789ad9d640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c102ef0 0x7f789c19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 -- 172.21.15.37:0/2706367808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f789c102570 msgr2=0x7f789c1a4d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f789c102570 0x7f789c1a4d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 -- 172.21.15.37:0/2706367808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c102ef0 msgr2=0x7f789c19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c102ef0 0x7f789c19ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 -- 172.21.15.37:0/2706367808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f789c19f440 con 0x7f789c100e20 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789ad9d640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c102ef0 0x7f789c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f789a59c640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c100e20 0x7f789c1a47f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f788c00d950 tx=0x7f788c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f788b7fe640 1 -- 172.21.15.37:0/2706367808 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c017070 con 0x7f789c100e20 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f789c19f730 con 0x7f789c100e20 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f788b7fe640 1 -- 172.21.15.37:0/2706367808 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788c009c30 con 0x7f789c100e20 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f788b7fe640 1 -- 172.21.15.37:0/2706367808 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c00f6b0 con 0x7f789c100e20 2024-09-19T06:43:32.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.984+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f789c1ab7b0 con 0x7f789c100e20 2024-09-19T06:43:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.987+0000 7f788b7fe640 1 -- 172.21.15.37:0/2706367808 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f788c0109d0 con 0x7f789c100e20 2024-09-19T06:43:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.987+0000 7f788b7fe640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f78840a33b0 0x7f78840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.988+0000 7f788b7fe640 1 -- 172.21.15.37:0/2706367808 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f788c0cfa40 con 0x7f789c100e20 2024-09-19T06:43:32.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.988+0000 7f7899d9b640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f78840a33b0 0x7f78840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.988+0000 7f7899d9b640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f78840a33b0 0x7f78840a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f789c19fc90 tx=0x7f7890005a20 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.988+0000 7f78a313b740 1 Processor -- start 2024-09-19T06:43:32.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.988+0000 7f78a313b740 1 -- start start 2024-09-19T06:43:32.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f78a313b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7060 0x557053ff7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f78a313b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054001d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f78a313b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x557054004630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f78a313b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557053f5c620 con 0x557053ff7060 2024-09-19T06:43:32.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f78a313b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557053f03e50 con 0x557054002240 2024-09-19T06:43:32.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f78a313b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557053f585e0 con 0x557053ff7980 2024-09-19T06:43:32.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7060 0x557053ff7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7060 0x557053ff7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53948/0 (socket says 172.21.15.37:53948) 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 -- 172.21.15.37:0/3656666177 learned_addr learned my addr 172.21.15.37:0/3656666177 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f7899d9b640 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054001d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789ad9d640 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x557054004630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 -- 172.21.15.37:0/3656666177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 msgr2=0x557054001d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054001d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 -- 172.21.15.37:0/3656666177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 msgr2=0x557054004630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x557054004630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 -- 172.21.15.37:0/3656666177 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557053ef4d40 con 0x557053ff7060 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.989+0000 7f789a59c640 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7060 0x557053ff7440 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f788c0986f0 tx=0x7f788c009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6268e69820541d20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.990+0000 7f7888ff9640 1 -- 172.21.15.37:0/3656666177 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c020040 con 0x557053ff7060 2024-09-19T06:43:32.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.990+0000 7f7888ff9640 1 -- 172.21.15.37:0/3656666177 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f788c0a00d0 con 0x557053ff7060 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.990+0000 7f7888ff9640 1 -- 172.21.15.37:0/3656666177 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c09b3d0 con 0x557053ff7060 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 -- 172.21.15.37:0/3656666177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7060 msgr2=0x557053ff7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7060 0x557053ff7440 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f788c0986f0 tx=0x7f788c009ea0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 -- 172.21.15.37:0/3656666177 shutdown_connections 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x557054004630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054001d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 --2- 172.21.15.37:0/3656666177 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7060 0x557053ff7440 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 -- 172.21.15.37:0/3656666177 >> 172.21.15.37:0/3656666177 conn(0x557053ff35f0 msgr2=0x557053ff39d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 -- 172.21.15.37:0/3656666177 shutdown_connections 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 -- 172.21.15.37:0/3656666177 wait complete. 2024-09-19T06:43:32.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 -- start start 2024-09-19T06:43:32.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.991+0000 7f78a313b740 1 Processor -- start 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f78a313b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7980 0x557053fff380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f78a313b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x557053fff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f78a313b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053fffe00 0x55705400e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f789a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7980 0x557053fff380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f789a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7980 0x557053fff380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53958/0 (socket says 172.21.15.37:53958) 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f789a59c640 1 -- 172.21.15.37:0/3885536905 learned_addr learned my addr 172.21.15.37:0/3885536905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557054004e10 con 0x557053ff7980 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557053ee4f10 con 0x557053fffe00 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557054004f90 con 0x557054002240 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f7899d9b640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x557053fff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.992+0000 7f789ad9d640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053fffe00 0x55705400e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:32.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f78a313b740 10 client.0 _ll_register_callbacks cb 0x557053f08db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f789a59c640 1 -- 172.21.15.37:0/3885536905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 msgr2=0x557053fff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f789a59c640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x557053fff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f789a59c640 1 -- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053fffe00 msgr2=0x55705400e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f789a59c640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053fffe00 0x55705400e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f789a59c640 1 -- 172.21.15.37:0/3885536905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557053c6b580 con 0x557053ff7980 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f7899d9b640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x557053fff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f788c0a00d0 con 0x557053ff7980 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f789a59c640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7980 0x557053fff380 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f788c0cd790 tx=0x7f788c00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597ef1425428271f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f7881ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f7881ffb640 1 -- 172.21.15.37:0/3885536905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c065060 con 0x557053ff7980 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f7881ffb640 1 -- 172.21.15.37:0/3885536905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f788c09b380 con 0x557053ff7980 2024-09-19T06:43:32.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f7881ffb640 1 -- 172.21.15.37:0/3885536905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f788c020070 con 0x557053ff7980 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f78a313b740 10 client.15942 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.993+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({fsmap=25}) -- 0x557053d35960 con 0x557053ff7980 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f7881ffb640 1 -- 172.21.15.37:0/3885536905 <== mon.0 v2:172.21.15.37:3300/0 4 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f788c0b3070 con 0x557053ff7980 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f78a313b740 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f78a313b740 10 client.15942 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f78a313b740 20 client.15942 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f78a313b740 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570540121a0 0x557054014590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f78a313b740 4 client.15942 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557053f69b90 con 0x5570540121a0 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.994+0000 7f7899d9b640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570540121a0 0x557054014590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f7881ffb640 10 client.15942 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f7899d9b640 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570540121a0 0x557054014590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e354d13a1d98e416 server_cookie=88985092ed44db26 in_seq=0 out_seq=0 2024-09-19T06:43:33.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f7881ffb640 1 -- 172.21.15.37:0/3885536905 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557053f69b90 con 0x5570540121a0 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f7881ffb640 10 client.15942 handle_command_reply: tid=0 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f78a313b740 1 client.15942 shutdown 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7980 msgr2=0x557053fff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f78a313b740 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7980 0x557053fff380 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f788c0cd790 tx=0x7f788c00bf60 comp rx=0 tx=0).stop 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 shutdown_connections 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f78a313b740 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570540121a0 0x557054014590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.995+0000 7f78a313b740 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053fffe00 0x55705400e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.996+0000 7f78a313b740 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x557053fff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.996+0000 7f78a313b740 1 --2- 172.21.15.37:0/3885536905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053ff7980 0x557053fff380 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.996+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 >> 172.21.15.37:0/3885536905 conn(0x557053ff35f0 msgr2=0x557053ff9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.996+0000 7f7881ffb640 0 client.15942 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.996+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 shutdown_connections 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.996+0000 7f78a313b740 1 -- 172.21.15.37:0/3885536905 wait complete. 2024-09-19T06:43:33.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:32.996+0000 7f78a313b740 20 client.15942 trim_cache size 0 max 16384 2024-09-19T06:43:33.002 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 Processor -- start 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 -- start start 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053ff7980 0x557053ff39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557053ff9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x5570540012f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1b6750 con 0x557053fffe00 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1b4020 con 0x557053ff7980 2024-09-19T06:43:33.019 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.017+0000 7f78a313b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1b1e70 con 0x557054002240 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557053ff9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557053ff9ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53966/0 (socket says 172.21.15.37:53966) 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 -- 172.21.15.37:0/1961795160 learned_addr learned my addr 172.21.15.37:0/1961795160 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f789ad9d640 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x5570540012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f789a59c640 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053ff7980 0x557053ff39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 -- 172.21.15.37:0/1961795160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 msgr2=0x5570540012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x5570540012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 -- 172.21.15.37:0/1961795160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053ff7980 msgr2=0x557053ff39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053ff7980 0x557053ff39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 -- 172.21.15.37:0/1961795160 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557053fff880 con 0x557053fffe00 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f789ad9d640 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x5570540012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7899d9b640 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557053ff9ac0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7890005c40 tx=0x7f789003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997214eebbb2c307 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7883fff640 1 -- 172.21.15.37:0/1961795160 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890035050 con 0x557053fffe00 2024-09-19T06:43:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7883fff640 1 -- 172.21.15.37:0/1961795160 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7890004c10 con 0x557053fffe00 2024-09-19T06:43:33.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.018+0000 7f7883fff640 1 -- 172.21.15.37:0/1961795160 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900327b0 con 0x557053fffe00 2024-09-19T06:43:33.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.019+0000 7f78a313b740 1 -- 172.21.15.37:0/1961795160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 msgr2=0x557053ff9ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.019+0000 7f78a313b740 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557053ff9ac0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f7890005c40 tx=0x7f789003f000 comp rx=0 tx=0).stop 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.019+0000 7f78a313b740 1 -- 172.21.15.37:0/1961795160 shutdown_connections 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.019+0000 7f78a313b740 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557054002240 0x5570540012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.019+0000 7f78a313b740 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557053ff9ac0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.019+0000 7f78a313b740 1 --2- 172.21.15.37:0/1961795160 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557053ff7980 0x557053ff39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.019+0000 7f78a313b740 1 -- 172.21.15.37:0/1961795160 >> 172.21.15.37:0/1961795160 conn(0x557053ff35f0 msgr2=0x557054000be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 -- 172.21.15.37:0/1961795160 shutdown_connections 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 -- 172.21.15.37:0/1961795160 wait complete. 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 -- start start 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 Processor -- start 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054017920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557054014c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x5570540151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1b24e0 con 0x557053fffe00 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1b3950 con 0x557054002240 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f789c1b2bb0 con 0x557053ff7980 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.020+0000 7f78a313b740 10 client.0 _ll_register_callbacks cb 0x7f788c097950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f7899d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557054014c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054017920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054017920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:53422/0 (socket says 172.21.15.37:53422) 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789a59c640 1 -- 172.21.15.37:0/1424253045 learned_addr learned my addr 172.21.15.37:0/1424253045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789ad9d640 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x5570540151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789ad9d640 1 -- 172.21.15.37:0/1424253045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 msgr2=0x557054017920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789ad9d640 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054017920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789ad9d640 1 -- 172.21.15.37:0/1424253045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 msgr2=0x557054014c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789ad9d640 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557054014c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789ad9d640 1 -- 172.21.15.37:0/1424253045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570540159e0 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f789ad9d640 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x5570540151b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7894005fd0 tx=0x7f7894012ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee67d97d8e53dd75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f7881ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f7881ffb640 1 -- 172.21.15.37:0/1424253045 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7894004d20 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7890004c10 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.021+0000 7f7881ffb640 1 -- 172.21.15.37:0/1424253045 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f7894011ad0 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f7881ffb640 1 -- 172.21.15.37:0/1424253045 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7894011cd0 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 10 client.25481 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55705407cbc0 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f7881ffb640 1 -- 172.21.15.37:0/1424253045 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7890004c10 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f7881ffb640 1 -- 172.21.15.37:0/1424253045 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f789401b070 con 0x557054002240 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 10 client.25481 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 10 client.25481 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 20 client.25481 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:33.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55705407d340 0x55705407f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 4 client.25481 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6fb810d1", "release": true, "await": true}] 2024-09-19T06:43:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.022+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6fb810d1", "release": true, "await": true}) -- 0x557053f640a0 con 0x55705407d340 2024-09-19T06:43:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.023+0000 7f789a59c640 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55705407d340 0x55705407f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.023+0000 7f7881ffb640 10 client.25481 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.023+0000 7f789a59c640 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55705407d340 0x55705407f760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e558a8dcaa280e1 server_cookie=d20d2c34f5714e45 in_seq=0 out_seq=0 2024-09-19T06:43:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f7881ffb640 1 -- 172.21.15.37:0/1424253045 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x557053f640a0 con 0x55705407d340 2024-09-19T06:43:33.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f7881ffb640 10 client.25481 handle_command_reply: tid=0 2024-09-19T06:43:33.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 client.25481 shutdown 2024-09-19T06:43:33.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 msgr2=0x5570540151b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x5570540151b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7894005fd0 tx=0x7f7894012ca0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 shutdown_connections 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55705407d340 0x55705407f760 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f7881ffb640 0 client.25481 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557054002240 0x5570540151b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557053fffe00 0x557054014c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 --2- 172.21.15.37:0/1424253045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557053ff7980 0x557054017920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 >> 172.21.15.37:0/1424253045 conn(0x557053ff35f0 msgr2=0x557054001400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 shutdown_connections 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.027+0000 7f78a313b740 1 -- 172.21.15.37:0/1424253045 wait complete. 2024-09-19T06:43:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a313b740 20 client.25481 trim_cache size 0 max 16384 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 25, 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 8, 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "6fb810d1": { 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 8, 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:33.030 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:43:33.031 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:33.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:43:33.031 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:43:33.031 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 140.3, 2024-09-19T06:43:33.031 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:43:33.031 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:43:33.031 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:33.032 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f78840a33b0 msgr2=0x7f78840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f78840a33b0 0x7f78840a5870 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f789c19fc90 tx=0x7f7890005a20 comp rx=0 tx=0).stop 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c100e20 msgr2=0x7f789c1a47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c100e20 0x7f789c1a47f0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f788c00d950 tx=0x7f788c00de10 comp rx=0 tx=0).stop 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 shutdown_connections 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f789c102ef0 0x7f789c19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f789c102570 0x7f789c1a4d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f78840a33b0 0x7f78840a5870 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 --2- 172.21.15.37:0/2706367808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f789c100e20 0x7f789c1a47f0 secure :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f788c00d950 tx=0x7f788c00de10 comp rx=0 tx=0).stop 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 >> 172.21.15.37:0/2706367808 conn(0x7f789c0fc7d0 msgr2=0x7f789c069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 shutdown_connections 2024-09-19T06:43:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.028+0000 7f78a0d62640 1 -- 172.21.15.37:0/2706367808 wait complete. 2024-09-19T06:43:33.034 INFO:tasks.workunit.client.0.smithi037.stderr:+ dbench 1 2024-09-19T06:43:33.065 INFO:tasks.workunit.client.0.smithi037.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:43:33.066 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-09-19T06:43:33.066 INFO:tasks.workunit.client.0.smithi037.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:43:33.088 INFO:tasks.workunit.client.0.smithi037.stdout:0 of 1 processes prepared for launch 0 sec 2024-09-19T06:43:33.188 INFO:tasks.workunit.client.0.smithi037.stdout:1 of 1 processes prepared for launch 0 sec 2024-09-19T06:43:33.189 INFO:tasks.workunit.client.0.smithi037.stdout:releasing clients 2024-09-19T06:43:33.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 -- 172.21.15.37:0/2849045920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c106f10 msgr2=0x7fa20c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 --2- 172.21.15.37:0/2849045920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c106f10 0x7fa20c10ea90 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fa204009ca0 tx=0x7fa20402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 -- 172.21.15.37:0/2849045920 shutdown_connections 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 --2- 172.21.15.37:0/2849045920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c106f10 0x7fa20c10ea90 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 --2- 172.21.15.37:0/2849045920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa20c102790 0x7fa20c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 --2- 172.21.15.37:0/2849045920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa20c101de0 0x7fa20c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 -- 172.21.15.37:0/2849045920 >> 172.21.15.37:0/2849045920 conn(0x7fa20c0fd8c0 msgr2=0x7fa20c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 -- 172.21.15.37:0/2849045920 shutdown_connections 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.765+0000 7fa211167640 1 -- 172.21.15.37:0/2849045920 wait complete. 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.766+0000 7fa211167640 1 Processor -- start 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.766+0000 7fa211167640 1 -- start start 2024-09-19T06:43:33.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa211167640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c101de0 0x7fa20c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa211167640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa20c102790 0x7fa20c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa211167640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa20c106f10 0x7fa20c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa211167640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c111f70 con 0x7fa20c101de0 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa211167640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c111df0 con 0x7fa20c102790 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa211167640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1120f0 con 0x7fa20c106f10 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c101de0 0x7fa20c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c101de0 0x7fa20c19a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53988/0 (socket says 172.21.15.37:53988) 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 -- 172.21.15.37:0/144533204 learned_addr learned my addr 172.21.15.37:0/144533204 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20a59c640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa20c102790 0x7fa20c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20b59e640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa20c106f10 0x7fa20c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 -- 172.21.15.37:0/144533204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa20c106f10 msgr2=0x7fa20c19ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa20c106f10 0x7fa20c19ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 -- 172.21.15.37:0/144533204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa20c102790 msgr2=0x7fa20c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa20c102790 0x7fa20c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 -- 172.21.15.37:0/144533204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa20c19f560 con 0x7fa20c101de0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20a59c640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa20c102790 0x7fa20c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c101de0 0x7fa20c19a5b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00ea60 tx=0x7fa1fc00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.767+0000 7fa1f3fff640 1 -- 172.21.15.37:0/144533204 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc00cde0 con 0x7fa20c101de0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.768+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa20c19f850 con 0x7fa20c101de0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.768+0000 7fa1f3fff640 1 -- 172.21.15.37:0/144533204 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc0035f0 con 0x7fa20c101de0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.768+0000 7fa1f3fff640 1 -- 172.21.15.37:0/144533204 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc010670 con 0x7fa20c101de0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.768+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa20c1a7130 con 0x7fa20c101de0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.770+0000 7fa213540740 1 Processor -- start 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.770+0000 7fa213540740 1 -- start start 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.770+0000 7fa1f3fff640 1 -- 172.21.15.37:0/144533204 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa1fc0037a0 con 0x7fa20c101de0 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa213540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927a2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa213540740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927a2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa213540740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927af760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa213540740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c5927082f0 con 0x55c5927a25d0 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa213540740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c5926afcf0 con 0x55c5927a3b40 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa213540740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5927042b0 con 0x55c5927ad370 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927a2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927a2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927a2a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53996/0 (socket says 172.21.15.37:53996) 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 -- 172.21.15.37:0/1410771336 learned_addr learned my addr 172.21.15.37:0/1410771336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20a59c640 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 -- 172.21.15.37:0/1410771336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 msgr2=0x55c5927af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 -- 172.21.15.37:0/1410771336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 msgr2=0x55c5927a2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927a2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 -- 172.21.15.37:0/1410771336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5926943a0 con 0x55c5927a25d0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.771+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927a2a10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0cd670 tx=0x7fa1fc0d8570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f888d68fe0b12a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.772+0000 7fa1f3fff640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa1f40a3430 0x7fa1f40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.772+0000 7fa1f17fa640 1 -- 172.21.15.37:0/1410771336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc0e3070 con 0x55c5927a25d0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.772+0000 7fa20b59e640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa1f40a3430 0x7fa1f40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.772+0000 7fa1f3fff640 1 -- 172.21.15.37:0/144533204 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fa1fc0cffb0 con 0x7fa20c101de0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.772+0000 7fa1f17fa640 1 -- 172.21.15.37:0/1410771336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc09c990 con 0x55c5927a25d0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.772+0000 7fa1f17fa640 1 -- 172.21.15.37:0/1410771336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc098530 con 0x55c5927a25d0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 -- 172.21.15.37:0/1410771336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 msgr2=0x55c5927a2a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa20b59e640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa1f40a3430 0x7fa1f40a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa204004af0 tx=0x7fa204004a80 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927a2a10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0cd670 tx=0x7fa1fc0d8570 comp rx=0 tx=0).stop 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 -- 172.21.15.37:0/1410771336 shutdown_connections 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927a2a10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 --2- 172.21.15.37:0/1410771336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927a2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 -- 172.21.15.37:0/1410771336 >> 172.21.15.37:0/1410771336 conn(0x55c59276b310 msgr2=0x55c59279ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 -- 172.21.15.37:0/1410771336 shutdown_connections 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 -- 172.21.15.37:0/1410771336 wait complete. 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 -- start start 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.773+0000 7fa213540740 1 Processor -- start 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927acce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927a3b40 0x55c5927a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927ad370 0x55c5927aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c5927b0070 con 0x55c5927ad370 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c5926c3060 con 0x55c5927a25d0 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c5927b01f0 con 0x55c5927a3b40 2024-09-19T06:43:33.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 10 client.0 _ll_register_callbacks cb 0x55c5924168c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927acce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53338/0 (socket says 172.21.15.37:53338) 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 -- 172.21.15.37:0/651381120 learned_addr learned my addr 172.21.15.37:0/651381120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20b59e640 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927ad370 0x55c5927aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20a59c640 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927a3b40 0x55c5927a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 -- 172.21.15.37:0/651381120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927a3b40 msgr2=0x55c5927a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927a3b40 0x55c5927a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 -- 172.21.15.37:0/651381120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927ad370 msgr2=0x55c5927aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927ad370 0x55c5927aa5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 -- 172.21.15.37:0/651381120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5924793c0 con 0x55c5927a25d0 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927acce0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0a49d0 tx=0x7fa1fc0d38d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92140544f87e3654 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa1eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:43:33.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa1eeff5640 1 -- 172.21.15.37:0/651381120 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1fc09bbf0 con 0x55c5927a25d0 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa1eeff5640 1 -- 172.21.15.37:0/651381120 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fa1fc09a590 con 0x55c5927a25d0 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.774+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1fc09c990 con 0x55c5927a25d0 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa1eeff5640 1 -- 172.21.15.37:0/651381120 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa1fc09bdd0 con 0x55c5927a25d0 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 10 client.25493 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c5927b0070 con 0x55c5927a25d0 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa1eeff5640 1 -- 172.21.15.37:0/651381120 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa1fc09c990 con 0x55c5927a25d0 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa1eeff5640 1 -- 172.21.15.37:0/651381120 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa1fc0cd980 con 0x55c5927a25d0 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 10 client.25493 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 10 client.25493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 20 client.25493 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5927bcc40 0x55c5927bf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 4 client.25493 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:33.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c592715860 con 0x55c5927bcc40 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.775+0000 7fa20a59c640 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5927bcc40 0x55c5927bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.776+0000 7fa20a59c640 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5927bcc40 0x55c5927bf030 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edfb12bfc52f3019 server_cookie=e30ec0caaeef1513 in_seq=0 out_seq=0 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.776+0000 7fa1eeff5640 10 client.25493 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.777+0000 7fa1eeff5640 1 -- 172.21.15.37:0/651381120 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c592715860 con 0x55c5927bcc40 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.777+0000 7fa1eeff5640 10 client.25493 handle_command_reply: tid=0 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.777+0000 7fa213540740 1 client.25493 shutdown 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 msgr2=0x55c5927acce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927acce0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc0a49d0 tx=0x7fa1fc0d38d0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 shutdown_connections 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5927bcc40 0x55c5927bf030 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927ad370 0x55c5927aa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927a3b40 0x55c5927a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa1eeff5640 0 client.25493 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 --2- 172.21.15.37:0/651381120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927acce0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 >> 172.21.15.37:0/651381120 conn(0x55c59276b310 msgr2=0x55c5927a4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 shutdown_connections 2024-09-19T06:43:33.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 1 -- 172.21.15.37:0/651381120 wait complete. 2024-09-19T06:43:33.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.778+0000 7fa213540740 20 client.25493 trim_cache size 0 max 16384 2024-09-19T06:43:33.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.792+0000 7fa213540740 1 Processor -- start 2024-09-19T06:43:33.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.792+0000 7fa213540740 1 -- start start 2024-09-19T06:43:33.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa213540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927b9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa213540740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a3b40 0x55c5927a5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa213540740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927a5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa213540740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1b2420 con 0x55c5927a3b40 2024-09-19T06:43:33.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa213540740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1afcf0 con 0x55c5927a25d0 2024-09-19T06:43:33.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa213540740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1adb40 con 0x55c5927ad370 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a3b40 0x55c5927a5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a3b40 0x55c5927a5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54016/0 (socket says 172.21.15.37:54016) 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 -- 172.21.15.37:0/1632354606 learned_addr learned my addr 172.21.15.37:0/1632354606 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20b59e640 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927a5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 -- 172.21.15.37:0/1632354606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 msgr2=0x55c5927a5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927a5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 -- 172.21.15.37:0/1632354606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 msgr2=0x55c5927b9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927b9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.793+0000 7fa20a59c640 1 -- 172.21.15.37:0/1632354606 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5927ad010 con 0x55c5927a3b40 2024-09-19T06:43:33.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.794+0000 7fa20a59c640 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a3b40 0x55c5927a5120 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa200000c00 tx=0x7fa200005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8d12d4516b840c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.794+0000 7fa1f0ff9640 1 -- 172.21.15.37:0/1632354606 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200004250 con 0x55c5927a3b40 2024-09-19T06:43:33.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.794+0000 7fa1f0ff9640 1 -- 172.21.15.37:0/1632354606 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2000043f0 con 0x55c5927a3b40 2024-09-19T06:43:33.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.794+0000 7fa1f0ff9640 1 -- 172.21.15.37:0/1632354606 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa200010830 con 0x55c5927a3b40 2024-09-19T06:43:33.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 -- 172.21.15.37:0/1632354606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a3b40 msgr2=0x55c5927a5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a3b40 0x55c5927a5120 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa200000c00 tx=0x7fa200005f90 comp rx=0 tx=0).stop 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 -- 172.21.15.37:0/1632354606 shutdown_connections 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927a5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a3b40 0x55c5927a5120 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 --2- 172.21.15.37:0/1632354606 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a25d0 0x55c5927b9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 -- 172.21.15.37:0/1632354606 >> 172.21.15.37:0/1632354606 conn(0x55c59276b310 msgr2=0x55c5927ac060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.795+0000 7fa213540740 1 -- 172.21.15.37:0/1632354606 shutdown_connections 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.796+0000 7fa213540740 1 -- 172.21.15.37:0/1632354606 wait complete. 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.796+0000 7fa213540740 1 -- start start 2024-09-19T06:43:33.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.796+0000 7fa213540740 1 Processor -- start 2024-09-19T06:43:33.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.796+0000 7fa213540740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927b5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.796+0000 7fa213540740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa213540740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927b0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53350/0 (socket says 172.21.15.37:53350) 2024-09-19T06:43:33.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 -- 172.21.15.37:0/1056488679 learned_addr learned my addr 172.21.15.37:0/1056488679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:33.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1ae1b0 con 0x55c5927a25d0 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1af620 con 0x55c5927a3b40 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa20c1ae880 con 0x55c5927ad370 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20b59e640 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa213540740 10 client.0 _ll_register_callbacks cb 0x7fa1fc0e1070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 -- 172.21.15.37:0/1056488679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 msgr2=0x55c5927b15e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927b15e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 -- 172.21.15.37:0/1056488679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 msgr2=0x55c5927b5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927b5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 -- 172.21.15.37:0/1056488679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5927b1de0 con 0x55c5927a3b40 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa20a59c640 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927b0ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa200012420 tx=0x7fa2000045a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493143585db512c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.797+0000 7fa1eeff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa1eeff5640 1 -- 172.21.15.37:0/1056488679 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2000037f0 con 0x55c5927a3b40 2024-09-19T06:43:33.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa1eeff5640 1 -- 172.21.15.37:0/1056488679 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fa200003990 con 0x55c5927a3b40 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa2000043f0 con 0x55c5927a3b40 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa1eeff5640 1 -- 172.21.15.37:0/1056488679 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa200003b90 con 0x55c5927a3b40 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa213540740 10 client.25499 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c59282ff10 con 0x55c5927a3b40 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa1eeff5640 1 -- 172.21.15.37:0/1056488679 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa20002e030 con 0x55c5927a3b40 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.798+0000 7fa1eeff5640 1 -- 172.21.15.37:0/1056488679 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7fa200028070 con 0x55c5927a3b40 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa213540740 10 client.25499 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa213540740 10 client.25499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa213540740 20 client.25499 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa213540740 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5928305b0 0x55c5928329d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa213540740 4 client.25499 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:43:33.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fa2000043f0 con 0x55c5928305b0 2024-09-19T06:43:33.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5928305b0 0x55c5928329d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa20ad9d640 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5928305b0 0x55c5928329d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59d31167f8eca2cc server_cookie=a14e129c941a2963 in_seq=0 out_seq=0 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.799+0000 7fa1eeff5640 10 client.25499 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa1eeff5640 1 -- 172.21.15.37:0/1056488679 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fa2000043f0 con 0x55c5928305b0 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa1eeff5640 10 client.25499 handle_command_reply: tid=0 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 client.25499 shutdown 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 msgr2=0x55c5927b0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927b0ff0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa200012420 tx=0x7fa2000045a0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 shutdown_connections 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c5928305b0 0x55c5928329d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c5927ad370 0x55c5927b15e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa1eeff5640 0 client.25499 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c5927a3b40 0x55c5927b0ff0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 --2- 172.21.15.37:0/1056488679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c5927a25d0 0x55c5927b5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 >> 172.21.15.37:0/1056488679 conn(0x55c59276b310 msgr2=0x55c59279d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 shutdown_connections 2024-09-19T06:43:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.800+0000 7fa213540740 1 -- 172.21.15.37:0/1056488679 wait complete. 2024-09-19T06:43:33.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa213540740 20 client.25499 trim_cache size 0 max 16384 2024-09-19T06:43:33.804 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa1f40a3430 msgr2=0x7fa1f40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa1f40a3430 0x7fa1f40a58f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa204004af0 tx=0x7fa204004a80 comp rx=0 tx=0).stop 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c101de0 msgr2=0x7fa20c19a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c101de0 0x7fa20c19a5b0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fa1fc00ea60 tx=0x7fa1fc00ef20 comp rx=0 tx=0).stop 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 shutdown_connections 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa20c106f10 0x7fa20c19ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa20c102790 0x7fa20c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa1f40a3430 0x7fa1f40a58f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 --2- 172.21.15.37:0/144533204 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa20c101de0 0x7fa20c19a5b0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.801+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 >> 172.21.15.37:0/144533204 conn(0x7fa20c0fd8c0 msgr2=0x7fa20c0ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.802+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 shutdown_connections 2024-09-19T06:43:33.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:33.802+0000 7fa211167640 1 -- 172.21.15.37:0/144533204 wait complete. 2024-09-19T06:43:34.188 INFO:tasks.workunit.client.0.smithi037.stdout: 1 3548 38.42 MB/sec warmup 1 sec latency 37.818 ms 2024-09-19T06:43:34.189 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6fb810d1', total seconds elapsed: 25.5 2024-09-19T06:43:34.189 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 20.35 2024-09-19T06:43:34.190 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:43:34.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:34 smithi096 ceph-mon[35004]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:43:34.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:34 smithi037 ceph-mon[31211]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:43:34.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:34 smithi151 ceph-mon[35236]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-09-19T06:43:35.021 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:35.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 6767 26.70 MB/sec warmup 2 sec latency 11.093 ms 2024-09-19T06:43:35.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 -- 172.21.15.37:0/3753535556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab8102d30 msgr2=0x7fbab8103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:35.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 --2- 172.21.15.37:0/3753535556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab8102d30 0x7fbab8103110 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8009cd0 tx=0x7fbaa802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 -- 172.21.15.37:0/3753535556 shutdown_connections 2024-09-19T06:43:35.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 --2- 172.21.15.37:0/3753535556 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 0x7fbab810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 --2- 172.21.15.37:0/3753535556 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab81036e0 0x7fbab8107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 --2- 172.21.15.37:0/3753535556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab8102d30 0x7fbab8103110 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 -- 172.21.15.37:0/3753535556 >> 172.21.15.37:0/3753535556 conn(0x7fbab80fc7d0 msgr2=0x7fbab80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 -- 172.21.15.37:0/3753535556 shutdown_connections 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.494+0000 7fbabf87c640 1 -- 172.21.15.37:0/3753535556 wait complete. 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 Processor -- start 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 -- start start 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab8102d30 0x7fbab81986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab81036e0 0x7fbab8198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 0x7fbab819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbab81120a0 con 0x7fbab81082a0 2024-09-19T06:43:35.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8111f20 con 0x7fbab81036e0 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabf87c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbab8112220 con 0x7fbab8102d30 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabde19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 0x7fbab819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.495+0000 7fbabce17640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab81036e0 0x7fbab8198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 0x7fbab819cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54050/0 (socket says 172.21.15.37:54050) 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 -- 172.21.15.37:0/251364278 learned_addr learned my addr 172.21.15.37:0/251364278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabd618640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab8102d30 0x7fbab81986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 -- 172.21.15.37:0/251364278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab8102d30 msgr2=0x7fbab81986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab8102d30 0x7fbab81986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 -- 172.21.15.37:0/251364278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab81036e0 msgr2=0x7fbab8198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab81036e0 0x7fbab8198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 -- 172.21.15.37:0/251364278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbab819d650 con 0x7fbab81082a0 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabce17640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab81036e0 0x7fbab8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:35.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabde19640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 0x7fbab819cf70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbab000e9b0 tx=0x7fbab000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbab67fc640 1 -- 172.21.15.37:0/251364278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab000cd80 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbab819d940 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbab81a5220 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.496+0000 7fbab67fc640 1 -- 172.21.15.37:0/251364278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbab00035f0 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.497+0000 7fbab67fc640 1 -- 172.21.15.37:0/251364278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0010690 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.498+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbab806a850 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.498+0000 7fbab67fc640 1 -- 172.21.15.37:0/251364278 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbab00037a0 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.499+0000 7fbab67fc640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbaa00a3430 0x7fbaa00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.499+0000 7fbab67fc640 1 -- 172.21.15.37:0/251364278 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fbab00cff60 con 0x7fbab81082a0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.499+0000 7fbabd618640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbaa00a3430 0x7fbaa00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.500+0000 7fbabd618640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbaa00a3430 0x7fbaa00a58f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006010 tx=0x7fbaa8009360 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:35.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.503+0000 7fbab67fc640 1 -- 172.21.15.37:0/251364278 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbab009c940 con 0x7fbab81082a0 2024-09-19T06:43:35.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.738+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbab810a230 con 0x7fbab81082a0 2024-09-19T06:43:35.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.738+0000 7fbab67fc640 1 -- 172.21.15.37:0/251364278 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11903 (secure 0 0 0) 0x7fbab009c240 con 0x7fbab81082a0 2024-09-19T06:43:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:43:35.739 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":25,"btime":"2024-09-19T06:41:45:400958+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:41:45.350782+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:43:35.740 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 25 2024-09-19T06:43:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbaa00a3430 msgr2=0x7fbaa00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbaa00a3430 0x7fbaa00a58f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006010 tx=0x7fbaa8009360 comp rx=0 tx=0).stop 2024-09-19T06:43:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 msgr2=0x7fbab819cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 0x7fbab819cf70 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fbab000e9b0 tx=0x7fbab000ee70 comp rx=0 tx=0).stop 2024-09-19T06:43:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 shutdown_connections 2024-09-19T06:43:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbaa00a3430 0x7fbaa00a58f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbab81082a0 0x7fbab819cf70 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbab81036e0 0x7fbab8198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 --2- 172.21.15.37:0/251364278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbab8102d30 0x7fbab81986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 >> 172.21.15.37:0/251364278 conn(0x7fbab80fc7d0 msgr2=0x7fbab8105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 shutdown_connections 2024-09-19T06:43:35.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:35.741+0000 7fbabf87c640 1 -- 172.21.15.37:0/251364278 wait complete. 2024-09-19T06:43:35.894 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:43:35.894 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:43:36.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 10016 23.90 MB/sec warmup 3 sec latency 14.082 ms 2024-09-19T06:43:36.341 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:36 smithi037 ceph-mon[31211]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T06:43:36.341 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:36 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/251364278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:36.369 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0' 2024-09-19T06:43:36.369 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 140.35 --await-for 45.0 2024-09-19T06:43:36.500 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:36.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:36 smithi096 ceph-mon[35004]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T06:43:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:36 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/251364278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:36.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:36 smithi151 ceph-mon[35236]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T06:43:36.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:36 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/251364278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:37.188 INFO:tasks.workunit.client.0.smithi037.stdout: 1 13070 21.36 MB/sec warmup 4 sec latency 11.104 ms 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 -- 172.21.15.37:0/1816577378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0100a80 msgr2=0x7f0dd0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 --2- 172.21.15.37:0/1816577378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0100a80 0x7f0dd0100ee0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0dc4009930 tx=0x7f0dc402a420 comp rx=0 tx=0).stop 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 -- 172.21.15.37:0/1816577378 shutdown_connections 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 --2- 172.21.15.37:0/1816577378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd0101420 0x7f0dd010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 --2- 172.21.15.37:0/1816577378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0100a80 0x7f0dd0100ee0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 --2- 172.21.15.37:0/1816577378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd0106a80 0x7f0dd0106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 -- 172.21.15.37:0/1816577378 >> 172.21.15.37:0/1816577378 conn(0x7f0dd00fc7c0 msgr2=0x7f0dd00febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:37.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 -- 172.21.15.37:0/1816577378 shutdown_connections 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.134+0000 7f0dd59db640 1 -- 172.21.15.37:0/1816577378 wait complete. 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 Processor -- start 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 -- start start 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd0100a80 0x7f0dd0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd0101420 0x7f0dd01989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0106a80 0x7f0dd019cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0110e90 con 0x7f0dd0106a80 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0110d10 con 0x7f0dd0101420 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dd59db640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd0111010 con 0x7f0dd0100a80 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0106a80 0x7f0dd019cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dcf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0106a80 0x7f0dd019cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54058/0 (socket says 172.21.15.37:54058) 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dce7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd0101420 0x7f0dd01989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dcf7fe640 1 -- 172.21.15.37:0/3568728781 learned_addr learned my addr 172.21.15.37:0/3568728781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.135+0000 7f0dceffd640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd0100a80 0x7f0dd0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dcf7fe640 1 -- 172.21.15.37:0/3568728781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd0100a80 msgr2=0x7f0dd0198460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dcf7fe640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd0100a80 0x7f0dd0198460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dcf7fe640 1 -- 172.21.15.37:0/3568728781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd0101420 msgr2=0x7f0dd01989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dcf7fe640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd0101420 0x7f0dd01989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dcf7fe640 1 -- 172.21.15.37:0/3568728781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dd019d410 con 0x7f0dd0106a80 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd0101420 0x7f0dd01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dcf7fe640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0106a80 0x7f0dd019cd30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800eb10 tx=0x7f0dc800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dd49d9640 1 -- 172.21.15.37:0/3568728781 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8004b20 con 0x7f0dd0106a80 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dd49d9640 1 -- 172.21.15.37:0/3568728781 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc8004cc0 con 0x7f0dd0106a80 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dd019d700 con 0x7f0dd0106a80 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dd49d9640 1 -- 172.21.15.37:0/3568728781 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc8010760 con 0x7f0dd0106a80 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.136+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dd01a4fe0 con 0x7f0dd0106a80 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 Processor -- start 2024-09-19T06:43:37.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd49d9640 1 -- 172.21.15.37:0/3568728781 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0dc8005020 con 0x7f0dd0106a80 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 -- start start 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b767f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b772800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b772d40 0x55a38b775130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a38b6cd280 con 0x55a38b772d40 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a38b674b10 con 0x55a38b768480 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a38b6c9240 con 0x55a38b767b60 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b772d40 0x55a38b775130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b772800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b772d40 0x55a38b775130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54066/0 (socket says 172.21.15.37:54066) 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dce7fc640 1 -- 172.21.15.37:0/589234704 learned_addr learned my addr 172.21.15.37:0/589234704 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b767f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.139+0000 7f0dd49d9640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0db80a33b0 0x7f0db80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dce7fc640 1 -- 172.21.15.37:0/589234704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 msgr2=0x55a38b767f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b767f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dcf7fe640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0db80a33b0 0x7f0db80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dce7fc640 1 -- 172.21.15.37:0/589234704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 msgr2=0x55a38b772800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b772800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dce7fc640 1 -- 172.21.15.37:0/589234704 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38b6811f0 con 0x55a38b772d40 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dceffd640 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b772800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dd49d9640 1 -- 172.21.15.37:0/3568728781 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0dc80d02b0 con 0x7f0dd0106a80 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b772d40 0x55a38b775130 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40096c0 tx=0x7f0dc4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244b420e008eea93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0db5ffb640 1 -- 172.21.15.37:0/589234704 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc4035030 con 0x55a38b772d40 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0db5ffb640 1 -- 172.21.15.37:0/589234704 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc4003c90 con 0x55a38b772d40 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0dcf7fe640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0db80a33b0 0x7f0db80a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80cdab0 tx=0x7f0dc806f3c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:37.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.140+0000 7f0db5ffb640 1 -- 172.21.15.37:0/589234704 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc403c680 con 0x55a38b772d40 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 -- 172.21.15.37:0/589234704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b772d40 msgr2=0x55a38b775130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b772d40 0x55a38b775130 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f0dc40096c0 tx=0x7f0dc4000bc0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 -- 172.21.15.37:0/589234704 shutdown_connections 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b772d40 0x55a38b775130 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b772800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/589234704 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b767f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 -- 172.21.15.37:0/589234704 >> 172.21.15.37:0/589234704 conn(0x55a38b7640f0 msgr2=0x55a38b7644d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 -- 172.21.15.37:0/589234704 shutdown_connections 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 -- 172.21.15.37:0/589234704 wait complete. 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.141+0000 7f0dd7db4740 1 -- start start 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 1 Processor -- start 2024-09-19T06:43:37.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b767b60 0x55a38b76fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b770180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 0x55a38b7706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a38b775910 con 0x55a38b767b60 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a38b687f10 con 0x55a38b768480 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a38b775a90 con 0x55a38b772d40 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dd7db4740 10 client.0 _ll_register_callbacks cb 0x55a38b650a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b767b60 0x55a38b76fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b770180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dce7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b767b60 0x55a38b76fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54074/0 (socket says 172.21.15.37:54074) 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dce7fc640 1 -- 172.21.15.37:0/3040508506 learned_addr learned my addr 172.21.15.37:0/3040508506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dceffd640 1 -- 172.21.15.37:0/3040508506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 msgr2=0x55a38b7706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dceffd640 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 0x55a38b7706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dceffd640 1 -- 172.21.15.37:0/3040508506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b767b60 msgr2=0x55a38b76fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dceffd640 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b767b60 0x55a38b76fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.142+0000 7f0dceffd640 1 -- 172.21.15.37:0/3040508506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38b43e6a0 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dceffd640 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b770180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000ab80 tx=0x7f0dc000c700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b62dca2017d17c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0daeffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0daeffd640 1 -- 172.21.15.37:0/3040508506 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc001f030 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc4003c90 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0daeffd640 1 -- 172.21.15.37:0/3040508506 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0dc000ac70 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0daeffd640 1 -- 172.21.15.37:0/3040508506 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0dc0029050 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 10 client.25517 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a38b781590 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0daeffd640 1 -- 172.21.15.37:0/3040508506 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc4003c90 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0daeffd640 1 -- 172.21.15.37:0/3040508506 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0dc001b500 con 0x55a38b768480 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 10 client.25517 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:37.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 10 client.25517 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 20 client.25517 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b782b40 0x55a38b784fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 4 client.25517 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a38b6da7f0 con 0x55a38b782b40 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.143+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b782b40 0x55a38b784fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.144+0000 7f0daeffd640 10 client.25517 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.144+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b782b40 0x55a38b784fe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e958c9b3e2e52b86 server_cookie=c19ed2053d649b8a in_seq=0 out_seq=0 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0daeffd640 1 -- 172.21.15.37:0/3040508506 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a38b6da7f0 con 0x55a38b782b40 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0daeffd640 10 client.25517 handle_command_reply: tid=0 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 client.25517 shutdown 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 msgr2=0x55a38b770180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b770180 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0dc000ab80 tx=0x7f0dc000c700 comp rx=0 tx=0).stop 2024-09-19T06:43:37.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 shutdown_connections 2024-09-19T06:43:37.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b782b40 0x55a38b784fe0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 0x55a38b7706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b768480 0x55a38b770180 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0daeffd640 0 client.25517 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/3040508506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b767b60 0x55a38b76fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 >> 172.21.15.37:0/3040508506 conn(0x55a38b7640f0 msgr2=0x55a38b769dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 shutdown_connections 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 1 -- 172.21.15.37:0/3040508506 wait complete. 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.145+0000 7f0dd7db4740 20 client.25517 trim_cache size 0 max 16384 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 Processor -- start 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 -- start start 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b767b60 0x55a38b769cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b78a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 0x55a38b78adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01adf70 con 0x55a38b768480 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01ab840 con 0x55a38b767b60 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01a9690 con 0x55a38b772d40 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b78a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b78a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54084/0 (socket says 172.21.15.37:54084) 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 -- 172.21.15.37:0/2599689 learned_addr learned my addr 172.21.15.37:0/2599689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dce7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b767b60 0x55a38b769cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 0x55a38b78adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 -- 172.21.15.37:0/2599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 msgr2=0x55a38b78adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 0x55a38b78adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 -- 172.21.15.37:0/2599689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b767b60 msgr2=0x55a38b769cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b767b60 0x55a38b769cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 -- 172.21.15.37:0/2599689 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a38b772510 con 0x55a38b768480 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.163+0000 7f0dceffd640 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b78a8b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00118b0 tx=0x7f0dc000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d0842c65e90a960 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0db57fa640 1 -- 172.21.15.37:0/2599689 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc001f040 con 0x55a38b768480 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0db57fa640 1 -- 172.21.15.37:0/2599689 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc000ae70 con 0x55a38b768480 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0db57fa640 1 -- 172.21.15.37:0/2599689 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc00049a0 con 0x55a38b768480 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 -- 172.21.15.37:0/2599689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 msgr2=0x55a38b78a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b78a8b0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f0dc00118b0 tx=0x7f0dc000cca0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 -- 172.21.15.37:0/2599689 shutdown_connections 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b772d40 0x55a38b78adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b78a8b0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/2599689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b767b60 0x55a38b769cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 -- 172.21.15.37:0/2599689 >> 172.21.15.37:0/2599689 conn(0x55a38b7640f0 msgr2=0x55a38b7695b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 -- 172.21.15.37:0/2599689 shutdown_connections 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.164+0000 7f0dd7db4740 1 -- 172.21.15.37:0/2599689 wait complete. 2024-09-19T06:43:37.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 -- start start 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 Processor -- start 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b776b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b7770d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b772d40 0x55a38b777610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01a9d00 con 0x55a38b768480 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01ab170 con 0x55a38b772d40 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0dd01aa3d0 con 0x55a38b767b60 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dd7db4740 10 client.0 _ll_register_callbacks cb 0x7f0dc00118e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:37.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b7770d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dceffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b7770d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54086/0 (socket says 172.21.15.37:54086) 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dceffd640 1 -- 172.21.15.37:0/696294538 learned_addr learned my addr 172.21.15.37:0/696294538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b776b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.165+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b772d40 0x55a38b777610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dceffd640 1 -- 172.21.15.37:0/696294538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 msgr2=0x55a38b776b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dceffd640 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b776b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dceffd640 1 -- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b772d40 msgr2=0x55a38b777610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dceffd640 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b772d40 0x55a38b777610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dceffd640 1 -- 172.21.15.37:0/696294538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a38b7ec980 con 0x55a38b768480 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dceffd640 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b7770d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0014040 tx=0x7f0dc000c700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f69d7b9bdfdf23c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0daeffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0daeffd640 1 -- 172.21.15.37:0/696294538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dc003a070 con 0x55a38b768480 2024-09-19T06:43:37.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0daeffd640 1 -- 172.21.15.37:0/696294538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0dc001b610 con 0x55a38b768480 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0dc000ae70 con 0x55a38b768480 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0daeffd640 1 -- 172.21.15.37:0/696294538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0dc0046050 con 0x55a38b768480 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dd7db4740 10 client.15987 fetch_fsmap learned FSMap version 25 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.166+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a38b7edca0 con 0x55a38b768480 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0daeffd640 1 -- 172.21.15.37:0/696294538 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0dc000ae70 con 0x55a38b768480 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0daeffd640 1 -- 172.21.15.37:0/696294538 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 25) ==== 8346+0+0 (secure 0 0 0) 0x7f0dc002d430 con 0x55a38b768480 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dd7db4740 10 client.15987 fetch_fsmap finished waiting for FSMap version 25 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dd7db4740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dd7db4740 20 client.15987 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b7ee200 0x55a38b7f0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dd7db4740 4 client.15987 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a38b6d4d00 con 0x55a38b7ee200 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b7ee200 0x55a38b7f0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:37.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0daeffd640 10 client.15987 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.167+0000 7f0dce7fc640 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b7ee200 0x55a38b7f0620 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=821c9799bdd9afe0 server_cookie=4c31b685f6c6a97d in_seq=0 out_seq=0 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.168+0000 7f0daeffd640 1 -- 172.21.15.37:0/696294538 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a38b6d4d00 con 0x55a38b7ee200 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.168+0000 7f0daeffd640 10 client.15987 handle_command_reply: tid=0 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.168+0000 7f0dd7db4740 1 client.15987 shutdown 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.168+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 msgr2=0x55a38b7770d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.168+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b7770d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f0dc0014040 tx=0x7f0dc000c700 comp rx=0 tx=0).stop 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 shutdown_connections 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a38b7ee200 0x55a38b7f0620 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a38b772d40 0x55a38b777610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a38b768480 0x55a38b7770d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 --2- 172.21.15.37:0/696294538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a38b767b60 0x55a38b776b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0daeffd640 0 client.15987 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:37.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 >> 172.21.15.37:0/696294538 conn(0x55a38b7640f0 msgr2=0x55a38b763270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:37.281 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:43:37.281 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:43:37.281 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "42b1367a-4566-4274-8bd8-64faaa963bd1", 2024-09-19T06:43:37.281 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:43:37.281 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:43:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 shutdown_connections 2024-09-19T06:43:37.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 1 -- 172.21.15.37:0/696294538 wait complete. 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd7db4740 20 client.15987 trim_cache size 0 max 16384 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0db80a33b0 msgr2=0x7f0db80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0db80a33b0 0x7f0db80a5870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0dc80cdab0 tx=0x7f0dc806f3c0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0106a80 msgr2=0x7f0dd019cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0106a80 0x7f0dd019cd30 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0dc800eb10 tx=0x7f0dc800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 shutdown_connections 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0db80a33b0 0x7f0db80a5870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0dd0106a80 0x7f0dd019cd30 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0dd0101420 0x7f0dd01989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 --2- 172.21.15.37:0/3568728781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0dd0100a80 0x7f0dd0198460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.169+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 >> 172.21.15.37:0/3568728781 conn(0x7f0dd00fc7c0 msgr2=0x7f0dd006aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.170+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 shutdown_connections 2024-09-19T06:43:37.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:37.170+0000 7f0dd59db640 1 -- 172.21.15.37:0/3568728781 wait complete. 2024-09-19T06:43:37.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:43:37.984 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:38.188 INFO:tasks.workunit.client.0.smithi037.stdout: 1 15647 20.50 MB/sec warmup 5 sec latency 18.793 ms 2024-09-19T06:43:38.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2558325915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102d30 msgr2=0x7f2f20103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2558325915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102d30 0x7f2f20103110 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009930 tx=0x7f2f1c02a420 comp rx=0 tx=0).stop 2024-09-19T06:43:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2558325915 shutdown_connections 2024-09-19T06:43:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2558325915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f201082a0 0x7f2f2010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2558325915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f201036e0 0x7f2f20107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2558325915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f20102d30 0x7f2f20103110 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2558325915 >> 172.21.15.37:0/2558325915 conn(0x7f2f200fc7d0 msgr2=0x7f2f200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2558325915 shutdown_connections 2024-09-19T06:43:38.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.475+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2558325915 wait complete. 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.476+0000 7f2f26d2a640 1 Processor -- start 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.476+0000 7f2f26d2a640 1 -- start start 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f26d2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f20102d30 0x7f2f201a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f26d2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201036e0 0x7f2f201a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f26d2a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f201082a0 0x7f2f2019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f26d2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201120a0 con 0x7f2f201036e0 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f26d2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20111f20 con 0x7f2f20102d30 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f26d2a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f20112220 con 0x7f2f201082a0 2024-09-19T06:43:38.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201036e0 0x7f2f201a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201036e0 0x7f2f201a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54106/0 (socket says 172.21.15.37:54106) 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 -- 172.21.15.37:0/2183265504 learned_addr learned my addr 172.21.15.37:0/2183265504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f20102d30 0x7f2f201a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f252c7640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f201082a0 0x7f2f2019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 -- 172.21.15.37:0/2183265504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f201082a0 msgr2=0x7f2f2019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f201082a0 0x7f2f2019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 -- 172.21.15.37:0/2183265504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f20102d30 msgr2=0x7f2f201a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f20102d30 0x7f2f201a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f17fff640 1 -- 172.21.15.37:0/2183265504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f2019ad00 con 0x7f2f201036e0 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.477+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f20102d30 0x7f2f201a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.478+0000 7f2f17fff640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201036e0 0x7f2f201a0950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000e970 tx=0x7f2f1000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:38.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.478+0000 7f2f15ffb640 1 -- 172.21.15.37:0/2183265504 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1000cd80 con 0x7f2f201036e0 2024-09-19T06:43:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.478+0000 7f2f15ffb640 1 -- 172.21.15.37:0/2183265504 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f100035f0 con 0x7f2f201036e0 2024-09-19T06:43:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.478+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f2019aff0 con 0x7f2f201036e0 2024-09-19T06:43:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.478+0000 7f2f15ffb640 1 -- 172.21.15.37:0/2183265504 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10010690 con 0x7f2f201036e0 2024-09-19T06:43:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.478+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f201a72e0 con 0x7f2f201036e0 2024-09-19T06:43:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f15ffb640 1 -- 172.21.15.37:0/2183265504 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2f100037a0 con 0x7f2f201036e0 2024-09-19T06:43:38.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 Processor -- start 2024-09-19T06:43:38.482 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 -- start start 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce477850 0x5650ce477c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce4824f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce482a30 0x5650ce484e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5650ce3dc160 con 0x5650ce482a30 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5650ce383720 con 0x5650ce478170 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.481+0000 7f2f29103740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650ce3d8120 con 0x5650ce477850 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce4824f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce4824f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53416/0 (socket says 172.21.15.37:53416) 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 -- 172.21.15.37:0/3255974574 learned_addr learned my addr 172.21.15.37:0/3255974574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f17fff640 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce482a30 0x5650ce484e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f252c7640 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce477850 0x5650ce477c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 -- 172.21.15.37:0/3255974574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce477850 msgr2=0x5650ce477c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce477850 0x5650ce477c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 -- 172.21.15.37:0/3255974574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce482a30 msgr2=0x5650ce484e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce482a30 0x5650ce484e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 -- 172.21.15.37:0/3255974574 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ce14dba0 con 0x5650ce478170 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f15ffb640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2f080a33b0 0x7f2f080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f252c7640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2f080a33b0 0x7f2f080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce4824f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c000c00 tx=0x7f2f1c005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e3cfb6fe24b29b3c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:38.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.482+0000 7f2f06ffd640 1 -- 172.21.15.37:0/3255974574 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c035040 con 0x5650ce478170 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f06ffd640 1 -- 172.21.15.37:0/3255974574 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1c031830 con 0x5650ce478170 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f06ffd640 1 -- 172.21.15.37:0/3255974574 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c003e60 con 0x5650ce478170 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f15ffb640 1 -- 172.21.15.37:0/2183265504 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f2f100cfff0 con 0x7f2f201036e0 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f29103740 1 -- 172.21.15.37:0/3255974574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 msgr2=0x5650ce4824f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f252c7640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2f080a33b0 0x7f2f080a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f18005fd0 tx=0x7f2f18009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f29103740 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce4824f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c000c00 tx=0x7f2f1c005d50 comp rx=0 tx=0).stop 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f29103740 1 -- 172.21.15.37:0/3255974574 shutdown_connections 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f29103740 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce482a30 0x5650ce484e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f29103740 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce4824f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f29103740 1 --2- 172.21.15.37:0/3255974574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce477850 0x5650ce477c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.483+0000 7f2f29103740 1 -- 172.21.15.37:0/3255974574 >> 172.21.15.37:0/3255974574 conn(0x5650ce473de0 msgr2=0x5650ce4741c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.484+0000 7f2f29103740 1 -- 172.21.15.37:0/3255974574 shutdown_connections 2024-09-19T06:43:38.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.484+0000 7f2f29103740 1 -- 172.21.15.37:0/3255974574 wait complete. 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.484+0000 7f2f29103740 1 -- start start 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.484+0000 7f2f29103740 1 Processor -- start 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f29103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce47f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f29103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce47fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f29103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce480320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f29103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5650ce485600 con 0x5650ce477850 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f29103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5650ce364900 con 0x5650ce478170 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f29103740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5650ce485780 con 0x5650ce482a30 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f29103740 10 client.0 _ll_register_callbacks cb 0x5650ce362780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce47f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce47fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce47f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54116/0 (socket says 172.21.15.37:54116) 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce47fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53418/0 (socket says 172.21.15.37:53418) 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f24ac6640 1 -- 172.21.15.37:0/50862666 learned_addr learned my addr 172.21.15.37:0/50862666 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f252c7640 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce480320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f17fff640 1 -- 172.21.15.37:0/50862666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 msgr2=0x5650ce480320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f17fff640 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce480320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f17fff640 1 -- 172.21.15.37:0/50862666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 msgr2=0x5650ce47f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f17fff640 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce47f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f17fff640 1 -- 172.21.15.37:0/50862666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ce3a4c50 con 0x5650ce478170 2024-09-19T06:43:38.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.485+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce47f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2f17fff640 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce47fde0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f100cd6e0 tx=0x7f2f10009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49589fd48d6942c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2efffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2efffff640 1 -- 172.21.15.37:0/50862666 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1009bdb0 con 0x5650ce478170 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2efffff640 1 -- 172.21.15.37:0/50862666 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f2f100d3c10 con 0x5650ce478170 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f1c031830 con 0x5650ce478170 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2efffff640 1 -- 172.21.15.37:0/50862666 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f1009bbd0 con 0x5650ce478170 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2f29103740 10 client.25535 fetch_fsmap learned FSMap version 26 2024-09-19T06:43:38.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650ce491290 con 0x5650ce478170 2024-09-19T06:43:38.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2efffff640 1 -- 172.21.15.37:0/50862666 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1c031830 con 0x5650ce478170 2024-09-19T06:43:38.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:38 smithi037 ceph-mon[31211]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T06:43:38.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:38 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:43:38.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:38 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:43:38.491 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:38 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:43:38.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:38 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:43:38.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:38 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:43:38.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.486+0000 7f2efffff640 1 -- 172.21.15.37:0/50862666 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f2f100a4a50 con 0x5650ce478170 2024-09-19T06:43:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f29103740 10 client.25535 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:43:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f29103740 10 client.25535 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f29103740 20 client.25535 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f29103740 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce492750 0x5650ce494be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f29103740 4 client.25535 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:38.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650ce3e9890 con 0x5650ce492750 2024-09-19T06:43:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce492750 0x5650ce494be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2efffff640 10 client.25535 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.487+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce492750 0x5650ce494be0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba1698dddac4963e server_cookie=b191b81fe55a5c95 in_seq=0 out_seq=0 2024-09-19T06:43:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.488+0000 7f2efffff640 1 -- 172.21.15.37:0/50862666 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5650ce3e9890 con 0x5650ce492750 2024-09-19T06:43:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.488+0000 7f2efffff640 10 client.25535 handle_command_reply: tid=0 2024-09-19T06:43:38.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 client.25535 shutdown 2024-09-19T06:43:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 msgr2=0x5650ce47fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce47fde0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2f100cd6e0 tx=0x7f2f10009550 comp rx=0 tx=0).stop 2024-09-19T06:43:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 shutdown_connections 2024-09-19T06:43:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce492750 0x5650ce494be0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce480320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2efffff640 0 client.25535 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce47fde0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 --2- 172.21.15.37:0/50862666 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce47f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 >> 172.21.15.37:0/50862666 conn(0x5650ce473de0 msgr2=0x5650ce479ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 shutdown_connections 2024-09-19T06:43:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 1 -- 172.21.15.37:0/50862666 wait complete. 2024-09-19T06:43:38.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.489+0000 7f2f29103740 20 client.25535 trim_cache size 0 max 16384 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 Processor -- start 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 -- start start 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce49a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce49a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce47cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201b2370 con 0x5650ce477850 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201afc40 con 0x5650ce478170 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f29103740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201ada90 con 0x5650ce482a30 2024-09-19T06:43:38.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce49a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce49a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54128/0 (socket says 172.21.15.37:54128) 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f24ac6640 1 -- 172.21.15.37:0/1185741380 learned_addr learned my addr 172.21.15.37:0/1185741380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.509+0000 7f2f17fff640 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce49a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f252c7640 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce47cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f17fff640 1 -- 172.21.15.37:0/1185741380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 msgr2=0x5650ce47cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f17fff640 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce47cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f17fff640 1 -- 172.21.15.37:0/1185741380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 msgr2=0x5650ce49a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f17fff640 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce49a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f17fff640 1 -- 172.21.15.37:0/1185741380 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650ce473c10 con 0x5650ce478170 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f17fff640 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce49a9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5650ce480e50 tx=0x7f2f10097eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a6fd90dc067f95a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f067fc640 1 -- 172.21.15.37:0/1185741380 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1009bdb0 con 0x5650ce478170 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f067fc640 1 -- 172.21.15.37:0/1185741380 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1009c970 con 0x5650ce478170 2024-09-19T06:43:38.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.510+0000 7f2f067fc640 1 -- 172.21.15.37:0/1185741380 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f10098510 con 0x5650ce478170 2024-09-19T06:43:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.511+0000 7f2f29103740 1 -- 172.21.15.37:0/1185741380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 msgr2=0x5650ce49a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.511+0000 7f2f29103740 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce49a9f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x5650ce480e50 tx=0x7f2f10097eb0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.511+0000 7f2f29103740 1 -- 172.21.15.37:0/1185741380 shutdown_connections 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.511+0000 7f2f29103740 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce47cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.511+0000 7f2f29103740 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce478170 0x5650ce49a9f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.511+0000 7f2f29103740 1 --2- 172.21.15.37:0/1185741380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce477850 0x5650ce49a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.512+0000 7f2f29103740 1 -- 172.21.15.37:0/1185741380 >> 172.21.15.37:0/1185741380 conn(0x5650ce473de0 msgr2=0x5650ce481920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.512+0000 7f2f29103740 1 -- 172.21.15.37:0/1185741380 shutdown_connections 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.512+0000 7f2f29103740 1 -- 172.21.15.37:0/1185741380 wait complete. 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.512+0000 7f2f29103740 1 -- start start 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.512+0000 7f2f29103740 1 Processor -- start 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f29103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce477850 0x5650ce486f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f29103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce478170 0x5650ce487470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f29103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce4fc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f29103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201ae100 con 0x5650ce478170 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f29103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201af570 con 0x5650ce477850 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f29103740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2f201ae7d0 con 0x5650ce482a30 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f29103740 10 client.0 _ll_register_callbacks cb 0x7f2f100d3790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce477850 0x5650ce486f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f17fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce478170 0x5650ce487470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce477850 0x5650ce486f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53446/0 (socket says 172.21.15.37:53446) 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 -- 172.21.15.37:0/4072506927 learned_addr learned my addr 172.21.15.37:0/4072506927 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f252c7640 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce4fc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 -- 172.21.15.37:0/4072506927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 msgr2=0x5650ce4fc480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce4fc480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 -- 172.21.15.37:0/4072506927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce478170 msgr2=0x5650ce487470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce478170 0x5650ce487470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f24ac6640 1 -- 172.21.15.37:0/4072506927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650ce47a6c0 con 0x5650ce477850 2024-09-19T06:43:38.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.513+0000 7f2f17fff640 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce478170 0x5650ce487470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2f24ac6640 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce477850 0x5650ce486f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009930 tx=0x7f2f1c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2ea3b75f42504c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2efffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2efffff640 1 -- 172.21.15.37:0/4072506927 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f1c035040 con 0x5650ce477850 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2efffff640 1 -- 172.21.15.37:0/4072506927 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f2f1c004240 con 0x5650ce477850 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f1009c970 con 0x5650ce477850 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2efffff640 1 -- 172.21.15.37:0/4072506927 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f2f1c004440 con 0x5650ce477850 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2f29103740 10 client.25541 fetch_fsmap learned FSMap version 26 2024-09-19T06:43:38.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5650ce4fdbf0 con 0x5650ce477850 2024-09-19T06:43:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2efffff640 1 -- 172.21.15.37:0/4072506927 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2f1009c970 con 0x5650ce477850 2024-09-19T06:43:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.514+0000 7f2efffff640 1 -- 172.21.15.37:0/4072506927 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f2f1c004c20 con 0x5650ce477850 2024-09-19T06:43:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f29103740 10 client.25541 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:43:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f29103740 10 client.25541 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f29103740 20 client.25541 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f29103740 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce4fdff0 0x5650ce5003e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:38.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f29103740 4 client.25541 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}] 2024-09-19T06:43:38.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 140.35, "await_for": 45.0}) -- 0x5650ce46f410 con 0x5650ce4fdff0 2024-09-19T06:43:38.519 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:38.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f17fff640 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce4fdff0 0x5650ce5003e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:38.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2efffff640 10 client.25541 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:38.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.515+0000 7f2f17fff640 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce4fdff0 0x5650ce5003e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ee23b0cf19e8327 server_cookie=841fa072e0414baa in_seq=0 out_seq=0 2024-09-19T06:43:38.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:38 smithi096 ceph-mon[35004]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T06:43:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:38 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:43:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:38 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:43:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:38 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:43:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:38 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:43:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:38 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:43:38.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:38 smithi151 ceph-mon[35236]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-19T06:43:38.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:38 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:43:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:38 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:43:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:38 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:43:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:38 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:43:38.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:38 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.633+0000 7f2efffff640 1 -- 172.21.15.37:0/4072506927 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5650ce46f410 con 0x5650ce4fdff0 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.633+0000 7f2efffff640 10 client.25541 handle_command_reply: tid=0 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 client.25541 shutdown 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce477850 msgr2=0x5650ce486f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce477850 0x5650ce486f30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c009930 tx=0x7f2f1c005be0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 shutdown_connections 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5650ce4fdff0 0x5650ce5003e0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5650ce482a30 0x5650ce4fc480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5650ce478170 0x5650ce487470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2efffff640 0 client.25541 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:38.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 --2- 172.21.15.37:0/4072506927 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5650ce477850 0x5650ce486f30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 >> 172.21.15.37:0/4072506927 conn(0x5650ce473de0 msgr2=0x5650ce47f860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 shutdown_connections 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 1 -- 172.21.15.37:0/4072506927 wait complete. 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.634+0000 7f2f29103740 20 client.25541 trim_cache size 0 max 16384 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 10, 2024-09-19T06:43:38.636 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "46e6b7a3": { 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 10, 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 140.3, 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:43:38.637 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:38.638 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2f080a33b0 msgr2=0x7f2f080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2f080a33b0 0x7f2f080a5870 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f18005fd0 tx=0x7f2f18009500 comp rx=0 tx=0).stop 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201036e0 msgr2=0x7f2f201a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201036e0 0x7f2f201a0950 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2f1000e970 tx=0x7f2f1000ee30 comp rx=0 tx=0).stop 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 shutdown_connections 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f201082a0 0x7f2f2019a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2f080a33b0 0x7f2f080a5870 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2f201036e0 0x7f2f201a0950 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 --2- 172.21.15.37:0/2183265504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2f20102d30 0x7f2f201a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 >> 172.21.15.37:0/2183265504 conn(0x7f2f200fc7d0 msgr2=0x7f2f2010cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 shutdown_connections 2024-09-19T06:43:38.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:38.635+0000 7f2f26d2a640 1 -- 172.21.15.37:0/2183265504 wait complete. 2024-09-19T06:43:39.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 18.22 MB/sec warmup 6 sec latency 559.036 ms 2024-09-19T06:43:39.444 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '46e6b7a3', quiesce took 5.3 seconds. Will release after: 15.05 2024-09-19T06:43:39.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 -- 172.21.15.37:0/532737082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 msgr2=0x7f4240106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 --2- 172.21.15.37:0/532737082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f4240106e60 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f4230009930 tx=0x7f423002a420 comp rx=0 tx=0).stop 2024-09-19T06:43:39.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 -- 172.21.15.37:0/532737082 shutdown_connections 2024-09-19T06:43:39.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 --2- 172.21.15.37:0/532737082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4240101420 0x7f424010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 --2- 172.21.15.37:0/532737082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4240100a80 0x7f4240100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 --2- 172.21.15.37:0/532737082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f4240106e60 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 -- 172.21.15.37:0/532737082 >> 172.21.15.37:0/532737082 conn(0x7f42400fc7a0 msgr2=0x7f42400febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:39.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 -- 172.21.15.37:0/532737082 shutdown_connections 2024-09-19T06:43:39.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.619+0000 7f424667d640 1 -- 172.21.15.37:0/532737082 wait complete. 2024-09-19T06:43:39.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 Processor -- start 2024-09-19T06:43:39.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 -- start start 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4240100a80 0x7f424010e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4240101420 0x7f424010e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f424010ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4240111fa0 con 0x7f4240106a80 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4240111e20 con 0x7f4240101420 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f424667d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4240112120 con 0x7f4240100a80 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f4244c1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f424010ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f4244c1a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f424010ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43994/0 (socket says 172.21.15.37:43994) 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f4244c1a640 1 -- 172.21.15.37:0/1528633034 learned_addr learned my addr 172.21.15.37:0/1528633034 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f423ffff640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4240100a80 0x7f424010e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.620+0000 7f423f7fe640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4240101420 0x7f424010e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f4244c1a640 1 -- 172.21.15.37:0/1528633034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4240100a80 msgr2=0x7f424010e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f4244c1a640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4240100a80 0x7f424010e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f4244c1a640 1 -- 172.21.15.37:0/1528633034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4240101420 msgr2=0x7f424010e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f4244c1a640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4240101420 0x7f424010e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f4244c1a640 1 -- 172.21.15.37:0/1528633034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42401a6960 con 0x7f4240106a80 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f4244c1a640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f424010ee60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f423800eb10 tx=0x7f423800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f423d7fa640 1 -- 172.21.15.37:0/1528633034 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238004b20 con 0x7f4240106a80 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f423d7fa640 1 -- 172.21.15.37:0/1528633034 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4238004cc0 con 0x7f4240106a80 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42401a6c50 con 0x7f4240106a80 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f423d7fa640 1 -- 172.21.15.37:0/1528633034 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238010760 con 0x7f4240106a80 2024-09-19T06:43:39.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.621+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42401a7140 con 0x7f4240106a80 2024-09-19T06:43:39.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.623+0000 7f423d7fa640 1 -- 172.21.15.37:0/1528633034 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4238005020 con 0x7f4240106a80 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 Processor -- start 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 -- start start 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd5e4a0 0x5643acd5e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd69140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd69680 0x5643acd6ba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5643accc42e0 con 0x5643acd5e4a0 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643acc6bce0 con 0x5643acd69680 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f4248a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643accc02a0 con 0x5643acd5edc0 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd5e4a0 0x5643acd5e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd5e4a0 0x5643acd5e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44006/0 (socket says 172.21.15.37:44006) 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f423d7fa640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f423f7fe640 1 -- 172.21.15.37:0/900612365 learned_addr learned my addr 172.21.15.37:0/900612365 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:39.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f423ffff640 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd69140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f423f7fe640 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd69680 0x5643acd6ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.624+0000 7f423d7fa640 1 -- 172.21.15.37:0/1528633034 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f42380d0210 con 0x7f4240106a80 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f4244c1a640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f423f7fe640 1 -- 172.21.15.37:0/900612365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 msgr2=0x5643acd69140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f423f7fe640 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd69140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f423f7fe640 1 -- 172.21.15.37:0/900612365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd69680 msgr2=0x5643acd6ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f423f7fe640 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd69680 0x5643acd6ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f423f7fe640 1 -- 172.21.15.37:0/900612365 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643acc50440 con 0x5643acd5e4a0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f423f7fe640 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd5e4a0 0x5643acd5e880 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4234004650 tx=0x7f4234004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad436d29967983b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f4244c1a640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42280a33b0 0x7f42280a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42380cda10 tx=0x7f423806e270 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f42267fc640 1 -- 172.21.15.37:0/900612365 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234005020 con 0x5643acd5e4a0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f42267fc640 1 -- 172.21.15.37:0/900612365 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42340056a0 con 0x5643acd5e4a0 2024-09-19T06:43:39.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.625+0000 7f42267fc640 1 -- 172.21.15.37:0/900612365 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42340106c0 con 0x5643acd5e4a0 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 -- 172.21.15.37:0/900612365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd5e4a0 msgr2=0x5643acd5e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd5e4a0 0x5643acd5e880 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f4234004650 tx=0x7f4234004250 comp rx=0 tx=0).stop 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 -- 172.21.15.37:0/900612365 shutdown_connections 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd69680 0x5643acd6ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd69140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 --2- 172.21.15.37:0/900612365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd5e4a0 0x5643acd5e880 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 -- 172.21.15.37:0/900612365 >> 172.21.15.37:0/900612365 conn(0x5643acd5aa30 msgr2=0x5643acd5ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 -- 172.21.15.37:0/900612365 shutdown_connections 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 -- 172.21.15.37:0/900612365 wait complete. 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 -- start start 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.626+0000 7f4248a56740 1 Processor -- start 2024-09-19T06:43:39.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f4248a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd68f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f4248a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd66220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f4248a56740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd66760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f4248a56740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5643acd6c250 con 0x5643acd69680 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f4248a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643acc7f050 con 0x5643acd5e4a0 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f4248a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5643acd6c3d0 con 0x5643acd5edc0 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f4248a56740 10 client.0 _ll_register_callbacks cb 0x5643accbf160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd66760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd66760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44014/0 (socket says 172.21.15.37:44014) 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 -- 172.21.15.37:0/3688185538 learned_addr learned my addr 172.21.15.37:0/3688185538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd68f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423ffff640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd66220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 -- 172.21.15.37:0/3688185538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 msgr2=0x5643acd66220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd66220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 -- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 msgr2=0x5643acd68f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd68f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423f7fe640 1 -- 172.21.15.37:0/3688185538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643aca353c0 con 0x5643acd69680 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.627+0000 7f423ffff640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd66220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f423f7fe640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd66760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f423400b490 tx=0x7f4234004cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b23a7d4776e9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f421f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f421f7fe640 1 -- 172.21.15.37:0/3688185538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42340050e0 con 0x5643acd69680 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f421f7fe640 1 -- 172.21.15.37:0/3688185538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f4234005280 con 0x5643acd69680 2024-09-19T06:43:39.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42340056a0 con 0x5643acd69680 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f421f7fe640 1 -- 172.21.15.37:0/3688185538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4234005480 con 0x5643acd69680 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f4248a56740 10 client.16017 fetch_fsmap learned FSMap version 26 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643acd77bc0 con 0x5643acd69680 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f421f7fe640 1 -- 172.21.15.37:0/3688185538 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42340056a0 con 0x5643acd69680 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f421f7fe640 1 -- 172.21.15.37:0/3688185538 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f423401f410 con 0x5643acd69680 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f4248a56740 10 client.16017 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f4248a56740 10 client.16017 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f4248a56740 20 client.16017 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.628+0000 7f4248a56740 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acd78f10 0x5643acd7b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.629+0000 7f4248a56740 4 client.16017 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.629+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643accd1850 con 0x5643acd78f10 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.629+0000 7f423ffff640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acd78f10 0x5643acd7b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.629+0000 7f423ffff640 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acd78f10 0x5643acd7b300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162d996e3eb23dd8 server_cookie=96cd9bfc4e43b2b8 in_seq=0 out_seq=0 2024-09-19T06:43:39.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.629+0000 7f421f7fe640 10 client.16017 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f421f7fe640 1 -- 172.21.15.37:0/3688185538 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5643accd1850 con 0x5643acd78f10 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f421f7fe640 10 client.16017 handle_command_reply: tid=0 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 client.16017 shutdown 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 msgr2=0x5643acd66760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd66760 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f423400b490 tx=0x7f4234004cb0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 shutdown_connections 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acd78f10 0x5643acd7b300 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd66760 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd66220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 --2- 172.21.15.37:0/3688185538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd68f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 >> 172.21.15.37:0/3688185538 conn(0x5643acd5aa30 msgr2=0x5643acd60710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f421f7fe640 0 client.16017 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 shutdown_connections 2024-09-19T06:43:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.630+0000 7f4248a56740 1 -- 172.21.15.37:0/3688185538 wait complete. 2024-09-19T06:43:39.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.631+0000 7f4248a56740 20 client.16017 trim_cache size 0 max 16384 2024-09-19T06:43:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 Processor -- start 2024-09-19T06:43:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 -- start start 2024-09-19T06:43:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5e4a0 0x5643acd59a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5edc0 0x5643acd80ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd81000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42401b2420 con 0x5643acd69680 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42401afcf0 con 0x5643acd5edc0 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f4248a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42401adb40 con 0x5643acd5e4a0 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd81000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd81000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44028/0 (socket says 172.21.15.37:44028) 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 -- 172.21.15.37:0/428420288 learned_addr learned my addr 172.21.15.37:0/428420288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5e4a0 0x5643acd59a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 -- 172.21.15.37:0/428420288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5e4a0 msgr2=0x5643acd59a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423ffff640 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5edc0 0x5643acd80ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5e4a0 0x5643acd59a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 -- 172.21.15.37:0/428420288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5edc0 msgr2=0x5643acd80ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5edc0 0x5643acd80ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 -- 172.21.15.37:0/428420288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643acd651d0 con 0x5643acd69680 2024-09-19T06:43:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.648+0000 7f423f7fe640 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd81000 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f423400bd80 tx=0x7f4234000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d724e206131a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4225ffb640 1 -- 172.21.15.37:0/428420288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234003e50 con 0x5643acd69680 2024-09-19T06:43:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4225ffb640 1 -- 172.21.15.37:0/428420288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4234004120 con 0x5643acd69680 2024-09-19T06:43:39.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4225ffb640 1 -- 172.21.15.37:0/428420288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423401f3e0 con 0x5643acd69680 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 -- 172.21.15.37:0/428420288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 msgr2=0x5643acd81000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd81000 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f423400bd80 tx=0x7f4234000bc0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 -- 172.21.15.37:0/428420288 shutdown_connections 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acd81000 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5edc0 0x5643acd80ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 --2- 172.21.15.37:0/428420288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5e4a0 0x5643acd59a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 -- 172.21.15.37:0/428420288 >> 172.21.15.37:0/428420288 conn(0x5643acd5aa30 msgr2=0x5643acd59370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 -- 172.21.15.37:0/428420288 shutdown_connections 2024-09-19T06:43:39.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.649+0000 7f4248a56740 1 -- 172.21.15.37:0/428420288 wait complete. 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 -- start start 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 Processor -- start 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd6ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd6f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acddba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42401ae1b0 con 0x5643acd69680 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42401af620 con 0x5643acd5e4a0 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42401ae880 con 0x5643acd5edc0 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f4248a56740 10 client.0 _ll_register_callbacks cb 0x7f4230009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acddba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acddba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44044/0 (socket says 172.21.15.37:44044) 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f423f7fe640 1 -- 172.21.15.37:0/2658409279 learned_addr learned my addr 172.21.15.37:0/2658409279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:39.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.650+0000 7f423ffff640 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd6f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f423f7fe640 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd6ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f423f7fe640 1 -- 172.21.15.37:0/2658409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 msgr2=0x5643acd6f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f423f7fe640 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd6f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f423f7fe640 1 -- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 msgr2=0x5643acd6ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f423f7fe640 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd6ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f423f7fe640 1 -- 172.21.15.37:0/2658409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643acddc100 con 0x5643acd69680 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f423f7fe640 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acddba60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f423400ba50 tx=0x7f4234024040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64d75fe55c925d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f421f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f421f7fe640 1 -- 172.21.15.37:0/2658409279 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423401f5e0 con 0x5643acd69680 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f421f7fe640 1 -- 172.21.15.37:0/2658409279 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f423401fa50 con 0x5643acd69680 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4234004120 con 0x5643acd69680 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f421f7fe640 1 -- 172.21.15.37:0/2658409279 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f42340056f0 con 0x5643acd69680 2024-09-19T06:43:39.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f4248a56740 10 client.16023 fetch_fsmap learned FSMap version 26 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.651+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5643acde34c0 con 0x5643acd69680 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f421f7fe640 1 -- 172.21.15.37:0/2658409279 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4234004120 con 0x5643acd69680 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f421f7fe640 1 -- 172.21.15.37:0/2658409279 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 26) ==== 8350+0+0 (secure 0 0 0) 0x7f4234010bb0 con 0x5643acd69680 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f4248a56740 10 client.16023 fetch_fsmap finished waiting for FSMap version 26 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f4248a56740 10 client.16023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f4248a56740 20 client.16023 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f4248a56740 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acde3b00 0x5643acde5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f4248a56740 4 client.16023 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5643acccbd60 con 0x5643acde3b00 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.652+0000 7f423ffff640 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acde3b00 0x5643acde5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f423ffff640 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acde3b00 0x5643acde5ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29bd8e5290a00976 server_cookie=2f0bd698a83cc523 in_seq=0 out_seq=0 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f421f7fe640 10 client.16023 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f421f7fe640 1 -- 172.21.15.37:0/2658409279 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5643acccbd60 con 0x5643acde3b00 2024-09-19T06:43:39.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f421f7fe640 10 client.16023 handle_command_reply: tid=0 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 client.16023 shutdown 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 msgr2=0x5643acddba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acddba60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f423400ba50 tx=0x7f4234024040 comp rx=0 tx=0).stop 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 shutdown_connections 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5643acde3b00 0x5643acde5ef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5643acd69680 0x5643acddba60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5643acd5edc0 0x5643acd6f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 --2- 172.21.15.37:0/2658409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643acd5e4a0 0x5643acd6ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 >> 172.21.15.37:0/2658409279 conn(0x5643acd5aa30 msgr2=0x5643acd59b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f421f7fe640 0 client.16023 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 shutdown_connections 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 1 -- 172.21.15.37:0/2658409279 wait complete. 2024-09-19T06:43:39.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.653+0000 7f4248a56740 20 client.16023 trim_cache size 0 max 16384 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "42b1367a-4566-4274-8bd8-64faaa963bd1": { 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "42b1367a-4566-4274-8bd8-64faaa963bd1", 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:39.660 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:43:39.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42280a33b0 msgr2=0x7f42280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42280a33b0 0x7f42280a5870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f42380cda10 tx=0x7f423806e270 comp rx=0 tx=0).stop 2024-09-19T06:43:39.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 msgr2=0x7f424010ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:39.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f424010ee60 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f423800eb10 tx=0x7f423800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 shutdown_connections 2024-09-19T06:43:39.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42280a33b0 0x7f42280a5870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4240106a80 0x7f424010ee60 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4240101420 0x7f424010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 --2- 172.21.15.37:0/1528633034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4240100a80 0x7f424010e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:39.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 >> 172.21.15.37:0/1528633034 conn(0x7f42400fc7a0 msgr2=0x7f42400fd040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:39.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 shutdown_connections 2024-09-19T06:43:39.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:39.654+0000 7f424667d640 1 -- 172.21.15.37:0/1528633034 wait complete. 2024-09-19T06:43:40.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 15.61 MB/sec warmup 7 sec latency 1559.145 ms 2024-09-19T06:43:40.422 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:40 smithi037 ceph-mon[31211]: pgmap v59: 97 pgs: 97 active+clean; 8.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 751 KiB/s wr, 92 op/s 2024-09-19T06:43:40.458 INFO:tasks.cephfs.filesystem:scrub status for tag:42b1367a-4566-4274-8bd8-64faaa963bd1 - {'path': '/', 'tag': '42b1367a-4566-4274-8bd8-64faaa963bd1', 'options': 'recursive,force'} 2024-09-19T06:43:40.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:40 smithi151 ceph-mon[35236]: pgmap v59: 97 pgs: 97 active+clean; 8.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 751 KiB/s wr, 92 op/s 2024-09-19T06:43:40.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:40 smithi096 ceph-mon[35004]: pgmap v59: 97 pgs: 97 active+clean; 8.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 443 KiB/s rd, 751 KiB/s wr, 92 op/s 2024-09-19T06:43:41.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 13.66 MB/sec warmup 8 sec latency 2559.263 ms 2024-09-19T06:43:42.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 12.14 MB/sec warmup 9 sec latency 3559.381 ms 2024-09-19T06:43:42.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:42 smithi151 ceph-mon[35236]: pgmap v60: 97 pgs: 97 active+clean; 8.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 751 KiB/s wr, 90 op/s 2024-09-19T06:43:42.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:42 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:43:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:42 smithi096 ceph-mon[35004]: pgmap v60: 97 pgs: 97 active+clean; 8.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 751 KiB/s wr, 90 op/s 2024-09-19T06:43:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:42 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:43:42.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:42 smithi037 ceph-mon[31211]: pgmap v60: 97 pgs: 97 active+clean; 8.4 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 442 KiB/s rd, 751 KiB/s wr, 90 op/s 2024-09-19T06:43:42.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:42 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:43:43.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 10.93 MB/sec warmup 10 sec latency 4559.488 ms 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 -- 172.21.15.37:0/1368495408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c41035e0 msgr2=0x7f56c410bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 --2- 172.21.15.37:0/1368495408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c41035e0 0x7f56c410bac0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009900 tx=0x7f56ac02a480 comp rx=0 tx=0).stop 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 -- 172.21.15.37:0/1368495408 shutdown_connections 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 --2- 172.21.15.37:0/1368495408 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c410c110 0x7f56c410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 --2- 172.21.15.37:0/1368495408 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c41035e0 0x7f56c410bac0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 --2- 172.21.15.37:0/1368495408 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c4102cc0 0x7f56c41030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 -- 172.21.15.37:0/1368495408 >> 172.21.15.37:0/1368495408 conn(0x7f56c40fc6b0 msgr2=0x7f56c40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 -- 172.21.15.37:0/1368495408 shutdown_connections 2024-09-19T06:43:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.392+0000 7f56c898d640 1 -- 172.21.15.37:0/1368495408 wait complete. 2024-09-19T06:43:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 Processor -- start 2024-09-19T06:43:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 -- start start 2024-09-19T06:43:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c4102cc0 0x7f56c4198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c41035e0 0x7f56c4198850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c410c110 0x7f56c419cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4111f40 con 0x7f56c410c110 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56c4111dc0 con 0x7f56c4102cc0 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c898d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41120c0 con 0x7f56c41035e0 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c410c110 0x7f56c419cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c410c110 0x7f56c419cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44056/0 (socket says 172.21.15.37:44056) 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 -- 172.21.15.37:0/3162509100 learned_addr learned my addr 172.21.15.37:0/3162509100 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c259c640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c4102cc0 0x7f56c4198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c41035e0 0x7f56c4198850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 -- 172.21.15.37:0/3162509100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c41035e0 msgr2=0x7f56c4198850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c41035e0 0x7f56c4198850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 -- 172.21.15.37:0/3162509100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c4102cc0 msgr2=0x7f56c4198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c4102cc0 0x7f56c4198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 -- 172.21.15.37:0/3162509100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56c419d300 con 0x7f56c410c110 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.393+0000 7f56c2d9d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c410c110 0x7f56c419cbd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00eaa0 tx=0x7f56bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.394+0000 7f56bb7fe640 1 -- 172.21.15.37:0/3162509100 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc00cdb0 con 0x7f56c410c110 2024-09-19T06:43:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.394+0000 7f56bb7fe640 1 -- 172.21.15.37:0/3162509100 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56bc005020 con 0x7f56c410c110 2024-09-19T06:43:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.394+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56c419d5f0 con 0x7f56c410c110 2024-09-19T06:43:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.394+0000 7f56bb7fe640 1 -- 172.21.15.37:0/3162509100 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56bc0116b0 con 0x7f56c410c110 2024-09-19T06:43:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.394+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56c41a4e80 con 0x7f56c410c110 2024-09-19T06:43:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.396+0000 7f56bb7fe640 1 -- 172.21.15.37:0/3162509100 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f56bc005560 con 0x7f56c410c110 2024-09-19T06:43:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.396+0000 7f56cb805740 1 Processor -- start 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56cb805740 1 -- start start 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56cb805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af54006db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56cb805740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54015b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56cb805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af540184b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56cb805740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55af53f6b2f0 con 0x55af540072f0 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56cb805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af53f13270 con 0x55af540160c0 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56cb805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af53f672b0 con 0x55af540069d0 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54015b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54015b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44068/0 (socket says 172.21.15.37:44068) 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 -- 172.21.15.37:0/3906888488 learned_addr learned my addr 172.21.15.37:0/3906888488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56bb7fe640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f56b00a34e0 0x7f56b00a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af54006db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c2d9d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f56b00a34e0 0x7f56b00a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 -- 172.21.15.37:0/3906888488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 msgr2=0x55af54006db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af540184b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af54006db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 -- 172.21.15.37:0/3906888488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 msgr2=0x55af540184b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af540184b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c259c640 1 -- 172.21.15.37:0/3906888488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af53f12340 con 0x55af540072f0 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af540184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.397+0000 7f56bb7fe640 1 -- 172.21.15.37:0/3162509100 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f56bc0cff10 con 0x7f56c410c110 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.398+0000 7f56c259c640 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54015b80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f56b400cc90 tx=0x7f56b400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33818c882e5cef71 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.398+0000 7f56c2d9d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f56b00a34e0 0x7f56b00a59a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cdc60 tx=0x7f56bc06df70 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.398+0000 7f56b8ff9640 1 -- 172.21.15.37:0/3906888488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4014070 con 0x55af540072f0 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.398+0000 7f56b8ff9640 1 -- 172.21.15.37:0/3906888488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b40047a0 con 0x55af540072f0 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.398+0000 7f56b8ff9640 1 -- 172.21.15.37:0/3906888488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4010760 con 0x55af540072f0 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 -- 172.21.15.37:0/3906888488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 msgr2=0x55af54015b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54015b80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f56b400cc90 tx=0x7f56b400e720 comp rx=0 tx=0).stop 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 -- 172.21.15.37:0/3906888488 shutdown_connections 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af540184b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54015b80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 --2- 172.21.15.37:0/3906888488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af54006db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 -- 172.21.15.37:0/3906888488 >> 172.21.15.37:0/3906888488 conn(0x55af54003060 msgr2=0x55af54003440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 -- 172.21.15.37:0/3906888488 shutdown_connections 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 -- 172.21.15.37:0/3906888488 wait complete. 2024-09-19T06:43:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 -- start start 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.399+0000 7f56cb805740 1 Processor -- start 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540069d0 0x55af54013040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54013580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540160c0 0x55af54013ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55af54018cf0 con 0x55af540072f0 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55af53ef4290 con 0x55af540069d0 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55af54018e70 con 0x55af540160c0 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 10 client.0 _ll_register_callbacks cb 0x55af53f25140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54013580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54013580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44080/0 (socket says 172.21.15.37:44080) 2024-09-19T06:43:43.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 -- 172.21.15.37:0/524191237 learned_addr learned my addr 172.21.15.37:0/524191237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540160c0 0x55af54013ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 -- 172.21.15.37:0/524191237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540160c0 msgr2=0x55af54013ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540160c0 0x55af54013ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 -- 172.21.15.37:0/524191237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540069d0 msgr2=0x55af54013040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540069d0 0x55af54013040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 -- 172.21.15.37:0/524191237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af53c7e210 con 0x55af540072f0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540160c0 0x55af54013ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56c259c640 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54013580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f56b4010c90 tx=0x7f56b4010cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f8dda8b3989f9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56a9ffb640 1 -- 172.21.15.37:0/524191237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b4027060 con 0x55af540072f0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56a9ffb640 1 -- 172.21.15.37:0/524191237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f56b4018620 con 0x55af540072f0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.400+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56b40047a0 con 0x55af540072f0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56a9ffb640 1 -- 172.21.15.37:0/524191237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56b4014070 con 0x55af540072f0 2024-09-19T06:43:43.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 10 client.16035 fetch_fsmap learned FSMap version 27 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af53d4a620 con 0x55af540072f0 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56a9ffb640 1 -- 172.21.15.37:0/524191237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56b40047a0 con 0x55af540072f0 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56a9ffb640 1 -- 172.21.15.37:0/524191237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f56b400f670 con 0x55af540072f0 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 10 client.16035 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 20 client.16035 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af540283a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 4 client.16035 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af53f78a20 con 0x55af54025fb0 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.401+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af540283a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.402+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af540283a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a0aa2e0786e5ff2 server_cookie=8cdcc46b3055e3f6 in_seq=0 out_seq=0 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.402+0000 7f56a9ffb640 10 client.16035 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:43.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.402+0000 7f56a9ffb640 1 -- 172.21.15.37:0/524191237 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55af53f78a20 con 0x55af54025fb0 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.402+0000 7f56a9ffb640 10 client.16035 handle_command_reply: tid=0 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.402+0000 7f56cb805740 1 client.16035 shutdown 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 msgr2=0x55af54013580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54013580 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f56b4010c90 tx=0x7f56b4010cc0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 shutdown_connections 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af540283a0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540160c0 0x55af54013ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af54013580 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56a9ffb640 0 client.16035 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 --2- 172.21.15.37:0/524191237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540069d0 0x55af54013040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 >> 172.21.15.37:0/524191237 conn(0x55af54003060 msgr2=0x55af54008dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 shutdown_connections 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 1 -- 172.21.15.37:0/524191237 wait complete. 2024-09-19T06:43:43.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.403+0000 7f56cb805740 20 client.16035 trim_cache size 0 max 16384 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.417+0000 7f56cb805740 1 Processor -- start 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56cb805740 1 -- start start 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56cb805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5402d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56cb805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540072f0 0x55af5402deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56cb805740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540160c0 0x55af54010280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56cb805740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41adf00 con 0x55af540160c0 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56cb805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41ab7d0 con 0x55af540072f0 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56cb805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41a9620 con 0x55af540069d0 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540160c0 0x55af54010280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540160c0 0x55af54010280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44092/0 (socket says 172.21.15.37:44092) 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 -- 172.21.15.37:0/2081222617 learned_addr learned my addr 172.21.15.37:0/2081222617 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5402d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c259c640 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540072f0 0x55af5402deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 -- 172.21.15.37:0/2081222617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 msgr2=0x55af5402d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5402d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 -- 172.21.15.37:0/2081222617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540072f0 msgr2=0x55af5402deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540072f0 0x55af5402deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 -- 172.21.15.37:0/2081222617 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af5400dd50 con 0x55af540160c0 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c259c640 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540072f0 0x55af5402deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.418+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540160c0 0x55af54010280 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005ab0 tx=0x7f56ac009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a17fd8831ae3a81 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56abfff640 1 -- 172.21.15.37:0/2081222617 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac04d070 con 0x55af540160c0 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56abfff640 1 -- 172.21.15.37:0/2081222617 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac0044a0 con 0x55af540160c0 2024-09-19T06:43:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56abfff640 1 -- 172.21.15.37:0/2081222617 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56ac035040 con 0x55af540160c0 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 -- 172.21.15.37:0/2081222617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540160c0 msgr2=0x55af54010280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540160c0 0x55af54010280 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f56ac005ab0 tx=0x7f56ac009650 comp rx=0 tx=0).stop 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 -- 172.21.15.37:0/2081222617 shutdown_connections 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540160c0 0x55af54010280 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540072f0 0x55af5402deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 --2- 172.21.15.37:0/2081222617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5402d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 -- 172.21.15.37:0/2081222617 >> 172.21.15.37:0/2081222617 conn(0x55af54003060 msgr2=0x55af540150c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 -- 172.21.15.37:0/2081222617 shutdown_connections 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.419+0000 7f56cb805740 1 -- 172.21.15.37:0/2081222617 wait complete. 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 -- start start 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 Processor -- start 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5401a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af5401a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af5401abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41a9c90 con 0x55af540072f0 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41ab100 con 0x55af540160c0 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f56c41aa360 con 0x55af540069d0 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56cb805740 10 client.0 _ll_register_callbacks cb 0x7f56b400cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af5401a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56c259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af5401a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44104/0 (socket says 172.21.15.37:44104) 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56c259c640 1 -- 172.21.15.37:0/3316237255 learned_addr learned my addr 172.21.15.37:0/3316237255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5401a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.420+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af5401abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56c259c640 1 -- 172.21.15.37:0/3316237255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 msgr2=0x55af5401a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56c259c640 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5401a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56c259c640 1 -- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 msgr2=0x55af5401abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56c259c640 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af5401abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56c259c640 1 -- 172.21.15.37:0/3316237255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af5408fc30 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56c259c640 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af5401a660 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f56b4000c00 tx=0x7f56b4010d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6108fb59a5632e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56a9ffb640 1 -- 172.21.15.37:0/3316237255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56b401c020 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56a9ffb640 1 -- 172.21.15.37:0/3316237255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f56b400f3c0 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56ac0044a0 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56a9ffb640 1 -- 172.21.15.37:0/3316237255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f56b403b050 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56cb805740 10 client.16041 fetch_fsmap learned FSMap version 27 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55af54090eb0 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.421+0000 7f56a9ffb640 1 -- 172.21.15.37:0/3316237255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f56ac0044a0 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56a9ffb640 1 -- 172.21.15.37:0/3316237255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f56b4004800 con 0x55af540072f0 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56cb805740 10 client.16041 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56cb805740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56cb805740 20 client.16041 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56cb805740 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af54093640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56cb805740 4 client.16041 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:43:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55af53f72d70 con 0x55af54025fb0 2024-09-19T06:43:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af54093640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56c1d9b640 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af54093640 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a22884961b91c8e2 server_cookie=4089f4d4402a8a14 in_seq=0 out_seq=0 2024-09-19T06:43:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.422+0000 7f56a9ffb640 10 client.16041 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56a9ffb640 1 -- 172.21.15.37:0/3316237255 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55af53f72d70 con 0x55af54025fb0 2024-09-19T06:43:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56a9ffb640 10 client.16041 handle_command_reply: tid=0 2024-09-19T06:43:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 client.16041 shutdown 2024-09-19T06:43:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 msgr2=0x55af5401a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af5401a660 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f56b4000c00 tx=0x7f56b4010d30 comp rx=0 tx=0).stop 2024-09-19T06:43:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 shutdown_connections 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55af54025fb0 0x55af54093640 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55af540160c0 0x55af5401abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55af540072f0 0x55af5401a660 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56a9ffb640 0 client.16041 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 --2- 172.21.15.37:0/3316237255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55af540069d0 0x55af5401a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 >> 172.21.15.37:0/3316237255 conn(0x55af54003060 msgr2=0x55af540148a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 shutdown_connections 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 1 -- 172.21.15.37:0/3316237255 wait complete. 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.490+0000 7f56cb805740 20 client.16041 trim_cache size 0 max 16384 2024-09-19T06:43:43.492 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:43:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:43:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:43:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:43:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f56b00a34e0 msgr2=0x7f56b00a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.493 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f56b00a34e0 0x7f56b00a59a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f56bc0cdc60 tx=0x7f56bc06df70 comp rx=0 tx=0).stop 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c410c110 msgr2=0x7f56c419cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c410c110 0x7f56c419cbd0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f56bc00eaa0 tx=0x7f56bc00ef60 comp rx=0 tx=0).stop 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 shutdown_connections 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f56b00a34e0 0x7f56b00a59a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f56c410c110 0x7f56c419cbd0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f56c41035e0 0x7f56c4198850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 --2- 172.21.15.37:0/3162509100 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f56c4102cc0 0x7f56c4198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 >> 172.21.15.37:0/3162509100 conn(0x7f56c40fc6b0 msgr2=0x7f56c40fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 shutdown_connections 2024-09-19T06:43:43.494 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:43.491+0000 7f56c898d640 1 -- 172.21.15.37:0/3162509100 wait complete. 2024-09-19T06:43:43.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:43 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:43:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:43 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:43:43.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:43 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:43:44.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 9.94 MB/sec warmup 11 sec latency 5559.585 ms 2024-09-19T06:43:44.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:44 smithi151 ceph-mon[35236]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T06:43:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:44 smithi096 ceph-mon[35004]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T06:43:44.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:44 smithi037 ceph-mon[31211]: pgmap v61: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 1.3 MiB/s wr, 119 op/s 2024-09-19T06:43:45.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 9.11 MB/sec warmup 12 sec latency 6559.717 ms 2024-09-19T06:43:46.189 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 8.41 MB/sec warmup 13 sec latency 7559.822 ms 2024-09-19T06:43:46.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:46 smithi151 ceph-mon[35236]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-09-19T06:43:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:46 smithi096 ceph-mon[35004]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-09-19T06:43:46.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:46 smithi037 ceph-mon[31211]: pgmap v62: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-09-19T06:43:47.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 7.81 MB/sec warmup 14 sec latency 8559.942 ms 2024-09-19T06:43:48.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 7.29 MB/sec warmup 15 sec latency 9560.048 ms 2024-09-19T06:43:48.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:48 smithi151 ceph-mon[35236]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-09-19T06:43:48.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:48 smithi096 ceph-mon[35004]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-09-19T06:43:48.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:48 smithi037 ceph-mon[31211]: pgmap v63: 97 pgs: 97 active+clean; 15 MiB data, 392 MiB used, 1.0 TiB / 1.0 TiB avail; 707 KiB/s rd, 1.3 MiB/s wr, 116 op/s 2024-09-19T06:43:49.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 6.83 MB/sec warmup 16 sec latency 10560.186 ms 2024-09-19T06:43:50.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 6.43 MB/sec warmup 17 sec latency 11560.335 ms 2024-09-19T06:43:50.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:50 smithi151 ceph-mon[35236]: pgmap v64: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-09-19T06:43:50.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:50 smithi096 ceph-mon[35004]: pgmap v64: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-09-19T06:43:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:50 smithi037 ceph-mon[31211]: pgmap v64: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 1.4 MiB/s wr, 177 op/s 2024-09-19T06:43:51.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 6.07 MB/sec warmup 18 sec latency 12560.436 ms 2024-09-19T06:43:52.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 5.75 MB/sec warmup 19 sec latency 13560.552 ms 2024-09-19T06:43:52.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:52 smithi151 ceph-mon[35236]: pgmap v65: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 644 KiB/s wr, 90 op/s 2024-09-19T06:43:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:52 smithi096 ceph-mon[35004]: pgmap v65: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 644 KiB/s wr, 90 op/s 2024-09-19T06:43:52.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:52 smithi037 ceph-mon[31211]: pgmap v65: 97 pgs: 97 active+clean; 11 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 271 KiB/s rd, 644 KiB/s wr, 90 op/s 2024-09-19T06:43:53.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 5.46 MB/sec warmup 20 sec latency 14560.669 ms 2024-09-19T06:43:54.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 5.20 MB/sec warmup 21 sec latency 15560.781 ms 2024-09-19T06:43:54.495 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:43:54.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:54 smithi151 ceph-mon[35236]: pgmap v66: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 645 KiB/s wr, 93 op/s 2024-09-19T06:43:54.751 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:54.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:54 smithi096 ceph-mon[35004]: pgmap v66: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 645 KiB/s wr, 93 op/s 2024-09-19T06:43:54.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:54 smithi037 ceph-mon[31211]: pgmap v66: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 645 KiB/s wr, 93 op/s 2024-09-19T06:43:55.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 4.97 MB/sec warmup 22 sec latency 16560.825 ms 2024-09-19T06:43:55.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 -- 172.21.15.37:0/1977517502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844106a80 msgr2=0x7f7844106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 --2- 172.21.15.37:0/1977517502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844106a80 0x7f7844106e60 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7834009930 tx=0x7f783402a420 comp rx=0 tx=0).stop 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 -- 172.21.15.37:0/1977517502 shutdown_connections 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 --2- 172.21.15.37:0/1977517502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7844101420 0x7f784410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 --2- 172.21.15.37:0/1977517502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7844100a80 0x7f7844100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 --2- 172.21.15.37:0/1977517502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844106a80 0x7f7844106e60 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 -- 172.21.15.37:0/1977517502 >> 172.21.15.37:0/1977517502 conn(0x7f78440fc7a0 msgr2=0x7f78440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 -- 172.21.15.37:0/1977517502 shutdown_connections 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.193+0000 7f784a10a640 1 -- 172.21.15.37:0/1977517502 wait complete. 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.194+0000 7f784a10a640 1 Processor -- start 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.194+0000 7f784a10a640 1 -- start start 2024-09-19T06:43:55.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.194+0000 7f784a10a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844100a80 0x7f7844198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.194+0000 7f784a10a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7844101420 0x7f78441989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f784a10a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7844106a80 0x7f784419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f784a10a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110e80 con 0x7f7844100a80 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f784a10a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7844110d00 con 0x7f7844101420 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f784a10a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7844111000 con 0x7f7844106a80 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844100a80 0x7f7844198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844100a80 0x7f7844198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58518/0 (socket says 172.21.15.37:58518) 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 -- 172.21.15.37:0/4200027072 learned_addr learned my addr 172.21.15.37:0/4200027072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f7842ffd640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7844101420 0x7f78441989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f7843fff640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7844106a80 0x7f784419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 -- 172.21.15.37:0/4200027072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7844106a80 msgr2=0x7f784419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7844106a80 0x7f784419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 -- 172.21.15.37:0/4200027072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7844101420 msgr2=0x7f78441989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7844101420 0x7f78441989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 -- 172.21.15.37:0/4200027072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784419d420 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f7842ffd640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7844101420 0x7f78441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f78437fe640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844100a80 0x7f7844198470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7834009a50 tx=0x7f7834004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f7840ff9640 1 -- 172.21.15.37:0/4200027072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f783403f030 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784419d6b0 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f7840ff9640 1 -- 172.21.15.37:0/4200027072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7834003ce0 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.195+0000 7f7840ff9640 1 -- 172.21.15.37:0/4200027072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7834035070 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.196+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78441a4ff0 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.196+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78441021c0 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.198+0000 7f7840ff9640 1 -- 172.21.15.37:0/4200027072 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7834042030 con 0x7f7844100a80 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.199+0000 7f7840ff9640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f782c0a33b0 0x7f782c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:55.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.199+0000 7f7842ffd640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f782c0a33b0 0x7f782c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.199+0000 7f7840ff9640 1 -- 172.21.15.37:0/4200027072 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f78340f0390 con 0x7f7844100a80 2024-09-19T06:43:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.199+0000 7f7842ffd640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f782c0a33b0 0x7f782c0a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f78440fecd0 tx=0x7f7838005ec0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:55.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.203+0000 7f7840ff9640 1 -- 172.21.15.37:0/4200027072 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f78340bcd70 con 0x7f7844100a80 2024-09-19T06:43:55.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.414+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78441a51c0 con 0x7f7844100a80 2024-09-19T06:43:55.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.415+0000 7f7840ff9640 1 -- 172.21.15.37:0/4200027072 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11906 (secure 0 0 0) 0x7f78340bc670 con 0x7f7844100a80 2024-09-19T06:43:55.416 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:43:55.416 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-19T06:43:42:276735+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:43:41.612230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:43:55.416 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f782c0a33b0 msgr2=0x7f782c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f782c0a33b0 0x7f782c0a5870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f78440fecd0 tx=0x7f7838005ec0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844100a80 msgr2=0x7f7844198470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844100a80 0x7f7844198470 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f7834009a50 tx=0x7f7834004270 comp rx=0 tx=0).stop 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 shutdown_connections 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7844106a80 0x7f784419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7844101420 0x7f78441989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f782c0a33b0 0x7f782c0a5870 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 --2- 172.21.15.37:0/4200027072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7844100a80 0x7f7844198470 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:55.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 >> 172.21.15.37:0/4200027072 conn(0x7f78440fc7a0 msgr2=0x7f784406a8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 shutdown_connections 2024-09-19T06:43:55.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:55.417+0000 7f784a10a640 1 -- 172.21.15.37:0/4200027072 wait complete. 2024-09-19T06:43:56.107 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 46e6b7a3 --release --await' 2024-09-19T06:43:56.107 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 46e6b7a3 --release --await 2024-09-19T06:43:56.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17095 4.75 MB/sec warmup 23 sec latency 17560.879 ms 2024-09-19T06:43:56.366 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:43:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.580+0000 7fdd7128d640 1 -- 172.21.15.37:0/1544953722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c101350 msgr2=0x7fdd6c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.580+0000 7fdd7128d640 1 --2- 172.21.15.37:0/1544953722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c101350 0x7fdd6c10d880 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009cd0 tx=0x7fdd6402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.580+0000 7fdd7128d640 1 -- 172.21.15.37:0/1544953722 shutdown_connections 2024-09-19T06:43:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.580+0000 7fdd7128d640 1 --2- 172.21.15.37:0/1544953722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c101350 0x7fdd6c10d880 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.580+0000 7fdd7128d640 1 --2- 172.21.15.37:0/1544953722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd6c1009b0 0x7fdd6c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.580+0000 7fdd7128d640 1 --2- 172.21.15.37:0/1544953722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd6c1069c0 0x7fdd6c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.580+0000 7fdd7128d640 1 -- 172.21.15.37:0/1544953722 >> 172.21.15.37:0/1544953722 conn(0x7fdd6c077990 msgr2=0x7fdd6c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.581+0000 7fdd7128d640 1 -- 172.21.15.37:0/1544953722 shutdown_connections 2024-09-19T06:43:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.581+0000 7fdd7128d640 1 -- 172.21.15.37:0/1544953722 wait complete. 2024-09-19T06:43:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.581+0000 7fdd7128d640 1 Processor -- start 2024-09-19T06:43:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd7128d640 1 -- start start 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd7128d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c1009b0 0x7fdd6c198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd7128d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd6c101350 0x7fdd6c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd7128d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd6c1069c0 0x7fdd6c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd7128d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c110e00 con 0x7fdd6c1009b0 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd7128d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c110c80 con 0x7fdd6c101350 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd7128d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c110f80 con 0x7fdd6c1069c0 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c1009b0 0x7fdd6c198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c1009b0 0x7fdd6c198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58528/0 (socket says 172.21.15.37:58528) 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 -- 172.21.15.37:0/3141163143 learned_addr learned my addr 172.21.15.37:0/3141163143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd627fc640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd6c101350 0x7fdd6c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd6c1069c0 0x7fdd6c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 -- 172.21.15.37:0/3141163143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd6c1069c0 msgr2=0x7fdd6c19cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd6c1069c0 0x7fdd6c19cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 -- 172.21.15.37:0/3141163143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd6c101350 msgr2=0x7fdd6c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd6c101350 0x7fdd6c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6affd640 1 -- 172.21.15.37:0/3141163143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c19d330 con 0x7fdd6c1009b0 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd627fc640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd6c101350 0x7fdd6c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.582+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd6c1069c0 0x7fdd6c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.583+0000 7fdd6affd640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c1009b0 0x7fdd6c198340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00d770 tx=0x7fdd5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.583+0000 7fdd68ff9640 1 -- 172.21.15.37:0/3141163143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c017070 con 0x7fdd6c1009b0 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.583+0000 7fdd68ff9640 1 -- 172.21.15.37:0/3141163143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd5c005500 con 0x7fdd6c1009b0 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.583+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c19d620 con 0x7fdd6c1009b0 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.583+0000 7fdd68ff9640 1 -- 172.21.15.37:0/3141163143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c004e00 con 0x7fdd6c1009b0 2024-09-19T06:43:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.583+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c1a4eb0 con 0x7fdd6c1009b0 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.585+0000 7fdd68ff9640 1 -- 172.21.15.37:0/3141163143 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fdd5c004fa0 con 0x7fdd6c1009b0 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.585+0000 7fdd74027740 1 Processor -- start 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.585+0000 7fdd74027740 1 -- start start 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd74027740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa940af0 0x55a2fa940ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd68ff9640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdd580a3460 0x7fdd580a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd74027740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa94b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd74027740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa94e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd74027740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2fa8a52f0 con 0x55a2fa94bcb0 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd74027740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2fa84d270 con 0x55a2fa941410 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd74027740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a2fa8a12b0 con 0x55a2fa940af0 2024-09-19T06:43:56.594 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd6affd640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdd580a3460 0x7fdd580a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa94e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa94e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58530/0 (socket says 172.21.15.37:58530) 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/845323245 learned_addr learned my addr 172.21.15.37:0/845323245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd68ff9640 1 -- 172.21.15.37:0/3141163143 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fdd5c0cf800 con 0x7fdd6c1009b0 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa940af0 0x55a2fa940ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.586+0000 7fdd627fc640 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa94b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/845323245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa940af0 msgr2=0x55a2fa940ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa940af0 0x55a2fa940ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/845323245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 msgr2=0x55a2fa94b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa94b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/845323245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2fa84c340 con 0x55a2fa94bcb0 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd6affd640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdd580a3460 0x7fdd580a5920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c098450 tx=0x7fdd5c086040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.595 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa94e0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009430 tx=0x7fdd6402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823008dbb66d3308 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd617fa640 1 -- 172.21.15.37:0/845323245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64031ac0 con 0x55a2fa94bcb0 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd617fa640 1 -- 172.21.15.37:0/845323245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd64036040 con 0x55a2fa94bcb0 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.587+0000 7fdd617fa640 1 -- 172.21.15.37:0/845323245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd640046a0 con 0x55a2fa94bcb0 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- 172.21.15.37:0/845323245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 msgr2=0x55a2fa94e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa94e0a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdd64009430 tx=0x7fdd6402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- 172.21.15.37:0/845323245 shutdown_connections 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa94e0a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa94b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 --2- 172.21.15.37:0/845323245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa940af0 0x55a2fa940ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- 172.21.15.37:0/845323245 >> 172.21.15.37:0/845323245 conn(0x55a2fa93d060 msgr2=0x55a2fa93d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- 172.21.15.37:0/845323245 shutdown_connections 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- 172.21.15.37:0/845323245 wait complete. 2024-09-19T06:43:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- start start 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 Processor -- start 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa948bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa941410 0x55a2fa9490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa949630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a2fa94e880 con 0x55a2fa941410 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a2fa82e290 con 0x55a2fa940af0 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.588+0000 7fdd74027740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a2fa94ea00 con 0x55a2fa94bcb0 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd74027740 10 client.0 _ll_register_callbacks cb 0x55a2fa85f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa941410 0x55a2fa9490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa941410 0x55a2fa9490f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58544/0 (socket says 172.21.15.37:58544) 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 -- 172.21.15.37:0/304020748 learned_addr learned my addr 172.21.15.37:0/304020748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa948bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa949630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 -- 172.21.15.37:0/304020748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 msgr2=0x55a2fa949630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa949630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 -- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 msgr2=0x55a2fa948bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa948bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 -- 172.21.15.37:0/304020748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2fa5b8210 con 0x55a2fa941410 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa949630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd627fc640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa941410 0x55a2fa9490f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdd540136e0 tx=0x7fdd54013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b426b37fe4d994 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd527fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd527fc640 1 -- 172.21.15.37:0/304020748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54011dd0 con 0x55a2fa941410 2024-09-19T06:43:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd527fc640 1 -- 172.21.15.37:0/304020748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fdd54004020 con 0x55a2fa941410 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.589+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd64036040 con 0x55a2fa941410 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd527fc640 1 -- 172.21.15.37:0/304020748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd54004220 con 0x55a2fa941410 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 10 client.16059 fetch_fsmap learned FSMap version 27 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2fa684620 con 0x55a2fa941410 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd527fc640 1 -- 172.21.15.37:0/304020748 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd64036040 con 0x55a2fa941410 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd527fc640 1 -- 172.21.15.37:0/304020748 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd5401a070 con 0x55a2fa941410 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 10 client.16059 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 20 client.16059 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa95bd20 0x55a2fa95e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 4 client.16059 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a2fa8b2a20 con 0x55a2fa95bd20 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.590+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa95bd20 0x55a2fa95e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.591+0000 7fdd527fc640 10 client.16059 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.591+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa95bd20 0x55a2fa95e170 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d04ee8089aefcb server_cookie=5b2e6566994e68e7 in_seq=0 out_seq=0 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.591+0000 7fdd527fc640 1 -- 172.21.15.37:0/304020748 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a2fa8b2a20 con 0x55a2fa95bd20 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.591+0000 7fdd527fc640 10 client.16059 handle_command_reply: tid=0 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 client.16059 shutdown 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa941410 msgr2=0x55a2fa9490f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa941410 0x55a2fa9490f0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdd540136e0 tx=0x7fdd54013ba0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 shutdown_connections 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa95bd20 0x55a2fa95e170 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa949630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa941410 0x55a2fa9490f0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd527fc640 0 client.16059 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 --2- 172.21.15.37:0/304020748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa948bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.600 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 >> 172.21.15.37:0/304020748 conn(0x55a2fa93d060 msgr2=0x55a2fa942d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 shutdown_connections 2024-09-19T06:43:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 1 -- 172.21.15.37:0/304020748 wait complete. 2024-09-19T06:43:56.601 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.592+0000 7fdd74027740 20 client.16059 trim_cache size 0 max 16384 2024-09-19T06:43:56.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:56 smithi151 ceph-mon[35236]: pgmap v67: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 24 KiB/s wr, 63 op/s 2024-09-19T06:43:56.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:56 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4200027072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.611+0000 7fdd74027740 1 Processor -- start 2024-09-19T06:43:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.611+0000 7fdd74027740 1 -- start start 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd74027740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa949460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd74027740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa941410 0x55a2fa963ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd74027740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa964010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd74027740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1b01b0 con 0x55a2fa94bcb0 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd74027740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1ada80 con 0x55a2fa940af0 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd74027740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1ab8d0 con 0x55a2fa941410 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa964010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa964010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58558/0 (socket says 172.21.15.37:58558) 2024-09-19T06:43:56.618 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3630239038 learned_addr learned my addr 172.21.15.37:0/3630239038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd627fc640 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa941410 0x55a2fa963ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa949460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3630239038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa941410 msgr2=0x55a2fa963ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa941410 0x55a2fa963ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3630239038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 msgr2=0x55a2fa949460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa949460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3630239038 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a2fa947b60 con 0x55a2fa94bcb0 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.612+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa964010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdd640042f0 tx=0x7fdd64041040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c28a07e4e164f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd60ff9640 1 -- 172.21.15.37:0/3630239038 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd640318c0 con 0x55a2fa94bcb0 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd60ff9640 1 -- 172.21.15.37:0/3630239038 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd64036070 con 0x55a2fa94bcb0 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd60ff9640 1 -- 172.21.15.37:0/3630239038 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd640047b0 con 0x55a2fa94bcb0 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 -- 172.21.15.37:0/3630239038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 msgr2=0x55a2fa964010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa964010 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdd640042f0 tx=0x7fdd64041040 comp rx=0 tx=0).stop 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 -- 172.21.15.37:0/3630239038 shutdown_connections 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa94bcb0 0x55a2fa964010 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa941410 0x55a2fa963ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 --2- 172.21.15.37:0/3630239038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa940af0 0x55a2fa949460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 -- 172.21.15.37:0/3630239038 >> 172.21.15.37:0/3630239038 conn(0x55a2fa93d060 msgr2=0x55a2fa94ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 -- 172.21.15.37:0/3630239038 shutdown_connections 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.613+0000 7fdd74027740 1 -- 172.21.15.37:0/3630239038 wait complete. 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 -- start start 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 Processor -- start 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa940af0 0x55a2fa953cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa954200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa950c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1abf40 con 0x55a2fa940af0 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1ad3b0 con 0x55a2fa941410 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1ac610 con 0x55a2fa94bcb0 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd74027740 10 client.0 _ll_register_callbacks cb 0x7fdd54005870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa940af0 0x55a2fa953cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd6b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa940af0 0x55a2fa953cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58574/0 (socket says 172.21.15.37:58574) 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3048021336 learned_addr learned my addr 172.21.15.37:0/3048021336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd627fc640 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa954200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.614+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa950c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3048021336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 msgr2=0x55a2fa950c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa950c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 msgr2=0x55a2fa954200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa954200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd6b7fe640 1 -- 172.21.15.37:0/3048021336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a2fa946b30 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd6b7fe640 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa940af0 0x55a2fa953cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006010 tx=0x7fdd64005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978ea8464f5b1ee2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd527fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd527fc640 1 -- 172.21.15.37:0/3048021336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64004730 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd527fc640 1 -- 172.21.15.37:0/3048021336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fdd640318c0 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd64036070 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd527fc640 1 -- 172.21.15.37:0/3048021336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fdd64031ac0 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd74027740 10 client.16065 fetch_fsmap learned FSMap version 27 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55a2fa9c7000 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd527fc640 1 -- 172.21.15.37:0/3048021336 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd64036070 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.615+0000 7fdd527fc640 1 -- 172.21.15.37:0/3048021336 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fdd64030070 con 0x55a2fa940af0 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd74027740 10 client.16065 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd74027740 10 client.16065 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:43:56.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd74027740 20 client.16065 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd74027740 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa9c7290 0x55a2fa9c9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd74027740 4 client.16065 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:43:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a2fa8acd70 con 0x55a2fa9c7290 2024-09-19T06:43:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd627fc640 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa9c7290 0x55a2fa9c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd527fc640 10 client.16065 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.616+0000 7fdd627fc640 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa9c7290 0x55a2fa9c9680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8012691af11d4012 server_cookie=35ba9eac8724ddc6 in_seq=0 out_seq=0 2024-09-19T06:43:56.625 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:56 smithi037 ceph-mon[31211]: pgmap v67: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 24 KiB/s wr, 63 op/s 2024-09-19T06:43:56.625 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:56 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4200027072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.691+0000 7fdd527fc640 1 -- 172.21.15.37:0/3048021336 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a2fa8acd70 con 0x55a2fa9c7290 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.691+0000 7fdd527fc640 10 client.16065 handle_command_reply: tid=0 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.691+0000 7fdd74027740 1 client.16065 shutdown 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa940af0 msgr2=0x55a2fa953cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa940af0 0x55a2fa953cc0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdd64006010 tx=0x7fdd64005ef0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 shutdown_connections 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a2fa9c7290 0x55a2fa9c9680 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a2fa94bcb0 0x55a2fa950c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd527fc640 0 client.16065 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a2fa941410 0x55a2fa954200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 --2- 172.21.15.37:0/3048021336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a2fa940af0 0x55a2fa953cc0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 >> 172.21.15.37:0/3048021336 conn(0x55a2fa93d060 msgr2=0x55a2fa94a410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 shutdown_connections 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 1 -- 172.21.15.37:0/3048021336 wait complete. 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd74027740 20 client.16065 trim_cache size 0 max 16384 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:43:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:43:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:43:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdd580a3460 msgr2=0x7fdd580a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdd580a3460 0x7fdd580a5920 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c098450 tx=0x7fdd5c086040 comp rx=0 tx=0).stop 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c1009b0 msgr2=0x7fdd6c198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c1009b0 0x7fdd6c198340 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00d770 tx=0x7fdd5c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 shutdown_connections 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdd6c1069c0 0x7fdd6c19cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd6c101350 0x7fdd6c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fdd580a3460 0x7fdd580a5920 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 --2- 172.21.15.37:0/3141163143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fdd6c1009b0 0x7fdd6c198340 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.692+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 >> 172.21.15.37:0/3141163143 conn(0x7fdd6c077990 msgr2=0x7fdd6c069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.693+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 shutdown_connections 2024-09-19T06:43:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:43:56.693+0000 7fdd7128d640 1 -- 172.21.15.37:0/3141163143 wait complete. 2024-09-19T06:43:56.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:56 smithi096 ceph-mon[35004]: pgmap v67: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 24 KiB/s wr, 63 op/s 2024-09-19T06:43:56.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:56 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4200027072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:43:56.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 -- 172.21.15.37:0/2770095975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b0102d30 msgr2=0x7f09b0103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2770095975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b0102d30 0x7f09b0103110 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f09a0009930 tx=0x7f09a002a440 comp rx=0 tx=0).stop 2024-09-19T06:43:56.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 -- 172.21.15.37:0/2770095975 shutdown_connections 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2770095975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09b01082a0 0x7f09b010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2770095975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b01036e0 0x7f09b0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2770095975 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b0102d30 0x7f09b0103110 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 -- 172.21.15.37:0/2770095975 >> 172.21.15.37:0/2770095975 conn(0x7f09b00fc7d0 msgr2=0x7f09b00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 -- 172.21.15.37:0/2770095975 shutdown_connections 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 -- 172.21.15.37:0/2770095975 wait complete. 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.833+0000 7f09b48b0640 1 Processor -- start 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09b48b0640 1 -- start start 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09b48b0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09b0102d30 0x7f09b0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09b48b0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b01036e0 0x7f09b0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09b48b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b01082a0 0x7f09b0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09b48b0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01120e0 con 0x7f09b01082a0 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09b48b0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09b0111f60 con 0x7f09b0102d30 2024-09-19T06:43:56.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09b48b0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09b0112260 con 0x7f09b01036e0 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b01082a0 0x7f09b0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b01082a0 0x7f09b0101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58594/0 (socket says 172.21.15.37:58594) 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 -- 172.21.15.37:0/2555138471 learned_addr learned my addr 172.21.15.37:0/2555138471 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09a7fff640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b01036e0 0x7f09b0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09ae59c640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09b0102d30 0x7f09b0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 -- 172.21.15.37:0/2555138471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b01036e0 msgr2=0x7f09b0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b01036e0 0x7f09b0100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 -- 172.21.15.37:0/2555138471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09b0102d30 msgr2=0x7f09b0100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09b0102d30 0x7f09b0100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 -- 172.21.15.37:0/2555138471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09b01a6930 con 0x7f09b01082a0 2024-09-19T06:43:56.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09ae59c640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09b0102d30 0x7f09b0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:56.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.834+0000 7f09aed9d640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b01082a0 0x7f09b0101410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cce0 tx=0x7f09a800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.835+0000 7f09a77fe640 1 -- 172.21.15.37:0/2555138471 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a800f450 con 0x7f09b01082a0 2024-09-19T06:43:56.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.835+0000 7f09a77fe640 1 -- 172.21.15.37:0/2555138471 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a800fad0 con 0x7f09b01082a0 2024-09-19T06:43:56.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.835+0000 7f09a77fe640 1 -- 172.21.15.37:0/2555138471 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a8018b60 con 0x7f09b01082a0 2024-09-19T06:43:56.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.835+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09b01a6c20 con 0x7f09b01082a0 2024-09-19T06:43:56.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.835+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09b01a7160 con 0x7f09b01082a0 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.838+0000 7f09b6c89740 1 Processor -- start 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.838+0000 7f09a77fe640 1 -- 172.21.15.37:0/2555138471 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f09a800fc80 con 0x7f09b01082a0 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.838+0000 7f09b6c89740 1 -- start start 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.838+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca980 0x56190ebd4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebd5240 0x56190ebd7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09b6c89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56190eb2f620 con 0x56190ebca980 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09b6c89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56190ead6e50 con 0x56190ebd5240 2024-09-19T06:43:56.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09b6c89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56190eb2b5e0 con 0x56190ebca060 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca980 0x56190ebd4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca980 0x56190ebd4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58600/0 (socket says 172.21.15.37:58600) 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 -- 172.21.15.37:0/654289746 learned_addr learned my addr 172.21.15.37:0/654289746 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09a7fff640 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebd5240 0x56190ebd7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09a7fff640 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 -- 172.21.15.37:0/654289746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 msgr2=0x56190ebca440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebca440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 -- 172.21.15.37:0/654289746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebd5240 msgr2=0x56190ebd7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebd5240 0x56190ebd7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 -- 172.21.15.37:0/654289746 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190eac7d40 con 0x56190ebca980 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09a77fe640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f099c0a3430 0x7f099c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09a7fff640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f099c0a3430 0x7f099c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09ae59c640 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca980 0x56190ebd4d00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f09a002aa30 tx=0x7f09a0004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172cf88e36311e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.839+0000 7f09a77fe640 1 -- 172.21.15.37:0/2555138471 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f09a80d0150 con 0x7f09b01082a0 2024-09-19T06:43:56.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.840+0000 7f09a4ff9640 1 -- 172.21.15.37:0/654289746 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a003f070 con 0x56190ebca980 2024-09-19T06:43:56.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.840+0000 7f09a4ff9640 1 -- 172.21.15.37:0/654289746 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a0003ad0 con 0x56190ebca980 2024-09-19T06:43:56.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.840+0000 7f09a4ff9640 1 -- 172.21.15.37:0/654289746 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0035070 con 0x56190ebca980 2024-09-19T06:43:56.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 -- 172.21.15.37:0/654289746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca980 msgr2=0x56190ebd4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca980 0x56190ebd4d00 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f09a002aa30 tx=0x7f09a0004180 comp rx=0 tx=0).stop 2024-09-19T06:43:56.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09a7fff640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f099c0a3430 0x7f099c0a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56190ebc7970 tx=0x7f0998013040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 -- 172.21.15.37:0/654289746 shutdown_connections 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebd5240 0x56190ebd7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca980 0x56190ebd4d00 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 --2- 172.21.15.37:0/654289746 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebca440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 -- 172.21.15.37:0/654289746 >> 172.21.15.37:0/654289746 conn(0x56190ebc65f0 msgr2=0x56190ebc69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 -- 172.21.15.37:0/654289746 shutdown_connections 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 -- 172.21.15.37:0/654289746 wait complete. 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 -- start start 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.841+0000 7f09b6c89740 1 Processor -- start 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebd2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebd2b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09b6c89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56190ebd7e10 con 0x56190ebd5240 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09b6c89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56190eab7f10 con 0x56190ebca980 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09b6c89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56190ebd7f90 con 0x56190ebca060 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09b6c89740 10 client.0 _ll_register_callbacks cb 0x56190eadbdb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebd2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebd2b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58614/0 (socket says 172.21.15.37:58614) 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 -- 172.21.15.37:0/3820766897 learned_addr learned my addr 172.21.15.37:0/3820766897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09a7fff640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09ae59c640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebd2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 -- 172.21.15.37:0/3820766897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 msgr2=0x56190ebd2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebd2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 -- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 msgr2=0x56190ebd25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 -- 172.21.15.37:0/3820766897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190e83e580 con 0x56190ebd5240 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09a7fff640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f09aed9d640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebd2b10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f09a8098890 tx=0x7f09a809be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c488c907ce6c4cc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f0995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f0995ffb640 1 -- 172.21.15.37:0/3820766897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80a07e0 con 0x56190ebd5240 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.842+0000 7f0995ffb640 1 -- 172.21.15.37:0/3820766897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f09a80b5220 con 0x56190ebd5240 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a0003ad0 con 0x56190ebd5240 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f0995ffb640 1 -- 172.21.15.37:0/3820766897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09a80b5420 con 0x56190ebd5240 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 10 client.16077 fetch_fsmap learned FSMap version 27 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56190e908960 con 0x56190ebd5240 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f0995ffb640 1 -- 172.21.15.37:0/3820766897 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a807f020 con 0x56190ebd5240 2024-09-19T06:43:56.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f0995ffb640 1 -- 172.21.15.37:0/3820766897 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f09a8020030 con 0x56190ebd5240 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 10 client.16077 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 10 client.16077 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 20 client.16077 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ebe4de0 0x56190ebe7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 4 client.16077 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56190eb3cb90 con 0x56190ebe4de0 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.843+0000 7f09ae59c640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ebe4de0 0x56190ebe7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.844+0000 7f09ae59c640 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ebe4de0 0x56190ebe7260 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463c5618ad80bb39 server_cookie=946ca36bff09e502 in_seq=0 out_seq=0 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.844+0000 7f0995ffb640 10 client.16077 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.844+0000 7f0995ffb640 1 -- 172.21.15.37:0/3820766897 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56190eb3cb90 con 0x56190ebe4de0 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.844+0000 7f0995ffb640 10 client.16077 handle_command_reply: tid=0 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.844+0000 7f09b6c89740 1 client.16077 shutdown 2024-09-19T06:43:56.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.844+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 msgr2=0x56190ebd2b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebd2b10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f09a8098890 tx=0x7f09a809be50 comp rx=0 tx=0).stop 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 shutdown_connections 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ebe4de0 0x56190ebe7260 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebd2b10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 --2- 172.21.15.37:0/3820766897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebd2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f0995ffb640 0 client.16077 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 >> 172.21.15.37:0/3820766897 conn(0x56190ebc65f0 msgr2=0x56190ebcc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 shutdown_connections 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 1 -- 172.21.15.37:0/3820766897 wait complete. 2024-09-19T06:43:56.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.845+0000 7f09b6c89740 20 client.16077 trim_cache size 0 max 16384 2024-09-19T06:43:56.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 Processor -- start 2024-09-19T06:43:56.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 -- start start 2024-09-19T06:43:56.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebccac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebc5640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebecb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01b2200 con 0x56190ebd5240 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01afad0 con 0x56190ebca980 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09b6c89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01ad920 con 0x56190ebca060 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebecb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebecb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58630/0 (socket says 172.21.15.37:58630) 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 -- 172.21.15.37:0/2614305893 learned_addr learned my addr 172.21.15.37:0/2614305893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09a7fff640 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebc5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 -- 172.21.15.37:0/2614305893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 msgr2=0x56190ebccac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 -- 172.21.15.37:0/2614305893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 msgr2=0x56190ebc5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebc5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 -- 172.21.15.37:0/2614305893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56190ebe78e0 con 0x56190ebd5240 2024-09-19T06:43:56.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f09aed9d640 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebecb30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f09a80cd950 tx=0x7f09a809bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937dab84f11b0647 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.862+0000 7f0997fff640 1 -- 172.21.15.37:0/2614305893 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a80983c0 con 0x56190ebd5240 2024-09-19T06:43:56.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f0997fff640 1 -- 172.21.15.37:0/2614305893 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a807f070 con 0x56190ebd5240 2024-09-19T06:43:56.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f0997fff640 1 -- 172.21.15.37:0/2614305893 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a809b970 con 0x56190ebd5240 2024-09-19T06:43:56.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 -- 172.21.15.37:0/2614305893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 msgr2=0x56190ebecb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebecb30 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f09a80cd950 tx=0x7f09a809bec0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 -- 172.21.15.37:0/2614305893 shutdown_connections 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebd5240 0x56190ebecb30 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebc5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 --2- 172.21.15.37:0/2614305893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebca060 0x56190ebccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 -- 172.21.15.37:0/2614305893 >> 172.21.15.37:0/2614305893 conn(0x56190ebc65f0 msgr2=0x56190ebc4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 -- 172.21.15.37:0/2614305893 shutdown_connections 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 -- 172.21.15.37:0/2614305893 wait complete. 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.863+0000 7f09b6c89740 1 -- start start 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 1 Processor -- start 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca060 0x56190ebd9690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebd5240 0x56190ec4eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01adf90 con 0x56190ebca060 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01af400 con 0x56190ebca980 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09b01ae660 con 0x56190ebd5240 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca060 0x56190ebd9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09b6c89740 10 client.0 _ll_register_callbacks cb 0x7f09a809bf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca060 0x56190ebd9690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58642/0 (socket says 172.21.15.37:58642) 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 -- 172.21.15.37:0/77935818 learned_addr learned my addr 172.21.15.37:0/77935818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09aed9d640 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebd5240 0x56190ec4eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 -- 172.21.15.37:0/77935818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebd5240 msgr2=0x56190ec4eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09a7fff640 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebd5240 0x56190ec4eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 -- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 msgr2=0x56190ebd9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 -- 172.21.15.37:0/77935818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56190ec4f040 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f09ae59c640 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca060 0x56190ebd9690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x56190ebc7450 tx=0x7f09a0031640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b47141161d17140 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.864+0000 7f0995ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f0995ffb640 1 -- 172.21.15.37:0/77935818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09a0035040 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f0995ffb640 1 -- 172.21.15.37:0/77935818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f09a00037f0 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09a807f070 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f0995ffb640 1 -- 172.21.15.37:0/77935818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f09a003f040 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 10 client.16083 fetch_fsmap learned FSMap version 27 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56190ec50350 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f0995ffb640 1 -- 172.21.15.37:0/77935818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09a807f070 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f0995ffb640 1 -- 172.21.15.37:0/77935818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f09a0031b10 con 0x56190ebca060 2024-09-19T06:43:56.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 10 client.16083 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 10 client.16083 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 20 client.16083 populate_metadata read hostname 'smithi037' 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ec507f0 0x56190ec52be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 4 client.16083 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "46e6b7a3", "release": true, "await": true}] 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "46e6b7a3", "release": true, "await": true}) -- 0x56190eb370a0 con 0x56190ec507f0 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.865+0000 7f09a7fff640 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ec507f0 0x56190ec52be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.866+0000 7f09a7fff640 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ec507f0 0x56190ec52be0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f949d0fcd565153 server_cookie=22c069d601c7c0ce in_seq=0 out_seq=0 2024-09-19T06:43:56.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.866+0000 7f0995ffb640 10 client.16083 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f0995ffb640 1 -- 172.21.15.37:0/77935818 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x56190eb370a0 con 0x56190ec507f0 2024-09-19T06:43:56.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f0995ffb640 10 client.16083 handle_command_reply: tid=0 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 client.16083 shutdown 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca060 msgr2=0x56190ebd9690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca060 0x56190ebd9690 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x56190ebc7450 tx=0x7f09a0031640 comp rx=0 tx=0).stop 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 shutdown_connections 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56190ec507f0 0x56190ec52be0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56190ebd5240 0x56190ec4eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f0995ffb640 0 client.16083 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56190ebca980 0x56190ebd9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 --2- 172.21.15.37:0/77935818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56190ebca060 0x56190ebd9690 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.875+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 >> 172.21.15.37:0/77935818 conn(0x56190ebc65f0 msgr2=0x56190ebd2870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 shutdown_connections 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b6c89740 1 -- 172.21.15.37:0/77935818 wait complete. 2024-09-19T06:43:56.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b6c89740 20 client.16083 trim_cache size 0 max 16384 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 27, 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 12, 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "46e6b7a3": { 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 12, 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:56.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 140.3, 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:56.880 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:56.881 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:43:56.881 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:43:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f099c0a3430 msgr2=0x7f099c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f099c0a3430 0x7f099c0a58f0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x56190ebc7970 tx=0x7f0998013040 comp rx=0 tx=0).stop 2024-09-19T06:43:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b01082a0 msgr2=0x7f09b0101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:43:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b01082a0 0x7f09b0101410 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f09a800cce0 tx=0x7f09a800a620 comp rx=0 tx=0).stop 2024-09-19T06:43:56.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 shutdown_connections 2024-09-19T06:43:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f099c0a3430 0x7f099c0a58f0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09b01082a0 0x7f09b0101410 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09b01036e0 0x7f09b0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 --2- 172.21.15.37:0/2555138471 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09b0102d30 0x7f09b0100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:43:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 >> 172.21.15.37:0/2555138471 conn(0x7f09b00fc7d0 msgr2=0x7f09b00fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:43:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 shutdown_connections 2024-09-19T06:43:56.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:43:56.876+0000 7f09b48b0640 1 -- 172.21.15.37:0/2555138471 wait complete. 2024-09-19T06:43:57.190 INFO:tasks.workunit.client.0.smithi037.stdout: 1 17967 4.84 MB/sec warmup 24 sec latency 18249.050 ms 2024-09-19T06:43:57.681 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '46e6b7a3', total seconds elapsed: 23.5 2024-09-19T06:43:58.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 20660 5.11 MB/sec warmup 25 sec latency 13.276 ms 2024-09-19T06:43:58.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:58 smithi151 ceph-mon[35236]: pgmap v68: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 24 KiB/s wr, 63 op/s 2024-09-19T06:43:58.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:58 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:58 smithi096 ceph-mon[35004]: pgmap v68: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 24 KiB/s wr, 63 op/s 2024-09-19T06:43:58.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:58.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:58 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:58 smithi037 ceph-mon[31211]: pgmap v68: 97 pgs: 97 active+clean; 11 MiB data, 393 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 24 KiB/s wr, 63 op/s 2024-09-19T06:43:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:43:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:58 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:59.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 22933 5.42 MB/sec warmup 26 sec latency 11.777 ms 2024-09-19T06:43:59.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:59.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:59.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:43:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:43:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:43:59.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:43:59.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:43:59.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:43:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:44:00.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 25758 5.76 MB/sec warmup 27 sec latency 17.232 ms 2024-09-19T06:44:00.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:00 smithi151 ceph-mon[35236]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-09-19T06:44:00.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:00 smithi096 ceph-mon[35004]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-09-19T06:44:00.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:00 smithi037 ceph-mon[31211]: pgmap v69: 97 pgs: 97 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 319 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-09-19T06:44:01.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 28425 6.15 MB/sec warmup 28 sec latency 31.495 ms 2024-09-19T06:44:02.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 30859 6.25 MB/sec warmup 29 sec latency 16.642 ms 2024-09-19T06:44:02.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:02 smithi151 ceph-mon[35236]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 2.4 MiB/s wr, 64 op/s 2024-09-19T06:44:02.787 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:02 smithi037 ceph-mon[31211]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 2.4 MiB/s wr, 64 op/s 2024-09-19T06:44:02.788 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:02 smithi096 ceph-mon[35004]: pgmap v70: 97 pgs: 97 active+clean; 31 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 2.4 MiB/s wr, 64 op/s 2024-09-19T06:44:03.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 33374 6.60 MB/sec warmup 30 sec latency 13.904 ms 2024-09-19T06:44:04.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 36169 6.85 MB/sec warmup 31 sec latency 11.054 ms 2024-09-19T06:44:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:04 smithi151 ceph-mon[35236]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-09-19T06:44:04.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:04 smithi096 ceph-mon[35004]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-09-19T06:44:04.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:04 smithi037 ceph-mon[31211]: pgmap v71: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 220 op/s 2024-09-19T06:44:05.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 39314 7.22 MB/sec warmup 32 sec latency 19.510 ms 2024-09-19T06:44:06.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 42078 7.39 MB/sec warmup 33 sec latency 10.739 ms 2024-09-19T06:44:06.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:06 smithi151 ceph-mon[35236]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-19T06:44:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:06 smithi096 ceph-mon[35004]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-19T06:44:06.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:06 smithi037 ceph-mon[31211]: pgmap v72: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-19T06:44:07.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 44289 7.52 MB/sec warmup 34 sec latency 62.856 ms 2024-09-19T06:44:08.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 47015 7.72 MB/sec warmup 35 sec latency 12.728 ms 2024-09-19T06:44:08.575 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:08 smithi096 ceph-mon[35004]: pgmap v73: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-19T06:44:08.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:08 smithi151 ceph-mon[35236]: pgmap v73: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-19T06:44:08.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:08 smithi037 ceph-mon[31211]: pgmap v73: 97 pgs: 97 active+clean; 31 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 216 op/s 2024-09-19T06:44:09.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 50070 8.02 MB/sec warmup 36 sec latency 14.258 ms 2024-09-19T06:44:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 -- 172.21.15.37:0/438869885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c1080f0 msgr2=0x7fb45c10ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 --2- 172.21.15.37:0/438869885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c1080f0 0x7fb45c10ea10 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fb454009900 tx=0x7fb45402a410 comp rx=0 tx=0).stop 2024-09-19T06:44:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 -- 172.21.15.37:0/438869885 shutdown_connections 2024-09-19T06:44:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 --2- 172.21.15.37:0/438869885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c1080f0 0x7fb45c10ea10 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 --2- 172.21.15.37:0/438869885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45c101250 0x7fb45c107990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 --2- 172.21.15.37:0/438869885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb45c1008a0 0x7fb45c100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 -- 172.21.15.37:0/438869885 >> 172.21.15.37:0/438869885 conn(0x7fb45c075ef0 msgr2=0x7fb45c076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 -- 172.21.15.37:0/438869885 shutdown_connections 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.784+0000 7fb463647640 1 -- 172.21.15.37:0/438869885 wait complete. 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 Processor -- start 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 -- start start 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45c1008a0 0x7fb45c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb45c1080f0 0x7fb45c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c19ed90 0x7fb45c19f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c111f20 con 0x7fb45c19ed90 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c111da0 con 0x7fb45c1008a0 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb463647640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1120a0 con 0x7fb45c1080f0 2024-09-19T06:44:09.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb461be4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c19ed90 0x7fb45c19f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.785+0000 7fb4613e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45c1008a0 0x7fb45c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c19ed90 0x7fb45c19f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49300/0 (socket says 172.21.15.37:49300) 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 -- 172.21.15.37:0/3415156590 learned_addr learned my addr 172.21.15.37:0/3415156590 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb460be2640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb45c1080f0 0x7fb45c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 -- 172.21.15.37:0/3415156590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb45c1080f0 msgr2=0x7fb45c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb45c1080f0 0x7fb45c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 -- 172.21.15.37:0/3415156590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45c1008a0 msgr2=0x7fb45c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45c1008a0 0x7fb45c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 -- 172.21.15.37:0/3415156590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb45c19f880 con 0x7fb45c19ed90 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb461be4640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c19ed90 0x7fb45c19f240 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb454005ab0 tx=0x7fb454004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb45a7fc640 1 -- 172.21.15.37:0/3415156590 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb45403f030 con 0x7fb45c19ed90 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb45a7fc640 1 -- 172.21.15.37:0/3415156590 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb454003ad0 con 0x7fb45c19ed90 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb45c19fb10 con 0x7fb45c19ed90 2024-09-19T06:44:09.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb45a7fc640 1 -- 172.21.15.37:0/3415156590 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb454035070 con 0x7fb45c19ed90 2024-09-19T06:44:09.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.786+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb45c1a7330 con 0x7fb45c19ed90 2024-09-19T06:44:09.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 Processor -- start 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb45a7fc640 1 -- 172.21.15.37:0/3415156590 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb454042030 con 0x7fb45c19ed90 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 -- start start 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd41770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56090dc9b2f0 con 0x56090dd37410 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56090dc43270 con 0x56090dd36af0 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4663db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56090dc972b0 con 0x56090dd41cb0 2024-09-19T06:44:09.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.789+0000 7fb4613e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd41770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb460be2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd41770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49312/0 (socket says 172.21.15.37:49312) 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 -- 172.21.15.37:0/1061505272 learned_addr learned my addr 172.21.15.37:0/1061505272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb460be2640 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 -- 172.21.15.37:0/1061505272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 msgr2=0x56090dd440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 -- 172.21.15.37:0/1061505272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 msgr2=0x56090dd36ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd36ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 -- 172.21.15.37:0/1061505272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090dc42340 con 0x56090dd37410 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb460be2640 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd36ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:09.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb45a7fc640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb4440a34e0 0x7fb4440a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4613e3640 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd41770 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb44c003a20 tx=0x7fb44c003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5478156fddeb5e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb460be2640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb4440a34e0 0x7fb4440a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb45a7fc640 1 -- 172.21.15.37:0/3415156590 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb4540f0340 con 0x7fb45c19ed90 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4437fe640 1 -- 172.21.15.37:0/1061505272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c011440 con 0x56090dd37410 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4437fe640 1 -- 172.21.15.37:0/1061505272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44c015030 con 0x56090dd37410 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.790+0000 7fb4437fe640 1 -- 172.21.15.37:0/1061505272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c0109e0 con 0x56090dd37410 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 -- 172.21.15.37:0/1061505272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 msgr2=0x56090dd41770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb460be2640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb4440a34e0 0x7fb4440a59a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x56090dd343e0 tx=0x7fb4500098d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd41770 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fb44c003a20 tx=0x7fb44c003e20 comp rx=0 tx=0).stop 2024-09-19T06:44:09.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 -- 172.21.15.37:0/1061505272 shutdown_connections 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd41770 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 --2- 172.21.15.37:0/1061505272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd36ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 -- 172.21.15.37:0/1061505272 >> 172.21.15.37:0/1061505272 conn(0x56090dd33060 msgr2=0x56090dd33440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 -- 172.21.15.37:0/1061505272 shutdown_connections 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 -- 172.21.15.37:0/1061505272 wait complete. 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.791+0000 7fb4663db740 1 -- start start 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 1 Processor -- start 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd37410 0x56090dd3f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd41cb0 0x56090dd3f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56090dd44880 con 0x56090dd36af0 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56090dc24290 con 0x56090dd41cb0 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56090dd44a00 con 0x56090dd37410 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4663db740 10 client.0 _ll_register_callbacks cb 0x56090dc55140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49324/0 (socket says 172.21.15.37:49324) 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 -- 172.21.15.37:0/2707630027 learned_addr learned my addr 172.21.15.37:0/2707630027 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb461be4640 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd41cb0 0x56090dd3f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb460be2640 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd37410 0x56090dd3f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 -- 172.21.15.37:0/2707630027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd37410 msgr2=0x56090dd3f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd37410 0x56090dd3f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 -- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd41cb0 msgr2=0x56090dd3f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd41cb0 0x56090dd3f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.792+0000 7fb4613e3640 1 -- 172.21.15.37:0/2707630027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090d9ae210 con 0x56090dd36af0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4613e3640 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3eaf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb44c004410 tx=0x7fb44c009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe59c26b7477aa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb440ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb440ff9640 1 -- 172.21.15.37:0/2707630027 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c029060 con 0x56090dd36af0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb440ff9640 1 -- 172.21.15.37:0/2707630027 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb44c010b30 con 0x56090dd36af0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb44c015030 con 0x56090dd36af0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb440ff9640 1 -- 172.21.15.37:0/2707630027 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb44c010d50 con 0x56090dd36af0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 10 client.16095 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56090da7a620 con 0x56090dd36af0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb440ff9640 1 -- 172.21.15.37:0/2707630027 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44c015030 con 0x56090dd36af0 2024-09-19T06:44:09.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb440ff9640 1 -- 172.21.15.37:0/2707630027 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb44c023070 con 0x56090dd36af0 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 10 client.16095 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 20 client.16095 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090dd53f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 4 client.16095 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.793+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56090dca8a20 con 0x56090dd51b30 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.794+0000 7fb460be2640 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090dd53f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.794+0000 7fb440ff9640 10 client.16095 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.794+0000 7fb460be2640 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090dd53f20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db079fd237005b82 server_cookie=620927e0060ed38e in_seq=0 out_seq=0 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb440ff9640 1 -- 172.21.15.37:0/2707630027 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56090dca8a20 con 0x56090dd51b30 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb440ff9640 10 client.16095 handle_command_reply: tid=0 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 client.16095 shutdown 2024-09-19T06:44:09.804 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 msgr2=0x56090dd3eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3eaf0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb44c004410 tx=0x7fb44c009d60 comp rx=0 tx=0).stop 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 shutdown_connections 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090dd53f20 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd41cb0 0x56090dd3f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb440ff9640 0 client.16095 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd37410 0x56090dd3f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 --2- 172.21.15.37:0/2707630027 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3eaf0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 >> 172.21.15.37:0/2707630027 conn(0x56090dd33060 msgr2=0x56090dd38dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 shutdown_connections 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 1 -- 172.21.15.37:0/2707630027 wait complete. 2024-09-19T06:44:09.805 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.795+0000 7fb4663db740 20 client.16095 trim_cache size 0 max 16384 2024-09-19T06:44:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.810+0000 7fb4663db740 1 Processor -- start 2024-09-19T06:44:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4663db740 1 -- start start 2024-09-19T06:44:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4663db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4663db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd37410 0x56090dd59610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4663db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd59b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4663db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1b2350 con 0x56090dd36af0 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4663db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1afc20 con 0x56090dd37410 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4663db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1ada70 con 0x56090dd41cb0 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49330/0 (socket says 172.21.15.37:49330) 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 -- 172.21.15.37:0/1437666335 learned_addr learned my addr 172.21.15.37:0/1437666335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb460be2640 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd37410 0x56090dd59610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb461be4640 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd59b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 -- 172.21.15.37:0/1437666335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 msgr2=0x56090dd59b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd59b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 -- 172.21.15.37:0/1437666335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd37410 msgr2=0x56090dd59610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd37410 0x56090dd59610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 -- 172.21.15.37:0/1437666335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56090dd36870 con 0x56090dd36af0 2024-09-19T06:44:09.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb461be4640 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd59b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb4613e3640 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3f3a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb44c006010 tx=0x7fb44c010bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3444914b322c5b3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.811+0000 7fb442ffd640 1 -- 172.21.15.37:0/1437666335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c009b80 con 0x56090dd36af0 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb442ffd640 1 -- 172.21.15.37:0/1437666335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44c015030 con 0x56090dd36af0 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb442ffd640 1 -- 172.21.15.37:0/1437666335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c004c00 con 0x56090dd36af0 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 -- 172.21.15.37:0/1437666335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 msgr2=0x56090dd3f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3f3a0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fb44c006010 tx=0x7fb44c010bc0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 -- 172.21.15.37:0/1437666335 shutdown_connections 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd59b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd37410 0x56090dd59610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 --2- 172.21.15.37:0/1437666335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd36af0 0x56090dd3f3a0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 -- 172.21.15.37:0/1437666335 >> 172.21.15.37:0/1437666335 conn(0x56090dd33060 msgr2=0x56090dd40b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 -- 172.21.15.37:0/1437666335 shutdown_connections 2024-09-19T06:44:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.812+0000 7fb4663db740 1 -- 172.21.15.37:0/1437666335 wait complete. 2024-09-19T06:44:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 -- start start 2024-09-19T06:44:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 Processor -- start 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd3b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd3be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1ae0e0 con 0x56090dd37410 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1af550 con 0x56090dd36af0 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb45c1ae7b0 con 0x56090dd41cb0 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4663db740 10 client.0 _ll_register_callbacks cb 0x7fb44c009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb460be2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd3b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb460be2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd3b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49336/0 (socket says 172.21.15.37:49336) 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb460be2640 1 -- 172.21.15.37:0/2002631032 learned_addr learned my addr 172.21.15.37:0/2002631032 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.813+0000 7fb4613e3640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb461be4640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd3be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb460be2640 1 -- 172.21.15.37:0/2002631032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 msgr2=0x56090dd3be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb460be2640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd3be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb460be2640 1 -- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 msgr2=0x56090dd4a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb460be2640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd4a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb460be2640 1 -- 172.21.15.37:0/2002631032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56090dd3c690 con 0x56090dd37410 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb461be4640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd3be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb4613e3640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd4a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb460be2640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd3b8c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0fede0 tx=0x7fb450004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b3a47a1f1b62e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb440ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb440ff9640 1 -- 172.21.15.37:0/2002631032 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb450005220 con 0x56090dd37410 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb44c015030 con 0x56090dd37410 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb440ff9640 1 -- 172.21.15.37:0/2002631032 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7fb4500053c0 con 0x56090dd37410 2024-09-19T06:44:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb440ff9640 1 -- 172.21.15.37:0/2002631032 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fb4500055c0 con 0x56090dd37410 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb4663db740 10 client.16101 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.814+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56090ddbc750 con 0x56090dd37410 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb440ff9640 1 -- 172.21.15.37:0/2002631032 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb44c015030 con 0x56090dd37410 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb440ff9640 1 -- 172.21.15.37:0/2002631032 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7fb45002c050 con 0x56090dd37410 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4663db740 10 client.16101 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4663db740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4663db740 20 client.16101 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4663db740 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090ddbef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4663db740 4 client.16101 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56090dca2d70 con 0x56090dd51b30 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4613e3640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090ddbef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb440ff9640 10 client.16101 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.815+0000 7fb4613e3640 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090ddbef30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a89751962a83255 server_cookie=fd8c085af2c4d184 in_seq=0 out_seq=0 2024-09-19T06:44:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.922+0000 7fb440ff9640 1 -- 172.21.15.37:0/2002631032 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56090dca2d70 con 0x56090dd51b30 2024-09-19T06:44:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.922+0000 7fb440ff9640 10 client.16101 handle_command_reply: tid=0 2024-09-19T06:44:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.922+0000 7fb4663db740 1 client.16101 shutdown 2024-09-19T06:44:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 msgr2=0x56090dd3b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd3b8c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0fede0 tx=0x7fb450004730 comp rx=0 tx=0).stop 2024-09-19T06:44:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 shutdown_connections 2024-09-19T06:44:09.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56090dd51b30 0x56090ddbef30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56090dd41cb0 0x56090dd3be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56090dd37410 0x56090dd3b8c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 --2- 172.21.15.37:0/2002631032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56090dd36af0 0x56090dd4a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 >> 172.21.15.37:0/2002631032 conn(0x56090dd33060 msgr2=0x56090dd319a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb440ff9640 0 client.16101 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 shutdown_connections 2024-09-19T06:44:09.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 1 -- 172.21.15.37:0/2002631032 wait complete. 2024-09-19T06:44:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.923+0000 7fb4663db740 20 client.16101 trim_cache size 0 max 16384 2024-09-19T06:44:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:44:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:44:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:44:09.927 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb4440a34e0 msgr2=0x7fb4440a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb4440a34e0 0x7fb4440a59a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x56090dd343e0 tx=0x7fb4500098d0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c19ed90 msgr2=0x7fb45c19f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c19ed90 0x7fb45c19f240 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fb454005ab0 tx=0x7fb454004270 comp rx=0 tx=0).stop 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 shutdown_connections 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb4440a34e0 0x7fb4440a59a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb45c19ed90 0x7fb45c19f240 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb45c1080f0 0x7fb45c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 --2- 172.21.15.37:0/3415156590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb45c1008a0 0x7fb45c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 >> 172.21.15.37:0/3415156590 conn(0x7fb45c075ef0 msgr2=0x7fb45c0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 shutdown_connections 2024-09-19T06:44:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:09.924+0000 7fb463647640 1 -- 172.21.15.37:0/3415156590 wait complete. 2024-09-19T06:44:10.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 52921 8.16 MB/sec warmup 37 sec latency 62.637 ms 2024-09-19T06:44:10.459 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:44:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:10 smithi151 ceph-mon[35236]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-09-19T06:44:10.733 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:10.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:10 smithi096 ceph-mon[35004]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-09-19T06:44:10.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:10 smithi037 ceph-mon[31211]: pgmap v74: 97 pgs: 97 active+clean; 39 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 4.3 MiB/s wr, 336 op/s 2024-09-19T06:44:11.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 55377 8.25 MB/sec warmup 38 sec latency 18.569 ms 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 -- 172.21.15.37:0/1983549593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0103740 msgr2=0x7f97e0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 --2- 172.21.15.37:0/1983549593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0103740 0x7f97e0107b80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009890 tx=0x7f97d402a380 comp rx=0 tx=0).stop 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 -- 172.21.15.37:0/1983549593 shutdown_connections 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 --2- 172.21.15.37:0/1983549593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 0x7f97e010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 --2- 172.21.15.37:0/1983549593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0103740 0x7f97e0107b80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 --2- 172.21.15.37:0/1983549593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e0102d90 0x7f97e0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 -- 172.21.15.37:0/1983549593 >> 172.21.15.37:0/1983549593 conn(0x7f97e00fc7d0 msgr2=0x7f97e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 -- 172.21.15.37:0/1983549593 shutdown_connections 2024-09-19T06:44:11.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.226+0000 7f97e554d640 1 -- 172.21.15.37:0/1983549593 wait complete. 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 Processor -- start 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 -- start start 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0102d90 0x7f97e01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e0103740 0x7f97e0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 0x7f97e0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0112090 con 0x7f97e0108290 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0111f10 con 0x7f97e0102d90 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97e554d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97e0112210 con 0x7f97e0103740 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 0x7f97e0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97df7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 0x7f97e0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49366/0 (socket says 172.21.15.37:49366) 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0102d90 0x7f97e01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.227+0000 7f97df7fe640 1 -- 172.21.15.37:0/1712363485 learned_addr learned my addr 172.21.15.37:0/1712363485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:11.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97de7fc640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e0103740 0x7f97e0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97df7fe640 1 -- 172.21.15.37:0/1712363485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e0103740 msgr2=0x7f97e0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97df7fe640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e0103740 0x7f97e0100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97df7fe640 1 -- 172.21.15.37:0/1712363485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0102d90 msgr2=0x7f97e01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97df7fe640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0102d90 0x7f97e01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97df7fe640 1 -- 172.21.15.37:0/1712363485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e01a6a70 con 0x7f97e0108290 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97deffd640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0102d90 0x7f97e01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97df7fe640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 0x7f97e0101470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f97d800efc0 tx=0x7f97d800c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97c7fff640 1 -- 172.21.15.37:0/1712363485 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8003970 con 0x7f97e0108290 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e01a6d60 con 0x7f97e0108290 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97c7fff640 1 -- 172.21.15.37:0/1712363485 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d8004020 con 0x7f97e0108290 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97c7fff640 1 -- 172.21.15.37:0/1712363485 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d800b9f0 con 0x7f97e0108290 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.228+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e01a7270 con 0x7f97e0108290 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.231+0000 7f97c7fff640 1 -- 172.21.15.37:0/1712363485 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f97d801d020 con 0x7f97e0108290 2024-09-19T06:44:11.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.231+0000 7f97e7926740 1 Processor -- start 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.231+0000 7f97e7926740 1 -- start start 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.231+0000 7f97e7926740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f5814a0 0x55f64f581880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.231+0000 7f97e7926740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f58c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97e7926740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f58ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97e7926740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f64f4e72e0 con 0x55f64f5814a0 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97e7926740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f64f48ece0 con 0x55f64f581dc0 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97e7926740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f64f4e32a0 con 0x55f64f58c680 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f5814a0 0x55f64f581880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f5814a0 0x55f64f581880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49376/0 (socket says 172.21.15.37:49376) 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97de7fc640 1 -- 172.21.15.37:0/3578804084 learned_addr learned my addr 172.21.15.37:0/3578804084 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97deffd640 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f58c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97de7fc640 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f58ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97c7fff640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97c80a3430 0x7f97c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97deffd640 1 -- 172.21.15.37:0/3578804084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 msgr2=0x55f64f58ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97deffd640 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f58ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97deffd640 1 -- 172.21.15.37:0/3578804084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f5814a0 msgr2=0x55f64f581880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97deffd640 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f5814a0 0x55f64f581880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97deffd640 1 -- 172.21.15.37:0/3578804084 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64f473440 con 0x55f64f581dc0 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97df7fe640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97c80a3430 0x7f97c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97deffd640 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f58c140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f97d0005fd0 tx=0x7f97d000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3be9f10287f4b2fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.232+0000 7f97c7fff640 1 -- 172.21.15.37:0/1712363485 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f97d80cfab0 con 0x7f97e0108290 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97c57fa640 1 -- 172.21.15.37:0/3578804084 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0011700 con 0x55f64f581dc0 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97c57fa640 1 -- 172.21.15.37:0/3578804084 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0015070 con 0x55f64f581dc0 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97c57fa640 1 -- 172.21.15.37:0/3578804084 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d001bd30 con 0x55f64f581dc0 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97df7fe640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97c80a3430 0x7f97c80a58f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97d809bd20 tx=0x7f97d8004840 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:11.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 -- 172.21.15.37:0/3578804084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 msgr2=0x55f64f58c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f58c140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f97d0005fd0 tx=0x7f97d000bf90 comp rx=0 tx=0).stop 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 -- 172.21.15.37:0/3578804084 shutdown_connections 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f58ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f58c140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 --2- 172.21.15.37:0/3578804084 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f5814a0 0x55f64f581880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 -- 172.21.15.37:0/3578804084 >> 172.21.15.37:0/3578804084 conn(0x55f64f57da30 msgr2=0x55f64f57de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 -- 172.21.15.37:0/3578804084 shutdown_connections 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.233+0000 7f97e7926740 1 -- 172.21.15.37:0/3578804084 wait complete. 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 -- start start 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 Processor -- start 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f5814a0 0x55f64f58bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f589170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f5896b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f64f58f250 con 0x55f64f581dc0 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f64f4a2050 con 0x55f64f5814a0 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f64f58f3d0 con 0x55f64f58c680 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97e7926740 10 client.0 _ll_register_callbacks cb 0x55f64f4e2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f589170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f589170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49382/0 (socket says 172.21.15.37:49382) 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.234+0000 7f97deffd640 1 -- 172.21.15.37:0/2974793679 learned_addr learned my addr 172.21.15.37:0/2974793679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f5814a0 0x55f64f58bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f5896b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 -- 172.21.15.37:0/2974793679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 msgr2=0x55f64f5896b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f5896b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 -- 172.21.15.37:0/2974793679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 msgr2=0x55f64f589170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f589170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 -- 172.21.15.37:0/2974793679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64f2583c0 con 0x55f64f5814a0 2024-09-19T06:44:11.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97deffd640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f589170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97de7fc640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f5814a0 0x55f64f58bec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009860 tx=0x7f97d40058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b72b91733e08eef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97c2ff5640 1 -- 172.21.15.37:0/2974793679 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4037040 con 0x55f64f5814a0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97c2ff5640 1 -- 172.21.15.37:0/2974793679 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f97d40035f0 con 0x55f64f5814a0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.235+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d0015070 con 0x55f64f5814a0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97c2ff5640 1 -- 172.21.15.37:0/2974793679 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97d40037f0 con 0x55f64f5814a0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 10 client.25619 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f64f59ab90 con 0x55f64f5814a0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97c2ff5640 1 -- 172.21.15.37:0/2974793679 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0015070 con 0x55f64f5814a0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97c2ff5640 1 -- 172.21.15.37:0/2974793679 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f97d403d040 con 0x55f64f5814a0 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 10 client.25619 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 10 client.25619 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:11.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 20 client.25619 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f59c050 0x55f64f59e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 4 client.25619 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f64f4f4850 con 0x55f64f59c050 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.236+0000 7f97deffd640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f59c050 0x55f64f59e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.237+0000 7f97deffd640 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f59c050 0x55f64f59e4f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b34a7d14f1a1b36 server_cookie=6de44aa91633c8d1 in_seq=0 out_seq=0 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.237+0000 7f97c2ff5640 10 client.25619 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.237+0000 7f97c2ff5640 1 -- 172.21.15.37:0/2974793679 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f64f4f4850 con 0x55f64f59c050 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.237+0000 7f97c2ff5640 10 client.25619 handle_command_reply: tid=0 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.237+0000 7f97e7926740 1 client.25619 shutdown 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.237+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f5814a0 msgr2=0x55f64f58bec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.237+0000 7f97e7926740 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f5814a0 0x55f64f58bec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f97d4009860 tx=0x7f97d40058d0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 shutdown_connections 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f59c050 0x55f64f59e4f0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f58c680 0x55f64f5896b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f589170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 --2- 172.21.15.37:0/2974793679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f5814a0 0x55f64f58bec0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97c2ff5640 0 client.25619 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 >> 172.21.15.37:0/2974793679 conn(0x55f64f57da30 msgr2=0x55f64f583760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 shutdown_connections 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 1 -- 172.21.15.37:0/2974793679 wait complete. 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.238+0000 7f97e7926740 20 client.25619 trim_cache size 0 max 16384 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 Processor -- start 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 -- start start 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f5a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f5a4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f58c680 0x55f64f594640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01b23e0 con 0x55f64f58c680 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01afcb0 con 0x55f64f581dc0 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97e7926740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01adb00 con 0x55f64f5814a0 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f58c680 0x55f64f594640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f5a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97de7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f58c680 0x55f64f594640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49392/0 (socket says 172.21.15.37:49392) 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97deffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f5a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36492/0 (socket says 172.21.15.37:36492) 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.258+0000 7f97de7fc640 1 -- 172.21.15.37:0/1519854745 learned_addr learned my addr 172.21.15.37:0/1519854745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97de7fc640 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f5a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97deffd640 1 -- 172.21.15.37:0/1519854745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 msgr2=0x55f64f5a3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97deffd640 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f5a3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97deffd640 1 -- 172.21.15.37:0/1519854745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f58c680 msgr2=0x55f64f594640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97deffd640 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f58c680 0x55f64f594640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97deffd640 1 -- 172.21.15.37:0/1519854745 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64f57d860 con 0x55f64f581dc0 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97deffd640 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f5a4300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55f64f58a7b0 tx=0x7f97d001e040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcfa9f46dff73527 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97c4ff9640 1 -- 172.21.15.37:0/1519854745 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0004020 con 0x55f64f581dc0 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97c4ff9640 1 -- 172.21.15.37:0/1519854745 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d0015040 con 0x55f64f581dc0 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.259+0000 7f97c4ff9640 1 -- 172.21.15.37:0/1519854745 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d0005420 con 0x55f64f581dc0 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 -- 172.21.15.37:0/1519854745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 msgr2=0x55f64f5a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f5a4300 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x55f64f58a7b0 tx=0x7f97d001e040 comp rx=0 tx=0).stop 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 -- 172.21.15.37:0/1519854745 shutdown_connections 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f58c680 0x55f64f594640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f581dc0 0x55f64f5a4300 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 --2- 172.21.15.37:0/1519854745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f5a3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 -- 172.21.15.37:0/1519854745 >> 172.21.15.37:0/1519854745 conn(0x55f64f57da30 msgr2=0x55f64f588260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 -- 172.21.15.37:0/1519854745 shutdown_connections 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 -- 172.21.15.37:0/1519854745 wait complete. 2024-09-19T06:44:11.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 -- start start 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.260+0000 7f97e7926740 1 Processor -- start 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97e7926740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f591a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97e7926740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f591fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97e7926740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f58c680 0x55f64f604d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97e7926740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01ae170 con 0x55f64f581dc0 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97e7926740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01af5e0 con 0x55f64f58c680 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97e7926740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f97e01ae840 con 0x55f64f5814a0 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97e7926740 10 client.0 _ll_register_callbacks cb 0x7f97d40056d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f591fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97deffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f591fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49400/0 (socket says 172.21.15.37:49400) 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97de7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f58c680 0x55f64f604d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97deffd640 1 -- 172.21.15.37:0/3809989090 learned_addr learned my addr 172.21.15.37:0/3809989090 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:11.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97de7fc640 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f591a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97de7fc640 1 -- 172.21.15.37:0/3809989090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 msgr2=0x55f64f591a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97de7fc640 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f591a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97de7fc640 1 -- 172.21.15.37:0/3809989090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 msgr2=0x55f64f591fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97de7fc640 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f591fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97de7fc640 1 -- 172.21.15.37:0/3809989090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64f6052d0 con 0x55f64f58c680 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.261+0000 7f97deffd640 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f591fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97de7fc640 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f58c680 0x55f64f604d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97d402a8a0 tx=0x7f97d4033f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6508ec7dc59a44a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97c2ff5640 1 -- 172.21.15.37:0/3809989090 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d4037040 con 0x55f64f58c680 2024-09-19T06:44:11.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97c2ff5640 1 -- 172.21.15.37:0/3809989090 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f97d4032640 con 0x55f64f58c680 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d0015040 con 0x55f64f58c680 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97c2ff5640 1 -- 172.21.15.37:0/3809989090 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f97d4003990 con 0x55f64f58c680 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97e7926740 10 client.25625 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55f64f606540 con 0x55f64f58c680 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.262+0000 7f97c2ff5640 1 -- 172.21.15.37:0/3809989090 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f97d4003b30 con 0x55f64f58c680 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97c2ff5640 1 -- 172.21.15.37:0/3809989090 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f97d403d070 con 0x55f64f58c680 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97e7926740 10 client.25625 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97e7926740 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97e7926740 20 client.25625 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97e7926740 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f606bb0 0x55f64f608fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97e7926740 4 client.25625 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f97d0015040 con 0x55f64f606bb0 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97deffd640 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f606bb0 0x55f64f608fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:11.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97c2ff5640 10 client.25625 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.263+0000 7f97deffd640 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f606bb0 0x55f64f608fd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee6f4d88e0dd4b50 server_cookie=8c4fee23f5f99de0 in_seq=0 out_seq=0 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97c2ff5640 1 -- 172.21.15.37:0/3809989090 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f97d0015040 con 0x55f64f606bb0 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97c2ff5640 10 client.25625 handle_command_reply: tid=0 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 client.25625 shutdown 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f58c680 msgr2=0x55f64f604d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f58c680 0x55f64f604d90 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f97d402a8a0 tx=0x7f97d4033f90 comp rx=0 tx=0).stop 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 shutdown_connections 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f64f606bb0 0x55f64f608fd0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f64f58c680 0x55f64f604d90 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97c2ff5640 0 client.25625 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f64f581dc0 0x55f64f591fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 --2- 172.21.15.37:0/3809989090 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f64f5814a0 0x55f64f591a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 >> 172.21.15.37:0/3809989090 conn(0x55f64f57da30 msgr2=0x55f64f57c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:11.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 shutdown_connections 2024-09-19T06:44:11.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 1 -- 172.21.15.37:0/3809989090 wait complete. 2024-09-19T06:44:11.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.265+0000 7f97e7926740 20 client.25625 trim_cache size 0 max 16384 2024-09-19T06:44:11.532 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:44:11.532 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:44:11.532 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:44:11.532 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97c80a3430 msgr2=0x7f97c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97c80a3430 0x7f97c80a58f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f97d809bd20 tx=0x7f97d8004840 comp rx=0 tx=0).stop 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 msgr2=0x7f97e0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 0x7f97e0101470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f97d800efc0 tx=0x7f97d800c800 comp rx=0 tx=0).stop 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 shutdown_connections 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f97c80a3430 0x7f97c80a58f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f97e0108290 0x7f97e0101470 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f97e0103740 0x7f97e0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 --2- 172.21.15.37:0/1712363485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f97e0102d90 0x7f97e01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 >> 172.21.15.37:0/1712363485 conn(0x7f97e00fc7d0 msgr2=0x7f97e0105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 shutdown_connections 2024-09-19T06:44:11.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:11.266+0000 7f97e554d640 1 -- 172.21.15.37:0/1712363485 wait complete. 2024-09-19T06:44:12.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 57785 8.41 MB/sec warmup 39 sec latency 21.451 ms 2024-09-19T06:44:12.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:44:12.510 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:44:12.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:12 smithi151 ceph-mon[35236]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-09-19T06:44:12.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:12 smithi096 ceph-mon[35004]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-09-19T06:44:12.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:12 smithi037 ceph-mon[31211]: pgmap v75: 97 pgs: 97 active+clean; 39 MiB data, 459 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.9 MiB/s wr, 275 op/s 2024-09-19T06:44:12.794 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:13.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 60627 8.64 MB/sec warmup 40 sec latency 80.456 ms 2024-09-19T06:44:13.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 -- 172.21.15.37:0/550702742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc106f10 msgr2=0x7ff7bc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:13.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/550702742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc106f10 0x7ff7bc10ea90 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4009ca0 tx=0x7ff7b402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 -- 172.21.15.37:0/550702742 shutdown_connections 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/550702742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc106f10 0x7ff7bc10ea90 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/550702742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7bc102790 0x7ff7bc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/550702742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 0x7ff7bc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 -- 172.21.15.37:0/550702742 >> 172.21.15.37:0/550702742 conn(0x7ff7bc0fd8c0 msgr2=0x7ff7bc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 -- 172.21.15.37:0/550702742 shutdown_connections 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.255+0000 7ff7c2a63640 1 -- 172.21.15.37:0/550702742 wait complete. 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 Processor -- start 2024-09-19T06:44:13.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 -- start start 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 0x7ff7bc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc102790 0x7ff7bc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7bc106f10 0x7ff7bc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc111f70 con 0x7ff7bc102790 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc111df0 con 0x7ff7bc106f10 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c2a63640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff7bc1120f0 con 0x7ff7bc101de0 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc102790 0x7ff7bc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7bbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 0x7ff7bc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7bbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 0x7ff7bc1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:55202/0 (socket says 172.21.15.37:55202) 2024-09-19T06:44:13.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7c1000640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7bc106f10 0x7ff7bc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7bbfff640 1 -- 172.21.15.37:0/4251435811 learned_addr learned my addr 172.21.15.37:0/4251435811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.256+0000 7ff7bb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc102790 0x7ff7bc198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49420/0 (socket says 172.21.15.37:49420) 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7bbfff640 1 -- 172.21.15.37:0/4251435811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7bc106f10 msgr2=0x7ff7bc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7bbfff640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7bc106f10 0x7ff7bc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7bbfff640 1 -- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc102790 msgr2=0x7ff7bc198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7bbfff640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc102790 0x7ff7bc198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7bbfff640 1 -- 172.21.15.37:0/4251435811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7bc19d380 con 0x7ff7bc101de0 2024-09-19T06:44:13.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7bbfff640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 0x7ff7bc1983d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00e9f0 tx=0x7ff7ac00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7b97fa640 1 -- 172.21.15.37:0/4251435811 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac00cd80 con 0x7ff7bc101de0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7b97fa640 1 -- 172.21.15.37:0/4251435811 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff7ac010910 con 0x7ff7bc101de0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7bc19d670 con 0x7ff7bc101de0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7b97fa640 1 -- 172.21.15.37:0/4251435811 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7ac018990 con 0x7ff7bc101de0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.257+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7bc1a4f50 con 0x7ff7bc101de0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.258+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7bc1021c0 con 0x7ff7bc101de0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.259+0000 7ff7b97fa640 1 -- 172.21.15.37:0/4251435811 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff7ac010ac0 con 0x7ff7bc101de0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.260+0000 7ff7b97fa640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff7a40a33b0 0x7ff7a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.260+0000 7ff7bb7fe640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff7a40a33b0 0x7ff7a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:13.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.260+0000 7ff7b97fa640 1 -- 172.21.15.37:0/4251435811 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7ff7ac0d0030 con 0x7ff7bc101de0 2024-09-19T06:44:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.261+0000 7ff7bb7fe640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff7a40a33b0 0x7ff7a40a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0fe0c0 tx=0x7ff7b0005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:13.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.265+0000 7ff7b97fa640 1 -- 172.21.15.37:0/4251435811 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff7ac09c3c0 con 0x7ff7bc101de0 2024-09-19T06:44:13.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.491+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7bc02d7c0 con 0x7ff7bc101de0 2024-09-19T06:44:13.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.492+0000 7ff7b97fa640 1 -- 172.21.15.37:0/4251435811 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+11906 (secure 0 0 0) 0x7ff7ac0a06c0 con 0x7ff7bc101de0 2024-09-19T06:44:13.494 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:44:13.494 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":27,"btime":"2024-09-19T06:43:42:276735+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":27,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:43:41.612230+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24445":{"gid":24445,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3546164673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3546164673},{"type":"v1","addr":"172.21.15.151:6839","nonce":3546164673}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:44:13.495 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 27 2024-09-19T06:44:13.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff7a40a33b0 msgr2=0x7ff7a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff7a40a33b0 0x7ff7a40a5870 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc0fe0c0 tx=0x7ff7b0005f90 comp rx=0 tx=0).stop 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 msgr2=0x7ff7bc1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 0x7ff7bc1983d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff7ac00e9f0 tx=0x7ff7ac00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 shutdown_connections 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7bc106f10 0x7ff7bc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff7a40a33b0 0x7ff7a40a5870 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff7bc102790 0x7ff7bc198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 --2- 172.21.15.37:0/4251435811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff7bc101de0 0x7ff7bc1983d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.494+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 >> 172.21.15.37:0/4251435811 conn(0x7ff7bc0fd8c0 msgr2=0x7ff7bc0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.495+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 shutdown_connections 2024-09-19T06:44:13.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:13.495+0000 7ff7c2a63640 1 -- 172.21.15.37:0/4251435811 wait complete. 2024-09-19T06:44:14.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 63228 8.73 MB/sec warmup 41 sec latency 53.321 ms 2024-09-19T06:44:14.397 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:44:14.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:14 smithi151 ceph-mon[35236]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 494 op/s 2024-09-19T06:44:14.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:14 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4251435811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:44:14.675 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:14.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:14 smithi096 ceph-mon[35004]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 494 op/s 2024-09-19T06:44:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:14 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4251435811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:44:14.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:14 smithi037 ceph-mon[31211]: pgmap v76: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 494 op/s 2024-09-19T06:44:14.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:14 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4251435811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:44:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.128+0000 7ff431d9b640 1 -- 172.21.15.37:0/2071307589 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280319b0 con 0x7ff434100a80 2024-09-19T06:44:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 -- 172.21.15.37:0/2071307589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434100a80 msgr2=0x7ff434100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 --2- 172.21.15.37:0/2071307589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434100a80 0x7ff434100ee0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff428009930 tx=0x7ff42802a420 comp rx=0 tx=0).stop 2024-09-19T06:44:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 -- 172.21.15.37:0/2071307589 shutdown_connections 2024-09-19T06:44:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 --2- 172.21.15.37:0/2071307589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434101420 0x7ff43410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 --2- 172.21.15.37:0/2071307589 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434100a80 0x7ff434100ee0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 --2- 172.21.15.37:0/2071307589 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff434106a80 0x7ff434106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 -- 172.21.15.37:0/2071307589 >> 172.21.15.37:0/2071307589 conn(0x7ff4340fc7a0 msgr2=0x7ff4340febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 -- 172.21.15.37:0/2071307589 shutdown_connections 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.129+0000 7ff439134640 1 -- 172.21.15.37:0/2071307589 wait complete. 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 Processor -- start 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 -- start start 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff434100a80 0x7ff43419a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434101420 0x7ff43419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434106a80 0x7ff43419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410f1d0 con 0x7ff434101420 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410f050 con 0x7ff434106a80 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff439134640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff43410f350 con 0x7ff434100a80 2024-09-19T06:44:15.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff43259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434101420 0x7ff43419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff434100a80 0x7ff43419a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff43259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434101420 0x7ff43419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49450/0 (socket says 172.21.15.37:49450) 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff43259c640 1 -- 172.21.15.37:0/3998883444 learned_addr learned my addr 172.21.15.37:0/3998883444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.130+0000 7ff43359e640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434106a80 0x7ff43419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff43259c640 1 -- 172.21.15.37:0/3998883444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff434100a80 msgr2=0x7ff43419a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff43259c640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff434100a80 0x7ff43419a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff43259c640 1 -- 172.21.15.37:0/3998883444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434106a80 msgr2=0x7ff43419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff43259c640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434106a80 0x7ff43419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff43259c640 1 -- 172.21.15.37:0/3998883444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43419f680 con 0x7ff434101420 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff432d9d640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff434100a80 0x7ff43419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff43259c640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434101420 0x7ff43419ac60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff428009230 tx=0x7ff428004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:15.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff41bfff640 1 -- 172.21.15.37:0/3998883444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42803f030 con 0x7ff434101420 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43419f910 con 0x7ff434101420 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff41bfff640 1 -- 172.21.15.37:0/3998883444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff428004440 con 0x7ff434101420 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff41bfff640 1 -- 172.21.15.37:0/3998883444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428035070 con 0x7ff434101420 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.131+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4341a7250 con 0x7ff434101420 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.133+0000 7ff41bfff640 1 -- 172.21.15.37:0/3998883444 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff428042030 con 0x7ff434101420 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.133+0000 7ff43b50d740 1 Processor -- start 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43b50d740 1 -- start start 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5d3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d35d0 0x55d50a5d3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5de370 0x55d50a5e0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43b50d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d50a5392f0 con 0x55d50a5de370 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43b50d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d50a4e0cf0 con 0x55d50a5d4b40 2024-09-19T06:44:15.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43b50d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d50a5352b0 con 0x55d50a5d35d0 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5d3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5de370 0x55d50a5e0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5d3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36528/0 (socket says 172.21.15.37:36528) 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 -- 172.21.15.37:0/2630357547 learned_addr learned my addr 172.21.15.37:0/2630357547 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff432d9d640 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d35d0 0x55d50a5d3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 -- 172.21.15.37:0/2630357547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d35d0 msgr2=0x55d50a5d3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d35d0 0x55d50a5d3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 -- 172.21.15.37:0/2630357547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5de370 msgr2=0x55d50a5e0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5de370 0x55d50a5e0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff43359e640 1 -- 172.21.15.37:0/2630357547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50a4c53a0 con 0x55d50a5d4b40 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff41bfff640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff41c0a33b0 0x7ff41c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.134+0000 7ff432d9d640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff41c0a33b0 0x7ff41c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff43359e640 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5d3090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009fd0 tx=0x7ff42c0118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5efbb9fe44f26718 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff41bfff640 1 -- 172.21.15.37:0/3998883444 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7ff4280f0000 con 0x7ff434101420 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff4197fa640 1 -- 172.21.15.37:0/2630357547 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004d20 con 0x55d50a5d4b40 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff4197fa640 1 -- 172.21.15.37:0/2630357547 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c004050 con 0x55d50a5d4b40 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff4197fa640 1 -- 172.21.15.37:0/2630357547 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42c005260 con 0x55d50a5d4b40 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff43b50d740 1 -- 172.21.15.37:0/2630357547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 msgr2=0x55d50a5d3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff43b50d740 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5d3090 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff42c009fd0 tx=0x7ff42c0118d0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.135+0000 7ff432d9d640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff41c0a33b0 0x7ff41c0a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55d50a5d0ac0 tx=0x7ff424005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- 172.21.15.37:0/2630357547 shutdown_connections 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5de370 0x55d50a5e0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d35d0 0x55d50a5d3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 --2- 172.21.15.37:0/2630357547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5d3090 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- 172.21.15.37:0/2630357547 >> 172.21.15.37:0/2630357547 conn(0x55d50a59c310 msgr2=0x55d50a5cfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- 172.21.15.37:0/2630357547 shutdown_connections 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- 172.21.15.37:0/2630357547 wait complete. 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- start start 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 Processor -- start 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5ddd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d4b40 0x55d50a5db000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5de370 0x55d50a5db5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d50a5e1070 con 0x55d50a5d35d0 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d50a4f4060 con 0x55d50a5de370 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d50a5e11f0 con 0x55d50a5d4b40 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.136+0000 7ff43b50d740 10 client.0 _ll_register_callbacks cb 0x55d50a2478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5ddd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5ddd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49480/0 (socket says 172.21.15.37:49480) 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 -- 172.21.15.37:0/3649406011 learned_addr learned my addr 172.21.15.37:0/3649406011 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff43359e640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5de370 0x55d50a5db5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff43259c640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d4b40 0x55d50a5db000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 -- 172.21.15.37:0/3649406011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d4b40 msgr2=0x55d50a5db000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d4b40 0x55d50a5db000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 -- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5de370 msgr2=0x55d50a5db5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5de370 0x55d50a5db5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 -- 172.21.15.37:0/3649406011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50a2aa3c0 con 0x55d50a5d35d0 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff43359e640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5de370 0x55d50a5db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff432d9d640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5ddd50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff424004650 tx=0x7ff4240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a1ca6cc15cae83 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff416ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:15.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff416ff5640 1 -- 172.21.15.37:0/3649406011 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42400f870 con 0x55d50a5d35d0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff42c004050 con 0x55d50a5d35d0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff416ff5640 1 -- 172.21.15.37:0/3649406011 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7ff42400fa10 con 0x55d50a5d35d0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff416ff5640 1 -- 172.21.15.37:0/3649406011 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff42400fc50 con 0x55d50a5d35d0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff43b50d740 10 client.16137 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.137+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d50a5ecba0 con 0x55d50a5d35d0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff416ff5640 1 -- 172.21.15.37:0/3649406011 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff42c004050 con 0x55d50a5d35d0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff416ff5640 1 -- 172.21.15.37:0/3649406011 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff424027070 con 0x55d50a5d35d0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff43b50d740 10 client.16137 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff43b50d740 10 client.16137 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff43b50d740 20 client.16137 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff43b50d740 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a5edfb0 0x55d50a5f03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff43b50d740 4 client.16137 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50a546860 con 0x55d50a5edfb0 2024-09-19T06:44:15.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.138+0000 7ff43259c640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a5edfb0 0x55d50a5f03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff416ff5640 10 client.16137 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff43259c640 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a5edfb0 0x55d50a5f03a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f932f7579d515931 server_cookie=9412c378e775cf68 in_seq=0 out_seq=0 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff416ff5640 1 -- 172.21.15.37:0/3649406011 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d50a546860 con 0x55d50a5edfb0 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff416ff5640 10 client.16137 handle_command_reply: tid=0 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff43b50d740 1 client.16137 shutdown 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 msgr2=0x55d50a5ddd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff43b50d740 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5ddd50 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7ff424004650 tx=0x7ff4240046c0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 shutdown_connections 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.139+0000 7ff43b50d740 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a5edfb0 0x55d50a5f03a0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff43b50d740 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5de370 0x55d50a5db5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff416ff5640 0 client.16137 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:15.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff43b50d740 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5d4b40 0x55d50a5db000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff43b50d740 1 --2- 172.21.15.37:0/3649406011 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5ddd50 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 >> 172.21.15.37:0/3649406011 conn(0x55d50a59c310 msgr2=0x55d50a5d5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 shutdown_connections 2024-09-19T06:44:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff43b50d740 1 -- 172.21.15.37:0/3649406011 wait complete. 2024-09-19T06:44:15.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.140+0000 7ff43b50d740 20 client.16137 trim_cache size 0 max 16384 2024-09-19T06:44:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 Processor -- start 2024-09-19T06:44:15.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 -- start start 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d35d0 0x55d50a5ea310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d4b40 0x55d50a5dcf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341b2410 con 0x55d50a5d4b40 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341afce0 con 0x55d50a5d35d0 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43b50d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341adb30 con 0x55d50a5de370 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d4b40 0x55d50a5dcf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d35d0 0x55d50a5ea310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d4b40 0x55d50a5dcf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49494/0 (socket says 172.21.15.37:49494) 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43259c640 1 -- 172.21.15.37:0/1169874993 learned_addr learned my addr 172.21.15.37:0/1169874993 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43359e640 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43259c640 1 -- 172.21.15.37:0/1169874993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 msgr2=0x55d50a5f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43259c640 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5f5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43259c640 1 -- 172.21.15.37:0/1169874993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d35d0 msgr2=0x55d50a5ea310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.154+0000 7ff43259c640 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d35d0 0x55d50a5ea310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.155+0000 7ff43259c640 1 -- 172.21.15.37:0/1169874993 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50a5cf390 con 0x55d50a5d4b40 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.155+0000 7ff432d9d640 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d35d0 0x55d50a5ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.155+0000 7ff43259c640 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d4b40 0x55d50a5dcf70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff4280edd50 tx=0x7ff4280dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3385e75e6a51942 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.155+0000 7ff418ff9640 1 -- 172.21.15.37:0/1169874993 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428049040 con 0x55d50a5d4b40 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.155+0000 7ff418ff9640 1 -- 172.21.15.37:0/1169874993 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4280bc390 con 0x55d50a5d4b40 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.155+0000 7ff418ff9640 1 -- 172.21.15.37:0/1169874993 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280f3d00 con 0x55d50a5d4b40 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- 172.21.15.37:0/1169874993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d4b40 msgr2=0x55d50a5dcf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d4b40 0x55d50a5dcf70 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7ff4280edd50 tx=0x7ff4280dc040 comp rx=0 tx=0).stop 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- 172.21.15.37:0/1169874993 shutdown_connections 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d4b40 0x55d50a5dcf70 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1169874993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d35d0 0x55d50a5ea310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- 172.21.15.37:0/1169874993 >> 172.21.15.37:0/1169874993 conn(0x55d50a59c310 msgr2=0x55d50a5dc860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- 172.21.15.37:0/1169874993 shutdown_connections 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- 172.21.15.37:0/1169874993 wait complete. 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- start start 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 Processor -- start 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5e22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5e2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341ae1a0 con 0x55d50a5d35d0 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341af610 con 0x55d50a5d4b40 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.156+0000 7ff43b50d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff4341ae870 con 0x55d50a5de370 2024-09-19T06:44:15.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff43b50d740 10 client.0 _ll_register_callbacks cb 0x7ff424009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5e22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5e22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49502/0 (socket says 172.21.15.37:49502) 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff43259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5e2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 -- 172.21.15.37:0/1459424322 learned_addr learned my addr 172.21.15.37:0/1459424322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff43359e640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 -- 172.21.15.37:0/1459424322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 msgr2=0x55d50a5e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5e2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 -- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 msgr2=0x55d50a5e2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5e2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 -- 172.21.15.37:0/1459424322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d50a657d40 con 0x55d50a5d35d0 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff43359e640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:44:15.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff43259c640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff432d9d640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5e22e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff42401f550 tx=0x7ff424004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34338f5b79824c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff416ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff416ff5640 1 -- 172.21.15.37:0/1459424322 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42401fd50 con 0x55d50a5d35d0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff416ff5640 1 -- 172.21.15.37:0/1459424322 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7ff424005040 con 0x55d50a5d35d0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.157+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4280bc390 con 0x55d50a5d35d0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff416ff5640 1 -- 172.21.15.37:0/1459424322 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff42400fa80 con 0x55d50a5d35d0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 10 client.16143 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55d50a659020 con 0x55d50a5d35d0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff416ff5640 1 -- 172.21.15.37:0/1459424322 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff4280bc390 con 0x55d50a5d35d0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff416ff5640 1 -- 172.21.15.37:0/1459424322 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7ff424027040 con 0x55d50a5d35d0 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 10 client.16143 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 10 client.16143 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:44:15.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 20 client.16143 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a6597a0 0x55d50a65bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 4 client.16143 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d50a540d70 con 0x55d50a6597a0 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.158+0000 7ff43259c640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a6597a0 0x55d50a65bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff416ff5640 10 client.16143 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43259c640 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a6597a0 0x55d50a65bbc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1da0326c8a1cba8d server_cookie=fd2167e6c3d7138f in_seq=0 out_seq=0 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff416ff5640 1 -- 172.21.15.37:0/1459424322 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d50a540d70 con 0x55d50a6597a0 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff416ff5640 10 client.16143 handle_command_reply: tid=0 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 client.16143 shutdown 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 msgr2=0x55d50a5e22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5e22e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7ff42401f550 tx=0x7ff424004490 comp rx=0 tx=0).stop 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 shutdown_connections 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d50a6597a0 0x55d50a65bbc0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d50a5de370 0x55d50a5e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d50a5d4b40 0x55d50a5e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff416ff5640 0 client.16143 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 --2- 172.21.15.37:0/1459424322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d50a5d35d0 0x55d50a5e22e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 >> 172.21.15.37:0/1459424322 conn(0x55d50a59c310 msgr2=0x55d50a5dd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 shutdown_connections 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 1 -- 172.21.15.37:0/1459424322 wait complete. 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.159+0000 7ff43b50d740 20 client.16143 trim_cache size 0 max 16384 2024-09-19T06:44:15.176 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:44:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff41c0a33b0 msgr2=0x7ff41c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff41c0a33b0 0x7ff41c0a5870 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x55d50a5d0ac0 tx=0x7ff424005f90 comp rx=0 tx=0).stop 2024-09-19T06:44:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434101420 msgr2=0x7ff43419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434101420 0x7ff43419ac60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7ff428009230 tx=0x7ff428004040 comp rx=0 tx=0).stop 2024-09-19T06:44:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 shutdown_connections 2024-09-19T06:44:15.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff434106a80 0x7ff43419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff41c0a33b0 0x7ff41c0a5870 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff434101420 0x7ff43419ac60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 --2- 172.21.15.37:0/3998883444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff434100a80 0x7ff43419a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 >> 172.21.15.37:0/3998883444 conn(0x7ff4340fc7a0 msgr2=0x7ff434104590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 shutdown_connections 2024-09-19T06:44:15.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:15.160+0000 7ff439134640 1 -- 172.21.15.37:0/3998883444 wait complete. 2024-09-19T06:44:15.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 65388 8.82 MB/sec warmup 42 sec latency 20.485 ms 2024-09-19T06:44:16.038 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:44:16.191 INFO:tasks.workunit.client.0.smithi037.stdout: 1 68212 8.95 MB/sec warmup 43 sec latency 12.380 ms 2024-09-19T06:44:16.302 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:16.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:16 smithi151 ceph-mon[35236]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-19T06:44:16.682 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:16 smithi037 ceph-mon[31211]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-19T06:44:16.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:16 smithi096 ceph-mon[35004]: pgmap v77: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-19T06:44:16.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 -- 172.21.15.37:0/41638670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 msgr2=0x7f8f34103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 --2- 172.21.15.37:0/41638670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f34103170 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009930 tx=0x7f8f2402a420 comp rx=0 tx=0).stop 2024-09-19T06:44:16.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 -- 172.21.15.37:0/41638670 shutdown_connections 2024-09-19T06:44:16.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 --2- 172.21.15.37:0/41638670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f34108290 0x7f8f3410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 --2- 172.21.15.37:0/41638670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f34103740 0x7f8f34107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 --2- 172.21.15.37:0/41638670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f34103170 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 -- 172.21.15.37:0/41638670 >> 172.21.15.37:0/41638670 conn(0x7f8f340fc7d0 msgr2=0x7f8f340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 -- 172.21.15.37:0/41638670 shutdown_connections 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.767+0000 7f8f39113640 1 -- 172.21.15.37:0/41638670 wait complete. 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 Processor -- start 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 -- start start 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f341985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f34103740 0x7f8f34198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f34108290 0x7f8f3419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341120b0 con 0x7f8f34102d90 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34111f30 con 0x7f8f34103740 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f39113640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f34112230 con 0x7f8f34108290 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f32d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f341985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f32d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f341985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49526/0 (socket says 172.21.15.37:49526) 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f32d9d640 1 -- 172.21.15.37:0/1560524077 learned_addr learned my addr 172.21.15.37:0/1560524077 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f3259c640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f34103740 0x7f8f34198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.768+0000 7f8f3359e640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f34108290 0x7f8f3419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f32d9d640 1 -- 172.21.15.37:0/1560524077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f34108290 msgr2=0x7f8f3419ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f34108290 0x7f8f3419ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f32d9d640 1 -- 172.21.15.37:0/1560524077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f34103740 msgr2=0x7f8f34198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f34103740 0x7f8f34198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f32d9d640 1 -- 172.21.15.37:0/1560524077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3419d570 con 0x7f8f34102d90 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f3259c640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f34103740 0x7f8f34198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f341985c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8f2402a940 tx=0x7f8f24003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f1bfff640 1 -- 172.21.15.37:0/1560524077 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2403f070 con 0x7f8f34102d90 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3419d800 con 0x7f8f34102d90 2024-09-19T06:44:16.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f1bfff640 1 -- 172.21.15.37:0/1560524077 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f24031c90 con 0x7f8f34102d90 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f1bfff640 1 -- 172.21.15.37:0/1560524077 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24035030 con 0x7f8f34102d90 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.769+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f341a5140 con 0x7f8f34102d90 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.771+0000 7f8f1bfff640 1 -- 172.21.15.37:0/1560524077 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8f24042070 con 0x7f8f34102d90 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 Processor -- start 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 -- start start 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f1bfff640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8f1c0a33b0 0x7f8f1c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d2766090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d2766a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3359e640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8f1c0a33b0 0x7f8f1c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d2773760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5579d26cc2f0 con 0x5579d2767b40 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5579d2673cf0 con 0x5579d27665d0 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5579d26c82b0 con 0x5579d2771370 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f1bfff640 1 -- 172.21.15.37:0/1560524077 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f8f240f03e0 con 0x7f8f34102d90 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d2766a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f32d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d2766090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d2766a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36592/0 (socket says 172.21.15.37:36592) 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f3259c640 1 -- 172.21.15.37:0/1476095375 learned_addr learned my addr 172.21.15.37:0/1476095375 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.772+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d2766090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49538/0 (socket says 172.21.15.37:49538) 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3359e640 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d2773760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3259c640 1 -- 172.21.15.37:0/1476095375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 msgr2=0x5579d2773760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3259c640 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d2773760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3259c640 1 -- 172.21.15.37:0/1476095375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 msgr2=0x5579d2766090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3259c640 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d2766090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3259c640 1 -- 172.21.15.37:0/1476095375 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d26583a0 con 0x5579d27665d0 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d2766090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3359e640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8f1c0a33b0 0x7f8f1c0a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c005fd0 tx=0x7f8f2c011810 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f3259c640 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d2766a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f28010480 tx=0x7f8f2800bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc293e1cbeb83db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f197fa640 1 -- 172.21.15.37:0/1476095375 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f28011760 con 0x5579d27665d0 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f197fa640 1 -- 172.21.15.37:0/1476095375 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f28011de0 con 0x5579d27665d0 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.773+0000 7f8f197fa640 1 -- 172.21.15.37:0/1476095375 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2801a380 con 0x5579d27665d0 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/1476095375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 msgr2=0x5579d2766a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d2766a10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8f28010480 tx=0x7f8f2800bf90 comp rx=0 tx=0).stop 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/1476095375 shutdown_connections 2024-09-19T06:44:16.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d2773760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d2766a10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/1476095375 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d2766090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/1476095375 >> 172.21.15.37:0/1476095375 conn(0x5579d272f310 msgr2=0x5579d2762c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/1476095375 shutdown_connections 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/1476095375 wait complete. 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.774+0000 7f8f3b4ec740 1 -- start start 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 1 Processor -- start 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d27665d0 0x5579d2770d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d276e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2771370 0x5579d276e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5579d2774070 con 0x5579d2771370 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5579d2687060 con 0x5579d2767b40 2024-09-19T06:44:16.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5579d27741f0 con 0x5579d27665d0 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3b4ec740 10 client.0 _ll_register_callbacks cb 0x5579d23da8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2771370 0x5579d276e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2771370 0x5579d276e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49550/0 (socket says 172.21.15.37:49550) 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 -- 172.21.15.37:0/3997369385 learned_addr learned my addr 172.21.15.37:0/3997369385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d276e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d27665d0 0x5579d2770d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 -- 172.21.15.37:0/3997369385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d27665d0 msgr2=0x5579d2770d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d27665d0 0x5579d2770d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 -- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 msgr2=0x5579d276e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d276e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3359e640 1 -- 172.21.15.37:0/3997369385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d243d3c0 con 0x5579d2771370 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d27665d0 0x5579d2770d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:16.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.775+0000 7f8f3259c640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d276e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f3359e640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2771370 0x5579d276e5d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c022e80 tx=0x7f8f2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ac10b5464c7b7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f16ff5640 1 -- 172.21.15.37:0/3997369385 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c017040 con 0x5579d2771370 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f16ff5640 1 -- 172.21.15.37:0/3997369385 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f8f2c003970 con 0x5579d2771370 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f28011de0 con 0x5579d2771370 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f16ff5640 1 -- 172.21.15.37:0/3997369385 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f2c003b70 con 0x5579d2771370 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f3b4ec740 10 client.16155 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579d277fb50 con 0x5579d2771370 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.776+0000 7f8f16ff5640 1 -- 172.21.15.37:0/3997369385 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f28011de0 con 0x5579d2771370 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f16ff5640 1 -- 172.21.15.37:0/3997369385 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f2c029070 con 0x5579d2771370 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f3b4ec740 10 client.16155 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f3b4ec740 10 client.16155 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f3b4ec740 20 client.16155 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:16.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d2781010 0x5579d2783470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f3b4ec740 4 client.16155 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579d26d9860 con 0x5579d2781010 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d2781010 0x5579d2783470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d2781010 0x5579d2783470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=73773b14e3b491c1 server_cookie=c91442ca7e0c4cc8 in_seq=0 out_seq=0 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.777+0000 7f8f16ff5640 10 client.16155 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f16ff5640 1 -- 172.21.15.37:0/3997369385 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5579d26d9860 con 0x5579d2781010 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f16ff5640 10 client.16155 handle_command_reply: tid=0 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 client.16155 shutdown 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2771370 msgr2=0x5579d276e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2771370 0x5579d276e5d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c022e80 tx=0x7f8f2c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 shutdown_connections 2024-09-19T06:44:16.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d2781010 0x5579d2783470 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2771370 0x5579d276e5d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d276e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/3997369385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d27665d0 0x5579d2770d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f16ff5640 0 client.16155 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 >> 172.21.15.37:0/3997369385 conn(0x5579d272f310 msgr2=0x5579d2768930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 shutdown_connections 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/3997369385 wait complete. 2024-09-19T06:44:16.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.778+0000 7f8f3b4ec740 20 client.16155 trim_cache size 0 max 16384 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 Processor -- start 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 -- start start 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d27665d0 0x5579d2762e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d277d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341ae010 con 0x5579d27665d0 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341ab8e0 con 0x5579d2767b40 2024-09-19T06:44:16.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341a9730 con 0x5579d2771370 2024-09-19T06:44:16.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f32d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d27665d0 0x5579d2762e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f32d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d27665d0 0x5579d2762e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49560/0 (socket says 172.21.15.37:49560) 2024-09-19T06:44:16.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f32d9d640 1 -- 172.21.15.37:0/135432206 learned_addr learned my addr 172.21.15.37:0/135432206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:16.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.792+0000 7f8f3259c640 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d277d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f3359e640 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f32d9d640 1 -- 172.21.15.37:0/135432206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 msgr2=0x5579d276d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f32d9d640 1 -- 172.21.15.37:0/135432206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 msgr2=0x5579d277d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d277d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f32d9d640 1 -- 172.21.15.37:0/135432206 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579d276cfd0 con 0x5579d27665d0 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f3259c640 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d277d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d27665d0 0x5579d2762e10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8f240edb30 tx=0x7f8f2402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb4d7134e5543959 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f18ff9640 1 -- 172.21.15.37:0/135432206 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f24049040 con 0x5579d27665d0 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f18ff9640 1 -- 172.21.15.37:0/135432206 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f240c52d0 con 0x5579d27665d0 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.793+0000 7f8f18ff9640 1 -- 172.21.15.37:0/135432206 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f240b8cd0 con 0x5579d27665d0 2024-09-19T06:44:16.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/135432206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d27665d0 msgr2=0x5579d2762e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d27665d0 0x5579d2762e10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8f240edb30 tx=0x7f8f2402a420 comp rx=0 tx=0).stop 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/135432206 shutdown_connections 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d2767b40 0x5579d277d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/135432206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d27665d0 0x5579d2762e10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/135432206 >> 172.21.15.37:0/135432206 conn(0x5579d272f310 msgr2=0x5579d2762700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/135432206 shutdown_connections 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/135432206 wait complete. 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.794+0000 7f8f3b4ec740 1 -- start start 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 1 Processor -- start 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d276b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d276b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341a9da0 con 0x5579d2767b40 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341ab210 con 0x5579d27665d0 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8f341aa470 con 0x5579d2771370 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3b4ec740 10 client.0 _ll_register_callbacks cb 0x7f8f2c00f400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d276b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f32d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d276b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d276b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49562/0 (socket says 172.21.15.37:49562) 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 -- 172.21.15.37:0/108775169 learned_addr learned my addr 172.21.15.37:0/108775169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f32d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d276b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36610/0 (socket says 172.21.15.37:36610) 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3359e640 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 -- 172.21.15.37:0/108775169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 msgr2=0x5579d276bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 -- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 msgr2=0x5579d276b0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d276b0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f3259c640 1 -- 172.21.15.37:0/108775169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579d27710c0 con 0x5579d2767b40 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.795+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d276b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f3259c640 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d276b620 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8f2801cfd0 tx=0x7f8f2801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8476ed9329abd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f16ff5640 1 -- 172.21.15.37:0/108775169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2802d060 con 0x5579d2767b40 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f16ff5640 1 -- 172.21.15.37:0/108775169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f8f28005450 con 0x5579d2767b40 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f240c52d0 con 0x5579d2767b40 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f16ff5640 1 -- 172.21.15.37:0/108775169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f8f28005650 con 0x5579d2767b40 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f3b4ec740 10 client.16161 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5579d27ebe70 con 0x5579d2767b40 2024-09-19T06:44:16.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f16ff5640 1 -- 172.21.15.37:0/108775169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8f240c52d0 con 0x5579d2767b40 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.796+0000 7f8f16ff5640 1 -- 172.21.15.37:0/108775169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f8f280035f0 con 0x5579d2767b40 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f3b4ec740 10 client.16161 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f3b4ec740 10 client.16161 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f3b4ec740 20 client.16161 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d27ec1a0 0x5579d27ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f3b4ec740 4 client.16161 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579d26d3d70 con 0x5579d27ec1a0 2024-09-19T06:44:16.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d27ec1a0 0x5579d27ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f32d9d640 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d27ec1a0 0x5579d27ee5c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b2cbad2e68819f88 server_cookie=7954487c33766e30 in_seq=0 out_seq=0 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f16ff5640 10 client.16161 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f16ff5640 1 -- 172.21.15.37:0/108775169 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579d26d3d70 con 0x5579d27ec1a0 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f16ff5640 10 client.16161 handle_command_reply: tid=0 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.797+0000 7f8f3b4ec740 1 client.16161 shutdown 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 msgr2=0x5579d276b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d276b620 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8f2801cfd0 tx=0x7f8f2801c890 comp rx=0 tx=0).stop 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 shutdown_connections 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5579d27ec1a0 0x5579d27ee5c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5579d2771370 0x5579d276bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5579d2767b40 0x5579d276b620 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 --2- 172.21.15.37:0/108775169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5579d27665d0 0x5579d276b0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 >> 172.21.15.37:0/108775169 conn(0x5579d272f310 msgr2=0x5579d2770080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:16.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f16ff5640 0 client.16161 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:16.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 shutdown_connections 2024-09-19T06:44:16.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 1 -- 172.21.15.37:0/108775169 wait complete. 2024-09-19T06:44:16.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f3b4ec740 20 client.16161 trim_cache size 0 max 16384 2024-09-19T06:44:16.805 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:44:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8f1c0a33b0 msgr2=0x7f8f1c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8f1c0a33b0 0x7f8f1c0a5870 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c005fd0 tx=0x7f8f2c011810 comp rx=0 tx=0).stop 2024-09-19T06:44:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 msgr2=0x7f8f341985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f341985c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8f2402a940 tx=0x7f8f24003b90 comp rx=0 tx=0).stop 2024-09-19T06:44:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 shutdown_connections 2024-09-19T06:44:16.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8f34108290 0x7f8f3419ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f34103740 0x7f8f34198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8f1c0a33b0 0x7f8f1c0a5870 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.798+0000 7f8f39113640 1 --2- 172.21.15.37:0/1560524077 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8f34102d90 0x7f8f341985c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.799+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 >> 172.21.15.37:0/1560524077 conn(0x7f8f340fc7d0 msgr2=0x7f8f34106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.799+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 shutdown_connections 2024-09-19T06:44:16.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:16.799+0000 7f8f39113640 1 -- 172.21.15.37:0/1560524077 wait complete. 2024-09-19T06:44:17.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 71206 9.16 MB/sec warmup 44 sec latency 38.993 ms 2024-09-19T06:44:17.641 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:44:17.911 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:18.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 73924 9.24 MB/sec warmup 45 sec latency 25.597 ms 2024-09-19T06:44:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 -- 172.21.15.37:0/1405649388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601082a0 msgr2=0x7f0d6010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 --2- 172.21.15.37:0/1405649388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601082a0 0x7f0d6010ec40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f0d58009cd0 tx=0x7f0d5802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 -- 172.21.15.37:0/1405649388 shutdown_connections 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 --2- 172.21.15.37:0/1405649388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601082a0 0x7f0d6010ec40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 --2- 172.21.15.37:0/1405649388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d601036e0 0x7f0d60107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 --2- 172.21.15.37:0/1405649388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d60102d30 0x7f0d60103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 -- 172.21.15.37:0/1405649388 >> 172.21.15.37:0/1405649388 conn(0x7f0d600fc7d0 msgr2=0x7f0d600febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.370+0000 7f0d65112640 1 -- 172.21.15.37:0/1405649388 shutdown_connections 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.371+0000 7f0d65112640 1 -- 172.21.15.37:0/1405649388 wait complete. 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.371+0000 7f0d65112640 1 Processor -- start 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.371+0000 7f0d65112640 1 -- start start 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d65112640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d60102d30 0x7f0d60198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d65112640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601036e0 0x7f0d60198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d65112640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d601082a0 0x7f0d6019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d65112640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601120a0 con 0x7f0d601036e0 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d65112640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60111f20 con 0x7f0d601082a0 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d65112640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d60112220 con 0x7f0d60102d30 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601036e0 0x7f0d60198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601036e0 0x7f0d60198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49592/0 (socket says 172.21.15.37:49592) 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 -- 172.21.15.37:0/433320610 learned_addr learned my addr 172.21.15.37:0/433320610 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d601082a0 0x7f0d6019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d60102d30 0x7f0d60198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 -- 172.21.15.37:0/433320610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d60102d30 msgr2=0x7f0d60198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d60102d30 0x7f0d60198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 -- 172.21.15.37:0/433320610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d601082a0 msgr2=0x7f0d6019ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d601082a0 0x7f0d6019ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 -- 172.21.15.37:0/433320610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d6019d4f0 con 0x7f0d601036e0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d601082a0 0x7f0d6019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d60102d30 0x7f0d60198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.372+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601036e0 0x7f0d60198a80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e9f0 tx=0x7f0d5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.373+0000 7f0d47fff640 1 -- 172.21.15.37:0/433320610 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5400ccc0 con 0x7f0d601036e0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.373+0000 7f0d47fff640 1 -- 172.21.15.37:0/433320610 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d54010910 con 0x7f0d601036e0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.373+0000 7f0d47fff640 1 -- 172.21.15.37:0/433320610 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d54018920 con 0x7f0d601036e0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.373+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d6019d7e0 con 0x7f0d601036e0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.373+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d601a50c0 con 0x7f0d601036e0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.375+0000 7f0d47fff640 1 -- 172.21.15.37:0/433320610 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0d54010ac0 con 0x7f0d601036e0 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.375+0000 7f0d674eb740 1 Processor -- start 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.375+0000 7f0d674eb740 1 -- start start 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0be090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0be5d0 0x56408f0bea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0c9370 0x56408f0cb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d674eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56408f0242f0 con 0x56408f0c9370 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d674eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56408efcbcf0 con 0x56408f0be5d0 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d674eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56408f0202b0 con 0x56408f0bfb40 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0be5d0 0x56408f0bea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0be5d0 0x56408f0bea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36644/0 (socket says 172.21.15.37:36644) 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5f59e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0be090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/3159254175 learned_addr learned my addr 172.21.15.37:0/3159254175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0c9370 0x56408f0cb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d47fff640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d480a33b0 0x7f0d480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/3159254175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 msgr2=0x56408f0be090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0be090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/3159254175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0c9370 msgr2=0x56408f0cb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0c9370 0x56408f0cb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/3159254175 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408efb03a0 con 0x56408f0be5d0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d480a33b0 0x7f0d480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d47fff640 1 -- 172.21.15.37:0/433320610 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0d540cfff0 con 0x7f0d601036e0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.376+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0be5d0 0x56408f0bea10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d60 tx=0x7f0d500117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b10d90921ee94301 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.377+0000 7f0d457fa640 1 -- 172.21.15.37:0/3159254175 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50012650 con 0x56408f0be5d0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.377+0000 7f0d457fa640 1 -- 172.21.15.37:0/3159254175 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d50009e50 con 0x56408f0be5d0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.377+0000 7f0d457fa640 1 -- 172.21.15.37:0/3159254175 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5000fc60 con 0x56408f0be5d0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.377+0000 7f0d674eb740 1 -- 172.21.15.37:0/3159254175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0be5d0 msgr2=0x56408f0bea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.377+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d480a33b0 0x7f0d480a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d58006010 tx=0x7f0d580095c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.377+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0be5d0 0x56408f0bea10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d60 tx=0x7f0d500117b0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- 172.21.15.37:0/3159254175 shutdown_connections 2024-09-19T06:44:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0c9370 0x56408f0cb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0be5d0 0x56408f0bea10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3159254175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0be090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- 172.21.15.37:0/3159254175 >> 172.21.15.37:0/3159254175 conn(0x56408f087310 msgr2=0x56408f0bac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- 172.21.15.37:0/3159254175 shutdown_connections 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- 172.21.15.37:0/3159254175 wait complete. 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- start start 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 Processor -- start 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56408f0cc070 con 0x56408f0be5d0 2024-09-19T06:44:18.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56408efdf060 con 0x56408f0c9370 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56408f0cc1f0 con 0x56408f0bfb40 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d674eb740 10 client.0 _ll_register_callbacks cb 0x56408ed328c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49616/0 (socket says 172.21.15.37:49616) 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.378+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/1090152116 learned_addr learned my addr 172.21.15.37:0/1090152116 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/1090152116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 msgr2=0x56408f0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 msgr2=0x56408f0c6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/1090152116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408ed953c0 con 0x56408f0be5d0 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0c8d10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d30 tx=0x7f0d5000fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17eaa80ffdd51ecf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:18.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d42ff5640 1 -- 172.21.15.37:0/1090152116 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50028030 con 0x56408f0be5d0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d50009e50 con 0x56408f0be5d0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d42ff5640 1 -- 172.21.15.37:0/1090152116 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0d5000fe40 con 0x56408f0be5d0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d42ff5640 1 -- 172.21.15.37:0/1090152116 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d50032050 con 0x56408f0be5d0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d674eb740 10 client.16173 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.379+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56408f0d7ac0 con 0x56408f0be5d0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d42ff5640 1 -- 172.21.15.37:0/1090152116 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d50009e50 con 0x56408f0be5d0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d42ff5640 1 -- 172.21.15.37:0/1090152116 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0d50012450 con 0x56408f0be5d0 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d674eb740 10 client.16173 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d674eb740 10 client.16173 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d674eb740 20 client.16173 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:18.402 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d674eb740 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f0d8f80 0x56408f0db370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d674eb740 4 client.16173 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56408f031860 con 0x56408f0d8f80 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f0d8f80 0x56408f0db370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f0d8f80 0x56408f0db370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d95a41c403c1be18 server_cookie=5c0533d6c3096623 in_seq=0 out_seq=0 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.380+0000 7f0d42ff5640 10 client.16173 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.381+0000 7f0d42ff5640 1 -- 172.21.15.37:0/1090152116 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56408f031860 con 0x56408f0d8f80 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.381+0000 7f0d42ff5640 10 client.16173 handle_command_reply: tid=0 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 client.16173 shutdown 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 msgr2=0x56408f0c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0c8d10 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d30 tx=0x7f0d5000fc50 comp rx=0 tx=0).stop 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 shutdown_connections 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f0d8f80 0x56408f0db370 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0bfb40 0x56408f0c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 --2- 172.21.15.37:0/1090152116 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0c8d10 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 >> 172.21.15.37:0/1090152116 conn(0x56408f087310 msgr2=0x56408f0c0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:18.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d42ff5640 0 client.16173 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:18.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 shutdown_connections 2024-09-19T06:44:18.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 1 -- 172.21.15.37:0/1090152116 wait complete. 2024-09-19T06:44:18.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.382+0000 7f0d674eb740 20 client.16173 trim_cache size 0 max 16384 2024-09-19T06:44:18.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 Processor -- start 2024-09-19T06:44:18.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 -- start start 2024-09-19T06:44:18.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0d5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0bfb40 0x56408f0b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0c9370 0x56408f0e0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601b0230 con 0x56408f0be5d0 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601adb00 con 0x56408f0bfb40 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d674eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601ab950 con 0x56408f0c9370 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0d5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0d5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49626/0 (socket says 172.21.15.37:49626) 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/589219175 learned_addr learned my addr 172.21.15.37:0/589219175 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0bfb40 0x56408f0b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0c9370 0x56408f0e0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/589219175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0c9370 msgr2=0x56408f0e0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0c9370 0x56408f0e0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/589219175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0bfb40 msgr2=0x56408f0b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0bfb40 0x56408f0b9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.396+0000 7f0d5ed9d640 1 -- 172.21.15.37:0/589219175 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56408f0c8830 con 0x56408f0be5d0 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0d5310 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d60 tx=0x7f0d500095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cba4977c1721438 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d44ff9640 1 -- 172.21.15.37:0/589219175 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50028040 con 0x56408f0be5d0 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d44ff9640 1 -- 172.21.15.37:0/589219175 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d5003b070 con 0x56408f0be5d0 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d44ff9640 1 -- 172.21.15.37:0/589219175 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d50009e50 con 0x56408f0be5d0 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d674eb740 1 -- 172.21.15.37:0/589219175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 msgr2=0x56408f0d5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d674eb740 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0d5310 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009d60 tx=0x7f0d500095b0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d674eb740 1 -- 172.21.15.37:0/589219175 shutdown_connections 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d674eb740 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0c9370 0x56408f0e0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d674eb740 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0bfb40 0x56408f0b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d674eb740 1 --2- 172.21.15.37:0/589219175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0be5d0 0x56408f0d5310 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.397+0000 7f0d674eb740 1 -- 172.21.15.37:0/589219175 >> 172.21.15.37:0/589219175 conn(0x56408f087310 msgr2=0x56408f0b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 -- 172.21.15.37:0/589219175 shutdown_connections 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 -- 172.21.15.37:0/589219175 wait complete. 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 -- start start 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 Processor -- start 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0be5d0 0x56408f0d1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0bfb40 0x56408f0d19c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601abfc0 con 0x56408f0bfb40 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601ad430 con 0x56408f0c9370 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d601ac690 con 0x56408f0be5d0 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0bfb40 0x56408f0d19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d674eb740 10 client.0 _ll_register_callbacks cb 0x7f0d540985f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:18.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d5e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0bfb40 0x56408f0d19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49630/0 (socket says 172.21.15.37:49630) 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.398+0000 7f0d5e59c640 1 -- 172.21.15.37:0/3750574778 learned_addr learned my addr 172.21.15.37:0/3750574778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5f59e640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0be5d0 0x56408f0d1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5e59c640 1 -- 172.21.15.37:0/3750574778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0be5d0 msgr2=0x56408f0d1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0be5d0 0x56408f0d1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5e59c640 1 -- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 msgr2=0x56408f0c25b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c25b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5e59c640 1 -- 172.21.15.37:0/3750574778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56408f0c2ba0 con 0x56408f0bfb40 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0be5d0 0x56408f0d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d5e59c640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0bfb40 0x56408f0d19c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0d5409bfd0 tx=0x7f0d540bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9de5735e11ce36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d42ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:18.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d42ff5640 1 -- 172.21.15.37:0/3750574778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d5409c2d0 con 0x56408f0bfb40 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d42ff5640 1 -- 172.21.15.37:0/3750574778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f0d54098940 con 0x56408f0bfb40 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d5003b070 con 0x56408f0bfb40 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d42ff5640 1 -- 172.21.15.37:0/3750574778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d540a0680 con 0x56408f0bfb40 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d674eb740 10 client.16179 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.399+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x56408f143370 con 0x56408f0bfb40 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d42ff5640 1 -- 172.21.15.37:0/3750574778 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d5003b070 con 0x56408f0bfb40 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d42ff5640 1 -- 172.21.15.37:0/3750574778 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f0d540a4b40 con 0x56408f0bfb40 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d674eb740 10 client.16179 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d674eb740 10 client.16179 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d674eb740 20 client.16179 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f143b00 0x56408f145f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d674eb740 4 client.16179 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:44:18.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56408f02bd70 con 0x56408f143b00 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f143b00 0x56408f145f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d5ed9d640 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f143b00 0x56408f145f20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8286499e7610b server_cookie=34e5b185d43d9c4b in_seq=0 out_seq=0 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.400+0000 7f0d42ff5640 10 client.16179 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.401+0000 7f0d42ff5640 1 -- 172.21.15.37:0/3750574778 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56408f02bd70 con 0x56408f143b00 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.401+0000 7f0d42ff5640 10 client.16179 handle_command_reply: tid=0 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 client.16179 shutdown 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0bfb40 msgr2=0x56408f0d19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0bfb40 0x56408f0d19c0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f0d5409bfd0 tx=0x7f0d540bc040 comp rx=0 tx=0).stop 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 shutdown_connections 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56408f143b00 0x56408f145f20 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d42ff5640 0 client.16179 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56408f0c9370 0x56408f0c25b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56408f0bfb40 0x56408f0d19c0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 --2- 172.21.15.37:0/3750574778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56408f0be5d0 0x56408f0d1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 >> 172.21.15.37:0/3750574778 conn(0x56408f087310 msgr2=0x56408f0c8040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:18.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 shutdown_connections 2024-09-19T06:44:18.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 1 -- 172.21.15.37:0/3750574778 wait complete. 2024-09-19T06:44:18.415 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d674eb740 20 client.16179 trim_cache size 0 max 16384 2024-09-19T06:44:18.415 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:44:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d480a33b0 msgr2=0x7f0d480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d65112640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d480a33b0 0x7f0d480a5870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f0d58006010 tx=0x7f0d580095c0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601036e0 msgr2=0x7f0d60198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.402+0000 7f0d65112640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601036e0 0x7f0d60198a80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f0d5400e9f0 tx=0x7f0d5400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 shutdown_connections 2024-09-19T06:44:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d601082a0 0x7f0d6019ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d480a33b0 0x7f0d480a5870 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d601036e0 0x7f0d60198a80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 --2- 172.21.15.37:0/433320610 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d60102d30 0x7f0d60198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:18.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 >> 172.21.15.37:0/433320610 conn(0x7f0d600fc7d0 msgr2=0x7f0d60105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:18.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 shutdown_connections 2024-09-19T06:44:18.417 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:18.403+0000 7f0d65112640 1 -- 172.21.15.37:0/433320610 wait complete. 2024-09-19T06:44:18.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:18 smithi151 ceph-mon[35236]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-19T06:44:18.673 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:18 smithi037 ceph-mon[31211]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-19T06:44:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:18 smithi096 ceph-mon[35004]: pgmap v78: 97 pgs: 97 active+clean; 35 MiB data, 462 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.9 MiB/s wr, 338 op/s 2024-09-19T06:44:19.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 76235 9.30 MB/sec warmup 46 sec latency 30.061 ms 2024-09-19T06:44:20.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 78989 9.41 MB/sec warmup 47 sec latency 11.621 ms 2024-09-19T06:44:20.413 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:44:20.413 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:44:20.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:20 smithi151 ceph-mon[35236]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 453 op/s 2024-09-19T06:44:20.679 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:20.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:20 smithi096 ceph-mon[35004]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 453 op/s 2024-09-19T06:44:20.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:20 smithi037 ceph-mon[31211]: pgmap v79: 97 pgs: 97 active+clean; 42 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.8 MiB/s wr, 453 op/s 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.143+0000 7f6773c81640 1 -- 172.21.15.37:0/3728743551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 msgr2=0x7f676c100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.143+0000 7f6773c81640 1 --2- 172.21.15.37:0/3728743551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c100cb0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6768009900 tx=0x7f676802a410 comp rx=0 tx=0).stop 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.143+0000 7f6773c81640 1 -- 172.21.15.37:0/3728743551 shutdown_connections 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.143+0000 7f6773c81640 1 --2- 172.21.15.37:0/3728743551 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f676c10e480 0x7f676c110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.143+0000 7f6773c81640 1 --2- 172.21.15.37:0/3728743551 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f676c101280 0x7f676c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.143+0000 7f6773c81640 1 --2- 172.21.15.37:0/3728743551 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c100cb0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.144+0000 7f6773c81640 1 -- 172.21.15.37:0/3728743551 >> 172.21.15.37:0/3728743551 conn(0x7f676c0fc7a0 msgr2=0x7f676c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.144+0000 7f6773c81640 1 -- 172.21.15.37:0/3728743551 shutdown_connections 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.144+0000 7f6773c81640 1 -- 172.21.15.37:0/3728743551 wait complete. 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.144+0000 7f6773c81640 1 Processor -- start 2024-09-19T06:44:21.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.144+0000 7f6773c81640 1 -- start start 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6773c81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6773c81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f676c101280 0x7f676c19abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6773c81640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f676c10e480 0x7f676c19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6773c81640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1141b0 con 0x7f676c1008d0 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6773c81640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f676c114030 con 0x7f676c101280 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6773c81640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f676c114330 con 0x7f676c10e480 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c19a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50250/0 (socket says 172.21.15.37:50250) 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f677121c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f676c101280 0x7f676c19abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 -- 172.21.15.37:0/185564493 learned_addr learned my addr 172.21.15.37:0/185564493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f677221e640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f676c10e480 0x7f676c19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 -- 172.21.15.37:0/185564493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f676c10e480 msgr2=0x7f676c19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f676c10e480 0x7f676c19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 -- 172.21.15.37:0/185564493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f676c101280 msgr2=0x7f676c19abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f676c101280 0x7f676c19abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 -- 172.21.15.37:0/185564493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f676c19f640 con 0x7f676c1008d0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.145+0000 7f6771a1d640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c19a690 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6768006010 tx=0x7f67680042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.146+0000 7f6762ffd640 1 -- 172.21.15.37:0/185564493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f676803f030 con 0x7f676c1008d0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.146+0000 7f6762ffd640 1 -- 172.21.15.37:0/185564493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6768003ce0 con 0x7f676c1008d0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.146+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f676c19f8d0 con 0x7f676c1008d0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.146+0000 7f6762ffd640 1 -- 172.21.15.37:0/185564493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768035070 con 0x7f676c1008d0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.146+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f676c1a7210 con 0x7f676c1008d0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.148+0000 7f6762ffd640 1 -- 172.21.15.37:0/185564493 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6768042030 con 0x7f676c1008d0 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.148+0000 7f677605a740 1 Processor -- start 2024-09-19T06:44:21.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.148+0000 7f677605a740 1 -- start start 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677605a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e51f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677605a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e5c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677605a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196e5f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677605a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559196db7280 con 0x559196e5cd40 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677605a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559196d5eb10 con 0x559196e51b60 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677605a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559196db3240 con 0x559196e52480 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f6762ffd640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67580a3430 0x7f67580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e51f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e51f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34160/0 (socket says 172.21.15.37:34160) 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677121c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196e5f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 -- 172.21.15.37:0/1070617821 learned_addr learned my addr 172.21.15.37:0/1070617821 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:21.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f6771a1d640 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e5c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67580a3430 0x7f67580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f6762ffd640 1 -- 172.21.15.37:0/185564493 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f67680f0230 con 0x7f676c1008d0 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 -- 172.21.15.37:0/1070617821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 msgr2=0x559196e5c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e5c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 -- 172.21.15.37:0/1070617821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 msgr2=0x559196e5f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196e5f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677221e640 1 -- 172.21.15.37:0/1070617821 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559196d6b1f0 con 0x559196e51b60 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f677121c640 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196e5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.149+0000 7f6771a1d640 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e5c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.150+0000 7f677221e640 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e51f40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67640094d0 tx=0x7f6764013eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bb04e9ad931c083 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.150+0000 7f6757fff640 1 -- 172.21.15.37:0/1070617821 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764022020 con 0x559196e51b60 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.150+0000 7f6757fff640 1 -- 172.21.15.37:0/1070617821 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6764012d00 con 0x559196e51b60 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.150+0000 7f6757fff640 1 -- 172.21.15.37:0/1070617821 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764010a50 con 0x559196e51b60 2024-09-19T06:44:21.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.150+0000 7f677221e640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67580a3430 0x7f67580a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6764023000 tx=0x7f6764021000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 -- 172.21.15.37:0/1070617821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 msgr2=0x559196e51f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e51f40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f67640094d0 tx=0x7f6764013eb0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 -- 172.21.15.37:0/1070617821 shutdown_connections 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196e5f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e5c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 --2- 172.21.15.37:0/1070617821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e51f40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 -- 172.21.15.37:0/1070617821 >> 172.21.15.37:0/1070617821 conn(0x559196e4e0f0 msgr2=0x559196e4e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 -- 172.21.15.37:0/1070617821 shutdown_connections 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 -- 172.21.15.37:0/1070617821 wait complete. 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 -- start start 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.151+0000 7f677605a740 1 Processor -- start 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677605a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e59bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677605a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e5a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677605a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e5a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677605a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559196e5f910 con 0x559196e52480 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677605a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559196d71f10 con 0x559196e51b60 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677605a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559196e5fa90 con 0x559196e5cd40 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677605a740 10 client.0 _ll_register_callbacks cb 0x559196d3aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677121c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e5a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677121c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e5a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50264/0 (socket says 172.21.15.37:50264) 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677121c640 1 -- 172.21.15.37:0/1335311487 learned_addr learned my addr 172.21.15.37:0/1335311487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f6771a1d640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e59bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677221e640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e5a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677221e640 1 -- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 msgr2=0x559196e59bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677221e640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e59bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677221e640 1 -- 172.21.15.37:0/1335311487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 msgr2=0x559196e5a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677221e640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e5a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677221e640 1 -- 172.21.15.37:0/1335311487 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559196b286a0 con 0x559196e5cd40 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677121c640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e5a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f677221e640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e5a650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f676400a010 tx=0x7f6764012f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6df4df8f9730c703 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.152+0000 7f67557fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f67557fa640 1 -- 172.21.15.37:0/1335311487 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6764022030 con 0x559196e5cd40 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f67557fa640 1 -- 172.21.15.37:0/1335311487 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f6764012820 con 0x559196e5cd40 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6764012d00 con 0x559196e5cd40 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f67557fa640 1 -- 172.21.15.37:0/1335311487 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6764012a60 con 0x559196e5cd40 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 10 client.25618 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559196e6b4c0 con 0x559196e5cd40 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f67557fa640 1 -- 172.21.15.37:0/1335311487 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6764012d00 con 0x559196e5cd40 2024-09-19T06:44:21.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f67557fa640 1 -- 172.21.15.37:0/1335311487 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f676402e020 con 0x559196e5cd40 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 10 client.25618 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 10 client.25618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 20 client.25618 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196e6c810 0x559196e6ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 4 client.25618 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559196dc47f0 con 0x559196e6c810 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.153+0000 7f6771a1d640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196e6c810 0x559196e6ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.154+0000 7f67557fa640 10 client.25618 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.154+0000 7f6771a1d640 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196e6c810 0x559196e6ec00 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7669d118e097fdef server_cookie=8191399dc4789079 in_seq=0 out_seq=0 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f67557fa640 1 -- 172.21.15.37:0/1335311487 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559196dc47f0 con 0x559196e6c810 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f67557fa640 10 client.25618 handle_command_reply: tid=0 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 client.25618 shutdown 2024-09-19T06:44:21.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 msgr2=0x559196e5a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e5a650 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f676400a010 tx=0x7f6764012f30 comp rx=0 tx=0).stop 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 shutdown_connections 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196e6c810 0x559196e6ec00 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e5a650 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e5a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f67557fa640 0 client.25618 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 --2- 172.21.15.37:0/1335311487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e59bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 >> 172.21.15.37:0/1335311487 conn(0x559196e4e0f0 msgr2=0x559196e53dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 shutdown_connections 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.155+0000 7f677605a740 1 -- 172.21.15.37:0/1335311487 wait complete. 2024-09-19T06:44:21.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.156+0000 7f677605a740 20 client.25618 trim_cache size 0 max 16384 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 Processor -- start 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 -- start start 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e5ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e74460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e749a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1b23a0 con 0x559196e52480 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1afc70 con 0x559196e51b60 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677605a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1adac0 con 0x559196e5cd40 2024-09-19T06:44:21.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e74460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e74460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50274/0 (socket says 172.21.15.37:50274) 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 -- 172.21.15.37:0/1834280199 learned_addr learned my addr 172.21.15.37:0/1834280199 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f6771a1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e5ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677221e640 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 -- 172.21.15.37:0/1834280199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 msgr2=0x559196e749a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e749a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 -- 172.21.15.37:0/1834280199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 msgr2=0x559196e5ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e5ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 -- 172.21.15.37:0/1834280199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559196e6f230 con 0x559196e52480 2024-09-19T06:44:21.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.174+0000 7f677121c640 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e74460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f675c00b990 tx=0x7f675c00bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56a71ae40f11d163 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:21.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f67577fe640 1 -- 172.21.15.37:0/1834280199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c003d50 con 0x559196e52480 2024-09-19T06:44:21.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f67577fe640 1 -- 172.21.15.37:0/1834280199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c020070 con 0x559196e52480 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f67577fe640 1 -- 172.21.15.37:0/1834280199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f675c01b660 con 0x559196e52480 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f677605a740 1 -- 172.21.15.37:0/1834280199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 msgr2=0x559196e74460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f677605a740 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e74460 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f675c00b990 tx=0x7f675c00bd90 comp rx=0 tx=0).stop 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f677605a740 1 -- 172.21.15.37:0/1834280199 shutdown_connections 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f677605a740 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e5cd40 0x559196e749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f677605a740 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e52480 0x559196e74460 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f677605a740 1 --2- 172.21.15.37:0/1834280199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e5ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.175+0000 7f677605a740 1 -- 172.21.15.37:0/1834280199 >> 172.21.15.37:0/1834280199 conn(0x559196e4e0f0 msgr2=0x559196e5b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 -- 172.21.15.37:0/1834280199 shutdown_connections 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 -- 172.21.15.37:0/1834280199 wait complete. 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 -- start start 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 Processor -- start 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e61180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e616c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196ed6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1ae130 con 0x559196e5cd40 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1af5a0 con 0x559196e51b60 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f676c1ae800 con 0x559196e52480 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.176+0000 7f677605a740 10 client.0 _ll_register_callbacks cb 0x7f675c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677221e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196ed6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e61180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677121c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677121c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e616c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:48868/0 (socket says 172.21.15.37:48868) 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677121c640 1 -- 172.21.15.37:0/2684685326 learned_addr learned my addr 172.21.15.37:0/2684685326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e61180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34182/0 (socket says 172.21.15.37:34182) 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677221e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196ed6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50286/0 (socket says 172.21.15.37:50286) 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 -- 172.21.15.37:0/2684685326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 msgr2=0x559196e616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 -- 172.21.15.37:0/2684685326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 msgr2=0x559196ed6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196ed6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 -- 172.21.15.37:0/2684685326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559196e4df20 con 0x559196e51b60 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677121c640 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677221e640 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196ed6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f6771a1d640 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e61180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67680ed920 tx=0x7f6768031a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75a057ca0d477f4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f67557fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f67557fa640 1 -- 172.21.15.37:0/2684685326 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6768049040 con 0x559196e51b60 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f67557fa640 1 -- 172.21.15.37:0/2684685326 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f67680b8c90 con 0x559196e51b60 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f675c020070 con 0x559196e51b60 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f67557fa640 1 -- 172.21.15.37:0/2684685326 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f67680b88c0 con 0x559196e51b60 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677605a740 10 client.25703 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.177+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x559196ed7ae0 con 0x559196e51b60 2024-09-19T06:44:21.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f67557fa640 1 -- 172.21.15.37:0/2684685326 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f675c020070 con 0x559196e51b60 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f67557fa640 1 -- 172.21.15.37:0/2684685326 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f67680edba0 con 0x559196e51b60 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f677605a740 10 client.25703 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f677605a740 10 client.25703 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f677605a740 20 client.25703 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f677605a740 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196ed8270 0x559196eda690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f677605a740 4 client.25703 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559196dbed00 con 0x559196ed8270 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.178+0000 7f677121c640 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196ed8270 0x559196eda690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.179+0000 7f677121c640 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196ed8270 0x559196eda690 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bd2117f09af7716 server_cookie=dacfdd429e52af4e in_seq=0 out_seq=0 2024-09-19T06:44:21.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.179+0000 7f67557fa640 10 client.25703 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:21.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.180+0000 7f67557fa640 1 -- 172.21.15.37:0/2684685326 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559196dbed00 con 0x559196ed8270 2024-09-19T06:44:21.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.180+0000 7f67557fa640 10 client.25703 handle_command_reply: tid=0 2024-09-19T06:44:21.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.180+0000 7f677605a740 1 client.25703 shutdown 2024-09-19T06:44:21.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 msgr2=0x559196e61180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e61180 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f67680ed920 tx=0x7f6768031a60 comp rx=0 tx=0).stop 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 shutdown_connections 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559196ed8270 0x559196eda690 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559196e5cd40 0x559196ed6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559196e52480 0x559196e616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f67557fa640 0 client.25703 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 --2- 172.21.15.37:0/2684685326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559196e51b60 0x559196e61180 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 >> 172.21.15.37:0/2684685326 conn(0x559196e4e0f0 msgr2=0x559196e5a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 shutdown_connections 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 1 -- 172.21.15.37:0/2684685326 wait complete. 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f677605a740 20 client.25703 trim_cache size 0 max 16384 2024-09-19T06:44:21.186 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:44:21.187 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:44:21.187 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "d3f528d4-d00b-4fd9-8e78-84bc716f53a3", 2024-09-19T06:44:21.187 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:44:21.187 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:44:21.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67580a3430 msgr2=0x7f67580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f6773c81640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67580a3430 0x7f67580a58f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6764023000 tx=0x7f6764021000 comp rx=0 tx=0).stop 2024-09-19T06:44:21.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 msgr2=0x7f676c19a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.181+0000 7f6773c81640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c19a690 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f6768006010 tx=0x7f67680042f0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 shutdown_connections 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f676c10e480 0x7f676c19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f676c101280 0x7f676c19abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f67580a3430 0x7f67580a58f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 --2- 172.21.15.37:0/185564493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f676c1008d0 0x7f676c19a690 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 >> 172.21.15.37:0/185564493 conn(0x7f676c0fc7a0 msgr2=0x7f676c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 shutdown_connections 2024-09-19T06:44:21.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:21.182+0000 7f6773c81640 1 -- 172.21.15.37:0/185564493 wait complete. 2024-09-19T06:44:21.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 81993 9.60 MB/sec warmup 48 sec latency 30.515 ms 2024-09-19T06:44:22.158 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:44:22.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 84869 9.68 MB/sec warmup 49 sec latency 31.161 ms 2024-09-19T06:44:22.424 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:22 smithi037 ceph-mon[31211]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.8 MiB/s wr, 333 op/s 2024-09-19T06:44:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:22 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:44:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:22 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:22 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:44:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:22 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:44:22.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:22 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:22 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:22 smithi151 ceph-mon[35236]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.8 MiB/s wr, 333 op/s 2024-09-19T06:44:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:22 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:44:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:22 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:22 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:44:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:22 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:44:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:22 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:22 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:22 smithi096 ceph-mon[35004]: pgmap v80: 97 pgs: 97 active+clean; 42 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.8 MiB/s wr, 333 op/s 2024-09-19T06:44:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:22 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:44:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:22 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:22 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:44:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:22 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:44:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:22 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:22 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 -- 172.21.15.37:0/4080073238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98381082a0 msgr2=0x7f983810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 --2- 172.21.15.37:0/4080073238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98381082a0 0x7f983810ec40 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f9828009930 tx=0x7f982802a440 comp rx=0 tx=0).stop 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 -- 172.21.15.37:0/4080073238 shutdown_connections 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 --2- 172.21.15.37:0/4080073238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f98381082a0 0x7f983810ec40 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 --2- 172.21.15.37:0/4080073238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 0x7f9838107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 --2- 172.21.15.37:0/4080073238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9838102d30 0x7f9838103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 -- 172.21.15.37:0/4080073238 >> 172.21.15.37:0/4080073238 conn(0x7f98380fc7d0 msgr2=0x7f98380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:22.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 -- 172.21.15.37:0/4080073238 shutdown_connections 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.878+0000 7f983d4e8640 1 -- 172.21.15.37:0/4080073238 wait complete. 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 Processor -- start 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 -- start start 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102d30 0x7f9838198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 0x7f9838198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98381082a0 0x7f983819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98381120a0 con 0x7f9838102d30 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9838111f20 con 0x7f98381036e0 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f983d4e8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9838112220 con 0x7f98381082a0 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102d30 0x7f9838198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f98367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 0x7f9838198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102d30 0x7f9838198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50294/0 (socket says 172.21.15.37:50294) 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f98367fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 0x7f9838198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34190/0 (socket says 172.21.15.37:34190) 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f9836ffd640 1 -- 172.21.15.37:0/668955164 learned_addr learned my addr 172.21.15.37:0/668955164 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.879+0000 7f98377fe640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98381082a0 0x7f983819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f98367fc640 1 -- 172.21.15.37:0/668955164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98381082a0 msgr2=0x7f983819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f98367fc640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98381082a0 0x7f983819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f98367fc640 1 -- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102d30 msgr2=0x7f9838198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f98367fc640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102d30 0x7f9838198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f98367fc640 1 -- 172.21.15.37:0/668955164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f983819d4e0 con 0x7f98381036e0 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f9836ffd640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102d30 0x7f9838198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f98367fc640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 0x7f9838198a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f98380697d0 tx=0x7f982c00c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f981ffff640 1 -- 172.21.15.37:0/668955164 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c003b70 con 0x7f98381036e0 2024-09-19T06:44:22.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f983819d7d0 con 0x7f98381036e0 2024-09-19T06:44:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f981ffff640 1 -- 172.21.15.37:0/668955164 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f982c003d10 con 0x7f98381036e0 2024-09-19T06:44:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f981ffff640 1 -- 172.21.15.37:0/668955164 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c0109a0 con 0x7f98381036e0 2024-09-19T06:44:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.880+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98381a50b0 con 0x7f98381036e0 2024-09-19T06:44:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.883+0000 7f981ffff640 1 -- 172.21.15.37:0/668955164 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f982c003ec0 con 0x7f98381036e0 2024-09-19T06:44:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.883+0000 7f983f8c1740 1 Processor -- start 2024-09-19T06:44:22.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f983f8c1740 1 -- start start 2024-09-19T06:44:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f981ffff640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f98200a33b0 0x7f98200a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75b7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75b7680 0x5583b75b9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f983f8c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5583b75122e0 con 0x5583b75ac4a0 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f983f8c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583b74b9ce0 con 0x5583b75acdc0 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f983f8c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583b750e2a0 con 0x5583b75b7680 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f981ffff640 1 -- 172.21.15.37:0/668955164 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f982c0d0090 con 0x7f98381036e0 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f98377fe640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f98200a33b0 0x7f98200a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75ac880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50298/0 (socket says 172.21.15.37:50298) 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 -- 172.21.15.37:0/2268368167 learned_addr learned my addr 172.21.15.37:0/2268368167 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f98367fc640 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f98377fe640 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75b7680 0x5583b75b9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 -- 172.21.15.37:0/2268368167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75b7680 msgr2=0x5583b75b9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75b7680 0x5583b75b9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 -- 172.21.15.37:0/2268368167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 msgr2=0x5583b75b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f9836ffd640 1 -- 172.21.15.37:0/2268368167 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b749e440 con 0x5583b75ac4a0 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.884+0000 7f98367fc640 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.885+0000 7f98377fe640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f98200a33b0 0x7f98200a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9828006010 tx=0x7f9828005be0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.885+0000 7f9836ffd640 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75ac880 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9830009fd0 tx=0x7f9830011ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bed9fbdcb7603d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.885+0000 7f981d7fa640 1 -- 172.21.15.37:0/2268368167 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983001e070 con 0x5583b75ac4a0 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.885+0000 7f981d7fa640 1 -- 172.21.15.37:0/2268368167 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983000fbd0 con 0x5583b75ac4a0 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.885+0000 7f981d7fa640 1 -- 172.21.15.37:0/2268368167 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830019d20 con 0x5583b75ac4a0 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.885+0000 7f983f8c1740 1 -- 172.21.15.37:0/2268368167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 msgr2=0x5583b75ac880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.885+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75ac880 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f9830009fd0 tx=0x7f9830011ae0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- 172.21.15.37:0/2268368167 shutdown_connections 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75b7680 0x5583b75b9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2268368167 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75ac880 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- 172.21.15.37:0/2268368167 >> 172.21.15.37:0/2268368167 conn(0x5583b75a8a30 msgr2=0x5583b75a8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- 172.21.15.37:0/2268368167 shutdown_connections 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- 172.21.15.37:0/2268368167 wait complete. 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- start start 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 Processor -- start 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75ac4a0 0x5583b75b6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75acdc0 0x5583b75b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75b7680 0x5583b75b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5583b75be660 con 0x5583b75acdc0 2024-09-19T06:44:22.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583b74cd050 con 0x5583b75b7680 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5583b75be7e0 con 0x5583b75ac4a0 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.886+0000 7f983f8c1740 10 client.0 _ll_register_callbacks cb 0x5583b750d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75acdc0 0x5583b75b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75b7680 0x5583b75b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75acdc0 0x5583b75b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50308/0 (socket says 172.21.15.37:50308) 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98377fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75b7680 0x5583b75b4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34196/0 (socket says 172.21.15.37:34196) 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 -- 172.21.15.37:0/2468231831 learned_addr learned my addr 172.21.15.37:0/2468231831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f9836ffd640 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75ac4a0 0x5583b75b6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 -- 172.21.15.37:0/2468231831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75ac4a0 msgr2=0x5583b75b6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75ac4a0 0x5583b75b6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 -- 172.21.15.37:0/2468231831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75b7680 msgr2=0x5583b75b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75b7680 0x5583b75b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 -- 172.21.15.37:0/2468231831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b72833c0 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f98367fc640 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75acdc0 0x5583b75b4200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f982c09c300 tx=0x7f982c09bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7abb05f72e5aa4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f981aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f981aff5640 1 -- 172.21.15.37:0/2468231831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c0a0720 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f981aff5640 1 -- 172.21.15.37:0/2468231831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f982c05f1f0 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.887+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983000fbd0 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f981aff5640 1 -- 172.21.15.37:0/2468231831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f982c0a4fe0 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 10 client.16209 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5583b75c9fb0 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f981aff5640 1 -- 172.21.15.37:0/2468231831 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983000fbd0 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f981aff5640 1 -- 172.21.15.37:0/2468231831 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f982c01f020 con 0x5583b75acdc0 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 10 client.16209 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:22.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 20 client.16209 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75cb3c0 0x5583b75cd820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 4 client.16209 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583b751f850 con 0x5583b75cb3c0 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.888+0000 7f9836ffd640 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75cb3c0 0x5583b75cd820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.889+0000 7f9836ffd640 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75cb3c0 0x5583b75cd820 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48976fb27a22fb server_cookie=f6acd4a03ec7670f in_seq=0 out_seq=0 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.889+0000 7f981aff5640 10 client.16209 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f981aff5640 1 -- 172.21.15.37:0/2468231831 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583b751f850 con 0x5583b75cb3c0 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f981aff5640 10 client.16209 handle_command_reply: tid=0 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 client.16209 shutdown 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75acdc0 msgr2=0x5583b75b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75acdc0 0x5583b75b4200 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f982c09c300 tx=0x7f982c09bcd0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 shutdown_connections 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75cb3c0 0x5583b75cd820 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75b7680 0x5583b75b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75acdc0 0x5583b75b4200 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 --2- 172.21.15.37:0/2468231831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75ac4a0 0x5583b75b6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 >> 172.21.15.37:0/2468231831 conn(0x5583b75a8a30 msgr2=0x5583b75ae760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f981aff5640 0 client.16209 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 shutdown_connections 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 1 -- 172.21.15.37:0/2468231831 wait complete. 2024-09-19T06:44:22.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.890+0000 7f983f8c1740 20 client.16209 trim_cache size 0 max 16384 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 Processor -- start 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 -- start start 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75a8a30 0x5583b75c78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75acdc0 0x5583b75d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98381adf50 con 0x5583b75ac4a0 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98381ab820 con 0x5583b75a8a30 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f983f8c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98381a9670 con 0x5583b75acdc0 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.908+0000 7f98367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75a8a30 0x5583b75c78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75a8a30 0x5583b75c78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34200/0 (socket says 172.21.15.37:34200) 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 -- 172.21.15.37:0/3919386684 learned_addr learned my addr 172.21.15.37:0/3919386684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f98367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75d30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50316/0 (socket says 172.21.15.37:50316) 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f98377fe640 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75acdc0 0x5583b75d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 -- 172.21.15.37:0/3919386684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75acdc0 msgr2=0x5583b75d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75acdc0 0x5583b75d3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 -- 172.21.15.37:0/3919386684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 msgr2=0x5583b75d30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75d30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 -- 172.21.15.37:0/3919386684 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583b75a8860 con 0x5583b75a8a30 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f98367fc640 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f9836ffd640 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75a8a30 0x5583b75c78c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98300096c0 tx=0x7f9830005d70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8fc39010a16c83a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f981cff9640 1 -- 172.21.15.37:0/3919386684 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f983001e040 con 0x5583b75a8a30 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f981cff9640 1 -- 172.21.15.37:0/3919386684 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983000f6f0 con 0x5583b75a8a30 2024-09-19T06:44:22.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.909+0000 7f981cff9640 1 -- 172.21.15.37:0/3919386684 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830003850 con 0x5583b75a8a30 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 -- 172.21.15.37:0/3919386684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75a8a30 msgr2=0x5583b75c78c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75a8a30 0x5583b75c78c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f98300096c0 tx=0x7f9830005d70 comp rx=0 tx=0).stop 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 -- 172.21.15.37:0/3919386684 shutdown_connections 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75acdc0 0x5583b75d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 --2- 172.21.15.37:0/3919386684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75a8a30 0x5583b75c78c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 -- 172.21.15.37:0/3919386684 >> 172.21.15.37:0/3919386684 conn(0x5583b75b48a0 msgr2=0x5583b75b2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 -- 172.21.15.37:0/3919386684 shutdown_connections 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 -- 172.21.15.37:0/3919386684 wait complete. 2024-09-19T06:44:22.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 -- start start 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.910+0000 7f983f8c1740 1 Processor -- start 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75a8a30 0x5583b75c3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f983f8c1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75c0a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f983f8c1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f98381a9ce0 con 0x5583b75ac4a0 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f983f8c1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98381ab150 con 0x5583b75acdc0 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f983f8c1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f98381aa3b0 con 0x5583b75a8a30 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f983f8c1740 10 client.0 _ll_register_callbacks cb 0x7f982c09c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75a8a30 0x5583b75c3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f9836ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75a8a30 0x5583b75c3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:48920/0 (socket says 172.21.15.37:48920) 2024-09-19T06:44:22.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50324/0 (socket says 172.21.15.37:50324) 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f9836ffd640 1 -- 172.21.15.37:0/4040502567 learned_addr learned my addr 172.21.15.37:0/4040502567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98377fe640 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75c0a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 -- 172.21.15.37:0/4040502567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75a8a30 msgr2=0x5583b75c3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75a8a30 0x5583b75c3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 -- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 msgr2=0x5583b75c0a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75c0a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 -- 172.21.15.37:0/4040502567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583b75c0f40 con 0x5583b75ac4a0 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f9836ffd640 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75a8a30 0x5583b75c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f98367fc640 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75c3f70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f982c09c070 tx=0x7f982c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b30a4d360688f3d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.911+0000 7f981aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f981aff5640 1 -- 172.21.15.37:0/4040502567 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c0ba060 con 0x5583b75ac4a0 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f981aff5640 1 -- 172.21.15.37:0/4040502567 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f982c0980c0 con 0x5583b75ac4a0 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f983000f6f0 con 0x5583b75ac4a0 2024-09-19T06:44:22.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f981aff5640 1 -- 172.21.15.37:0/4040502567 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f982c098610 con 0x5583b75ac4a0 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 10 client.16215 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x5583b7636860 con 0x5583b75ac4a0 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f981aff5640 1 -- 172.21.15.37:0/4040502567 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f983000f6f0 con 0x5583b75ac4a0 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f981aff5640 1 -- 172.21.15.37:0/4040502567 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f982c01f070 con 0x5583b75ac4a0 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 10 client.16215 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 20 client.16215 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75b7680 0x5583b7638e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 4 client.16215 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583b7519d60 con 0x5583b75b7680 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.912+0000 7f9836ffd640 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75b7680 0x5583b7638e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.913+0000 7f9836ffd640 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75b7680 0x5583b7638e90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9839e25d9f93f0dd server_cookie=b46d60611ff2454d in_seq=0 out_seq=0 2024-09-19T06:44:22.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.913+0000 7f981aff5640 10 client.16215 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.913+0000 7f981aff5640 1 -- 172.21.15.37:0/4040502567 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5583b7519d60 con 0x5583b75b7680 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.913+0000 7f981aff5640 10 client.16215 handle_command_reply: tid=0 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.913+0000 7f983f8c1740 1 client.16215 shutdown 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 msgr2=0x5583b75c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75c3f70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f982c09c070 tx=0x7f982c010430 comp rx=0 tx=0).stop 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 shutdown_connections 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5583b75b7680 0x5583b7638e90 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583b75acdc0 0x5583b75c0a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5583b75ac4a0 0x5583b75c3f70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 --2- 172.21.15.37:0/4040502567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5583b75a8a30 0x5583b75c3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 >> 172.21.15.37:0/4040502567 conn(0x5583b75b48a0 msgr2=0x5583b75b5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f981aff5640 0 client.16215 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 shutdown_connections 2024-09-19T06:44:22.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 1 -- 172.21.15.37:0/4040502567 wait complete. 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983f8c1740 20 client.16215 trim_cache size 0 max 16384 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: "d3f528d4-d00b-4fd9-8e78-84bc716f53a3": { 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "d3f528d4-d00b-4fd9-8e78-84bc716f53a3", 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:44:22.921 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:44:22.922 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:44:22.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f98200a33b0 msgr2=0x7f98200a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f98200a33b0 0x7f98200a5870 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9828006010 tx=0x7f9828005be0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 msgr2=0x7f9838198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 0x7f9838198a70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f98380697d0 tx=0x7f982c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 shutdown_connections 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f98381082a0 0x7f983819ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98381036e0 0x7f9838198a70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f98200a33b0 0x7f98200a5870 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 --2- 172.21.15.37:0/668955164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9838102d30 0x7f9838198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.914+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 >> 172.21.15.37:0/668955164 conn(0x7f98380fc7d0 msgr2=0x7f9838105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.915+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 shutdown_connections 2024-09-19T06:44:22.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:22.915+0000 7f983d4e8640 1 -- 172.21.15.37:0/668955164 wait complete. 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.016+0000 7f07dda15640 1 -- 172.21.15.37:0/790267733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 msgr2=0x7f07d810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.016+0000 7f07dda15640 1 --2- 172.21.15.37:0/790267733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 0x7f07d810d880 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009c10 tx=0x7f07d002a6e0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.016+0000 7f07dda15640 1 -- 172.21.15.37:0/790267733 shutdown_connections 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.016+0000 7f07dda15640 1 --2- 172.21.15.37:0/790267733 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 0x7f07d810d880 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.016+0000 7f07dda15640 1 --2- 172.21.15.37:0/790267733 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 0x7f07d8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.017+0000 7f07dda15640 1 --2- 172.21.15.37:0/790267733 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d81069c0 0x7f07d8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.017+0000 7f07dda15640 1 -- 172.21.15.37:0/790267733 >> 172.21.15.37:0/790267733 conn(0x7f07d8077990 msgr2=0x7f07d80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.017+0000 7f07dda15640 1 -- 172.21.15.37:0/790267733 shutdown_connections 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.017+0000 7f07dda15640 1 -- 172.21.15.37:0/790267733 wait complete. 2024-09-19T06:44:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.017+0000 7f07dda15640 1 Processor -- start 2024-09-19T06:44:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.017+0000 7f07dda15640 1 -- start start 2024-09-19T06:44:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07dda15640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 0x7f07d8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07dda15640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 0x7f07d81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07dda15640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d81069c0 0x7f07d819cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07dda15640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07d8110db0 con 0x7f07d8101350 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07dda15640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f07d8110c30 con 0x7f07d81009b0 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07dda15640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07d8110f30 con 0x7f07d81069c0 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 0x7f07d8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 0x7f07d8198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34230/0 (socket says 172.21.15.37:34230) 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 -- 172.21.15.37:0/1079309937 learned_addr learned my addr 172.21.15.37:0/1079309937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d77fe640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d81069c0 0x7f07d819cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 -- 172.21.15.37:0/1079309937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d81069c0 msgr2=0x7f07d819cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d81069c0 0x7f07d819cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d67fc640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 0x7f07d81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 -- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 msgr2=0x7f07d81987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 0x7f07d81987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 -- 172.21.15.37:0/1079309937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07d819d280 con 0x7f07d81009b0 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d67fc640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 0x7f07d81987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.018+0000 7f07d6ffd640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 0x7f07d8198290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c920 tx=0x7f07c800cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.019+0000 7f07dca13640 1 -- 172.21.15.37:0/1079309937 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8007c70 con 0x7f07d81009b0 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.019+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07d819d570 con 0x7f07d81009b0 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.019+0000 7f07dca13640 1 -- 172.21.15.37:0/1079309937 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07c8007e10 con 0x7f07d81009b0 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.019+0000 7f07dca13640 1 -- 172.21.15.37:0/1079309937 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07c8005260 con 0x7f07d81009b0 2024-09-19T06:44:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.019+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07d81a4e00 con 0x7f07d81009b0 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 Processor -- start 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07dca13640 1 -- 172.21.15.37:0/1079309937 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f07c8005400 con 0x7f07d81009b0 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 -- start start 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f983ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f98e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f9910a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e18f8e82f0 con 0x55e18f983af0 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e18f890270 con 0x55e18f98ecb0 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07e088a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e18f8e42b0 con 0x55e18f984410 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.021+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f983ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f983ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50350/0 (socket says 172.21.15.37:50350) 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 -- 172.21.15.37:0/1482262420 learned_addr learned my addr 172.21.15.37:0/1482262420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d67fc640 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f9910a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d6ffd640 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f98e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 -- 172.21.15.37:0/1482262420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 msgr2=0x55e18f98e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f98e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 -- 172.21.15.37:0/1482262420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 msgr2=0x55e18f9910a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f9910a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 -- 172.21.15.37:0/1482262420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18f88f340 con 0x55e18f983af0 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07dca13640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07c00a3460 0x7f07c00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d77fe640 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f983ed0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009be0 tx=0x7f07d0030ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8daf200192c0a9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07d6ffd640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07c00a3460 0x7f07c00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07dca13640 1 -- 172.21.15.37:0/1079309937 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f07c80cfdf0 con 0x7f07d81009b0 2024-09-19T06:44:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.022+0000 7f07bdffb640 1 -- 172.21.15.37:0/1482262420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0042030 con 0x55e18f983af0 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07bdffb640 1 -- 172.21.15.37:0/1482262420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d0037040 con 0x55e18f983af0 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07bdffb640 1 -- 172.21.15.37:0/1482262420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0036070 con 0x55e18f983af0 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 -- 172.21.15.37:0/1482262420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 msgr2=0x55e18f983ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07d6ffd640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07c00a3460 0x7f07c00a5920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07c80cd5f0 tx=0x7f07c80bc040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f983ed0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009be0 tx=0x7f07d0030ac0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 -- 172.21.15.37:0/1482262420 shutdown_connections 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f9910a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f98e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 --2- 172.21.15.37:0/1482262420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f983ed0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 -- 172.21.15.37:0/1482262420 >> 172.21.15.37:0/1482262420 conn(0x55e18f980060 msgr2=0x55e18f980440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 -- 172.21.15.37:0/1482262420 shutdown_connections 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.023+0000 7f07e088a740 1 -- 172.21.15.37:0/1482262420 wait complete. 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 -- start start 2024-09-19T06:44:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 Processor -- start 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f993170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f984410 0x55e18f9936b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f98ecb0 0x55e18f997810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e18f991880 con 0x55e18f984410 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e18f871290 con 0x55e18f983af0 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e18f991a00 con 0x55e18f98ecb0 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07e088a740 10 client.0 _ll_register_callbacks cb 0x55e18f8a2140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f984410 0x55e18f9936b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f993170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07d67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f984410 0x55e18f9936b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50362/0 (socket says 172.21.15.37:50362) 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07d67fc640 1 -- 172.21.15.37:0/231090352 learned_addr learned my addr 172.21.15.37:0/231090352 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f993170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34250/0 (socket says 172.21.15.37:34250) 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.024+0000 7f07d77fe640 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f98ecb0 0x55e18f997810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07d67fc640 1 -- 172.21.15.37:0/231090352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f98ecb0 msgr2=0x55e18f997810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07d67fc640 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f98ecb0 0x55e18f997810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07d67fc640 1 -- 172.21.15.37:0/231090352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 msgr2=0x55e18f993170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07d67fc640 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f993170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07d67fc640 1 -- 172.21.15.37:0/231090352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18f5fb210 con 0x55e18f984410 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07d67fc640 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f984410 0x55e18f9936b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099a0 tx=0x7f07cc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544b6fa6f7f21856 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07b6ffd640 1 -- 172.21.15.37:0/231090352 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc01b440 con 0x55e18f984410 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07b6ffd640 1 -- 172.21.15.37:0/231090352 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f07cc012440 con 0x55e18f984410 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d0037040 con 0x55e18f984410 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07b6ffd640 1 -- 172.21.15.37:0/231090352 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07cc012640 con 0x55e18f984410 2024-09-19T06:44:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07e088a740 10 client.16224 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.025+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e18f6c7620 con 0x55e18f984410 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07b6ffd640 1 -- 172.21.15.37:0/231090352 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d0037040 con 0x55e18f984410 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07b6ffd640 1 -- 172.21.15.37:0/231090352 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f07cc019020 con 0x55e18f984410 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07e088a740 10 client.16224 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07e088a740 10 client.16224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07e088a740 20 client.16224 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07e088a740 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18f9a3020 0x55e18f9a5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07e088a740 4 client.16224 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e18f8f5a20 con 0x55e18f9a3020 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07d77fe640 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18f9a3020 0x55e18f9a5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07d77fe640 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18f9a3020 0x55e18f9a5450 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92a812666636777 server_cookie=be0281e8b5369872 in_seq=0 out_seq=0 2024-09-19T06:44:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.026+0000 7f07b6ffd640 10 client.16224 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.027+0000 7f07b6ffd640 1 -- 172.21.15.37:0/231090352 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e18f8f5a20 con 0x55e18f9a3020 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.027+0000 7f07b6ffd640 10 client.16224 handle_command_reply: tid=0 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.027+0000 7f07e088a740 1 client.16224 shutdown 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f984410 msgr2=0x55e18f9936b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f984410 0x55e18f9936b0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f07cc0099a0 tx=0x7f07cc009e60 comp rx=0 tx=0).stop 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 shutdown_connections 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18f9a3020 0x55e18f9a5450 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f98ecb0 0x55e18f997810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f984410 0x55e18f9936b0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07b6ffd640 0 client.16224 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 --2- 172.21.15.37:0/231090352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f993170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 >> 172.21.15.37:0/231090352 conn(0x55e18f980060 msgr2=0x55e18f985d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 shutdown_connections 2024-09-19T06:44:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 1 -- 172.21.15.37:0/231090352 wait complete. 2024-09-19T06:44:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.028+0000 7f07e088a740 20 client.16224 trim_cache size 0 max 16384 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 Processor -- start 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 -- start start 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f997800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f993650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f98ecb0 0x55e18f9aad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81ade50 con 0x55e18f98ecb0 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81ab720 con 0x55e18f983af0 2024-09-19T06:44:23.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07e088a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81a9570 con 0x55e18f984410 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f997800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f997800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34254/0 (socket says 172.21.15.37:34254) 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 -- 172.21.15.37:0/265253575 learned_addr learned my addr 172.21.15.37:0/265253575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d67fc640 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f993650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f98ecb0 0x55e18f9aad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 -- 172.21.15.37:0/265253575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 msgr2=0x55e18f993650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f993650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 -- 172.21.15.37:0/265253575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 msgr2=0x55e18f997800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f997800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 -- 172.21.15.37:0/265253575 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18f981110 con 0x55e18f98ecb0 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d67fc640 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f993650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.043+0000 7f07d77fe640 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f98ecb0 0x55e18f9aad20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009560 tx=0x7f07d002a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8514e61c29f4686 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:23.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07bd7fa640 1 -- 172.21.15.37:0/265253575 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d005a070 con 0x55e18f98ecb0 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07bd7fa640 1 -- 172.21.15.37:0/265253575 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d0004040 con 0x55e18f98ecb0 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07bd7fa640 1 -- 172.21.15.37:0/265253575 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0036040 con 0x55e18f98ecb0 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 -- 172.21.15.37:0/265253575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f98ecb0 msgr2=0x55e18f9aad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f98ecb0 0x55e18f9aad20 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009560 tx=0x7f07d002a6e0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 -- 172.21.15.37:0/265253575 shutdown_connections 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f98ecb0 0x55e18f9aad20 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f993650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 --2- 172.21.15.37:0/265253575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f983af0 0x55e18f997800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 -- 172.21.15.37:0/265253575 >> 172.21.15.37:0/265253575 conn(0x55e18f980060 msgr2=0x55e18f992f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 -- 172.21.15.37:0/265253575 shutdown_connections 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.044+0000 7f07e088a740 1 -- 172.21.15.37:0/265253575 wait complete. 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 -- start start 2024-09-19T06:44:23.051 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 Processor -- start 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f99a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f99aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f99ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81a9be0 con 0x55e18f983af0 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81ab050 con 0x55e18f98ecb0 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f07d81aa2b0 con 0x55e18f984410 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07e088a740 10 client.0 _ll_register_callbacks cb 0x7f07cc004080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f99a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07d77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f99a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50386/0 (socket says 172.21.15.37:50386) 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07d77fe640 1 -- 172.21.15.37:0/2626872446 learned_addr learned my addr 172.21.15.37:0/2626872446 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07d67fc640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f99aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.045+0000 7f07d77fe640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f99ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07d67fc640 1 -- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 msgr2=0x55e18f99ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07d67fc640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f99ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07d67fc640 1 -- 172.21.15.37:0/2626872446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 msgr2=0x55e18f99a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07d67fc640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f99a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07d67fc640 1 -- 172.21.15.37:0/2626872446 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18f99f270 con 0x55e18f984410 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07d77fe640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f99a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07d67fc640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f99aad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f07cc01bd20 tx=0x7f07cc009470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5ec158fc54a67f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07b6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07b6ffd640 1 -- 172.21.15.37:0/2626872446 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07cc02a070 con 0x55e18f984410 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f07d0004040 con 0x55e18f984410 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07b6ffd640 1 -- 172.21.15.37:0/2626872446 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8527+0+0 (secure 0 0 0) 0x7f07cc005240 con 0x55e18f984410 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07b6ffd640 1 -- 172.21.15.37:0/2626872446 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f07cc005440 con 0x55e18f984410 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07e088a740 10 client.25657 fetch_fsmap learned FSMap version 27 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) -- 0x55e18fa0d7d0 con 0x55e18f984410 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.046+0000 7f07b6ffd640 1 -- 172.21.15.37:0/2626872446 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f07d0004040 con 0x55e18f984410 2024-09-19T06:44:23.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07b6ffd640 1 -- 172.21.15.37:0/2626872446 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 27) ==== 8354+0+0 (secure 0 0 0) 0x7f07cc019040 con 0x55e18f984410 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07e088a740 10 client.25657 fetch_fsmap finished waiting for FSMap version 27 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07e088a740 10 client.25657 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07e088a740 20 client.25657 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07e088a740 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18fa0dfb0 0x55e18fa103d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07e088a740 4 client.25657 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e18f8efd70 con 0x55e18fa0dfb0 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07d77fe640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18fa0dfb0 0x55e18fa103d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07b6ffd640 10 client.25657 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:23.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.047+0000 7f07d77fe640 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18fa0dfb0 0x55e18fa103d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=305b103d13a29d88 server_cookie=ffbe87b76c89fa02 in_seq=0 out_seq=0 2024-09-19T06:44:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.153+0000 7f07b6ffd640 1 -- 172.21.15.37:0/2626872446 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e18f8efd70 con 0x55e18fa0dfb0 2024-09-19T06:44:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.153+0000 7f07b6ffd640 10 client.25657 handle_command_reply: tid=0 2024-09-19T06:44:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 client.25657 shutdown 2024-09-19T06:44:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 msgr2=0x55e18f99aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f99aad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f07cc01bd20 tx=0x7f07cc009470 comp rx=0 tx=0).stop 2024-09-19T06:44:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 shutdown_connections 2024-09-19T06:44:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18fa0dfb0 0x55e18fa103d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18f98ecb0 0x55e18f99ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07b6ffd640 0 client.25657 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18f984410 0x55e18f99aad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 --2- 172.21.15.37:0/2626872446 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18f983af0 0x55e18f99a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 >> 172.21.15.37:0/2626872446 conn(0x55e18f980060 msgr2=0x55e18f97e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 shutdown_connections 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 1 -- 172.21.15.37:0/2626872446 wait complete. 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.154+0000 7f07e088a740 20 client.25657 trim_cache size 0 max 16384 2024-09-19T06:44:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:44:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:44:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:44:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:44:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07c00a3460 msgr2=0x7f07c00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07c00a3460 0x7f07c00a5920 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f07c80cd5f0 tx=0x7f07c80bc040 comp rx=0 tx=0).stop 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 msgr2=0x7f07d8198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 0x7f07d8198290 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f07c800c920 tx=0x7f07c800cde0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 shutdown_connections 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07d81069c0 0x7f07d819cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07c00a3460 0x7f07c00a5920 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f07d8101350 0x7f07d81987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 --2- 172.21.15.37:0/1079309937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07d81009b0 0x7f07d8198290 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 >> 172.21.15.37:0/1079309937 conn(0x7f07d8077990 msgr2=0x7f07d80ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 shutdown_connections 2024-09-19T06:44:23.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:23.155+0000 7f07dda15640 1 -- 172.21.15.37:0/1079309937 wait complete. 2024-09-19T06:44:23.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 86770 9.70 MB/sec warmup 50 sec latency 54.989 ms 2024-09-19T06:44:23.654 INFO:tasks.cephfs.filesystem:scrub status for tag:d3f528d4-d00b-4fd9-8e78-84bc716f53a3 - {'path': '/', 'tag': 'd3f528d4-d00b-4fd9-8e78-84bc716f53a3', 'options': 'recursive,force'} 2024-09-19T06:44:24.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 89528 9.80 MB/sec warmup 51 sec latency 11.306 ms 2024-09-19T06:44:24.349 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:44:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setuser ceph since I am not root 2024-09-19T06:44:24.349 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:44:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setgroup ceph since I am not root 2024-09-19T06:44:24.350 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:44:24 smithi151 ceph-mds[66270]: -- 172.21.15.151:0/2111983778 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2d50fd860 con 0x55c2d50da800 2024-09-19T06:44:24.350 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:44:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: starting mds.i at 2024-09-19T06:44:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:24 smithi151 ceph-mon[35236]: pgmap v81: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 460 op/s 2024-09-19T06:44:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:24 smithi151 ceph-mon[35236]: daemon mds.i restarted 2024-09-19T06:44:24.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:24 smithi096 ceph-mon[35004]: pgmap v81: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 460 op/s 2024-09-19T06:44:24.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:24 smithi096 ceph-mon[35004]: daemon mds.i restarted 2024-09-19T06:44:24.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:24 smithi037 ceph-mon[31211]: pgmap v81: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 2.7 MiB/s wr, 460 op/s 2024-09-19T06:44:24.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:24 smithi037 ceph-mon[31211]: daemon mds.i restarted 2024-09-19T06:44:25.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 92610 9.96 MB/sec warmup 52 sec latency 12.726 ms 2024-09-19T06:44:25.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:25 smithi151 ceph-mon[35236]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:44:25.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:25 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6838/3248587852,v1:172.21.15.151:6839/3248587852] up:boot 2024-09-19T06:44:25.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:25 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:44:25.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:25 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:44:25.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:25 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:44:25.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:25 smithi096 ceph-mon[35004]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:44:25.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:25 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6838/3248587852,v1:172.21.15.151:6839/3248587852] up:boot 2024-09-19T06:44:25.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:25 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:44:25.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:25 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:44:25.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:25 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:44:25.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:25 smithi037 ceph-mon[31211]: osdmap e85: 12 total, 12 up, 12 in 2024-09-19T06:44:25.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:25 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6838/3248587852,v1:172.21.15.151:6839/3248587852] up:boot 2024-09-19T06:44:25.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:25 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:44:25.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:25 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:44:25.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:25 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:44:26.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 95789 10.11 MB/sec warmup 53 sec latency 11.026 ms 2024-09-19T06:44:26.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:26 smithi151 ceph-mon[35236]: pgmap v83: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-09-19T06:44:26.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:26 smithi096 ceph-mon[35004]: pgmap v83: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-09-19T06:44:26.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:26 smithi037 ceph-mon[31211]: pgmap v83: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-09-19T06:44:27.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 98254 10.08 MB/sec warmup 54 sec latency 19.381 ms 2024-09-19T06:44:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:27 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:44:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:27 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:44:27.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:27 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:44:27.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:27 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:44:27.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:27 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:44:27.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:27 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:44:28.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 100582 10.16 MB/sec warmup 55 sec latency 12.637 ms 2024-09-19T06:44:28.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:28 smithi151 ceph-mon[35236]: pgmap v84: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-09-19T06:44:28.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:28 smithi096 ceph-mon[35004]: pgmap v84: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-09-19T06:44:28.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:28 smithi037 ceph-mon[31211]: pgmap v84: 97 pgs: 97 active+clean; 49 MiB data, 520 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.1 MiB/s wr, 290 op/s 2024-09-19T06:44:29.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 103529 10.28 MB/sec warmup 56 sec latency 10.885 ms 2024-09-19T06:44:30.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 106692 10.43 MB/sec warmup 57 sec latency 13.091 ms 2024-09-19T06:44:30.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:30 smithi096 ceph-mon[35004]: pgmap v85: 97 pgs: 97 active+clean; 42 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-09-19T06:44:30.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:30 smithi037 ceph-mon[31211]: pgmap v85: 97 pgs: 97 active+clean; 42 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-09-19T06:44:30.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:30 smithi151 ceph-mon[35236]: pgmap v85: 97 pgs: 97 active+clean; 42 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-09-19T06:44:31.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 109626 10.48 MB/sec warmup 58 sec latency 13.867 ms 2024-09-19T06:44:32.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 112338 10.52 MB/sec warmup 59 sec latency 11.872 ms 2024-09-19T06:44:32.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:32 smithi096 ceph-mon[35004]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-09-19T06:44:32.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:32 smithi037 ceph-mon[31211]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-09-19T06:44:32.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:32 smithi151 ceph-mon[35236]: pgmap v86: 97 pgs: 97 active+clean; 42 MiB data, 517 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-09-19T06:44:33.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 114418 10.56 MB/sec warmup 60 sec latency 79.966 ms 2024-09-19T06:44:34.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 117375 10.68 MB/sec warmup 61 sec latency 28.626 ms 2024-09-19T06:44:34.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:34 smithi096 ceph-mon[35004]: pgmap v87: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-09-19T06:44:34.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:34 smithi037 ceph-mon[31211]: pgmap v87: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-09-19T06:44:34.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:34 smithi151 ceph-mon[35236]: pgmap v87: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.5 MiB/s wr, 411 op/s 2024-09-19T06:44:35.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 120011 10.72 MB/sec warmup 62 sec latency 15.339 ms 2024-09-19T06:44:36.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 122165 10.73 MB/sec warmup 63 sec latency 69.472 ms 2024-09-19T06:44:36.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 -- 172.21.15.37:0/3175883040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 msgr2=0x7fd718106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 --2- 172.21.15.37:0/3175883040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 0x7fd718106da0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fd708009890 tx=0x7fd70802a380 comp rx=0 tx=0).stop 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 -- 172.21.15.37:0/3175883040 shutdown_connections 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 --2- 172.21.15.37:0/3175883040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd718101350 0x7fd71810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 --2- 172.21.15.37:0/3175883040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 0x7fd718100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 --2- 172.21.15.37:0/3175883040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 0x7fd718106da0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 -- 172.21.15.37:0/3175883040 >> 172.21.15.37:0/3175883040 conn(0x7fd718077990 msgr2=0x7fd7180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 -- 172.21.15.37:0/3175883040 shutdown_connections 2024-09-19T06:44:36.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.249+0000 7fd71cf49640 1 -- 172.21.15.37:0/3175883040 wait complete. 2024-09-19T06:44:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.250+0000 7fd71cf49640 1 Processor -- start 2024-09-19T06:44:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.250+0000 7fd71cf49640 1 -- start start 2024-09-19T06:44:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.250+0000 7fd71cf49640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 0x7fd7181991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71cf49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd718101350 0x7fd718199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71cf49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 0x7fd7181a4270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71cf49640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd718110e00 con 0x7fd7181069c0 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71cf49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd718110c80 con 0x7fd718101350 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71cf49640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd718110f80 con 0x7fd7181009b0 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 0x7fd7181991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd716d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 0x7fd7181a4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 0x7fd7181991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:42916/0 (socket says 172.21.15.37:42916) 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71659c640 1 -- 172.21.15.37:0/770649783 learned_addr learned my addr 172.21.15.37:0/770649783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd715d9b640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd718101350 0x7fd718199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd716d9d640 1 -- 172.21.15.37:0/770649783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 msgr2=0x7fd7181991f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd716d9d640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 0x7fd7181991f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd716d9d640 1 -- 172.21.15.37:0/770649783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd718101350 msgr2=0x7fd718199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd716d9d640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd718101350 0x7fd718199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd716d9d640 1 -- 172.21.15.37:0/770649783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7181a47b0 con 0x7fd7181069c0 2024-09-19T06:44:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71659c640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 0x7fd7181991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd716d9d640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 0x7fd7181a4270 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd71000ef00 tx=0x7fd71000c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd7077fe640 1 -- 172.21.15.37:0/770649783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd710004b20 con 0x7fd7181069c0 2024-09-19T06:44:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd7077fe640 1 -- 172.21.15.37:0/770649783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd710003e50 con 0x7fd7181069c0 2024-09-19T06:44:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd7077fe640 1 -- 172.21.15.37:0/770649783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd71000b570 con 0x7fd7181069c0 2024-09-19T06:44:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7181a4980 con 0x7fd7181069c0 2024-09-19T06:44:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.251+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7181a4e50 con 0x7fd7181069c0 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd7077fe640 1 -- 172.21.15.37:0/770649783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd71001d020 con 0x7fd7181069c0 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 Processor -- start 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 -- start start 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7faed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f805770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f8080a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d80f75f2f0 con 0x55d80f7faaf0 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d80f707270 con 0x55d80f805cb0 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.254+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d80f75b2b0 con 0x55d80f7fb410 2024-09-19T06:44:36.262 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd715d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7faed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd715d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7faed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48212/0 (socket says 172.21.15.37:48212) 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd715d9b640 1 -- 172.21.15.37:0/578933053 learned_addr learned my addr 172.21.15.37:0/578933053 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd71659c640 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f805770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd715d9b640 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f8080a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd7077fe640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7000a3460 0x7fd7000a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd71659c640 1 -- 172.21.15.37:0/578933053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 msgr2=0x55d80f8080a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd71659c640 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f8080a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd71659c640 1 -- 172.21.15.37:0/578933053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 msgr2=0x55d80f7faed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd71659c640 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7faed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd71659c640 1 -- 172.21.15.37:0/578933053 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d80f706340 con 0x55d80f7fb410 2024-09-19T06:44:36.263 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd716d9d640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7000a3460 0x7fd7000a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd7077fe640 1 -- 172.21.15.37:0/770649783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd7100cfad0 con 0x7fd7181069c0 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd71659c640 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f805770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd70802a970 tx=0x7fd708009590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=118ad0169a9bfda6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd704ff9640 1 -- 172.21.15.37:0/578933053 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708037040 con 0x55d80f7fb410 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd716d9d640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7000a3460 0x7fd7000a5920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd7100cd2d0 tx=0x7fd71006aaa0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd704ff9640 1 -- 172.21.15.37:0/578933053 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd708004230 con 0x55d80f7fb410 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.255+0000 7fd704ff9640 1 -- 172.21.15.37:0/578933053 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708003c30 con 0x55d80f7fb410 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 -- 172.21.15.37:0/578933053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 msgr2=0x55d80f805770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f805770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd70802a970 tx=0x7fd708009590 comp rx=0 tx=0).stop 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 -- 172.21.15.37:0/578933053 shutdown_connections 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f8080a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f805770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/578933053 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7faed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 -- 172.21.15.37:0/578933053 >> 172.21.15.37:0/578933053 conn(0x55d80f7f7060 msgr2=0x55d80f7f7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 -- 172.21.15.37:0/578933053 shutdown_connections 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.256+0000 7fd71fdbc740 1 -- 172.21.15.37:0/578933053 wait complete. 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 -- start start 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 Processor -- start 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f802b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f7fb410 0x55d80f803070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f805cb0 0x55d80f8035b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d80f808880 con 0x55d80f805cb0 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d80f6e8290 con 0x55d80f7fb410 2024-09-19T06:44:36.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d80f808a00 con 0x55d80f7faaf0 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71fdbc740 10 client.0 _ll_register_callbacks cb 0x55d80f719140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f805cb0 0x55d80f8035b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f805cb0 0x55d80f8035b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48218/0 (socket says 172.21.15.37:48218) 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 -- 172.21.15.37:0/2616816328 learned_addr learned my addr 172.21.15.37:0/2616816328 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f802b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd71659c640 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f7fb410 0x55d80f803070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 -- 172.21.15.37:0/2616816328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 msgr2=0x55d80f802b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f802b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 -- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f7fb410 msgr2=0x55d80f803070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f7fb410 0x55d80f803070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.257+0000 7fd715d9b640 1 -- 172.21.15.37:0/2616816328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d80f472210 con 0x55d80f805cb0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f805cb0 0x55d80f8035b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd70c013430 tx=0x7fd70c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ac0265def3eee0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd6fdffb640 1 -- 172.21.15.37:0/2616816328 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c004030 con 0x55d80f805cb0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd6fdffb640 1 -- 172.21.15.37:0/2616816328 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd70c0041d0 con 0x55d80f805cb0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd708004230 con 0x55d80f805cb0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd6fdffb640 1 -- 172.21.15.37:0/2616816328 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd70c0043d0 con 0x55d80f805cb0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd71fdbc740 10 client.16248 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d80f53e620 con 0x55d80f805cb0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd6fdffb640 1 -- 172.21.15.37:0/2616816328 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd708004230 con 0x55d80f805cb0 2024-09-19T06:44:36.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd6fdffb640 1 -- 172.21.15.37:0/2616816328 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd70c003ab0 con 0x55d80f805cb0 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd71fdbc740 10 client.16248 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.258+0000 7fd71fdbc740 10 client.16248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.259+0000 7fd71fdbc740 20 client.16248 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.259+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f815d00 0x55d80f818110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.259+0000 7fd71fdbc740 4 client.16248 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.259+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d80f76ca20 con 0x55d80f815d00 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.259+0000 7fd71659c640 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f815d00 0x55d80f818110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.259+0000 7fd6fdffb640 10 client.16248 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:36.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.259+0000 7fd71659c640 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f815d00 0x55d80f818110 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920775f607875c71 server_cookie=750cecc072e4b174 in_seq=0 out_seq=0 2024-09-19T06:44:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd6fdffb640 1 -- 172.21.15.37:0/2616816328 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d80f76ca20 con 0x55d80f815d00 2024-09-19T06:44:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd6fdffb640 10 client.16248 handle_command_reply: tid=0 2024-09-19T06:44:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 client.16248 shutdown 2024-09-19T06:44:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f805cb0 msgr2=0x55d80f8035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f805cb0 0x55d80f8035b0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd70c013430 tx=0x7fd70c005f90 comp rx=0 tx=0).stop 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 shutdown_connections 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f815d00 0x55d80f818110 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f805cb0 0x55d80f8035b0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f7fb410 0x55d80f803070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd6fdffb640 0 client.16248 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2616816328 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f802b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 >> 172.21.15.37:0/2616816328 conn(0x55d80f7f7060 msgr2=0x55d80f7fcdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 shutdown_connections 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2616816328 wait complete. 2024-09-19T06:44:36.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.261+0000 7fd71fdbc740 20 client.16248 trim_cache size 0 max 16384 2024-09-19T06:44:36.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 Processor -- start 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 -- start start 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f8033e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f81d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181b0100 con 0x55d80f7faaf0 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ad970 con 0x55d80f805cb0 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ab7c0 con 0x55d80f7fb410 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48226/0 (socket says 172.21.15.37:48226) 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 -- 172.21.15.37:0/2760193323 learned_addr learned my addr 172.21.15.37:0/2760193323 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:36.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd71659c640 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f8033e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f81d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 -- 172.21.15.37:0/2760193323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 msgr2=0x55d80f8033e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f8033e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 -- 172.21.15.37:0/2760193323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 msgr2=0x55d80f81d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f81d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 -- 172.21.15.37:0/2760193323 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d80f7f7cf0 con 0x55d80f7faaf0 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd715d9b640 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7fd560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0058c0 tx=0x7fd70c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe451cc6ddeb059 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.276+0000 7fd6fffff640 1 -- 172.21.15.37:0/2760193323 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c02f040 con 0x55d80f7faaf0 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd6fffff640 1 -- 172.21.15.37:0/2760193323 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd70c013710 con 0x55d80f7faaf0 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd6fffff640 1 -- 172.21.15.37:0/2760193323 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70c0052f0 con 0x55d80f7faaf0 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2760193323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 msgr2=0x55d80f7fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7fd560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fd70c0058c0 tx=0x7fd70c005850 comp rx=0 tx=0).stop 2024-09-19T06:44:36.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2760193323 shutdown_connections 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f81d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7fb410 0x55d80f8033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/2760193323 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7faaf0 0x55d80f7fd560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2760193323 >> 172.21.15.37:0/2760193323 conn(0x55d80f7f7060 msgr2=0x55d80f804bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2760193323 shutdown_connections 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 -- 172.21.15.37:0/2760193323 wait complete. 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.277+0000 7fd71fdbc740 1 -- start start 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 1 Processor -- start 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f80e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7fb410 0x55d80f7ff920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f7ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181abe30 con 0x55d80f7fb410 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ad2a0 con 0x55d80f805cb0 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7181ac500 con 0x55d80f7faaf0 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71fdbc740 10 client.0 _ll_register_callbacks cb 0x7fd70c013050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7fb410 0x55d80f7ff920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7fb410 0x55d80f7ff920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48230/0 (socket says 172.21.15.37:48230) 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 -- 172.21.15.37:0/1218739725 learned_addr learned my addr 172.21.15.37:0/1218739725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd715d9b640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f7ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd715d9b640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f80e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 -- 172.21.15.37:0/1218739725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 msgr2=0x55d80f80e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f80e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 -- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 msgr2=0x55d80f7ffec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:36.285 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f7ffec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 -- 172.21.15.37:0/1218739725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d80f7f22d0 con 0x55d80f7fb410 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd715d9b640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f80e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd71659c640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7fb410 0x55d80f7ff920 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd708005d90 tx=0x7fd708003850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6bd2a701f92ad0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.278+0000 7fd6fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd6fdffb640 1 -- 172.21.15.37:0/1218739725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd708037040 con 0x55d80f7fb410 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd6fdffb640 1 -- 172.21.15.37:0/1218739725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd708030750 con 0x55d80f7fb410 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd70c013710 con 0x55d80f7fb410 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd6fdffb640 1 -- 172.21.15.37:0/1218739725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd70805f050 con 0x55d80f7fb410 2024-09-19T06:44:36.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d80f884c60 con 0x55d80f7fb410 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd6fdffb640 1 -- 172.21.15.37:0/1218739725 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd70c013710 con 0x55d80f7fb410 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd6fdffb640 1 -- 172.21.15.37:0/1218739725 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd70804f330 con 0x55d80f7fb410 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 20 client.16254 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f8850b0 0x55d80f8874a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 4 client.16254 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d80f766d70 con 0x55d80f8850b0 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.279+0000 7fd715d9b640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f8850b0 0x55d80f8874a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.280+0000 7fd715d9b640 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f8850b0 0x55d80f8874a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a7a669ac9e3569e server_cookie=1ede5d67361357bc in_seq=0 out_seq=0 2024-09-19T06:44:36.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.280+0000 7fd6fdffb640 10 client.16254 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd6fdffb640 1 -- 172.21.15.37:0/1218739725 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d80f766d70 con 0x55d80f8850b0 2024-09-19T06:44:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd6fdffb640 10 client.16254 handle_command_reply: tid=0 2024-09-19T06:44:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 client.16254 shutdown 2024-09-19T06:44:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7fb410 msgr2=0x55d80f7ff920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7fb410 0x55d80f7ff920 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fd708005d90 tx=0x7fd708003850 comp rx=0 tx=0).stop 2024-09-19T06:44:36.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 shutdown_connections 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d80f8850b0 0x55d80f8874a0 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd6fdffb640 0 client.16254 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d80f805cb0 0x55d80f7ffec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d80f7fb410 0x55d80f7ff920 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 --2- 172.21.15.37:0/1218739725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d80f7faaf0 0x55d80f80e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 >> 172.21.15.37:0/1218739725 conn(0x55d80f7f7060 msgr2=0x55d80f804390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 shutdown_connections 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 1 -- 172.21.15.37:0/1218739725 wait complete. 2024-09-19T06:44:36.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.401+0000 7fd71fdbc740 20 client.16254 trim_cache size 0 max 16384 2024-09-19T06:44:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:44:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:44:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:44:36.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:44:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7000a3460 msgr2=0x7fd7000a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7000a3460 0x7fd7000a5920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd7100cd2d0 tx=0x7fd71006aaa0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 msgr2=0x7fd7181a4270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 0x7fd7181a4270 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fd71000ef00 tx=0x7fd71000c740 comp rx=0 tx=0).stop 2024-09-19T06:44:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 shutdown_connections 2024-09-19T06:44:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7000a3460 0x7fd7000a5920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.406 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd7181069c0 0x7fd7181a4270 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd718101350 0x7fd718199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 --2- 172.21.15.37:0/770649783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd7181009b0 0x7fd7181991f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 >> 172.21.15.37:0/770649783 conn(0x7fd718077990 msgr2=0x7fd718103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 shutdown_connections 2024-09-19T06:44:36.407 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:36.402+0000 7fd71cf49640 1 -- 172.21.15.37:0/770649783 wait complete. 2024-09-19T06:44:36.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:36 smithi096 ceph-mon[35004]: pgmap v88: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-09-19T06:44:36.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:36 smithi037 ceph-mon[31211]: pgmap v88: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-09-19T06:44:36.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:36 smithi151 ceph-mon[35236]: pgmap v88: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-09-19T06:44:37.192 INFO:tasks.workunit.client.0.smithi037.stdout: 1 124680 10.79 MB/sec warmup 64 sec latency 72.022 ms 2024-09-19T06:44:38.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 127533 10.84 MB/sec warmup 65 sec latency 57.904 ms 2024-09-19T06:44:38.664 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:38 smithi151 ceph-mon[35236]: pgmap v89: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-09-19T06:44:38.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:38 smithi096 ceph-mon[35004]: pgmap v89: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-09-19T06:44:38.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:38 smithi037 ceph-mon[31211]: pgmap v89: 97 pgs: 97 active+clean; 48 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 2.1 MiB/s wr, 342 op/s 2024-09-19T06:44:39.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 129419 10.83 MB/sec warmup 66 sec latency 44.607 ms 2024-09-19T06:44:40.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 132179 10.88 MB/sec warmup 67 sec latency 10.665 ms 2024-09-19T06:44:40.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:40 smithi096 ceph-mon[35004]: pgmap v90: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 601 op/s 2024-09-19T06:44:40.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:40 smithi037 ceph-mon[31211]: pgmap v90: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 601 op/s 2024-09-19T06:44:40.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:40 smithi151 ceph-mon[35236]: pgmap v90: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 3.9 MiB/s wr, 601 op/s 2024-09-19T06:44:41.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 135145 10.99 MB/sec warmup 68 sec latency 41.459 ms 2024-09-19T06:44:42.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 138026 11.03 MB/sec warmup 69 sec latency 14.352 ms 2024-09-19T06:44:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:42 smithi096 ceph-mon[35004]: pgmap v91: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 377 op/s 2024-09-19T06:44:42.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:42 smithi037 ceph-mon[31211]: pgmap v91: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 377 op/s 2024-09-19T06:44:42.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:42 smithi151 ceph-mon[35236]: pgmap v91: 97 pgs: 97 active+clean; 42 MiB data, 565 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 2.8 MiB/s wr, 377 op/s 2024-09-19T06:44:43.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 139926 11.03 MB/sec warmup 70 sec latency 50.407 ms 2024-09-19T06:44:44.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 142508 11.07 MB/sec warmup 71 sec latency 55.124 ms 2024-09-19T06:44:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:44 smithi096 ceph-mon[35004]: pgmap v92: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 530 op/s 2024-09-19T06:44:44.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:44 smithi037 ceph-mon[31211]: pgmap v92: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 530 op/s 2024-09-19T06:44:44.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:44 smithi151 ceph-mon[35236]: pgmap v92: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 4.6 MiB/s wr, 530 op/s 2024-09-19T06:44:45.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 145693 11.18 MB/sec warmup 72 sec latency 15.565 ms 2024-09-19T06:44:46.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 148432 11.20 MB/sec warmup 73 sec latency 53.480 ms 2024-09-19T06:44:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:46 smithi096 ceph-mon[35004]: pgmap v93: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-09-19T06:44:46.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:46 smithi037 ceph-mon[31211]: pgmap v93: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-09-19T06:44:46.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:46 smithi151 ceph-mon[35236]: pgmap v93: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-09-19T06:44:47.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 150979 11.21 MB/sec warmup 74 sec latency 15.446 ms 2024-09-19T06:44:48.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 153217 11.24 MB/sec warmup 75 sec latency 80.309 ms 2024-09-19T06:44:48.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:48 smithi096 ceph-mon[35004]: pgmap v94: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-09-19T06:44:48.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:48 smithi037 ceph-mon[31211]: pgmap v94: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-09-19T06:44:48.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:48 smithi151 ceph-mon[35236]: pgmap v94: 97 pgs: 97 active+clean; 46 MiB data, 564 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 3.6 MiB/s wr, 411 op/s 2024-09-19T06:44:49.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 156180 11.33 MB/sec warmup 76 sec latency 72.836 ms 2024-09-19T06:44:49.501 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 -- 172.21.15.37:0/1559666317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce8101350 msgr2=0x7f2ce810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/1559666317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce8101350 0x7f2ce810d880 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009cd0 tx=0x7f2ce002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 -- 172.21.15.37:0/1559666317 shutdown_connections 2024-09-19T06:44:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/1559666317 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce8101350 0x7f2ce810d880 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/1559666317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce81009b0 0x7f2ce8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/1559666317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81069c0 0x7f2ce8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 -- 172.21.15.37:0/1559666317 >> 172.21.15.37:0/1559666317 conn(0x7f2ce8077990 msgr2=0x7f2ce80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:49.502 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 -- 172.21.15.37:0/1559666317 shutdown_connections 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 -- 172.21.15.37:0/1559666317 wait complete. 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.498+0000 7f2ceef9f640 1 Processor -- start 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ceef9f640 1 -- start start 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ceef9f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81009b0 0x7f2ce8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ceef9f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce8101350 0x7f2ce8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ceef9f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce81069c0 0x7f2ce819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ceef9f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8110e50 con 0x7f2ce81069c0 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ceef9f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8110cd0 con 0x7f2ce81009b0 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ceef9f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce8110fd0 con 0x7f2ce8101350 2024-09-19T06:44:49.503 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81009b0 0x7f2ce8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ced53c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce81069c0 0x7f2ce819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81009b0 0x7f2ce8198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47454/0 (socket says 172.21.15.37:47454) 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2cecd3b640 1 -- 172.21.15.37:0/3568832826 learned_addr learned my addr 172.21.15.37:0/3568832826 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce8101350 0x7f2ce8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ced53c640 1 -- 172.21.15.37:0/3568832826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce8101350 msgr2=0x7f2ce8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ced53c640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce8101350 0x7f2ce8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ced53c640 1 -- 172.21.15.37:0/3568832826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81009b0 msgr2=0x7f2ce8198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ced53c640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81009b0 0x7f2ce8198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.504 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ced53c640 1 -- 172.21.15.37:0/3568832826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce819d320 con 0x7f2ce81069c0 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce8101350 0x7f2ce8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.499+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81009b0 0x7f2ce8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.500+0000 7f2ced53c640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce81069c0 0x7f2ce819cbf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009ca0 tx=0x7f2ce0031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.500+0000 7f2ce5ffb640 1 -- 172.21.15.37:0/3568832826 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0004040 con 0x7f2ce81069c0 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.500+0000 7f2ce5ffb640 1 -- 172.21.15.37:0/3568832826 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce00046c0 con 0x7f2ce81069c0 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.500+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce819d5b0 con 0x7f2ce81069c0 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.500+0000 7f2ce5ffb640 1 -- 172.21.15.37:0/3568832826 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce0030e40 con 0x7f2ce81069c0 2024-09-19T06:44:49.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.500+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce81a4ea0 con 0x7f2ce81069c0 2024-09-19T06:44:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.502+0000 7f2ce5ffb640 1 -- 172.21.15.37:0/3568832826 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2ce00305a0 con 0x7f2ce81069c0 2024-09-19T06:44:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 Processor -- start 2024-09-19T06:44:49.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 -- start start 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2ce5ffb640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cd80a3460 0x7f2cd80a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e51d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e634b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c9f5db62f0 con 0x55c9f5e519b0 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9f5d5e270 con 0x55c9f5e522d0 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9f5db22b0 con 0x55c9f5e610c0 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e51d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2ce5ffb640 1 -- 172.21.15.37:0/3568832826 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2ce002e030 con 0x7f2ce81069c0 2024-09-19T06:44:49.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e51d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55442/0 (socket says 172.21.15.37:55442) 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cd80a3460 0x7f2cd80a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.503+0000 7f2cecd3b640 1 -- 172.21.15.37:0/2201834379 learned_addr learned my addr 172.21.15.37:0/2201834379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ced53c640 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e634b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ced53c640 1 -- 172.21.15.37:0/2201834379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 msgr2=0x55c9f5e60b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ced53c640 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ced53c640 1 -- 172.21.15.37:0/2201834379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 msgr2=0x55c9f5e51d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ced53c640 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e51d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ced53c640 1 -- 172.21.15.37:0/2201834379 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f5d5d340 con 0x55c9f5e610c0 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e51d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:49.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cd80a3460 0x7f2cd80a5920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009fd0 tx=0x7f2cd4009b00 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2ced53c640 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e634b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00ee7e0 tx=0x7f2ce00bd1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1c30d1932195e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2cd2ffd640 1 -- 172.21.15.37:0/2201834379 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce00a0ea0 con 0x55c9f5e610c0 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2cd2ffd640 1 -- 172.21.15.37:0/2201834379 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce0036020 con 0x55c9f5e610c0 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.504+0000 7f2cd2ffd640 1 -- 172.21.15.37:0/2201834379 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ce004e7a0 con 0x55c9f5e610c0 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/2201834379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 msgr2=0x55c9f5e634b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e634b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2ce00ee7e0 tx=0x7f2ce00bd1d0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/2201834379 shutdown_connections 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e634b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/2201834379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e51d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/2201834379 >> 172.21.15.37:0/2201834379 conn(0x55c9f5e4e060 msgr2=0x55c9f5e4e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/2201834379 shutdown_connections 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/2201834379 wait complete. 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.505+0000 7f2cf1e1a740 1 -- start start 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 1 Processor -- start 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e5e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c9f5e63a50 con 0x55c9f5e519b0 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9f5d3f290 con 0x55c9f5e610c0 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9f5e63bd0 con 0x55c9f5e522d0 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cf1e1a740 10 client.0 _ll_register_callbacks cb 0x55c9f5d70140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:49.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55454/0 (socket says 172.21.15.37:55454) 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cecd3b640 1 -- 172.21.15.37:0/24259725 learned_addr learned my addr 172.21.15.37:0/24259725 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2ced53c640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e5e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.506+0000 7f2cecd3b640 1 -- 172.21.15.37:0/24259725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 msgr2=0x55c9f5e5e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e5e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cecd3b640 1 -- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 msgr2=0x55c9f5e5eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e5eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cecd3b640 1 -- 172.21.15.37:0/24259725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f5ac9210 con 0x55c9f5e519b0 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2ced53c640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e5eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e0b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc012d20 tx=0x7f2cdc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=122d209b58bb53bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:49.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cc7fff640 1 -- 172.21.15.37:0/24259725 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0120d0 con 0x55c9f5e519b0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cc7fff640 1 -- 172.21.15.37:0/24259725 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2cdc004030 con 0x55c9f5e519b0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ce0036020 con 0x55c9f5e519b0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cc7fff640 1 -- 172.21.15.37:0/24259725 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2cdc012270 con 0x55c9f5e519b0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cf1e1a740 10 client.16266 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9f5b95620 con 0x55c9f5e519b0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cc7fff640 1 -- 172.21.15.37:0/24259725 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ce0036020 con 0x55c9f5e519b0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.507+0000 7f2cc7fff640 1 -- 172.21.15.37:0/24259725 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2cdc01c620 con 0x55c9f5e519b0 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2cf1e1a740 10 client.16266 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2cf1e1a740 10 client.16266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2cf1e1a740 20 client.16266 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5e70ef0 0x55c9f5e73330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2cf1e1a740 4 client.16266 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9f5dc3a20 con 0x55c9f5e70ef0 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5e70ef0 0x55c9f5e73330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5e70ef0 0x55c9f5e73330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4030d61b0baefd3f server_cookie=2dea1ecd9370c24c in_seq=0 out_seq=0 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.508+0000 7f2cc7fff640 10 client.16266 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.509+0000 7f2cc7fff640 1 -- 172.21.15.37:0/24259725 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9f5dc3a20 con 0x55c9f5e70ef0 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.509+0000 7f2cc7fff640 10 client.16266 handle_command_reply: tid=0 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.509+0000 7f2cf1e1a740 1 client.16266 shutdown 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 msgr2=0x55c9f5e5e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e0b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc012d20 tx=0x7f2cdc00f790 comp rx=0 tx=0).stop 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 shutdown_connections 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5e70ef0 0x55c9f5e73330 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e5eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e5e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cc7fff640 0 client.16266 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/24259725 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e0b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 >> 172.21.15.37:0/24259725 conn(0x55c9f5e4e060 msgr2=0x55c9f5e4d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 shutdown_connections 2024-09-19T06:44:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/24259725 wait complete. 2024-09-19T06:44:49.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.510+0000 7f2cf1e1a740 20 client.16266 trim_cache size 0 max 16384 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 Processor -- start 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 -- start start 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e78c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81aded0 con 0x55c9f5e519b0 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81ab740 con 0x55c9f5e610c0 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81a9590 con 0x55c9f5e522d0 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.524+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55466/0 (socket says 172.21.15.37:55466) 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1078291728 learned_addr learned my addr 172.21.15.37:0/1078291728 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2ced53c640 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e78c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1078291728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 msgr2=0x55c9f5e60020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1078291728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 msgr2=0x55c9f5e78c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e78c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1078291728 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9f5e58e90 con 0x55c9f5e519b0 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e960 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009ee0 tx=0x7f2cdc005d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feceaed0eb3752eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:49.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cd27fc640 1 -- 172.21.15.37:0/1078291728 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc029040 con 0x55c9f5e519b0 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cd27fc640 1 -- 172.21.15.37:0/1078291728 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cdc005070 con 0x55c9f5e519b0 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.525+0000 7f2cd27fc640 1 -- 172.21.15.37:0/1078291728 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc0046c0 con 0x55c9f5e519b0 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1078291728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 msgr2=0x55c9f5e5e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e960 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc009ee0 tx=0x7f2cdc005d30 comp rx=0 tx=0).stop 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1078291728 shutdown_connections 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e610c0 0x55c9f5e78c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e522d0 0x55c9f5e60020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1078291728 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e5e960 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1078291728 >> 172.21.15.37:0/1078291728 conn(0x55c9f5e4e060 msgr2=0x55c9f5e5f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1078291728 shutdown_connections 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1078291728 wait complete. 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 -- start start 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.526+0000 7f2cf1e1a740 1 Processor -- start 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e692a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e64610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e64b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81a9c00 con 0x55c9f5e519b0 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81ab070 con 0x55c9f5e522d0 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce81aa2d0 con 0x55c9f5e610c0 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 10 client.0 _ll_register_callbacks cb 0x7f2cdc012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e692a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e692a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55480/0 (socket says 172.21.15.37:55480) 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1718145428 learned_addr learned my addr 172.21.15.37:0/1718145428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e64610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2ced53c640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e64b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1718145428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 msgr2=0x55c9f5e64b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e64b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 msgr2=0x55c9f5e64610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e64610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 -- 172.21.15.37:0/1718145428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9f5e65090 con 0x55c9f5e519b0 2024-09-19T06:44:49.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e64610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cecd3b640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e692a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0092b0 tx=0x7f2cdc005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=992d1c04dc63111c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cc7fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cc7fff640 1 -- 172.21.15.37:0/1718145428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cdc029040 con 0x55c9f5e519b0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cc7fff640 1 -- 172.21.15.37:0/1718145428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2cdc0055f0 con 0x55c9f5e519b0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.527+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cdc005070 con 0x55c9f5e519b0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cc7fff640 1 -- 172.21.15.37:0/1718145428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2cdc010430 con 0x55c9f5e519b0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 10 client.16272 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c9f5edb950 con 0x55c9f5e519b0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cc7fff640 1 -- 172.21.15.37:0/1718145428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cdc005070 con 0x55c9f5e519b0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cc7fff640 1 -- 172.21.15.37:0/1718145428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2cdc025840 con 0x55c9f5e519b0 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 10 client.16272 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:49.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 10 client.16272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 20 client.16272 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5edbf50 0x55c9f5ede370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 4 client.16272 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:44:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9f5dbdd70 con 0x55c9f5edbf50 2024-09-19T06:44:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.528+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5edbf50 0x55c9f5ede370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.529+0000 7f2cc7fff640 10 client.16272 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:49.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.529+0000 7f2ce7fff640 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5edbf50 0x55c9f5ede370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b3920d9ba548199 server_cookie=d075d4d78dfd331c in_seq=0 out_seq=0 2024-09-19T06:44:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cc7fff640 1 -- 172.21.15.37:0/1718145428 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9f5dbdd70 con 0x55c9f5edbf50 2024-09-19T06:44:49.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cc7fff640 10 client.16272 handle_command_reply: tid=0 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 client.16272 shutdown 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 msgr2=0x55c9f5e692a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e692a0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f2cdc0092b0 tx=0x7f2cdc005f10 comp rx=0 tx=0).stop 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 shutdown_connections 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9f5edbf50 0x55c9f5ede370 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9f5e610c0 0x55c9f5e64b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cc7fff640 0 client.16272 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9f5e522d0 0x55c9f5e64610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 --2- 172.21.15.37:0/1718145428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9f5e519b0 0x55c9f5e692a0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 >> 172.21.15.37:0/1718145428 conn(0x55c9f5e4e060 msgr2=0x55c9f5e4d1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 shutdown_connections 2024-09-19T06:44:49.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 1 -- 172.21.15.37:0/1718145428 wait complete. 2024-09-19T06:44:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.725+0000 7f2cf1e1a740 20 client.16272 trim_cache size 0 max 16384 2024-09-19T06:44:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:44:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:44:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:44:49.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cd80a3460 msgr2=0x7f2cd80a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cd80a3460 0x7f2cd80a5920 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2cd4009fd0 tx=0x7f2cd4009b00 comp rx=0 tx=0).stop 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce81069c0 msgr2=0x7f2ce819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce81069c0 0x7f2ce819cbf0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f2ce0009ca0 tx=0x7f2ce0031e60 comp rx=0 tx=0).stop 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 shutdown_connections 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cd80a3460 0x7f2cd80a5920 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ce81069c0 0x7f2ce819cbf0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ce8101350 0x7f2ce8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 --2- 172.21.15.37:0/3568832826 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ce81009b0 0x7f2ce8198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:49.729 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 >> 172.21.15.37:0/3568832826 conn(0x7f2ce8077990 msgr2=0x7f2ce8069a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 shutdown_connections 2024-09-19T06:44:49.730 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:44:49.726+0000 7f2ceef9f640 1 -- 172.21.15.37:0/3568832826 wait complete. 2024-09-19T06:44:50.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 158483 11.29 MB/sec warmup 77 sec latency 85.611 ms 2024-09-19T06:44:50.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:50 smithi096 ceph-mon[35004]: pgmap v95: 97 pgs: 97 active+clean; 52 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 5.5 MiB/s wr, 565 op/s 2024-09-19T06:44:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:50 smithi037 ceph-mon[31211]: pgmap v95: 97 pgs: 97 active+clean; 52 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 5.5 MiB/s wr, 565 op/s 2024-09-19T06:44:50.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:50 smithi151 ceph-mon[35236]: pgmap v95: 97 pgs: 97 active+clean; 52 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 5.5 MiB/s wr, 565 op/s 2024-09-19T06:44:51.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 160826 11.33 MB/sec warmup 78 sec latency 11.470 ms 2024-09-19T06:44:52.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 163655 11.40 MB/sec warmup 79 sec latency 25.729 ms 2024-09-19T06:44:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:52 smithi096 ceph-mon[35004]: pgmap v96: 97 pgs: 97 active+clean; 52 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2024-09-19T06:44:52.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:52 smithi037 ceph-mon[31211]: pgmap v96: 97 pgs: 97 active+clean; 52 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2024-09-19T06:44:52.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:52 smithi151 ceph-mon[35236]: pgmap v96: 97 pgs: 97 active+clean; 52 MiB data, 609 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 3.7 MiB/s wr, 307 op/s 2024-09-19T06:44:53.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 166824 11.48 MB/sec warmup 80 sec latency 27.330 ms 2024-09-19T06:44:53.655 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:44:53.951 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:54.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 168891 11.43 MB/sec warmup 81 sec latency 58.342 ms 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.429+0000 7fc688a54640 1 -- 172.21.15.37:0/4032532623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684102d90 msgr2=0x7fc684103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.429+0000 7fc688a54640 1 --2- 172.21.15.37:0/4032532623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684102d90 0x7fc684103170 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009cd0 tx=0x7fc67c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 -- 172.21.15.37:0/4032532623 shutdown_connections 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 --2- 172.21.15.37:0/4032532623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc684108290 0x7fc68410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 --2- 172.21.15.37:0/4032532623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc684103740 0x7fc684107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 --2- 172.21.15.37:0/4032532623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684102d90 0x7fc684103170 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 -- 172.21.15.37:0/4032532623 >> 172.21.15.37:0/4032532623 conn(0x7fc6840fc7d0 msgr2=0x7fc6840febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:54.446 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 -- 172.21.15.37:0/4032532623 shutdown_connections 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 -- 172.21.15.37:0/4032532623 wait complete. 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 Processor -- start 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.430+0000 7fc688a54640 1 -- start start 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc688a54640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc684102d90 0x7fc6841009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc688a54640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684103740 0x7fc684100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc688a54640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc684108290 0x7fc684101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc688a54640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc684112090 con 0x7fc684103740 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc688a54640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc684111f10 con 0x7fc684102d90 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc688a54640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc684112210 con 0x7fc684108290 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684103740 0x7fc684100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc68259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc684102d90 0x7fc6841009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684103740 0x7fc684100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55488/0 (socket says 172.21.15.37:55488) 2024-09-19T06:44:54.447 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 -- 172.21.15.37:0/1137774690 learned_addr learned my addr 172.21.15.37:0/1137774690 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc684108290 0x7fc684101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 -- 172.21.15.37:0/1137774690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc684108290 msgr2=0x7fc684101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc684108290 0x7fc684101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 -- 172.21.15.37:0/1137774690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc684102d90 msgr2=0x7fc6841009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc684102d90 0x7fc6841009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 -- 172.21.15.37:0/1137774690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6841a69e0 con 0x7fc684103740 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc68259c640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc684102d90 0x7fc6841009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.431+0000 7fc681d9b640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684103740 0x7fc684100f30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc67800c970 tx=0x7fc67800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.432+0000 7fc6737fe640 1 -- 172.21.15.37:0/1137774690 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc678014070 con 0x7fc684103740 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.432+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6841a6cd0 con 0x7fc684103740 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.432+0000 7fc6737fe640 1 -- 172.21.15.37:0/1137774690 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc678004440 con 0x7fc684103740 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.432+0000 7fc6737fe640 1 -- 172.21.15.37:0/1137774690 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67800f450 con 0x7fc684103740 2024-09-19T06:44:54.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.432+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6841a71c0 con 0x7fc684103740 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.434+0000 7fc6737fe640 1 -- 172.21.15.37:0/1137774690 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc678017070 con 0x7fc684103740 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.434+0000 7fc68ae2d740 1 Processor -- start 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.434+0000 7fc68ae2d740 1 -- start start 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6584f1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584f1dc0 0x55e6584fc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584fc680 0x55e6584fea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6584572e0 con 0x55e6584f14a0 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6583fece0 con 0x55e6584fc680 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e6584532a0 con 0x55e6584f1dc0 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6584f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6584f1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55496/0 (socket says 172.21.15.37:55496) 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 -- 172.21.15.37:0/1595975905 learned_addr learned my addr 172.21.15.37:0/1595975905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:54.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc681d9b640 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584fc680 0x55e6584fea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68259c640 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584f1dc0 0x55e6584fc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc6737fe640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc66c0a33b0 0x7fc66c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 -- 172.21.15.37:0/1595975905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584f1dc0 msgr2=0x55e6584fc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584f1dc0 0x55e6584fc140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 -- 172.21.15.37:0/1595975905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584fc680 msgr2=0x55e6584fea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584fc680 0x55e6584fea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 -- 172.21.15.37:0/1595975905 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6583e3440 con 0x55e6584f14a0 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc68259c640 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584f1dc0 0x55e6584fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc6737fe640 1 -- 172.21.15.37:0/1137774690 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc6780cfb20 con 0x7fc684103740 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.435+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6584f1880 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc674009c20 tx=0x7fc674011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2289069c1a2c094 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.436+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc66c0a33b0 0x7fc66c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.436+0000 7fc670ff9640 1 -- 172.21.15.37:0/1595975905 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67401e070 con 0x55e6584f14a0 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.436+0000 7fc670ff9640 1 -- 172.21.15.37:0/1595975905 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc674019d50 con 0x55e6584f14a0 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.436+0000 7fc670ff9640 1 -- 172.21.15.37:0/1595975905 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6740185d0 con 0x55e6584f14a0 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1595975905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 msgr2=0x55e6584f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6584f1880 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc674009c20 tx=0x7fc674011810 comp rx=0 tx=0).stop 2024-09-19T06:44:54.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc66c0a33b0 0x7fc66c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc67401b000 tx=0x7fc674018cf0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1595975905 shutdown_connections 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584fc680 0x55e6584fea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584f1dc0 0x55e6584fc140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1595975905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6584f1880 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1595975905 >> 172.21.15.37:0/1595975905 conn(0x55e6584eda30 msgr2=0x55e6584ede10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1595975905 shutdown_connections 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1595975905 wait complete. 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 -- start start 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.437+0000 7fc68ae2d740 1 Processor -- start 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6584fbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e6584f9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e6584f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6584ff250 con 0x55e6584f1dc0 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e658412050 con 0x55e6584f14a0 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e6584ff3d0 con 0x55e6584fc680 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68ae2d740 10 client.0 _ll_register_callbacks cb 0x55e658452160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6584fbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e6584f9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6584fbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47556/0 (socket says 172.21.15.37:47556) 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 -- 172.21.15.37:0/1387744253 learned_addr learned my addr 172.21.15.37:0/1387744253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e6584f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 -- 172.21.15.37:0/1387744253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 msgr2=0x55e6584f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e6584f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 -- 172.21.15.37:0/1387744253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 msgr2=0x55e6584f9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e6584f9180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 -- 172.21.15.37:0/1387744253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6581c83c0 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e6584f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc68259c640 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6584fbed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009cd0 tx=0x7fc67c003970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a1a64620d008d59 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.438+0000 7fc669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc669ffb640 1 -- 172.21.15.37:0/1387744253 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c003d30 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc669ffb640 1 -- 172.21.15.37:0/1387744253 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc67c004040 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc674019d50 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc669ffb640 1 -- 172.21.15.37:0/1387744253 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc67c004b40 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 10 client.25793 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e65850aac0 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc669ffb640 1 -- 172.21.15.37:0/1387744253 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc674019d50 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc669ffb640 1 -- 172.21.15.37:0/1387744253 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc67c030070 con 0x55e6584f14a0 2024-09-19T06:44:54.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 10 client.25793 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 10 client.25793 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 20 client.25793 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e65850bfd0 0x55e65850e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 4 client.25793 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.439+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e658464850 con 0x55e65850bfd0 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.440+0000 7fc681d9b640 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e65850bfd0 0x55e65850e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.440+0000 7fc669ffb640 10 client.25793 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.440+0000 7fc681d9b640 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e65850bfd0 0x55e65850e450 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a460aeb6fd6072d server_cookie=160896844381a80e in_seq=0 out_seq=0 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc669ffb640 1 -- 172.21.15.37:0/1387744253 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e658464850 con 0x55e65850bfd0 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc669ffb640 10 client.25793 handle_command_reply: tid=0 2024-09-19T06:44:54.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 client.25793 shutdown 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 msgr2=0x55e6584fbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6584fbed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc67c009cd0 tx=0x7fc67c003970 comp rx=0 tx=0).stop 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 shutdown_connections 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e65850bfd0 0x55e65850e450 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e6584f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e6584f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc669ffb640 0 client.25793 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1387744253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6584fbed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 >> 172.21.15.37:0/1387744253 conn(0x55e6584eda30 msgr2=0x55e6584ecb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 shutdown_connections 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1387744253 wait complete. 2024-09-19T06:44:54.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.441+0000 7fc68ae2d740 20 client.25793 trim_cache size 0 max 16384 2024-09-19T06:44:54.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 Processor -- start 2024-09-19T06:44:54.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 -- start start 2024-09-19T06:44:54.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6585084e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f1dc0 0x55e658513d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658514260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6841b2470 con 0x55e6584f14a0 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc6841afce0 con 0x55e6584f1dc0 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6841adb30 con 0x55e6584fc680 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6585084e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f1dc0 0x55e658513d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6585084e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55522/0 (socket says 172.21.15.37:55522) 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f1dc0 0x55e658513d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47568/0 (socket says 172.21.15.37:47568) 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 -- 172.21.15.37:0/1935226511 learned_addr learned my addr 172.21.15.37:0/1935226511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc682d9d640 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658514260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 -- 172.21.15.37:0/1935226511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 msgr2=0x55e658514260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658514260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 -- 172.21.15.37:0/1935226511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f1dc0 msgr2=0x55e658513d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f1dc0 0x55e658513d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.460+0000 7fc68259c640 1 -- 172.21.15.37:0/1935226511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6584fc240 con 0x55e6584f14a0 2024-09-19T06:44:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc68259c640 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6585084e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc67c031d50 tx=0x7fc67c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2e2f510791e809 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc66bfff640 1 -- 172.21.15.37:0/1935226511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c030ac0 con 0x55e6584f14a0 2024-09-19T06:44:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc66bfff640 1 -- 172.21.15.37:0/1935226511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c036040 con 0x55e6584f14a0 2024-09-19T06:44:54.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc66bfff640 1 -- 172.21.15.37:0/1935226511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c04c400 con 0x55e6584f14a0 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1935226511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 msgr2=0x55e6585084e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6585084e0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fc67c031d50 tx=0x7fc67c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1935226511 shutdown_connections 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.461+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658514260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f1dc0 0x55e658513d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/1935226511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f14a0 0x55e6585084e0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1935226511 >> 172.21.15.37:0/1935226511 conn(0x55e6584eda30 msgr2=0x55e6584f7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1935226511 shutdown_connections 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 -- 172.21.15.37:0/1935226511 wait complete. 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 -- start start 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 Processor -- start 2024-09-19T06:44:54.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6585003c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e658500900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658500e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc6841ae1a0 con 0x55e6584f1dc0 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc6841af610 con 0x55e6584f14a0 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc6841ae870 con 0x55e6584fc680 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68ae2d740 10 client.0 _ll_register_callbacks cb 0x7fc67c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.462+0000 7fc68259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6585003c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e658500900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc68259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6585003c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47570/0 (socket says 172.21.15.37:47570) 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e658500900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55528/0 (socket says 172.21.15.37:55528) 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc68259c640 1 -- 172.21.15.37:0/2212864189 learned_addr learned my addr 172.21.15.37:0/2212864189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc682d9d640 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658500e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 -- 172.21.15.37:0/2212864189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 msgr2=0x55e658500e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658500e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 -- 172.21.15.37:0/2212864189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 msgr2=0x55e6585003c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6585003c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 -- 172.21.15.37:0/2212864189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e658575d20 con 0x55e6584f1dc0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc681d9b640 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e658500900 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6780cd1e0 tx=0x7fc6780045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=166e0fa9cdd4ddcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc669ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc669ffb640 1 -- 172.21.15.37:0/2212864189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6780d5040 con 0x55e6584f1dc0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc669ffb640 1 -- 172.21.15.37:0/2212864189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc6780035f0 con 0x55e6584f1dc0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc67c036040 con 0x55e6584f1dc0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc669ffb640 1 -- 172.21.15.37:0/2212864189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc67809b700 con 0x55e6584f1dc0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc68ae2d740 10 client.16290 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.463+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e658577050 con 0x55e6584f1dc0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc669ffb640 1 -- 172.21.15.37:0/2212864189 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc67c036040 con 0x55e6584f1dc0 2024-09-19T06:44:54.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc669ffb640 1 -- 172.21.15.37:0/2212864189 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc678098410 con 0x55e6584f1dc0 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68ae2d740 10 client.16290 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68ae2d740 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68ae2d740 20 client.16290 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e658577830 0x55e658579c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68ae2d740 4 client.16290 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e65845ed60 con 0x55e658577830 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68259c640 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e658577830 0x55e658579c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc68259c640 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e658577830 0x55e658579c50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ba0775ed9d780 server_cookie=7c2201a548adf1ea in_seq=0 out_seq=0 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.464+0000 7fc669ffb640 10 client.16290 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.465+0000 7fc669ffb640 1 -- 172.21.15.37:0/2212864189 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e65845ed60 con 0x55e658577830 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.465+0000 7fc669ffb640 10 client.16290 handle_command_reply: tid=0 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 client.16290 shutdown 2024-09-19T06:44:54.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 msgr2=0x55e658500900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e658500900 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6780cd1e0 tx=0x7fc6780045f0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 shutdown_connections 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e658577830 0x55e658579c50 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6584fc680 0x55e658500e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6584f1dc0 0x55e658500900 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 --2- 172.21.15.37:0/2212864189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6584f14a0 0x55e6585003c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc669ffb640 0 client.16290 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 >> 172.21.15.37:0/2212864189 conn(0x55e6584eda30 msgr2=0x55e6584ecb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 shutdown_connections 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 1 -- 172.21.15.37:0/2212864189 wait complete. 2024-09-19T06:44:54.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc68ae2d740 20 client.16290 trim_cache size 0 max 16384 2024-09-19T06:44:54.472 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:44:54.472 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:44:54.472 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:44:54.472 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc66c0a33b0 msgr2=0x7fc66c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc688a54640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc66c0a33b0 0x7fc66c0a5870 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc67401b000 tx=0x7fc674018cf0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684103740 msgr2=0x7fc684100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.466+0000 7fc688a54640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684103740 0x7fc684100f30 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc67800c970 tx=0x7fc67800ce30 comp rx=0 tx=0).stop 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 shutdown_connections 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc684108290 0x7fc684101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc66c0a33b0 0x7fc66c0a5870 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc684103740 0x7fc684100f30 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 --2- 172.21.15.37:0/1137774690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc684102d90 0x7fc6841009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 >> 172.21.15.37:0/1137774690 conn(0x7fc6840fc7d0 msgr2=0x7fc684106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 shutdown_connections 2024-09-19T06:44:54.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:54.467+0000 7fc688a54640 1 -- 172.21.15.37:0/1137774690 wait complete. 2024-09-19T06:44:54.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:54 smithi096 ceph-mon[35004]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-09-19T06:44:54.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:54 smithi037 ceph-mon[31211]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-09-19T06:44:54.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:54 smithi151 ceph-mon[35236]: pgmap v97: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 5.4 MiB/s wr, 576 op/s 2024-09-19T06:44:55.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 171185 11.46 MB/sec warmup 82 sec latency 35.701 ms 2024-09-19T06:44:55.306 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:44:55.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:44:55.585 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 -- 172.21.15.37:0/173647722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 msgr2=0x7fe16410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 --2- 172.21.15.37:0/173647722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 0x7fe16410ec70 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fe15c009cd0 tx=0x7fe15c02a8d0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 -- 172.21.15.37:0/173647722 shutdown_connections 2024-09-19T06:44:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 --2- 172.21.15.37:0/173647722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 0x7fe16410ec70 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 --2- 172.21.15.37:0/173647722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe164103780 0x7fe164107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 --2- 172.21.15.37:0/173647722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 0x7fe1641031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 -- 172.21.15.37:0/173647722 >> 172.21.15.37:0/173647722 conn(0x7fe1640fc7d0 msgr2=0x7fe1640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.062+0000 7fe169375640 1 -- 172.21.15.37:0/173647722 shutdown_connections 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.063+0000 7fe169375640 1 -- 172.21.15.37:0/173647722 wait complete. 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.063+0000 7fe169375640 1 Processor -- start 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.063+0000 7fe169375640 1 -- start start 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe169375640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 0x7fe1641984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe169375640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe164103780 0x7fe164198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe169375640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 0x7fe16419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe169375640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe1641120d0 con 0x7fe164108300 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe169375640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe164111f50 con 0x7fe164102dd0 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe169375640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe164112250 con 0x7fe164103780 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 0x7fe1641984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:56.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 0x7fe1641984f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47576/0 (socket says 172.21.15.37:47576) 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 -- 172.21.15.37:0/3595134675 learned_addr learned my addr 172.21.15.37:0/3595134675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe1637fe640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 0x7fe16419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe1627fc640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe164103780 0x7fe164198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 -- 172.21.15.37:0/3595134675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe164103780 msgr2=0x7fe164198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe164103780 0x7fe164198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 -- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 msgr2=0x7fe16419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 0x7fe16419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe162ffd640 1 -- 172.21.15.37:0/3595134675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe16419d4a0 con 0x7fe164102dd0 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe1627fc640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe164103780 0x7fe164198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.064+0000 7fe1637fe640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 0x7fe16419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.065+0000 7fe162ffd640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 0x7fe1641984f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e9f0 tx=0x7fe15400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.065+0000 7fe14bfff640 1 -- 172.21.15.37:0/3595134675 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15400cd80 con 0x7fe164102dd0 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.065+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe16419d790 con 0x7fe164102dd0 2024-09-19T06:44:56.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.065+0000 7fe14bfff640 1 -- 172.21.15.37:0/3595134675 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe154010910 con 0x7fe164102dd0 2024-09-19T06:44:56.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.065+0000 7fe14bfff640 1 -- 172.21.15.37:0/3595134675 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1540188f0 con 0x7fe164102dd0 2024-09-19T06:44:56.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.065+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1641a5070 con 0x7fe164102dd0 2024-09-19T06:44:56.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.066+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1641031b0 con 0x7fe164102dd0 2024-09-19T06:44:56.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.067+0000 7fe14bfff640 1 -- 172.21.15.37:0/3595134675 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe154010ac0 con 0x7fe164102dd0 2024-09-19T06:44:56.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.068+0000 7fe14bfff640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe14c0a33b0 0x7fe14c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:56.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.068+0000 7fe14bfff640 1 -- 172.21.15.37:0/3595134675 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fe1540d05c0 con 0x7fe164102dd0 2024-09-19T06:44:56.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.068+0000 7fe1627fc640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe14c0a33b0 0x7fe14c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:56.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.068+0000 7fe1627fc640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe14c0a33b0 0x7fe14c0a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe158005e10 tx=0x7fe158005da0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:56.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.071+0000 7fe14bfff640 1 -- 172.21.15.37:0/3595134675 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe15409cf40 con 0x7fe164102dd0 2024-09-19T06:44:56.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 174083 11.54 MB/sec warmup 83 sec latency 102.920 ms 2024-09-19T06:44:56.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.297+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe164199850 con 0x7fe164102dd0 2024-09-19T06:44:56.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.298+0000 7fe14bfff640 1 -- 172.21.15.37:0/3595134675 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11906 (secure 0 0 0) 0x7fe15409c840 con 0x7fe164102dd0 2024-09-19T06:44:56.299 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:44:56.299 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"btime":"2024-09-19T06:44:24:345285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:44:24.345273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:44:56.299 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe14c0a33b0 msgr2=0x7fe14c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe14c0a33b0 0x7fe14c0a5870 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe158005e10 tx=0x7fe158005da0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 msgr2=0x7fe1641984f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 0x7fe1641984f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe15400e9f0 tx=0x7fe15400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 shutdown_connections 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe14c0a33b0 0x7fe14c0a5870 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe164108300 0x7fe16419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe164103780 0x7fe164198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 --2- 172.21.15.37:0/3595134675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe164102dd0 0x7fe1641984f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 >> 172.21.15.37:0/3595134675 conn(0x7fe1640fc7d0 msgr2=0x7fe164105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:56.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 shutdown_connections 2024-09-19T06:44:56.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:56.300+0000 7fe169375640 1 -- 172.21.15.37:0/3595134675 wait complete. 2024-09-19T06:44:56.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:56 smithi037 ceph-mon[31211]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-09-19T06:44:56.669 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:56 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3595134675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:44:56.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:56 smithi096 ceph-mon[35004]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-09-19T06:44:56.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:56 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3595134675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:44:56.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:56 smithi151 ceph-mon[35236]: pgmap v98: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-09-19T06:44:56.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:56 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3595134675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:44:57.076 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:44:57.193 INFO:tasks.workunit.client.0.smithi037.stdout: 1 176834 11.55 MB/sec warmup 84 sec latency 21.628 ms 2024-09-19T06:44:57.357 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:57.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/566423094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8106f10 msgr2=0x7f4cd810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/566423094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8106f10 0x7f4cd810ea90 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0009ca0 tx=0x7f4cd002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/566423094 shutdown_connections 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/566423094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8106f10 0x7f4cd810ea90 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/566423094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd8102790 0x7f4cd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/566423094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cd8101de0 0x7f4cd81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/566423094 >> 172.21.15.37:0/566423094 conn(0x7f4cd80fd8c0 msgr2=0x7f4cd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/566423094 shutdown_connections 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.827+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/566423094 wait complete. 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 Processor -- start 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 -- start start 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8101de0 0x7f4cd81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cd8102790 0x7f4cd8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd8106f10 0x7f4cd819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8111f70 con 0x7f4cd8101de0 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd8111df0 con 0x7f4cd8106f10 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.828+0000 7f4cdf6c1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81120f0 con 0x7f4cd8102790 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8101de0 0x7f4cd81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8101de0 0x7f4cd81983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55572/0 (socket says 172.21.15.37:55572) 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4143283976 learned_addr learned my addr 172.21.15.37:0/4143283976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cddc5e640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd8106f10 0x7f4cd819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cd8102790 0x7f4cd8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4143283976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cd8102790 msgr2=0x7f4cd8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cd8102790 0x7f4cd8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4143283976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd8106f10 msgr2=0x7f4cd819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd8106f10 0x7f4cd819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4143283976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cd819d380 con 0x7f4cd8101de0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8101de0 0x7f4cd81983d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400d770 tx=0x7f4cd400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cce7fc640 1 -- 172.21.15.37:0/4143283976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4017070 con 0x7f4cd8101de0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cce7fc640 1 -- 172.21.15.37:0/4143283976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd4005500 con 0x7f4cd8101de0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cd819d670 con 0x7f4cd8101de0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cce7fc640 1 -- 172.21.15.37:0/4143283976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4004e00 con 0x7f4cd8101de0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.829+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cd81a4f50 con 0x7f4cd8101de0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.832+0000 7f4cce7fc640 1 -- 172.21.15.37:0/4143283976 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4cd4004fa0 con 0x7f4cd8101de0 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 Processor -- start 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 -- start start 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cce7fc640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4cc40a3430 0x7f4cc40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71ac090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71aca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71b9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c8f71122f0 con 0x55c8f71ac5d0 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8f70b9cf0 con 0x55c8f71adb40 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c8f710e2b0 con 0x55c8f71b7370 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cddc5e640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4cc40a3430 0x7f4cc40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cce7fc640 1 -- 172.21.15.37:0/4143283976 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4cd40cf6e0 con 0x7f4cd8101de0 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdcc5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47616/0 (socket says 172.21.15.37:47616) 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdd45d640 1 -- 172.21.15.37:0/3233290342 learned_addr learned my addr 172.21.15.37:0/3233290342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:57.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cddc5e640 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71b9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdd45d640 1 -- 172.21.15.37:0/3233290342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 msgr2=0x55c8f71b9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71b9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdd45d640 1 -- 172.21.15.37:0/3233290342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 msgr2=0x55c8f71aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.833+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.834+0000 7f4cdd45d640 1 -- 172.21.15.37:0/3233290342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8f709e3a0 con 0x55c8f71adb40 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.834+0000 7f4cddc5e640 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.834+0000 7f4cddc5e640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4cc40a3430 0x7f4cc40a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004af0 tx=0x7f4cd0031c90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.834+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71ac090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4097e70 tx=0x7f4cd4097d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8ae1c167ab66ca1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.834+0000 7f4cc37fe640 1 -- 172.21.15.37:0/3233290342 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4020070 con 0x55c8f71adb40 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.834+0000 7f4cc37fe640 1 -- 172.21.15.37:0/3233290342 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd409fd10 con 0x55c8f71adb40 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.834+0000 7f4cc37fe640 1 -- 172.21.15.37:0/3233290342 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4098080 con 0x55c8f71adb40 2024-09-19T06:44:57.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.835+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/3233290342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 msgr2=0x55c8f71ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.835+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71ac090 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4097e70 tx=0x7f4cd4097d30 comp rx=0 tx=0).stop 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/3233290342 shutdown_connections 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71b9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/3233290342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71ac090 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/3233290342 >> 172.21.15.37:0/3233290342 conn(0x55c8f7175310 msgr2=0x55c8f71a8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/3233290342 shutdown_connections 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/3233290342 wait complete. 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 -- start start 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.836+0000 7f4ce1a9a740 1 Processor -- start 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71b6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71b4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c8f71be3c0 con 0x55c8f71b7370 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c8f70cd060 con 0x55c8f71ac5d0 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c8f71be540 con 0x55c8f71adb40 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4ce1a9a740 10 client.0 _ll_register_callbacks cb 0x55c8f6e208c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71b6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71b6d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47624/0 (socket says 172.21.15.37:47624) 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 -- 172.21.15.37:0/655714019 learned_addr learned my addr 172.21.15.37:0/655714019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cddc5e640 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 -- 172.21.15.37:0/655714019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 msgr2=0x55c8f71b4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71b4020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 -- 172.21.15.37:0/655714019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 msgr2=0x55c8f71b4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 -- 172.21.15.37:0/655714019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8f6e833c0 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71b6d70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4cd409b710 tx=0x7f4cd4005bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3037fac29583076 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.837+0000 7f4cc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/655714019 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40b4060 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/655714019 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4cd4098080 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd409fd10 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/655714019 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cd4020050 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 10 client.25817 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8f71c9e60 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/655714019 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd404e070 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/655714019 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4cd4052bc0 con 0x55c8f71ac5d0 2024-09-19T06:44:57.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 10 client.25817 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 10 client.25817 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 20 client.25817 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f71cb1b0 0x55c8f71cd5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 4 client.25817 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.838+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8f711f860 con 0x55c8f71cb1b0 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.839+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f71cb1b0 0x55c8f71cd5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.839+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f71cb1b0 0x55c8f71cd5a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6a196b82ad5e904c server_cookie=fc77c6f845748270 in_seq=0 out_seq=0 2024-09-19T06:44:57.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.839+0000 7f4cc0ff9640 10 client.25817 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/655714019 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c8f711f860 con 0x55c8f71cb1b0 2024-09-19T06:44:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4cc0ff9640 10 client.25817 handle_command_reply: tid=0 2024-09-19T06:44:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 client.25817 shutdown 2024-09-19T06:44:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 msgr2=0x55c8f71b6d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71b6d70 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4cd409b710 tx=0x7f4cd4005bb0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 shutdown_connections 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f71cb1b0 0x55c8f71cd5a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71b4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/655714019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71b6d70 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 >> 172.21.15.37:0/655714019 conn(0x55c8f7175310 msgr2=0x55c8f71ae960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4cc0ff9640 0 client.25817 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 shutdown_connections 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/655714019 wait complete. 2024-09-19T06:44:57.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.840+0000 7f4ce1a9a740 20 client.25817 trim_cache size 0 max 16384 2024-09-19T06:44:57.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 Processor -- start 2024-09-19T06:44:57.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 -- start start 2024-09-19T06:44:57.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71af120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71af660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81adfe0 con 0x55c8f71ac5d0 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81ab850 con 0x55c8f71adb40 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81a96a0 con 0x55c8f71b7370 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55592/0 (socket says 172.21.15.37:55592) 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 -- 172.21.15.37:0/82123906 learned_addr learned my addr 172.21.15.37:0/82123906 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71af120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cddc5e640 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71af660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 -- 172.21.15.37:0/82123906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 msgr2=0x55c8f71af660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71af660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 -- 172.21.15.37:0/82123906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 msgr2=0x55c8f71af120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71af120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.856+0000 7f4cdd45d640 1 -- 172.21.15.37:0/82123906 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8f71b6990 con 0x55c8f71ac5d0 2024-09-19T06:44:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.857+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c7660 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400d740 tx=0x7f4cd409bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32289100c1ee240a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.857+0000 7f4cc2ffd640 1 -- 172.21.15.37:0/82123906 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40b4070 con 0x55c8f71ac5d0 2024-09-19T06:44:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.857+0000 7f4cc2ffd640 1 -- 172.21.15.37:0/82123906 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd404e030 con 0x55c8f71ac5d0 2024-09-19T06:44:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.857+0000 7f4cc2ffd640 1 -- 172.21.15.37:0/82123906 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4009cc0 con 0x55c8f71ac5d0 2024-09-19T06:44:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.857+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/82123906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 msgr2=0x55c8f71c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.857+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c7660 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400d740 tx=0x7f4cd409bb90 comp rx=0 tx=0).stop 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.857+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/82123906 shutdown_connections 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71b7370 0x55c8f71af660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71adb40 0x55c8f71af120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/82123906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c7660 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/82123906 >> 172.21.15.37:0/82123906 conn(0x55c8f7175310 msgr2=0x55c8f71b3130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/82123906 shutdown_connections 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/82123906 wait complete. 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 -- start start 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 Processor -- start 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81a9d10 con 0x55c8f71b7370 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81ab180 con 0x55c8f71ac5d0 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.858+0000 7f4ce1a9a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4cd81aa3e0 con 0x55c8f71adb40 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4ce1a9a740 10 client.0 _ll_register_callbacks cb 0x7f4cc80097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cddc5e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c3a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47646/0 (socket says 172.21.15.37:47646) 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4025608667 learned_addr learned my addr 172.21.15.37:0/4025608667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4025608667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 msgr2=0x55c8f71c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4025608667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 msgr2=0x55c8f71b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 -- 172.21.15.37:0/4025608667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8f71b11d0 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cdd45d640 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c3a90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4cd409ae80 tx=0x7f4cd400df40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4348ce5db4ce076 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cc0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/4025608667 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4080060 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/4025608667 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4cd40cd050 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd404e030 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/4025608667 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4cd40b4070 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4ce1a9a740 10 client.25823 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.859+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8f71b11d0 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/4025608667 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4cd404e030 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/4025608667 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4cd40b4b60 con 0x55c8f71ac5d0 2024-09-19T06:44:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4ce1a9a740 10 client.25823 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4ce1a9a740 10 client.25823 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4ce1a9a740 20 client.25823 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f7235ab0 0x55c8f7237ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4ce1a9a740 4 client.25823 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8f7119d70 con 0x55c8f7235ab0 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.860+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f7235ab0 0x55c8f7237ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4cdcc5c640 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f7235ab0 0x55c8f7237ea0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=74b636787a8d284b server_cookie=f99072b20e760f94 in_seq=0 out_seq=0 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4cc0ff9640 10 client.25823 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4cc0ff9640 1 -- 172.21.15.37:0/4025608667 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8f7119d70 con 0x55c8f7235ab0 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4cc0ff9640 10 client.25823 handle_command_reply: tid=0 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 client.25823 shutdown 2024-09-19T06:44:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 msgr2=0x55c8f71c3a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c3a90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4cd409ae80 tx=0x7f4cd400df40 comp rx=0 tx=0).stop 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 shutdown_connections 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c8f7235ab0 0x55c8f7237ea0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c8f71b7370 0x55c8f71b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c8f71adb40 0x55c8f71c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 --2- 172.21.15.37:0/4025608667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c8f71ac5d0 0x55c8f71c3a90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4cc0ff9640 0 client.25823 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 >> 172.21.15.37:0/4025608667 conn(0x55c8f7175310 msgr2=0x55c8f71b5880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 shutdown_connections 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 1 -- 172.21.15.37:0/4025608667 wait complete. 2024-09-19T06:44:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.861+0000 7f4ce1a9a740 20 client.25823 trim_cache size 0 max 16384 2024-09-19T06:44:57.868 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:44:57.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4cc40a3430 msgr2=0x7f4cc40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4cc40a3430 0x7f4cc40a58f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f4cd0004af0 tx=0x7f4cd0031c90 comp rx=0 tx=0).stop 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8101de0 msgr2=0x7f4cd81983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8101de0 0x7f4cd81983d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f4cd400d770 tx=0x7f4cd400dc30 comp rx=0 tx=0).stop 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 shutdown_connections 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4cd8106f10 0x7f4cd819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4cd8102790 0x7f4cd8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4cc40a3430 0x7f4cc40a58f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 --2- 172.21.15.37:0/4143283976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4cd8101de0 0x7f4cd81983d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 >> 172.21.15.37:0/4143283976 conn(0x7f4cd80fd8c0 msgr2=0x7f4cd80feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 shutdown_connections 2024-09-19T06:44:57.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:57.862+0000 7f4cdf6c1640 1 -- 172.21.15.37:0/4143283976 wait complete. 2024-09-19T06:44:58.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 179058 11.56 MB/sec warmup 85 sec latency 39.599 ms 2024-09-19T06:44:58.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:44:58.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:58 smithi096 ceph-mon[35004]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-09-19T06:44:58.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:58 smithi037 ceph-mon[31211]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-09-19T06:44:58.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:58 smithi151 ceph-mon[35236]: pgmap v99: 97 pgs: 97 active+clean; 45 MiB data, 620 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 3.6 MiB/s wr, 422 op/s 2024-09-19T06:44:58.932 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:44:59.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 181345 11.58 MB/sec warmup 86 sec latency 60.630 ms 2024-09-19T06:44:59.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:59.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:59.652 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:59.653 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:44:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:44:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 -- 172.21.15.37:0/1039831293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1082a0 msgr2=0x7f09fc10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 --2- 172.21.15.37:0/1039831293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1082a0 0x7f09fc10ec40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009cd0 tx=0x7f09f402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 -- 172.21.15.37:0/1039831293 shutdown_connections 2024-09-19T06:44:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 --2- 172.21.15.37:0/1039831293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1082a0 0x7f09fc10ec40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 --2- 172.21.15.37:0/1039831293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09fc1036e0 0x7f09fc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 --2- 172.21.15.37:0/1039831293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc102d30 0x7f09fc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 -- 172.21.15.37:0/1039831293 >> 172.21.15.37:0/1039831293 conn(0x7f09fc0fc7d0 msgr2=0x7f09fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 -- 172.21.15.37:0/1039831293 shutdown_connections 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 -- 172.21.15.37:0/1039831293 wait complete. 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 Processor -- start 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.647+0000 7f0a01f02640 1 -- start start 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f0a01f02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09fc102d30 0x7f09fc19a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f0a01f02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1036e0 0x7f09fc19ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f0a01f02640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc1082a0 0x7f09fc19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f0a01f02640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1120a0 con 0x7f09fc1036e0 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f0a01f02640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc111f20 con 0x7f09fc102d30 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f0a01f02640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc112220 con 0x7f09fc1082a0 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1036e0 0x7f09fc19ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1036e0 0x7f09fc19ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36870/0 (socket says 172.21.15.37:36870) 2024-09-19T06:44:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 -- 172.21.15.37:0/3478257321 learned_addr learned my addr 172.21.15.37:0/3478257321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09fc102d30 0x7f09fc19a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09fbfff640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc1082a0 0x7f09fc19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 -- 172.21.15.37:0/3478257321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc1082a0 msgr2=0x7f09fc19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc1082a0 0x7f09fc19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 -- 172.21.15.37:0/3478257321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09fc102d30 msgr2=0x7f09fc19a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09fc102d30 0x7f09fc19a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 -- 172.21.15.37:0/3478257321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc19f700 con 0x7f09fc1036e0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09faffd640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1036e0 0x7f09fc19ac90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f09f000e9f0 tx=0x7f09f000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.648+0000 7f09f8ff9640 1 -- 172.21.15.37:0/3478257321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f000ccc0 con 0x7f09fc1036e0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.649+0000 7f09f8ff9640 1 -- 172.21.15.37:0/3478257321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f0010910 con 0x7f09fc1036e0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.649+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc19f9f0 con 0x7f09fc1036e0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.649+0000 7f09f8ff9640 1 -- 172.21.15.37:0/3478257321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0018920 con 0x7f09fc1036e0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.649+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc1a72d0 con 0x7f09fc1036e0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.651+0000 7f09f8ff9640 1 -- 172.21.15.37:0/3478257321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f09f0010ac0 con 0x7f09fc1036e0 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.651+0000 7f0a042db740 1 Processor -- start 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.651+0000 7f0a042db740 1 -- start start 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.651+0000 7f0a042db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.651+0000 7f0a042db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d2974ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f0a042db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d29759760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f0a042db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555d296b22f0 con 0x555d29757370 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f0a042db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555d29659cf0 con 0x555d2974db40 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f0a042db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d296ae2b0 con 0x555d2974c5d0 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44248/0 (socket says 172.21.15.37:44248) 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 -- 172.21.15.37:0/2996288994 learned_addr learned my addr 172.21.15.37:0/2996288994 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09faffd640 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d29759760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d2974ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 -- 172.21.15.37:0/2996288994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 msgr2=0x555d2974ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d2974ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 -- 172.21.15.37:0/2996288994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 msgr2=0x555d29759760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d29759760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 -- 172.21.15.37:0/2996288994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d2963e3a0 con 0x555d2974db40 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09f8ff9640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09e40a33b0 0x7f09e40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09e40a33b0 0x7f09e40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09fbfff640 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974c090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009cd0 tx=0x7f09f4009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e3a37a9875075cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.652+0000 7f09f8ff9640 1 -- 172.21.15.37:0/3478257321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f09f00cffa0 con 0x7f09fc1036e0 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f09e1ffb640 1 -- 172.21.15.37:0/2996288994 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4048070 con 0x555d2974db40 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f09e1ffb640 1 -- 172.21.15.37:0/2996288994 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f4036070 con 0x555d2974db40 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f09e1ffb640 1 -- 172.21.15.37:0/2996288994 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f4004470 con 0x555d2974db40 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 -- 172.21.15.37:0/2996288994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 msgr2=0x555d2974c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09e40a33b0 0x7f09e40a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x555d29749ac0 tx=0x7f09ec00b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:59.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974c090 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f09f4009cd0 tx=0x7f09f4009a20 comp rx=0 tx=0).stop 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 -- 172.21.15.37:0/2996288994 shutdown_connections 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d29759760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d2974ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 --2- 172.21.15.37:0/2996288994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974c090 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 -- 172.21.15.37:0/2996288994 >> 172.21.15.37:0/2996288994 conn(0x555d29715310 msgr2=0x555d29748c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 -- 172.21.15.37:0/2996288994 shutdown_connections 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.653+0000 7f0a042db740 1 -- 172.21.15.37:0/2996288994 wait complete. 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 -- start start 2024-09-19T06:44:59.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 Processor -- start 2024-09-19T06:44:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d29756de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d29754090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d297547d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555d2975a010 con 0x555d2974c5d0 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555d2966d060 con 0x555d29757370 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555d2975a190 con 0x555d2974db40 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f0a042db740 10 client.0 _ll_register_callbacks cb 0x555d293c08c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d29756de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d29756de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36892/0 (socket says 172.21.15.37:36892) 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.654+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d297547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09fb7fe640 1 -- 172.21.15.37:0/2346119489 learned_addr learned my addr 172.21.15.37:0/2346119489 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09faffd640 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d29754090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09fb7fe640 1 -- 172.21.15.37:0/2346119489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 msgr2=0x555d29754090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d29754090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09fb7fe640 1 -- 172.21.15.37:0/2346119489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 msgr2=0x555d297547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d297547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09fb7fe640 1 -- 172.21.15.37:0/2346119489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d294233c0 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d29756de0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003e90 tx=0x7f09ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac01bf649db0c19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09daffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09daffd640 1 -- 172.21.15.37:0/2346119489 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec01b660 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09daffd640 1 -- 172.21.15.37:0/2346119489 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f09ec01b800 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f4036070 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f09daffd640 1 -- 172.21.15.37:0/2346119489 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09ec01ba00 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f0a042db740 10 client.16326 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.655+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d29765bf0 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f09daffd640 1 -- 172.21.15.37:0/2346119489 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09f4036070 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f09daffd640 1 -- 172.21.15.37:0/2346119489 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f09ec024070 con 0x555d2974c5d0 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f0a042db740 10 client.16326 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f0a042db740 10 client.16326 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:44:59.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f0a042db740 20 client.16326 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f0a042db740 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297670b0 0x555d297694a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f0a042db740 4 client.16326 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555d296bf860 con 0x555d297670b0 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f09faffd640 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297670b0 0x555d297694a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f09daffd640 10 client.16326 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.656+0000 7f09faffd640 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297670b0 0x555d297694a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b4f2f638a23cdca1 server_cookie=c79d185716e0eb64 in_seq=0 out_seq=0 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.657+0000 7f09daffd640 1 -- 172.21.15.37:0/2346119489 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555d296bf860 con 0x555d297670b0 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.657+0000 7f09daffd640 10 client.16326 handle_command_reply: tid=0 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.657+0000 7f0a042db740 1 client.16326 shutdown 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.657+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 msgr2=0x555d29756de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.657+0000 7f0a042db740 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d29756de0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003e90 tx=0x7f09ec0046c0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.657+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 shutdown_connections 2024-09-19T06:44:59.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.657+0000 7f0a042db740 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297670b0 0x555d297694a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f0a042db740 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d297547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f0a042db740 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d29754090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f0a042db740 1 --2- 172.21.15.37:0/2346119489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d29756de0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 >> 172.21.15.37:0/2346119489 conn(0x555d29715310 msgr2=0x555d2974e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f09daffd640 0 client.16326 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 shutdown_connections 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f0a042db740 1 -- 172.21.15.37:0/2346119489 wait complete. 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.658+0000 7f0a042db740 20 client.16326 trim_cache size 0 max 16384 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 Processor -- start 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 -- start start 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d297632b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d2974f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1b23e0 con 0x555d29757370 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1afc50 con 0x555d2974db40 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.672+0000 7f0a042db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1adaa0 con 0x555d2974c5d0 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d2974f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d297632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d297632b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:43518/0 (socket says 172.21.15.37:43518) 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d2974f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36894/0 (socket says 172.21.15.37:36894) 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 -- 172.21.15.37:0/3898578761 learned_addr learned my addr 172.21.15.37:0/3898578761 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09faffd640 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 -- 172.21.15.37:0/3898578761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 msgr2=0x555d2974f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 -- 172.21.15.37:0/3898578761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 msgr2=0x555d2974f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d2974f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 -- 172.21.15.37:0/3898578761 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555d29748ed0 con 0x555d2974c5d0 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fbfff640 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d2974f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d297632b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003af0 tx=0x7f09ec0104a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a63d621676f295b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.673+0000 7f09e17fa640 1 -- 172.21.15.37:0/3898578761 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec01b660 con 0x555d2974c5d0 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f09e17fa640 1 -- 172.21.15.37:0/3898578761 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09ec01be30 con 0x555d2974c5d0 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f09e17fa640 1 -- 172.21.15.37:0/3898578761 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec010ca0 con 0x555d2974c5d0 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 -- 172.21.15.37:0/3898578761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 msgr2=0x555d297632b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d297632b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f09ec003af0 tx=0x7f09ec0104a0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 -- 172.21.15.37:0/3898578761 shutdown_connections 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d29757370 0x555d2974f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d2974db40 0x555d2974f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 --2- 172.21.15.37:0/3898578761 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974c5d0 0x555d297632b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 -- 172.21.15.37:0/3898578761 >> 172.21.15.37:0/3898578761 conn(0x555d29715310 msgr2=0x555d29753180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.674+0000 7f0a042db740 1 -- 172.21.15.37:0/3898578761 shutdown_connections 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 -- 172.21.15.37:0/3898578761 wait complete. 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 -- start start 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 Processor -- start 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d2975f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d2975acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d2975b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1ae110 con 0x555d2974c5d0 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1af580 con 0x555d29757370 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc1ae7e0 con 0x555d2974db40 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f0a042db740 10 client.0 _ll_register_callbacks cb 0x7f09ec004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:44:59.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d2975f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f09fb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d2975f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36910/0 (socket says 172.21.15.37:36910) 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f09fb7fe640 1 -- 172.21.15.37:0/589455297 learned_addr learned my addr 172.21.15.37:0/589455297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f09faffd640 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d2975acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.675+0000 7f09fbfff640 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d2975b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09fb7fe640 1 -- 172.21.15.37:0/589455297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 msgr2=0x555d2975acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d2975acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09fb7fe640 1 -- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 msgr2=0x555d2975b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d2975b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09fb7fe640 1 -- 172.21.15.37:0/589455297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555d2975b740 con 0x555d2974c5d0 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09fb7fe640 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d2975f820 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec010c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e9797c30aa568d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09daffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:44:59.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09daffd640 1 -- 172.21.15.37:0/589455297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09ec03b070 con 0x555d2974c5d0 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09ec01be30 con 0x555d2974c5d0 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09daffd640 1 -- 172.21.15.37:0/589455297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f09ec005450 con 0x555d2974c5d0 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f09daffd640 1 -- 172.21.15.37:0/589455297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09ec047050 con 0x555d2974c5d0 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f0a042db740 10 client.16332 fetch_fsmap learned FSMap version 29 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.676+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555d297d19f0 con 0x555d2974c5d0 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f09daffd640 1 -- 172.21.15.37:0/589455297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f09ec01be30 con 0x555d2974c5d0 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f09daffd640 1 -- 172.21.15.37:0/589455297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f09ec035020 con 0x555d2974c5d0 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f0a042db740 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f0a042db740 10 client.16332 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f0a042db740 20 client.16332 populate_metadata read hostname 'smithi037' 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f0a042db740 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297d2000 0x555d297d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f0a042db740 4 client.16332 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555d296b9d70 con 0x555d297d2000 2024-09-19T06:44:59.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f09faffd640 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297d2000 0x555d297d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f09faffd640 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297d2000 0x555d297d4420 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=802412d2001135b0 server_cookie=2e251ff8d6185b9c in_seq=0 out_seq=0 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.677+0000 7f09daffd640 10 client.16332 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f09daffd640 1 -- 172.21.15.37:0/589455297 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555d296b9d70 con 0x555d297d2000 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f09daffd640 10 client.16332 handle_command_reply: tid=0 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 client.16332 shutdown 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 msgr2=0x555d2975f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d2975f820 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f09ec000c00 tx=0x7f09ec010c50 comp rx=0 tx=0).stop 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 shutdown_connections 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555d297d2000 0x555d297d4420 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555d29757370 0x555d2975b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555d2974db40 0x555d2975acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 --2- 172.21.15.37:0/589455297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555d2974c5d0 0x555d2975f820 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 >> 172.21.15.37:0/589455297 conn(0x555d29715310 msgr2=0x555d297474e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:59.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f09daffd640 0 client.16332 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:44:59.739 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 shutdown_connections 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 1 -- 172.21.15.37:0/589455297 wait complete. 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.678+0000 7f0a042db740 20 client.16332 trim_cache size 0 max 16384 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09e40a33b0 msgr2=0x7f09e40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09e40a33b0 0x7f09e40a5870 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x555d29749ac0 tx=0x7f09ec00b390 comp rx=0 tx=0).stop 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1036e0 msgr2=0x7f09fc19ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1036e0 0x7f09fc19ac90 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f09f000e9f0 tx=0x7f09f000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 shutdown_connections 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09fc1082a0 0x7f09fc19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09e40a33b0 0x7f09e40a5870 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09fc1036e0 0x7f09fc19ac90 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 --2- 172.21.15.37:0/3478257321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09fc102d30 0x7f09fc19a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:44:59.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 >> 172.21.15.37:0/3478257321 conn(0x7f09fc0fc7d0 msgr2=0x7f09fc1008c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:44:59.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 shutdown_connections 2024-09-19T06:44:59.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:44:59.679+0000 7f0a01f02640 1 -- 172.21.15.37:0/3478257321 wait complete. 2024-09-19T06:44:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:59.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:44:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:44:59.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:44:59.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:44:59.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:44:59.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:44:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:45:00.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 184454 11.65 MB/sec warmup 87 sec latency 11.586 ms 2024-09-19T06:45:00.588 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:45:00.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:00 smithi096 ceph-mon[35004]: pgmap v100: 97 pgs: 97 active+clean; 50 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 563 op/s 2024-09-19T06:45:00.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:00 smithi037 ceph-mon[31211]: pgmap v100: 97 pgs: 97 active+clean; 50 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 563 op/s 2024-09-19T06:45:00.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:00 smithi151 ceph-mon[35236]: pgmap v100: 97 pgs: 97 active+clean; 50 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.3 MiB/s wr, 563 op/s 2024-09-19T06:45:00.877 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:01.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 185753 11.56 MB/sec warmup 88 sec latency 21.491 ms 2024-09-19T06:45:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 -- 172.21.15.37:0/1791947508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb09410e4e0 msgr2=0x7fb0941108d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 --2- 172.21.15.37:0/1791947508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb09410e4e0 0x7fb0941108d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fb08c009ca0 tx=0x7fb08c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 -- 172.21.15.37:0/1791947508 shutdown_connections 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 --2- 172.21.15.37:0/1791947508 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb09410e4e0 0x7fb0941108d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 --2- 172.21.15.37:0/1791947508 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0941012e0 0x7fb09410dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 --2- 172.21.15.37:0/1791947508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb094100930 0x7fb094100d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 -- 172.21.15.37:0/1791947508 >> 172.21.15.37:0/1791947508 conn(0x7fb0940fc7a0 msgr2=0x7fb0940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 -- 172.21.15.37:0/1791947508 shutdown_connections 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.336+0000 7fb09b040640 1 -- 172.21.15.37:0/1791947508 wait complete. 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 Processor -- start 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 -- start start 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb094100930 0x7fb09419a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0941012e0 0x7fb09419ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb09410e4e0 0x7fb09419eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941141c0 con 0x7fb094100930 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb094114040 con 0x7fb0941012e0 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb09b040640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb094114340 con 0x7fb09410e4e0 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb098ddc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb094100930 0x7fb09419a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb098ddc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb094100930 0x7fb09419a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36924/0 (socket says 172.21.15.37:36924) 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb098ddc640 1 -- 172.21.15.37:0/3399195314 learned_addr learned my addr 172.21.15.37:0/3399195314 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb093fff640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0941012e0 0x7fb09419ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.337+0000 7fb0995dd640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb09410e4e0 0x7fb09419eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb098ddc640 1 -- 172.21.15.37:0/3399195314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb09410e4e0 msgr2=0x7fb09419eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb09410e4e0 0x7fb09419eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb098ddc640 1 -- 172.21.15.37:0/3399195314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0941012e0 msgr2=0x7fb09419ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0941012e0 0x7fb09419ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb098ddc640 1 -- 172.21.15.37:0/3399195314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb09419f590 con 0x7fb094100930 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb093fff640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0941012e0 0x7fb09419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb094100930 0x7fb09419a5e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb08800e9f0 tx=0x7fb08800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb091ffb640 1 -- 172.21.15.37:0/3399195314 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08800cd80 con 0x7fb094100930 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb09419f880 con 0x7fb094100930 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb091ffb640 1 -- 172.21.15.37:0/3399195314 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb088010910 con 0x7fb094100930 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb091ffb640 1 -- 172.21.15.37:0/3399195314 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880188f0 con 0x7fb094100930 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.338+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0941a7160 con 0x7fb094100930 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.341+0000 7fb091ffb640 1 -- 172.21.15.37:0/3399195314 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb088010ac0 con 0x7fb094100930 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.341+0000 7fb09d419740 1 Processor -- start 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.341+0000 7fb09d419740 1 -- start start 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.341+0000 7fb091ffb640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0840a33b0 0x7fb0840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.341+0000 7fb091ffb640 1 -- 172.21.15.37:0/3399195314 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb0880d0160 con 0x7fb094100930 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb09d419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73fb40 0x55bcac73e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb0995dd640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0840a33b0 0x7fb0840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb09d419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac73ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb09d419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac74b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb09d419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcac6a42f0 con 0x55bcac73fb40 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb09d419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bcac64bcf0 con 0x55bcac749370 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb09d419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bcac6a02b0 con 0x55bcac73e5d0 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73fb40 0x55bcac73e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73fb40 0x55bcac73e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36926/0 (socket says 172.21.15.37:36926) 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 -- 172.21.15.37:0/1878544495 learned_addr learned my addr 172.21.15.37:0/1878544495 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb093fff640 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac73ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb0995dd640 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac74b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 -- 172.21.15.37:0/1878544495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 msgr2=0x55bcac73ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac73ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 -- 172.21.15.37:0/1878544495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 msgr2=0x55bcac74b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac74b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 -- 172.21.15.37:0/1878544495 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcac6303a0 con 0x55bcac73fb40 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb093fff640 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac73ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb098ddc640 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73fb40 0x55bcac73e090 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb088098e10 tx=0x7fb08809cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=665e1637bc9283ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb0995dd640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0840a33b0 0x7fb0840a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb08c006010 tx=0x7fb08c009a20 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb07effd640 1 -- 172.21.15.37:0/1878544495 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880cdb30 con 0x55bcac73fb40 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb07effd640 1 -- 172.21.15.37:0/1878544495 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb088098ab0 con 0x55bcac73fb40 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.342+0000 7fb07effd640 1 -- 172.21.15.37:0/1878544495 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880983f0 con 0x55bcac73fb40 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 -- 172.21.15.37:0/1878544495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73fb40 msgr2=0x55bcac73e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73fb40 0x55bcac73e090 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb088098e10 tx=0x7fb08809cc80 comp rx=0 tx=0).stop 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 -- 172.21.15.37:0/1878544495 shutdown_connections 2024-09-19T06:45:01.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac74b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac73ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 --2- 172.21.15.37:0/1878544495 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73fb40 0x55bcac73e090 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 -- 172.21.15.37:0/1878544495 >> 172.21.15.37:0/1878544495 conn(0x55bcac707310 msgr2=0x55bcac73ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 -- 172.21.15.37:0/1878544495 shutdown_connections 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.343+0000 7fb09d419740 1 -- 172.21.15.37:0/1878544495 wait complete. 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 -- start start 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 Processor -- start 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73e5d0 0x55bcac748d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac7466d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bcac74c070 con 0x55bcac73e5d0 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bcac65f060 con 0x55bcac749370 2024-09-19T06:45:01.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bcac74c1f0 con 0x55bcac73fb40 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb09d419740 10 client.0 _ll_register_callbacks cb 0x55bcac3b28c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb098ddc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73e5d0 0x55bcac748d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb098ddc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73e5d0 0x55bcac748d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36934/0 (socket says 172.21.15.37:36934) 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb0995dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac7466d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb098ddc640 1 -- 172.21.15.37:0/3498929824 learned_addr learned my addr 172.21.15.37:0/3498929824 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.344+0000 7fb093fff640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb098ddc640 1 -- 172.21.15.37:0/3498929824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 msgr2=0x55bcac746070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb098ddc640 1 -- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 msgr2=0x55bcac7466d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac7466d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb098ddc640 1 -- 172.21.15.37:0/3498929824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcac4153c0 con 0x55bcac73e5d0 2024-09-19T06:45:01.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb093fff640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb0995dd640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac7466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73e5d0 0x55bcac748d70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb0880cdeb0 tx=0x7fb08809bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c34d1898e80499b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb073fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb073fff640 1 -- 172.21.15.37:0/3498929824 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0880b4060 con 0x55bcac73e5d0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb073fff640 1 -- 172.21.15.37:0/3498929824 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb088010640 con 0x55bcac73e5d0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb088098ab0 con 0x55bcac73e5d0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb073fff640 1 -- 172.21.15.37:0/3498929824 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0880b5250 con 0x55bcac73e5d0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb09d419740 10 client.16344 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bcac757ba0 con 0x55bcac73e5d0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.345+0000 7fb073fff640 1 -- 172.21.15.37:0/3498929824 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb088098ab0 con 0x55bcac73e5d0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb073fff640 1 -- 172.21.15.37:0/3498929824 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb0880a1980 con 0x55bcac73e5d0 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb09d419740 10 client.16344 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:01.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb09d419740 10 client.16344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb09d419740 20 client.16344 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb09d419740 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac758ef0 0x55bcac75b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb09d419740 4 client.16344 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcac6b1860 con 0x55bcac758ef0 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb093fff640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac758ef0 0x55bcac75b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb073fff640 10 client.16344 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.346+0000 7fb093fff640 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac758ef0 0x55bcac75b2e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c264bb443cf5458b server_cookie=1ed3e8e4596ea0d3 in_seq=0 out_seq=0 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.347+0000 7fb073fff640 1 -- 172.21.15.37:0/3498929824 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bcac6b1860 con 0x55bcac758ef0 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.347+0000 7fb073fff640 10 client.16344 handle_command_reply: tid=0 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.347+0000 7fb09d419740 1 client.16344 shutdown 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73e5d0 msgr2=0x55bcac748d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73e5d0 0x55bcac748d70 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb0880cdeb0 tx=0x7fb08809bb20 comp rx=0 tx=0).stop 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 shutdown_connections 2024-09-19T06:45:01.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac758ef0 0x55bcac75b2e0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac749370 0x55bcac7466d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 --2- 172.21.15.37:0/3498929824 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac73e5d0 0x55bcac748d70 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 >> 172.21.15.37:0/3498929824 conn(0x55bcac707310 msgr2=0x55bcac740990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb073fff640 0 client.16344 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 shutdown_connections 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 1 -- 172.21.15.37:0/3498929824 wait complete. 2024-09-19T06:45:01.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.348+0000 7fb09d419740 20 client.16344 trim_cache size 0 max 16384 2024-09-19T06:45:01.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 Processor -- start 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 -- start start 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73e5d0 0x55bcac7554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941b24a0 con 0x55bcac749370 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941afd10 con 0x55bcac73e5d0 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb09d419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941adb60 con 0x55bcac73fb40 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb0995dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb0995dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36942/0 (socket says 172.21.15.37:36942) 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb0995dd640 1 -- 172.21.15.37:0/2005893702 learned_addr learned my addr 172.21.15.37:0/2005893702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:01.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb098ddc640 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73e5d0 0x55bcac7554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb093fff640 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb0995dd640 1 -- 172.21.15.37:0/2005893702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 msgr2=0x55bcac746040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb0995dd640 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.362+0000 7fb0995dd640 1 -- 172.21.15.37:0/2005893702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73e5d0 msgr2=0x55bcac7554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb0995dd640 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73e5d0 0x55bcac7554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb0995dd640 1 -- 172.21.15.37:0/2005893702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcac741120 con 0x55bcac749370 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb093fff640 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb098ddc640 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73e5d0 0x55bcac7554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb0995dd640 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75df20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb08c035000 tx=0x7fb08c048000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2253cd864e65a13e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb07e7fc640 1 -- 172.21.15.37:0/2005893702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c004230 con 0x55bcac749370 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb07e7fc640 1 -- 172.21.15.37:0/2005893702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c036040 con 0x55bcac749370 2024-09-19T06:45:01.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.363+0000 7fb07e7fc640 1 -- 172.21.15.37:0/2005893702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c02fc10 con 0x55bcac749370 2024-09-19T06:45:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 -- 172.21.15.37:0/2005893702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 msgr2=0x55bcac75df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75df20 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb08c035000 tx=0x7fb08c048000 comp rx=0 tx=0).stop 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 -- 172.21.15.37:0/2005893702 shutdown_connections 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75df20 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73fb40 0x55bcac746040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 --2- 172.21.15.37:0/2005893702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73e5d0 0x55bcac7554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 -- 172.21.15.37:0/2005893702 >> 172.21.15.37:0/2005893702 conn(0x55bcac707310 msgr2=0x55bcac745930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 -- 172.21.15.37:0/2005893702 shutdown_connections 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 -- 172.21.15.37:0/2005893702 wait complete. 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 -- start start 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.364+0000 7fb09d419740 1 Processor -- start 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb09d419740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac75af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb09d419740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73fb40 0x55bcac75b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb09d419740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb09d419740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ae1d0 con 0x55bcac749370 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb09d419740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941af640 con 0x55bcac73fb40 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb09d419740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0941ae8a0 con 0x55bcac73e5d0 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb09d419740 10 client.0 _ll_register_callbacks cb 0x7fb0880a0070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36946/0 (socket says 172.21.15.37:36946) 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 -- 172.21.15.37:0/3553265385 learned_addr learned my addr 172.21.15.37:0/3553265385 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb093fff640 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73fb40 0x55bcac75b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 -- 172.21.15.37:0/3553265385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 msgr2=0x55bcac75af00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac75af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 -- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73fb40 msgr2=0x55bcac75b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73fb40 0x55bcac75b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 -- 172.21.15.37:0/3553265385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcac7c1a20 con 0x55bcac749370 2024-09-19T06:45:01.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb093fff640 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73fb40 0x55bcac75b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb0995dd640 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75b980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb08c02a900 tx=0x7fb08c02ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b2bb303173ea22a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb073fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.365+0000 7fb073fff640 1 -- 172.21.15.37:0/3553265385 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c05b020 con 0x55bcac749370 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08c036040 con 0x55bcac749370 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb073fff640 1 -- 172.21.15.37:0/3553265385 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb08c009800 con 0x55bcac749370 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb073fff640 1 -- 172.21.15.37:0/3553265385 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb08c004240 con 0x55bcac749370 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 10 client.16350 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bcac7c2d00 con 0x55bcac749370 2024-09-19T06:45:01.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb073fff640 1 -- 172.21.15.37:0/3553265385 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb08c036040 con 0x55bcac749370 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb073fff640 1 -- 172.21.15.37:0/3553265385 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb08c030b60 con 0x55bcac749370 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 10 client.16350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 20 client.16350 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac7c33e0 0x55bcac7c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 4 client.16350 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bcac6abd70 con 0x55bcac7c33e0 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.366+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac7c33e0 0x55bcac7c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.367+0000 7fb098ddc640 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac7c33e0 0x55bcac7c5800 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b6d81a2b71ca34 server_cookie=3518715f5b155dde in_seq=0 out_seq=0 2024-09-19T06:45:01.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.367+0000 7fb073fff640 10 client.16350 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.368+0000 7fb073fff640 1 -- 172.21.15.37:0/3553265385 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bcac6abd70 con 0x55bcac7c33e0 2024-09-19T06:45:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.368+0000 7fb073fff640 10 client.16350 handle_command_reply: tid=0 2024-09-19T06:45:01.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.368+0000 7fb09d419740 1 client.16350 shutdown 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 msgr2=0x55bcac75b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75b980 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fb08c02a900 tx=0x7fb08c02ff40 comp rx=0 tx=0).stop 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 shutdown_connections 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bcac7c33e0 0x55bcac7c5800 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb073fff640 0 client.16350 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bcac749370 0x55bcac75b980 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bcac73fb40 0x55bcac75b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 --2- 172.21.15.37:0/3553265385 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bcac73e5d0 0x55bcac75af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 >> 172.21.15.37:0/3553265385 conn(0x55bcac707310 msgr2=0x55bcac745160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 shutdown_connections 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 1 -- 172.21.15.37:0/3553265385 wait complete. 2024-09-19T06:45:01.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.369+0000 7fb09d419740 20 client.16350 trim_cache size 0 max 16384 2024-09-19T06:45:01.375 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:45:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0840a33b0 msgr2=0x7fb0840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0840a33b0 0x7fb0840a5870 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb08c006010 tx=0x7fb08c009a20 comp rx=0 tx=0).stop 2024-09-19T06:45:01.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb094100930 msgr2=0x7fb09419a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb094100930 0x7fb09419a5e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb08800e9f0 tx=0x7fb08800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 shutdown_connections 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb09410e4e0 0x7fb09419eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0941012e0 0x7fb09419ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb0840a33b0 0x7fb0840a5870 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 --2- 172.21.15.37:0/3399195314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb094100930 0x7fb09419a5e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 >> 172.21.15.37:0/3399195314 conn(0x7fb0940fc7a0 msgr2=0x7fb0940feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 shutdown_connections 2024-09-19T06:45:01.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:01.370+0000 7fb09b040640 1 -- 172.21.15.37:0/3399195314 wait complete. 2024-09-19T06:45:02.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 188530 11.62 MB/sec warmup 89 sec latency 15.576 ms 2024-09-19T06:45:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:02 smithi096 ceph-mon[35004]: pgmap v101: 97 pgs: 97 active+clean; 50 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-19T06:45:02.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:02 smithi037 ceph-mon[31211]: pgmap v101: 97 pgs: 97 active+clean; 50 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-19T06:45:02.836 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 -- 172.21.15.37:0/3085520364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d41069e0 msgr2=0x7f77d4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 --2- 172.21.15.37:0/3085520364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d41069e0 0x7f77d4106dc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f77d000d960 tx=0x7f77d002a0a0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 -- 172.21.15.37:0/3085520364 shutdown_connections 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 --2- 172.21.15.37:0/3085520364 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d4101370 0x7f77d410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 --2- 172.21.15.37:0/3085520364 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d41009d0 0x7f77d4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 --2- 172.21.15.37:0/3085520364 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d41069e0 0x7f77d4106dc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 -- 172.21.15.37:0/3085520364 >> 172.21.15.37:0/3085520364 conn(0x7f77d40fc6f0 msgr2=0x7f77d40feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 -- 172.21.15.37:0/3085520364 shutdown_connections 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 -- 172.21.15.37:0/3085520364 wait complete. 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.834+0000 7f77db95f640 1 Processor -- start 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77db95f640 1 -- start start 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77db95f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d41009d0 0x7f77d4196150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77db95f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d4101370 0x7f77d4196690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77db95f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d41069e0 0x7f77d419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77db95f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77d410fd20 con 0x7f77d41009d0 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77db95f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77d410fba0 con 0x7f77d4101370 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77db95f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77d410fea0 con 0x7f77d41069e0 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d41009d0 0x7f77d4196150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d41009d0 0x7f77d4196150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36970/0 (socket says 172.21.15.37:36970) 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 -- 172.21.15.37:0/678094240 learned_addr learned my addr 172.21.15.37:0/678094240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d8efa640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d4101370 0x7f77d4196690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d9efc640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d41069e0 0x7f77d419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 -- 172.21.15.37:0/678094240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d41069e0 msgr2=0x7f77d419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d41069e0 0x7f77d419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 -- 172.21.15.37:0/678094240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d4101370 msgr2=0x7f77d4196690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d4101370 0x7f77d4196690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 -- 172.21.15.37:0/678094240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d419b140 con 0x7f77d41009d0 2024-09-19T06:45:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.835+0000 7f77d96fb640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d41009d0 0x7f77d4196150 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f77d0035040 tx=0x7f77d00045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.836+0000 7f77ca7fc640 1 -- 172.21.15.37:0/678094240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0037050 con 0x7f77d41009d0 2024-09-19T06:45:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.836+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d419b3d0 con 0x7f77d41009d0 2024-09-19T06:45:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.836+0000 7f77ca7fc640 1 -- 172.21.15.37:0/678094240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77d0043070 con 0x7f77d41009d0 2024-09-19T06:45:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.836+0000 7f77ca7fc640 1 -- 172.21.15.37:0/678094240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77d0032840 con 0x7f77d41009d0 2024-09-19T06:45:02.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.836+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d41a2cc0 con 0x7f77d41009d0 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 Processor -- start 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 -- start start 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77ca7fc640 1 -- 172.21.15.37:0/678094240 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f77d002abe0 con 0x7f77d41009d0 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f56ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f61770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b88ebb2f0 con 0x560b88f56af0 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560b88e63270 con 0x560b88f61cb0 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77de7d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b88eb72b0 con 0x560b88f57410 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f56ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f56ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36976/0 (socket says 172.21.15.37:36976) 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 -- 172.21.15.37:0/3108301487 learned_addr learned my addr 172.21.15.37:0/3108301487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d8efa640 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d96fb640 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f61770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 -- 172.21.15.37:0/3108301487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 msgr2=0x560b88f61770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f61770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 -- 172.21.15.37:0/3108301487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 msgr2=0x560b88f640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.839+0000 7f77d9efc640 1 -- 172.21.15.37:0/3108301487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b88e62340 con 0x560b88f56af0 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.840+0000 7f77d9efc640 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f56ed0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e730 tx=0x7f77cc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4f7c6719a50951 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.840+0000 7f77ca7fc640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77c00a3460 0x7f77c00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.840+0000 7f77d96fb640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77c00a3460 0x7f77c00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.840+0000 7f77bf7fe640 1 -- 172.21.15.37:0/3108301487 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01e070 con 0x560b88f56af0 2024-09-19T06:45:02.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.840+0000 7f77bf7fe640 1 -- 172.21.15.37:0/3108301487 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77cc003e50 con 0x560b88f56af0 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.840+0000 7f77bf7fe640 1 -- 172.21.15.37:0/3108301487 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc004220 con 0x560b88f56af0 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.840+0000 7f77ca7fc640 1 -- 172.21.15.37:0/678094240 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f77d00efa70 con 0x7f77d41009d0 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 -- 172.21.15.37:0/3108301487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 msgr2=0x560b88f56ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77d96fb640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77c00a3460 0x7f77c00a5920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f77d00ed380 tx=0x7f77d002a0a0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f56ed0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00e730 tx=0x7f77cc0095c0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 -- 172.21.15.37:0/3108301487 shutdown_connections 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f61770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 --2- 172.21.15.37:0/3108301487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f56ed0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 -- 172.21.15.37:0/3108301487 >> 172.21.15.37:0/3108301487 conn(0x560b88f53060 msgr2=0x560b88f53440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 -- 172.21.15.37:0/3108301487 shutdown_connections 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 -- 172.21.15.37:0/3108301487 wait complete. 2024-09-19T06:45:02.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 -- start start 2024-09-19T06:45:02.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.841+0000 7f77de7d5740 1 Processor -- start 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f5eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f5f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77de7d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b88f64880 con 0x560b88f56af0 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77de7d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560b88e44290 con 0x560b88f57410 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77de7d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b88f64a00 con 0x560b88f61cb0 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77de7d5740 10 client.0 _ll_register_callbacks cb 0x560b88e75140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f5eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f5eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36986/0 (socket says 172.21.15.37:36986) 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44344/0 (socket says 172.21.15.37:44344) 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d9efc640 1 -- 172.21.15.37:0/2164411550 learned_addr learned my addr 172.21.15.37:0/2164411550 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d9efc640 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f5f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 -- 172.21.15.37:0/2164411550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 msgr2=0x560b88f5f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f5f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 -- 172.21.15.37:0/2164411550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 msgr2=0x560b88f5eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f5eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 -- 172.21.15.37:0/2164411550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b88bce210 con 0x560b88f57410 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d9efc640 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77d8efa640 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f77d40fec20 tx=0x7f77c4012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f939b23f90e7d6a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.842+0000 7f77bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:02.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77bcff9640 1 -- 172.21.15.37:0/2164411550 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c4004020 con 0x560b88f57410 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77cc003e50 con 0x560b88f57410 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77bcff9640 1 -- 172.21.15.37:0/2164411550 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f77c40041c0 con 0x560b88f57410 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77bcff9640 1 -- 172.21.15.37:0/2164411550 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77c40043c0 con 0x560b88f57410 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 10 client.25871 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b88c9a620 con 0x560b88f57410 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77bcff9640 1 -- 172.21.15.37:0/2164411550 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77cc003e50 con 0x560b88f57410 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77bcff9640 1 -- 172.21.15.37:0/2164411550 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f77c4003c20 con 0x560b88f57410 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 10 client.25871 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 10 client.25871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 20 client.25871 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88f71c00 0x560b88f74010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 4 client.25871 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b88ec8a20 con 0x560b88f71c00 2024-09-19T06:45:02.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.843+0000 7f77d9efc640 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88f71c00 0x560b88f74010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.844+0000 7f77d9efc640 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88f71c00 0x560b88f74010 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94879e57d53d3156 server_cookie=91fef675abd3bfc3 in_seq=0 out_seq=0 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.844+0000 7f77bcff9640 10 client.25871 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77bcff9640 1 -- 172.21.15.37:0/2164411550 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b88ec8a20 con 0x560b88f71c00 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77bcff9640 10 client.25871 handle_command_reply: tid=0 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 client.25871 shutdown 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 msgr2=0x560b88f5f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f040 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f77d40fec20 tx=0x7f77c4012b60 comp rx=0 tx=0).stop 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 shutdown_connections 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88f71c00 0x560b88f74010 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f040 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77bcff9640 0 client.25871 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:02.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 --2- 172.21.15.37:0/2164411550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f5eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.845+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 >> 172.21.15.37:0/2164411550 conn(0x560b88f53060 msgr2=0x560b88f58d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.846+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 shutdown_connections 2024-09-19T06:45:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.846+0000 7f77de7d5740 1 -- 172.21.15.37:0/2164411550 wait complete. 2024-09-19T06:45:02.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.846+0000 7f77de7d5740 20 client.25871 trim_cache size 0 max 16384 2024-09-19T06:45:02.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:02 smithi151 ceph-mon[35236]: pgmap v101: 97 pgs: 97 active+clean; 50 MiB data, 641 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 3.5 MiB/s wr, 409 op/s 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 Processor -- start 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 -- start start 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f59560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f79920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41adf60 con 0x560b88f56af0 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41ab7d0 con 0x560b88f57410 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77de7d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41a9620 con 0x560b88f61cb0 2024-09-19T06:45:02.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f59560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f59560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36990/0 (socket says 172.21.15.37:36990) 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77d9efc640 1 -- 172.21.15.37:0/571920941 learned_addr learned my addr 172.21.15.37:0/571920941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77d8efa640 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.862+0000 7f77d9efc640 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f79920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77d9efc640 1 -- 172.21.15.37:0/571920941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 msgr2=0x560b88f79920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77d9efc640 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f79920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77d9efc640 1 -- 172.21.15.37:0/571920941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 msgr2=0x560b88f5f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77d9efc640 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77d9efc640 1 -- 172.21.15.37:0/571920941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b88f5a6a0 con 0x560b88f56af0 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77d9efc640 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f59560 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f77cc015d60 tx=0x7f77cc027040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ccebae5bd672eb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77beffd640 1 -- 172.21.15.37:0/571920941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01e040 con 0x560b88f56af0 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77beffd640 1 -- 172.21.15.37:0/571920941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77cc003d00 con 0x560b88f56af0 2024-09-19T06:45:02.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.863+0000 7f77beffd640 1 -- 172.21.15.37:0/571920941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc031300 con 0x560b88f56af0 2024-09-19T06:45:02.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- 172.21.15.37:0/571920941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 msgr2=0x560b88f59560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f59560 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f77cc015d60 tx=0x7f77cc027040 comp rx=0 tx=0).stop 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- 172.21.15.37:0/571920941 shutdown_connections 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f61cb0 0x560b88f79920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f57410 0x560b88f5f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 --2- 172.21.15.37:0/571920941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f59560 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- 172.21.15.37:0/571920941 >> 172.21.15.37:0/571920941 conn(0x560b88f53060 msgr2=0x560b88f60b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- 172.21.15.37:0/571920941 shutdown_connections 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- 172.21.15.37:0/571920941 wait complete. 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- start start 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 Processor -- start 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f65be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f66120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f66660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41a9c90 con 0x560b88f56af0 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41ab100 con 0x560b88f61cb0 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.864+0000 7f77de7d5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77d41aa360 con 0x560b88f57410 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77de7d5740 10 client.0 _ll_register_callbacks cb 0x7f77c4009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f65be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f65be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36998/0 (socket says 172.21.15.37:36998) 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 -- 172.21.15.37:0/1251409686 learned_addr learned my addr 172.21.15.37:0/1251409686 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f66660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d8efa640 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 -- 172.21.15.37:0/1251409686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 msgr2=0x560b88f66120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f66120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 -- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 msgr2=0x560b88f66660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:02.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f66660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 -- 172.21.15.37:0/1251409686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b88fdbaf0 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77d9efc640 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f65be0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f77cc016010 tx=0x7f77cc005860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928d20eb94a66871 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77bcff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.865+0000 7f77bcff9640 1 -- 172.21.15.37:0/1251409686 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77cc01e070 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77bcff9640 1 -- 172.21.15.37:0/1251409686 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f77cc005300 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77cc003d00 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77bcff9640 1 -- 172.21.15.37:0/1251409686 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77cc03f050 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b88fdcda0 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77bcff9640 1 -- 172.21.15.37:0/1251409686 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77cc005500 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77bcff9640 1 -- 172.21.15.37:0/1251409686 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f77cc01a450 con 0x560b88f56af0 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:02.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 10 client.16368 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 20 client.16368 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88fdd210 0x560b88fdf600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 4 client.16368 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f77cc003d00 con 0x560b88fdd210 2024-09-19T06:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.866+0000 7f77d8efa640 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88fdd210 0x560b88fdf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.867+0000 7f77bcff9640 10 client.16368 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:02.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.867+0000 7f77d8efa640 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88fdd210 0x560b88fdf600 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711ff9740fe6f8d server_cookie=8d87f11279aa8391 in_seq=0 out_seq=0 2024-09-19T06:45:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77bcff9640 1 -- 172.21.15.37:0/1251409686 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f77cc003d00 con 0x560b88fdd210 2024-09-19T06:45:02.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77bcff9640 10 client.16368 handle_command_reply: tid=0 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 client.16368 shutdown 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 msgr2=0x560b88f65be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f65be0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f77cc016010 tx=0x7f77cc005860 comp rx=0 tx=0).stop 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 shutdown_connections 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b88fdd210 0x560b88fdf600 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77bcff9640 0 client.16368 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b88f61cb0 0x560b88f66660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b88f57410 0x560b88f66120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 --2- 172.21.15.37:0/1251409686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b88f56af0 0x560b88f65be0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 >> 172.21.15.37:0/1251409686 conn(0x560b88f53060 msgr2=0x560b88f519a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:02.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.995+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 shutdown_connections 2024-09-19T06:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77de7d5740 1 -- 172.21.15.37:0/1251409686 wait complete. 2024-09-19T06:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77de7d5740 20 client.16368 trim_cache size 0 max 16384 2024-09-19T06:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:45:02.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77c00a3460 msgr2=0x7f77c00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77db95f640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77c00a3460 0x7f77c00a5920 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f77d00ed380 tx=0x7f77d002a0a0 comp rx=0 tx=0).stop 2024-09-19T06:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d41009d0 msgr2=0x7f77d4196150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:02.999 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77db95f640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d41009d0 0x7f77d4196150 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f77d0035040 tx=0x7f77d00045b0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 shutdown_connections 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.996+0000 7f77db95f640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77d41069e0 0x7f77d419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.997+0000 7f77db95f640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77d4101370 0x7f77d4196690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.997+0000 7f77db95f640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77c00a3460 0x7f77c00a5920 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.997+0000 7f77db95f640 1 --2- 172.21.15.37:0/678094240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77d41009d0 0x7f77d4196150 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.997+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 >> 172.21.15.37:0/678094240 conn(0x7f77d40fc6f0 msgr2=0x7f77d40fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.997+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 shutdown_connections 2024-09-19T06:45:03.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:02.997+0000 7f77db95f640 1 -- 172.21.15.37:0/678094240 wait complete. 2024-09-19T06:45:03.042 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:45:03.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:45:03.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 191368 11.65 MB/sec warmup 90 sec latency 46.954 ms 2024-09-19T06:45:03.323 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/18939102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b41082a0 msgr2=0x7fb3b410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/18939102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b41082a0 0x7fb3b410ec40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009cd0 tx=0x7fb3ac02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/18939102 shutdown_connections 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/18939102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b41082a0 0x7fb3b410ec40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/18939102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b41036e0 0x7fb3b4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/18939102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b4102d30 0x7fb3b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/18939102 >> 172.21.15.37:0/18939102 conn(0x7fb3b40fc7d0 msgr2=0x7fb3b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/18939102 shutdown_connections 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.928+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/18939102 wait complete. 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 Processor -- start 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 -- start start 2024-09-19T06:45:03.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102d30 0x7fb3b4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b41036e0 0x7fb3b4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b41082a0 0x7fb3b419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41120f0 con 0x7fb3b4102d30 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4111f70 con 0x7fb3b41082a0 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.929+0000 7fb3bc5cc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b4112270 con 0x7fb3b41036e0 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102d30 0x7fb3b4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102d30 0x7fb3b4198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37022/0 (socket says 172.21.15.37:37022) 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 -- 172.21.15.37:0/1005766890 learned_addr learned my addr 172.21.15.37:0/1005766890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3bab69640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b41082a0 0x7fb3b419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b41036e0 0x7fb3b4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 -- 172.21.15.37:0/1005766890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b41036e0 msgr2=0x7fb3b4198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b41036e0 0x7fb3b4198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 -- 172.21.15.37:0/1005766890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b41082a0 msgr2=0x7fb3b419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b41082a0 0x7fb3b419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 -- 172.21.15.37:0/1005766890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3b419d5e0 con 0x7fb3b4102d30 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3bab69640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b41082a0 0x7fb3b419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ba368640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102d30 0x7fb3b4198630 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000e9f0 tx=0x7fb3b000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.930+0000 7fb3ab7fe640 1 -- 172.21.15.37:0/1005766890 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b000cd80 con 0x7fb3b4102d30 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.931+0000 7fb3ab7fe640 1 -- 172.21.15.37:0/1005766890 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b0005500 con 0x7fb3b4102d30 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.931+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3b419d8d0 con 0x7fb3b4102d30 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.931+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3b41a51b0 con 0x7fb3b4102d30 2024-09-19T06:45:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.932+0000 7fb3ab7fe640 1 -- 172.21.15.37:0/1005766890 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0010690 con 0x7fb3b4102d30 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.933+0000 7fb3be9a5740 1 Processor -- start 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.933+0000 7fb3ab7fe640 1 -- 172.21.15.37:0/1005766890 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb3b0010830 con 0x7fb3b4102d30 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.933+0000 7fb3be9a5740 1 -- start start 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6db60 0x555f29e6df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6e480 0x555f29e78800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e7b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555f29dd3280 con 0x555f29e6e480 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f29d7ab10 con 0x555f29e78d40 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f29dcf240 con 0x555f29e6db60 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6e480 0x555f29e78800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6e480 0x555f29e78800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37034/0 (socket says 172.21.15.37:37034) 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3ba368640 1 -- 172.21.15.37:0/1008462752 learned_addr learned my addr 172.21.15.37:0/1008462752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3b9b67640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e7b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3bab69640 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6db60 0x555f29e6df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3b9b67640 1 -- 172.21.15.37:0/1008462752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6db60 msgr2=0x555f29e6df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6db60 0x555f29e6df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3b9b67640 1 -- 172.21.15.37:0/1008462752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6e480 msgr2=0x555f29e78800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6e480 0x555f29e78800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3b9b67640 1 -- 172.21.15.37:0/1008462752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f29d871f0 con 0x555f29e78d40 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3bab69640 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6db60 0x555f29e6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.934+0000 7fb3ab7fe640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a00a3430 0x7fb3a00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.935+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e7b130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406a370 tx=0x7fb3a4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a89ca14a63d2792e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.935+0000 7fb3ab7fe640 1 -- 172.21.15.37:0/1005766890 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb3b00d0070 con 0x7fb3b4102d30 2024-09-19T06:45:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.935+0000 7fb3bab69640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a00a3430 0x7fb3a00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.935+0000 7fb3a8ff9640 1 -- 172.21.15.37:0/1008462752 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4004050 con 0x555f29e78d40 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.935+0000 7fb3a8ff9640 1 -- 172.21.15.37:0/1008462752 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a4005500 con 0x555f29e78d40 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.935+0000 7fb3a8ff9640 1 -- 172.21.15.37:0/1008462752 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4004d90 con 0x555f29e78d40 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 -- 172.21.15.37:0/1008462752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 msgr2=0x555f29e7b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3bab69640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a00a3430 0x7fb3a00a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac005e30 tx=0x7fb3ac03d040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e7b130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb3b406a370 tx=0x7fb3a4011810 comp rx=0 tx=0).stop 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 -- 172.21.15.37:0/1008462752 shutdown_connections 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e7b130 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6e480 0x555f29e78800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/1008462752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6db60 0x555f29e6df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 -- 172.21.15.37:0/1008462752 >> 172.21.15.37:0/1008462752 conn(0x555f29e6a0f0 msgr2=0x555f29e6a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 -- 172.21.15.37:0/1008462752 shutdown_connections 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.936+0000 7fb3be9a5740 1 -- 172.21.15.37:0/1008462752 wait complete. 2024-09-19T06:45:03.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 -- start start 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 Processor -- start 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e75c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e76150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e76690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555f29e7fd20 con 0x555f29e6db60 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f29d8df10 con 0x555f29e78d40 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555f29e7fea0 con 0x555f29e6e480 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3be9a5740 10 client.0 _ll_register_callbacks cb 0x555f29d56a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e75c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3b9b67640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e76150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e75c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37048/0 (socket says 172.21.15.37:37048) 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 -- 172.21.15.37:0/242704213 learned_addr learned my addr 172.21.15.37:0/242704213 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:03.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3bab69640 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e76690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 -- 172.21.15.37:0/242704213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 msgr2=0x555f29e76150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e76150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 -- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 msgr2=0x555f29e76690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e76690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.937+0000 7fb3ba368640 1 -- 172.21.15.37:0/242704213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f29b446a0 con 0x555f29e6db60 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e76150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb3ba368640 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e75c10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00cd870 tx=0x7fb3b0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6dfa08f2bd1554a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:03.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb39dffb640 1 -- 172.21.15.37:0/242704213 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b009c120 con 0x555f29e6db60 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb39dffb640 1 -- 172.21.15.37:0/242704213 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb3b00046c0 con 0x555f29e6db60 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3a4005500 con 0x555f29e6db60 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb39dffb640 1 -- 172.21.15.37:0/242704213 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb3b009bcb0 con 0x555f29e6db60 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb3be9a5740 10 client.16380 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f29e8b980 con 0x555f29e6db60 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb39dffb640 1 -- 172.21.15.37:0/242704213 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3a4005500 con 0x555f29e6db60 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.938+0000 7fb39dffb640 1 -- 172.21.15.37:0/242704213 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb3b00b5140 con 0x555f29e6db60 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3be9a5740 10 client.16380 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:03.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3be9a5740 10 client.16380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3be9a5740 20 client.16380 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29e8ce40 0x555f29e8f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3be9a5740 4 client.16380 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f29de07f0 con 0x555f29e8ce40 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29e8ce40 0x555f29e8f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb39dffb640 10 client.16380 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.939+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29e8ce40 0x555f29e8f290 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de7d29205c3cf99a server_cookie=fe12fefd29ffd51 in_seq=0 out_seq=0 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb39dffb640 1 -- 172.21.15.37:0/242704213 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555f29de07f0 con 0x555f29e8ce40 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb39dffb640 10 client.16380 handle_command_reply: tid=0 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 client.16380 shutdown 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 msgr2=0x555f29e75c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e75c10 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00cd870 tx=0x7fb3b0009310 comp rx=0 tx=0).stop 2024-09-19T06:45:03.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 shutdown_connections 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29e8ce40 0x555f29e8f290 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e76690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e76150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/242704213 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e75c10 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 >> 172.21.15.37:0/242704213 conn(0x555f29e6a0f0 msgr2=0x555f29e6fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb39dffb640 0 client.16380 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 shutdown_connections 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 1 -- 172.21.15.37:0/242704213 wait complete. 2024-09-19T06:45:03.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.941+0000 7fb3be9a5740 20 client.16380 trim_cache size 0 max 16384 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.962+0000 7fb3be9a5740 1 Processor -- start 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.962+0000 7fb3be9a5740 1 -- start start 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e6fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e94b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41ae110 con 0x555f29e6db60 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41ab980 con 0x555f29e78d40 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41a97d0 con 0x555f29e6e480 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e6fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e6fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37064/0 (socket says 172.21.15.37:37064) 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 -- 172.21.15.37:0/3297541570 learned_addr learned my addr 172.21.15.37:0/3297541570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:03.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3b9b67640 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e94b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3bab69640 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 -- 172.21.15.37:0/3297541570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 msgr2=0x555f29e94b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e94b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 -- 172.21.15.37:0/3297541570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 msgr2=0x555f29e950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.963+0000 7fb3ba368640 1 -- 172.21.15.37:0/3297541570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f29e65830 con 0x555f29e6db60 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.964+0000 7fb3ba368640 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e6fd10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0098d20 tx=0x7fb3b00d23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c313d7b18f774 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.964+0000 7fb39ffff640 1 -- 172.21.15.37:0/3297541570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b009be90 con 0x555f29e6db60 2024-09-19T06:45:03.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.964+0000 7fb39ffff640 1 -- 172.21.15.37:0/3297541570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b009ca50 con 0x555f29e6db60 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.964+0000 7fb39ffff640 1 -- 172.21.15.37:0/3297541570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b00a0700 con 0x555f29e6db60 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 -- 172.21.15.37:0/3297541570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 msgr2=0x555f29e6fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e6fd10 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0098d20 tx=0x7fb3b00d23b0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 -- 172.21.15.37:0/3297541570 shutdown_connections 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e94b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/3297541570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e6fd10 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 -- 172.21.15.37:0/3297541570 >> 172.21.15.37:0/3297541570 conn(0x555f29e6a0f0 msgr2=0x555f29e6f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 -- 172.21.15.37:0/3297541570 shutdown_connections 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.965+0000 7fb3be9a5740 1 -- 172.21.15.37:0/3297541570 wait complete. 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 -- start start 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 Processor -- start 2024-09-19T06:45:03.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e7f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e85150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e856b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41a9e40 con 0x555f29e6db60 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41ab2b0 con 0x555f29e78d40 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3b41aa510 con 0x555f29e6e480 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3be9a5740 10 client.0 _ll_register_callbacks cb 0x7fb3b00056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e7f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3b9b67640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e85150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3bab69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e856b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3ba368640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e7f450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37080/0 (socket says 172.21.15.37:37080) 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3ba368640 1 -- 172.21.15.37:0/2495651872 learned_addr learned my addr 172.21.15.37:0/2495651872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:03.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.966+0000 7fb3b9b67640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e85150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:43668/0 (socket says 172.21.15.37:43668) 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3bab69640 1 -- 172.21.15.37:0/2495651872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 msgr2=0x555f29e85150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3bab69640 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e85150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3bab69640 1 -- 172.21.15.37:0/2495651872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 msgr2=0x555f29e7f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3bab69640 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e7f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3bab69640 1 -- 172.21.15.37:0/2495651872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f29e7b790 con 0x555f29e78d40 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3ba368640 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e7f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3bab69640 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e856b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009ca0 tx=0x7fb3ac02a8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d9d31c19fe81f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb39dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb39dffb640 1 -- 172.21.15.37:0/2495651872 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac057020 con 0x555f29e78d40 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb39dffb640 1 -- 172.21.15.37:0/2495651872 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb3ac0309b0 con 0x555f29e78d40 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.967+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3b009ca50 con 0x555f29e78d40 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb39dffb640 1 -- 172.21.15.37:0/2495651872 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb3ac062050 con 0x555f29e78d40 2024-09-19T06:45:03.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 10 client.25895 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f29efc610 con 0x555f29e78d40 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb39dffb640 1 -- 172.21.15.37:0/2495651872 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b009ca50 con 0x555f29e78d40 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb39dffb640 1 -- 172.21.15.37:0/2495651872 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb3ac030070 con 0x555f29e78d40 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 10 client.25895 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 20 client.25895 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29efcde0 0x555f29eff200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 4 client.25895 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x555f29ddad00 con 0x555f29efcde0 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.968+0000 7fb3ba368640 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29efcde0 0x555f29eff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.969+0000 7fb3ba368640 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29efcde0 0x555f29eff200 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dabc838ab1d1980a server_cookie=4302663298fead6e in_seq=0 out_seq=0 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.969+0000 7fb39dffb640 10 client.25895 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.969+0000 7fb39dffb640 1 -- 172.21.15.37:0/2495651872 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x555f29ddad00 con 0x555f29efcde0 2024-09-19T06:45:03.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.969+0000 7fb39dffb640 10 client.25895 handle_command_reply: tid=0 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 client.25895 shutdown 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 msgr2=0x555f29e856b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e856b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac009ca0 tx=0x7fb3ac02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 shutdown_connections 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555f29efcde0 0x555f29eff200 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f29e78d40 0x555f29e856b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555f29e6e480 0x555f29e85150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 --2- 172.21.15.37:0/2495651872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555f29e6db60 0x555f29e7f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb39dffb640 0 client.25895 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 >> 172.21.15.37:0/2495651872 conn(0x555f29e6a0f0 msgr2=0x555f29e763f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 shutdown_connections 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 1 -- 172.21.15.37:0/2495651872 wait complete. 2024-09-19T06:45:03.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.970+0000 7fb3be9a5740 20 client.25895 trim_cache size 0 max 16384 2024-09-19T06:45:03.984 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:45:03.984 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:45:03.984 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "51916c16-6273-414b-9158-036f91fb19d6", 2024-09-19T06:45:03.984 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:45:03.984 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:45:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a00a3430 msgr2=0x7fb3a00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a00a3430 0x7fb3a00a58f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac005e30 tx=0x7fb3ac03d040 comp rx=0 tx=0).stop 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102d30 msgr2=0x7fb3b4198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102d30 0x7fb3b4198630 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000e9f0 tx=0x7fb3b000edf0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 shutdown_connections 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3b41082a0 0x7fb3b419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3b41036e0 0x7fb3b4198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a00a3430 0x7fb3a00a58f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 --2- 172.21.15.37:0/1005766890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3b4102d30 0x7fb3b4198630 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 >> 172.21.15.37:0/1005766890 conn(0x7fb3b40fc7d0 msgr2=0x7fb3b40fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 shutdown_connections 2024-09-19T06:45:03.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:03.971+0000 7fb3bc5cc640 1 -- 172.21.15.37:0/1005766890 wait complete. 2024-09-19T06:45:04.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 193040 11.63 MB/sec warmup 91 sec latency 52.106 ms 2024-09-19T06:45:04.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:04 smithi096 ceph-mon[35004]: pgmap v102: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 578 op/s 2024-09-19T06:45:04.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:04 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:45:04.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:04 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:04.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:04 smithi037 ceph-mon[31211]: pgmap v102: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 578 op/s 2024-09-19T06:45:04.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:04 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:45:04.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:04 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:04.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:04 smithi151 ceph-mon[35236]: pgmap v102: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 5.4 MiB/s wr, 578 op/s 2024-09-19T06:45:04.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:04 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:45:04.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:04 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:05.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 195763 11.66 MB/sec warmup 92 sec latency 17.619 ms 2024-09-19T06:45:05.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:45:05.677 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:05.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:05 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:45:05.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:05 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:05.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:05 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:45:05.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:05 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:05.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:05 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:45:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:05 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:05 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:45:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:05 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:05.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:05 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:45:05.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:05 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:05.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:05 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:45:05.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:05 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/4159426410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 msgr2=0x7ff0dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/4159426410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc101200 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc009900 tx=0x7ff0cc02a480 comp rx=0 tx=0).stop 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/4159426410 shutdown_connections 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/4159426410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0dc102ef0 0x7ff0dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/4159426410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc102570 0x7ff0dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/4159426410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc101200 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/4159426410 >> 172.21.15.37:0/4159426410 conn(0x7ff0dc0fc7d0 msgr2=0x7ff0dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/4159426410 shutdown_connections 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.134+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/4159426410 wait complete. 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 Processor -- start 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 -- start start 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc1984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0dc102570 0x7ff0dc198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc102ef0 0x7ff0dc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1120a0 con 0x7ff0dc100e20 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc111f20 con 0x7ff0dc102570 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0e1bd4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc112220 con 0x7ff0dc102ef0 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc1984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc1984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37094/0 (socket says 172.21.15.37:37094) 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0db7fe640 1 -- 172.21.15.37:0/604967329 learned_addr learned my addr 172.21.15.37:0/604967329 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc102ef0 0x7ff0dc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0d2ffd640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0dc102570 0x7ff0dc198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0db7fe640 1 -- 172.21.15.37:0/604967329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc102ef0 msgr2=0x7ff0dc19cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc102ef0 0x7ff0dc19cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0db7fe640 1 -- 172.21.15.37:0/604967329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0dc102570 msgr2=0x7ff0dc198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.135+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0dc102570 0x7ff0dc198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0db7fe640 1 -- 172.21.15.37:0/604967329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0dc19d480 con 0x7ff0dc100e20 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc102ef0 0x7ff0dc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc1984d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc006010 tx=0x7ff0cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0d97fa640 1 -- 172.21.15.37:0/604967329 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc03f070 con 0x7ff0dc100e20 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0dc19d710 con 0x7ff0dc100e20 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0d97fa640 1 -- 172.21.15.37:0/604967329 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0cc031c30 con 0x7ff0dc100e20 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0d97fa640 1 -- 172.21.15.37:0/604967329 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc035030 con 0x7ff0dc100e20 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.136+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0dc1a5050 con 0x7ff0dc100e20 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.138+0000 7ff0d97fa640 1 -- 172.21.15.37:0/604967329 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff0cc042070 con 0x7ff0dc100e20 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 Processor -- start 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 -- start start 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6340 0x564189ff6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6c60 0x56418a005550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x56418a007e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564189f5c2e0 con 0x564189ff6c60 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564189f03ce0 con 0x564189ff6340 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.139+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564189f582a0 con 0x56418a005a90 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x56418a007e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6340 0x564189ff6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x56418a007e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:43694/0 (socket says 172.21.15.37:43694) 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6340 0x564189ff6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44448/0 (socket says 172.21.15.37:44448) 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0d2ffd640 1 -- 172.21.15.37:0/3788028155 learned_addr learned my addr 172.21.15.37:0/3788028155 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6c60 0x56418a005550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 -- 172.21.15.37:0/3788028155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 msgr2=0x56418a007e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x56418a007e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 -- 172.21.15.37:0/3788028155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6c60 msgr2=0x56418a005550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6c60 0x56418a005550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0d97fa640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff0c80a33b0 0x7ff0c80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 -- 172.21.15.37:0/3788028155 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564189ee8440 con 0x564189ff6340 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff0c80a33b0 0x7ff0c80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0d97fa640 1 -- 172.21.15.37:0/604967329 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff0cc0f0380 con 0x7ff0dc100e20 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.140+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6340 0x564189ff6720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005fd0 tx=0x7ff0d4011bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8842bcea09f8b0f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0d1ffb640 1 -- 172.21.15.37:0/3788028155 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d400f9a0 con 0x564189ff6340 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0d1ffb640 1 -- 172.21.15.37:0/3788028155 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d400fc80 con 0x564189ff6340 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0d1ffb640 1 -- 172.21.15.37:0/3788028155 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d401cc60 con 0x564189ff6340 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 -- 172.21.15.37:0/3788028155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6340 msgr2=0x564189ff6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6340 0x564189ff6720 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005fd0 tx=0x7ff0d4011bf0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff0c80a33b0 0x7ff0c80a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4020000 tx=0x7ff0d4021000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:06.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 -- 172.21.15.37:0/3788028155 shutdown_connections 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x56418a007e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6c60 0x56418a005550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/3788028155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6340 0x564189ff6720 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 -- 172.21.15.37:0/3788028155 >> 172.21.15.37:0/3788028155 conn(0x564189ff2a30 msgr2=0x564189ff2e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.141+0000 7ff0e3fad740 1 -- 172.21.15.37:0/3788028155 shutdown_connections 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 -- 172.21.15.37:0/3788028155 wait complete. 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 -- start start 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 Processor -- start 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x564189ffe910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x564189ffee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x564189fff390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564189ff9380 con 0x564189ff6340 2024-09-19T06:45:06.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564189f17050 con 0x564189ff6c60 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564189ff9500 con 0x56418a005a90 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.142+0000 7ff0e3fad740 10 client.0 _ll_register_callbacks cb 0x564189f57160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x564189ffe910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0dbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x564189fff390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x564189ffe910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37112/0 (socket says 172.21.15.37:37112) 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0db7fe640 1 -- 172.21.15.37:0/2643495940 learned_addr learned my addr 172.21.15.37:0/2643495940 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x564189ffee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0dbfff640 1 -- 172.21.15.37:0/2643495940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 msgr2=0x564189ffee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x564189ffee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0dbfff640 1 -- 172.21.15.37:0/2643495940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 msgr2=0x564189ffe910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x564189ffe910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0dbfff640 1 -- 172.21.15.37:0/2643495940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564189ccd3c0 con 0x56418a005a90 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x564189fff390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005fd0 tx=0x7ff0d401c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30cff61d070eb0c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0baffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0baffd640 1 -- 172.21.15.37:0/2643495940 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4036060 con 0x56418a005a90 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d400fc80 con 0x56418a005a90 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.143+0000 7ff0baffd640 1 -- 172.21.15.37:0/2643495940 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff0d402e070 con 0x56418a005a90 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0baffd640 1 -- 172.21.15.37:0/2643495940 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0d400fe30 con 0x56418a005a90 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 10 client.25810 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56418a0183e0 con 0x56418a005a90 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0baffd640 1 -- 172.21.15.37:0/2643495940 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d400fc80 con 0x56418a005a90 2024-09-19T06:45:06.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0baffd640 1 -- 172.21.15.37:0/2643495940 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff0d4004cd0 con 0x56418a005a90 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 10 client.25810 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 10 client.25810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 20 client.25810 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a019840 0x56418a01bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 4 client.25810 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564189f69850 con 0x56418a019840 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.144+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a019840 0x56418a01bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.145+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a019840 0x56418a01bc30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=712b4807dc60a6a4 server_cookie=9bdd5d0e549dbd1d in_seq=0 out_seq=0 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.145+0000 7ff0baffd640 10 client.25810 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0baffd640 1 -- 172.21.15.37:0/2643495940 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564189f69850 con 0x56418a019840 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0baffd640 10 client.25810 handle_command_reply: tid=0 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 client.25810 shutdown 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 msgr2=0x564189fff390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x564189fff390 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4005fd0 tx=0x7ff0d401c980 comp rx=0 tx=0).stop 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 shutdown_connections 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a019840 0x56418a01bc30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56418a005a90 0x564189fff390 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x564189ffee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2643495940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x564189ffe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 >> 172.21.15.37:0/2643495940 conn(0x564189ff2a30 msgr2=0x564189ff8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0baffd640 0 client.25810 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 shutdown_connections 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2643495940 wait complete. 2024-09-19T06:45:06.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.146+0000 7ff0e3fad740 20 client.25810 trim_cache size 0 max 16384 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 Processor -- start 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 -- start start 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6340 0x564189fff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x56418a0214b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56418a005a90 0x56418a0219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1ae040 con 0x56418a005a90 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1ab8b0 con 0x564189ff6c60 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1a9700 con 0x564189ff6340 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56418a005a90 0x56418a0219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6340 0x564189fff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x56418a0214b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56418a005a90 0x56418a0219f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37118/0 (socket says 172.21.15.37:37118) 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6340 0x564189fff1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:43702/0 (socket says 172.21.15.37:43702) 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 -- 172.21.15.37:0/2572999498 learned_addr learned my addr 172.21.15.37:0/2572999498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 -- 172.21.15.37:0/2572999498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6340 msgr2=0x564189fff1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6340 0x564189fff1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 -- 172.21.15.37:0/2572999498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 msgr2=0x56418a0214b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x56418a0214b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0dbfff640 1 -- 172.21.15.37:0/2572999498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564189ffce60 con 0x56418a005a90 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.164+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6340 0x564189fff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.165+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56418a005a90 0x56418a0219f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff0d401ca30 tx=0x7ff0d4009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf650fb20a58021 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.165+0000 7ff0d17fa640 1 -- 172.21.15.37:0/2572999498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4036070 con 0x56418a005a90 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.165+0000 7ff0d17fa640 1 -- 172.21.15.37:0/2572999498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d400fc80 con 0x56418a005a90 2024-09-19T06:45:06.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.165+0000 7ff0d17fa640 1 -- 172.21.15.37:0/2572999498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0d40057d0 con 0x56418a005a90 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2572999498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56418a005a90 msgr2=0x56418a0219f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56418a005a90 0x56418a0219f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7ff0d401ca30 tx=0x7ff0d4009500 comp rx=0 tx=0).stop 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2572999498 shutdown_connections 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56418a005a90 0x56418a0219f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564189ff6c60 0x56418a0214b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/2572999498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6340 0x564189fff1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2572999498 >> 172.21.15.37:0/2572999498 conn(0x564189ff2a30 msgr2=0x56418a0029a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2572999498 shutdown_connections 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 -- 172.21.15.37:0/2572999498 wait complete. 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 -- start start 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.166+0000 7ff0e3fad740 1 Processor -- start 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x56418a0165c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6c60 0x56418a0118f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56418a005a90 0x56418a011ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1a9d70 con 0x564189ff6340 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1ab1e0 con 0x56418a005a90 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff0dc1aa440 con 0x564189ff6c60 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 10 client.0 _ll_register_callbacks cb 0x7ff0d401f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x56418a0165c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x56418a0165c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37132/0 (socket says 172.21.15.37:37132) 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 -- 172.21.15.37:0/917260678 learned_addr learned my addr 172.21.15.37:0/917260678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56418a005a90 0x56418a011ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0d2ffd640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6c60 0x56418a0118f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 -- 172.21.15.37:0/917260678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6c60 msgr2=0x56418a0118f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6c60 0x56418a0118f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 -- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56418a005a90 msgr2=0x56418a011ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56418a005a90 0x56418a011ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 -- 172.21.15.37:0/917260678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56418a012710 con 0x564189ff6340 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0dbfff640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56418a005a90 0x56418a011ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0db7fe640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x56418a0165c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc005ab0 tx=0x7ff0cc0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3468e240d24db33 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0baffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0baffd640 1 -- 172.21.15.37:0/917260678 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc049040 con 0x564189ff6340 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0baffd640 1 -- 172.21.15.37:0/917260678 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff0cc0edd50 con 0x564189ff6340 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.167+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0d400fc80 con 0x564189ff6340 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0baffd640 1 -- 172.21.15.37:0/917260678 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0cc0bc710 con 0x564189ff6340 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 10 client.16404 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56418a0845d0 con 0x564189ff6340 2024-09-19T06:45:06.175 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0baffd640 1 -- 172.21.15.37:0/917260678 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0d400fc80 con 0x564189ff6340 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0baffd640 1 -- 172.21.15.37:0/917260678 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff0cc0a1e80 con 0x564189ff6340 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 10 client.16404 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 20 client.16404 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a084d20 0x56418a087140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 4 client.16404 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564189f63d60 con 0x56418a084d20 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.168+0000 7ff0d2ffd640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a084d20 0x56418a087140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0baffd640 10 client.16404 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0d2ffd640 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a084d20 0x56418a087140 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d5fed52342d4823 server_cookie=96cfab005c8b15e2 in_seq=0 out_seq=0 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0baffd640 1 -- 172.21.15.37:0/917260678 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564189f63d60 con 0x56418a084d20 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0baffd640 10 client.16404 handle_command_reply: tid=0 2024-09-19T06:45:06.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 client.16404 shutdown 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 msgr2=0x56418a0165c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x56418a0165c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc005ab0 tx=0x7ff0cc0058b0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 shutdown_connections 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56418a084d20 0x56418a087140 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56418a005a90 0x56418a011ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564189ff6c60 0x56418a0118f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 --2- 172.21.15.37:0/917260678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564189ff6340 0x56418a0165c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 >> 172.21.15.37:0/917260678 conn(0x564189ff2a30 msgr2=0x564189ff1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0baffd640 0 client.16404 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 shutdown_connections 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 1 -- 172.21.15.37:0/917260678 wait complete. 2024-09-19T06:45:06.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.169+0000 7ff0e3fad740 20 client.16404 trim_cache size 0 max 16384 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "51916c16-6273-414b-9158-036f91fb19d6": { 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "51916c16-6273-414b-9158-036f91fb19d6", 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:06.178 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:45:06.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff0c80a33b0 msgr2=0x7ff0c80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff0c80a33b0 0x7ff0c80a5870 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4020000 tx=0x7ff0d4021000 comp rx=0 tx=0).stop 2024-09-19T06:45:06.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 msgr2=0x7ff0dc1984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc1984d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc006010 tx=0x7ff0cc003c10 comp rx=0 tx=0).stop 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 shutdown_connections 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff0dc102ef0 0x7ff0dc19cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0dc102570 0x7ff0dc198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff0c80a33b0 0x7ff0c80a5870 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 --2- 172.21.15.37:0/604967329 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff0dc100e20 0x7ff0dc1984d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 >> 172.21.15.37:0/604967329 conn(0x7ff0dc0fc7d0 msgr2=0x7ff0dc0fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 shutdown_connections 2024-09-19T06:45:06.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:06.170+0000 7ff0e1bd4640 1 -- 172.21.15.37:0/604967329 wait complete. 2024-09-19T06:45:06.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 198807 11.73 MB/sec warmup 93 sec latency 68.404 ms 2024-09-19T06:45:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:06 smithi096 ceph-mon[35004]: pgmap v103: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 310 op/s 2024-09-19T06:45:06.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:06 smithi037 ceph-mon[31211]: pgmap v103: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 310 op/s 2024-09-19T06:45:06.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:06 smithi151 ceph-mon[35236]: pgmap v103: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 310 op/s 2024-09-19T06:45:06.972 INFO:tasks.cephfs.filesystem:scrub status for tag:51916c16-6273-414b-9158-036f91fb19d6 - {'path': '/', 'tag': '51916c16-6273-414b-9158-036f91fb19d6', 'options': 'recursive,force'} 2024-09-19T06:45:07.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 201567 11.75 MB/sec warmup 94 sec latency 12.679 ms 2024-09-19T06:45:08.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 203878 11.75 MB/sec warmup 95 sec latency 32.136 ms 2024-09-19T06:45:08.665 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:08 smithi151 ceph-mon[35236]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 310 op/s 2024-09-19T06:45:08.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:08 smithi096 ceph-mon[35004]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 310 op/s 2024-09-19T06:45:08.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:08 smithi037 ceph-mon[31211]: pgmap v104: 97 pgs: 97 active+clean; 55 MiB data, 751 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 MiB/s rd, 3.7 MiB/s wr, 310 op/s 2024-09-19T06:45:09.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 206239 11.77 MB/sec warmup 96 sec latency 107.965 ms 2024-09-19T06:45:10.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 209286 11.83 MB/sec warmup 97 sec latency 11.204 ms 2024-09-19T06:45:10.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:10 smithi096 ceph-mon[35004]: pgmap v105: 97 pgs: 97 active+clean; 45 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-09-19T06:45:10.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:10 smithi037 ceph-mon[31211]: pgmap v105: 97 pgs: 97 active+clean; 45 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-09-19T06:45:10.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:10 smithi151 ceph-mon[35236]: pgmap v105: 97 pgs: 97 active+clean; 45 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.6 MiB/s wr, 578 op/s 2024-09-19T06:45:11.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 211714 11.80 MB/sec warmup 98 sec latency 11.244 ms 2024-09-19T06:45:12.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 214006 11.82 MB/sec warmup 99 sec latency 15.397 ms 2024-09-19T06:45:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:12 smithi096 ceph-mon[35004]: pgmap v106: 97 pgs: 97 active+clean; 45 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 436 op/s 2024-09-19T06:45:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:12 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:45:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:12 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:45:12.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:12 smithi037 ceph-mon[31211]: pgmap v106: 97 pgs: 97 active+clean; 45 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 436 op/s 2024-09-19T06:45:12.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:12 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:45:12.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:12 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:45:12.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:12 smithi151 ceph-mon[35236]: pgmap v106: 97 pgs: 97 active+clean; 45 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.9 MiB/s wr, 436 op/s 2024-09-19T06:45:12.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:12 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:45:12.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:12 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:45:13.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 216969 11.87 MB/sec warmup 100 sec latency 16.735 ms 2024-09-19T06:45:13.268 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 16.6 2024-09-19T06:45:13.269 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:45:13.535 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:14.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 -- 172.21.15.37:0/1110567750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 msgr2=0x7f785010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 --2- 172.21.15.37:0/1110567750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 0x7f785010e610 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f7848009ca0 tx=0x7f784802a950 comp rx=0 tx=0).stop 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 -- 172.21.15.37:0/1110567750 shutdown_connections 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 --2- 172.21.15.37:0/1110567750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 0x7f785010e610 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 --2- 172.21.15.37:0/1110567750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7850103660 0x7f785010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 --2- 172.21.15.37:0/1110567750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 0x7f7850103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 -- 172.21.15.37:0/1110567750 >> 172.21.15.37:0/1110567750 conn(0x7f78500fc7a0 msgr2=0x7f78500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.015+0000 7f78585a2640 1 -- 172.21.15.37:0/1110567750 shutdown_connections 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 -- 172.21.15.37:0/1110567750 wait complete. 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 Processor -- start 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 -- start start 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 0x7f78501009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7850103660 0x7f7850100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:14.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 0x7f7850101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7850111f30 con 0x7f785010c170 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7850111db0 con 0x7f7850102d40 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.016+0000 7f78585a2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f78501120b0 con 0x7f7850103660 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 0x7f78501009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f7856b3f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 0x7f7850101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 0x7f78501009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54840/0 (socket says 172.21.15.37:54840) 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 -- 172.21.15.37:0/52680311 learned_addr learned my addr 172.21.15.37:0/52680311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f7855b3d640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7850103660 0x7f7850100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 -- 172.21.15.37:0/52680311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7850103660 msgr2=0x7f7850100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7850103660 0x7f7850100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 -- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 msgr2=0x7f7850101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 0x7f7850101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 -- 172.21.15.37:0/52680311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78501a6900 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f785633e640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 0x7f78501009a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f784c00ea60 tx=0x7f784c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f78477fe640 1 -- 172.21.15.37:0/52680311 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c00cde0 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78501a6bf0 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f78477fe640 1 -- 172.21.15.37:0/52680311 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f784c0035f0 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f78477fe640 1 -- 172.21.15.37:0/52680311 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c010670 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.017+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78501a7130 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.018+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7850103120 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.019+0000 7f78477fe640 1 -- 172.21.15.37:0/52680311 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f784c0037a0 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.020+0000 7f78477fe640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f783c0a33b0 0x7f783c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.020+0000 7f7855b3d640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f783c0a33b0 0x7f783c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.020+0000 7f78477fe640 1 -- 172.21.15.37:0/52680311 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f784c0d0280 con 0x7f7850102d40 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.021+0000 7f7855b3d640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f783c0a33b0 0x7f783c0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7850102430 tx=0x7f7840005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:14.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.023+0000 7f78477fe640 1 -- 172.21.15.37:0/52680311 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f784c09cc60 con 0x7f7850102d40 2024-09-19T06:45:14.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 219420 11.88 MB/sec warmup 101 sec latency 79.907 ms 2024-09-19T06:45:14.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.243+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7850101ed0 con 0x7f7850102d40 2024-09-19T06:45:14.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.244+0000 7f78477fe640 1 -- 172.21.15.37:0/52680311 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11906 (secure 0 0 0) 0x7f784c09c560 con 0x7f7850102d40 2024-09-19T06:45:14.246 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:45:14.246 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"btime":"2024-09-19T06:44:24:345285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:44:24.345273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:45:14.246 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-09-19T06:45:14.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f783c0a33b0 msgr2=0x7f783c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:14.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f783c0a33b0 0x7f783c0a5870 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7850102430 tx=0x7f7840005f90 comp rx=0 tx=0).stop 2024-09-19T06:45:14.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 msgr2=0x7f78501009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:14.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 0x7f78501009a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f784c00ea60 tx=0x7f784c00ef20 comp rx=0 tx=0).stop 2024-09-19T06:45:14.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 shutdown_connections 2024-09-19T06:45:14.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f783c0a33b0 0x7f783c0a5870 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f785010c170 0x7f7850101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7850103660 0x7f7850100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 --2- 172.21.15.37:0/52680311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7850102d40 0x7f78501009a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:14.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 >> 172.21.15.37:0/52680311 conn(0x7f78500fc7a0 msgr2=0x7f78500fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:14.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 shutdown_connections 2024-09-19T06:45:14.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:14.247+0000 7f78585a2640 1 -- 172.21.15.37:0/52680311 wait complete. 2024-09-19T06:45:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:14 smithi037 ceph-mon[31211]: pgmap v107: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.8 MiB/s wr, 582 op/s 2024-09-19T06:45:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:14 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/52680311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:14 smithi096 ceph-mon[35004]: pgmap v107: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.8 MiB/s wr, 582 op/s 2024-09-19T06:45:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:14 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/52680311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:14.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:14 smithi151 ceph-mon[35236]: pgmap v107: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.8 MiB/s wr, 582 op/s 2024-09-19T06:45:14.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:14 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/52680311' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:15.029 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0' 2024-09-19T06:45:15.030 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 136.6 --await-for 45.0 2024-09-19T06:45:15.194 INFO:tasks.workunit.client.0.smithi037.stdout: 1 221945 11.88 MB/sec warmup 102 sec latency 19.221 ms 2024-09-19T06:45:15.305 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:15.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 -- 172.21.15.37:0/2658538164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101082a0 msgr2=0x7ff31010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 --2- 172.21.15.37:0/2658538164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101082a0 0x7ff31010ec40 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7ff308009930 tx=0x7ff30802a420 comp rx=0 tx=0).stop 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 -- 172.21.15.37:0/2658538164 shutdown_connections 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 --2- 172.21.15.37:0/2658538164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101082a0 0x7ff31010ec40 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 --2- 172.21.15.37:0/2658538164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3101036e0 0x7ff310107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 --2- 172.21.15.37:0/2658538164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff310102d30 0x7ff310103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 -- 172.21.15.37:0/2658538164 >> 172.21.15.37:0/2658538164 conn(0x7ff3100fc7d0 msgr2=0x7ff3100febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 -- 172.21.15.37:0/2658538164 shutdown_connections 2024-09-19T06:45:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.829+0000 7ff315116640 1 -- 172.21.15.37:0/2658538164 wait complete. 2024-09-19T06:45:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 Processor -- start 2024-09-19T06:45:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 -- start start 2024-09-19T06:45:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff310102d30 0x7ff310198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101036e0 0x7ff310198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3101082a0 0x7ff31019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101120a0 con 0x7ff3101036e0 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff310111f20 con 0x7ff310102d30 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.830+0000 7ff315116640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff310112220 con 0x7ff3101082a0 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101036e0 0x7ff310198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff310102d30 0x7ff310198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101036e0 0x7ff310198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58240/0 (socket says 172.21.15.37:58240) 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3101082a0 0x7ff31019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff310102d30 0x7ff310198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54850/0 (socket says 172.21.15.37:54850) 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 -- 172.21.15.37:0/1476985207 learned_addr learned my addr 172.21.15.37:0/1476985207 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 -- 172.21.15.37:0/1476985207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3101082a0 msgr2=0x7ff31019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3101082a0 0x7ff31019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 -- 172.21.15.37:0/1476985207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff310102d30 msgr2=0x7ff310198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff310102d30 0x7ff310198550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 -- 172.21.15.37:0/1476985207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff31019d500 con 0x7ff3101036e0 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30f59e640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3101082a0 0x7ff31019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff30e59c640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101036e0 0x7ff310198a90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800e9c0 tx=0x7ff2f800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff307fff640 1 -- 172.21.15.37:0/1476985207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f800ccc0 con 0x7ff3101036e0 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff31019d7f0 con 0x7ff3101036e0 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff307fff640 1 -- 172.21.15.37:0/1476985207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2f8005500 con 0x7ff3101036e0 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff307fff640 1 -- 172.21.15.37:0/1476985207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8010690 con 0x7ff3101036e0 2024-09-19T06:45:15.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.831+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3101a50d0 con 0x7ff3101036e0 2024-09-19T06:45:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.834+0000 7ff307fff640 1 -- 172.21.15.37:0/1476985207 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff2f8004e80 con 0x7ff3101036e0 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.834+0000 7ff3174ef740 1 Processor -- start 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff3174ef740 1 -- start start 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b8f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff3174ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559e0b84b2a0 con 0x559e0b8f0940 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff3174ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e0b7f2b10 con 0x559e0b8e5760 2024-09-19T06:45:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff3174ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e0b847260 con 0x559e0b8e6080 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b8f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8e5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54866/0 (socket says 172.21.15.37:54866) 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 -- 172.21.15.37:0/263614065 learned_addr learned my addr 172.21.15.37:0/263614065 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 -- 172.21.15.37:0/263614065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 msgr2=0x559e0b8f0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 -- 172.21.15.37:0/263614065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 msgr2=0x559e0b8f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b8f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff307fff640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff2fc0a33b0 0x7ff2fc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.835+0000 7ff30f59e640 1 -- 172.21.15.37:0/263614065 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0b5bcb90 con 0x559e0b8e5760 2024-09-19T06:45:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.836+0000 7ff307fff640 1 -- 172.21.15.37:0/1476985207 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff2f80d00b0 con 0x7ff3101036e0 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.836+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff2fc0a33b0 0x7ff2fc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.836+0000 7ff30f59e640 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8e5b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff308009930 tx=0x7ff308005d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a129094a95a08189 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.836+0000 7ff3057fa640 1 -- 172.21.15.37:0/263614065 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308035040 con 0x559e0b8e5760 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.836+0000 7ff3057fa640 1 -- 172.21.15.37:0/263614065 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3080317b0 con 0x559e0b8e5760 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.836+0000 7ff3057fa640 1 -- 172.21.15.37:0/263614065 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff308003850 con 0x559e0b8e5760 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 -- 172.21.15.37:0/263614065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 msgr2=0x559e0b8e5b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff2fc0a33b0 0x7ff2fc0a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x559e0b8e2b50 tx=0x7ff300009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8e5b40 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff308009930 tx=0x7ff308005d50 comp rx=0 tx=0).stop 2024-09-19T06:45:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 -- 172.21.15.37:0/263614065 shutdown_connections 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b8f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 --2- 172.21.15.37:0/263614065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8e5b40 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 -- 172.21.15.37:0/263614065 >> 172.21.15.37:0/263614065 conn(0x559e0b8e1cf0 msgr2=0x559e0b8e20d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 -- 172.21.15.37:0/263614065 shutdown_connections 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 -- 172.21.15.37:0/263614065 wait complete. 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 -- start start 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.837+0000 7ff3174ef740 1 Processor -- start 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e6080 0x559e0b8eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8f0940 0x559e0b8ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff3174ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559e0b8f3510 con 0x559e0b8e6080 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff3174ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559e0b7d36e0 con 0x559e0b8e5760 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff3174ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559e0b8f3690 con 0x559e0b8f0940 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff3174ef740 10 client.0 _ll_register_callbacks cb 0x559e0b7f7a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e6080 0x559e0b8eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e6080 0x559e0b8eddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58254/0 (socket says 172.21.15.37:58254) 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 -- 172.21.15.37:0/4264549262 learned_addr learned my addr 172.21.15.37:0/4264549262 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30f59e640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8f0940 0x559e0b8ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 -- 172.21.15.37:0/4264549262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8f0940 msgr2=0x559e0b8ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8f0940 0x559e0b8ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 -- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 msgr2=0x559e0b8ed870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ed870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30e59c640 1 -- 172.21.15.37:0/4264549262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0b8fca50 con 0x559e0b8e6080 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ed870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.838+0000 7ff30f59e640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8f0940 0x559e0b8ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:15.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff30e59c640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e6080 0x559e0b8eddb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800c6a0 tx=0x7ff2f80d3750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a24256ccd9a5772 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff2f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff2f67fc640 1 -- 172.21.15.37:0/4264549262 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f809be70 con 0x559e0b8e6080 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff2f67fc640 1 -- 172.21.15.37:0/4264549262 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff2f8005170 con 0x559e0b8e6080 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3080317b0 con 0x559e0b8e6080 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff2f67fc640 1 -- 172.21.15.37:0/4264549262 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2f80056b0 con 0x559e0b8e6080 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff3174ef740 10 client.16422 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e0b8ff440 con 0x559e0b8e6080 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff2f67fc640 1 -- 172.21.15.37:0/4264549262 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3080317b0 con 0x559e0b8e6080 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff2f67fc640 1 -- 172.21.15.37:0/4264549262 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff2f80cda20 con 0x559e0b8e6080 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff3174ef740 10 client.16422 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff3174ef740 10 client.16422 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:45:15.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.839+0000 7ff3174ef740 20 client.16422 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.840+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b900950 0x559e0b902d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.840+0000 7ff3174ef740 4 client.16422 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.840+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e0b8589d0 con 0x559e0b900950 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.840+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b900950 0x559e0b902d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.840+0000 7ff2f67fc640 10 client.16422 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.840+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b900950 0x559e0b902d40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=452e9694d686d1fc server_cookie=4ebecf3267301378 in_seq=0 out_seq=0 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff2f67fc640 1 -- 172.21.15.37:0/4264549262 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559e0b8589d0 con 0x559e0b900950 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff2f67fc640 10 client.16422 handle_command_reply: tid=0 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 client.16422 shutdown 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e6080 msgr2=0x559e0b8eddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e6080 0x559e0b8eddb0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800c6a0 tx=0x7ff2f80d3750 comp rx=0 tx=0).stop 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 shutdown_connections 2024-09-19T06:45:15.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b900950 0x559e0b902d40 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8f0940 0x559e0b8ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e6080 0x559e0b8eddb0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4264549262 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ed870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 >> 172.21.15.37:0/4264549262 conn(0x559e0b8e1cf0 msgr2=0x559e0b8e0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff2f67fc640 0 client.16422 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 shutdown_connections 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 1 -- 172.21.15.37:0/4264549262 wait complete. 2024-09-19T06:45:15.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.842+0000 7ff3174ef740 20 client.16422 trim_cache size 0 max 16384 2024-09-19T06:45:15.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 Processor -- start 2024-09-19T06:45:15.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 -- start start 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b908610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b908b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ae020 con 0x559e0b8f0940 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ab890 con 0x559e0b8e5760 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff3174ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101a96e0 con 0x559e0b8e6080 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff30e59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b908610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ec140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54896/0 (socket says 172.21.15.37:54896) 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff30ed9d640 1 -- 172.21.15.37:0/4001374517 learned_addr learned my addr 172.21.15.37:0/4001374517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.860+0000 7ff30f59e640 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b908b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff30ed9d640 1 -- 172.21.15.37:0/4001374517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 msgr2=0x559e0b908610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b908610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff30ed9d640 1 -- 172.21.15.37:0/4001374517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 msgr2=0x559e0b908b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b908b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff30ed9d640 1 -- 172.21.15.37:0/4001374517 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e0b8dd260 con 0x559e0b8e5760 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff30e59c640 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b908610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ec140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff300005fd0 tx=0x7ff30001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=854eb3ef4ed5a45e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff304ff9640 1 -- 172.21.15.37:0/4001374517 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff300004b20 con 0x559e0b8e5760 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff304ff9640 1 -- 172.21.15.37:0/4001374517 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30000fcf0 con 0x559e0b8e5760 2024-09-19T06:45:15.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.861+0000 7ff304ff9640 1 -- 172.21.15.37:0/4001374517 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff30002aac0 con 0x559e0b8e5760 2024-09-19T06:45:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 -- 172.21.15.37:0/4001374517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 msgr2=0x559e0b8ec140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ec140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff300005fd0 tx=0x7ff30001a000 comp rx=0 tx=0).stop 2024-09-19T06:45:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 -- 172.21.15.37:0/4001374517 shutdown_connections 2024-09-19T06:45:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8f0940 0x559e0b908b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b908610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 --2- 172.21.15.37:0/4001374517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8e5760 0x559e0b8ec140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 -- 172.21.15.37:0/4001374517 >> 172.21.15.37:0/4001374517 conn(0x559e0b8e1cf0 msgr2=0x559e0b8ef8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 -- 172.21.15.37:0/4001374517 shutdown_connections 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 -- 172.21.15.37:0/4001374517 wait complete. 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.862+0000 7ff3174ef740 1 -- start start 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 1 Processor -- start 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e5760 0x559e0b8f4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8f0940 0x559e0b8f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101a9d50 con 0x559e0b8e5760 2024-09-19T06:45:15.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101ab1c0 con 0x559e0b8f0940 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3101aa420 con 0x559e0b8e6080 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff3174ef740 10 client.0 _ll_register_callbacks cb 0x7ff2f8004a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e5760 0x559e0b8f4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e5760 0x559e0b8f4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58280/0 (socket says 172.21.15.37:58280) 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30ed9d640 1 -- 172.21.15.37:0/1732376413 learned_addr learned my addr 172.21.15.37:0/1732376413 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30e59c640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30f59e640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8f0940 0x559e0b8f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30e59c640 1 -- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8f0940 msgr2=0x559e0b8f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30e59c640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8f0940 0x559e0b8f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30e59c640 1 -- 172.21.15.37:0/1732376413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e5760 msgr2=0x559e0b8f4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30e59c640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e5760 0x559e0b8f4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30e59c640 1 -- 172.21.15.37:0/1732376413 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e0b96a630 con 0x559e0b8e6080 2024-09-19T06:45:15.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.863+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e5760 0x559e0b8f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff30e59c640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f4bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8004650 tx=0x7ff2f80b5420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d3dbd054b933635 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff2f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff2f67fc640 1 -- 172.21.15.37:0/1732376413 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f809bc90 con 0x559e0b8e6080 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff2f67fc640 1 -- 172.21.15.37:0/1732376413 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff2f80a4d10 con 0x559e0b8e6080 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff30000fcf0 con 0x559e0b8e6080 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff2f67fc640 1 -- 172.21.15.37:0/1732376413 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2f809c330 con 0x559e0b8e6080 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff3174ef740 10 client.25840 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559e0b96b8d0 con 0x559e0b8e6080 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff2f67fc640 1 -- 172.21.15.37:0/1732376413 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff30000fcf0 con 0x559e0b8e6080 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff2f67fc640 1 -- 172.21.15.37:0/1732376413 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff2f809b780 con 0x559e0b8e6080 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff3174ef740 10 client.25840 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff3174ef740 10 client.25840 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff3174ef740 20 client.25840 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:15.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.864+0000 7ff3174ef740 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b96bf80 0x559e0b96e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.865+0000 7ff3174ef740 4 client.25840 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}] 2024-09-19T06:45:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.865+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 136.6, "await_for": 45.0}) -- 0x559e0b852d20 con 0x559e0b96bf80 2024-09-19T06:45:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.865+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b96bf80 0x559e0b96e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.865+0000 7ff2f67fc640 10 client.25840 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:15.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.865+0000 7ff30ed9d640 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b96bf80 0x559e0b96e3a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc5524950258a709 server_cookie=bcfc8404d88f877d in_seq=0 out_seq=0 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.960+0000 7ff2f67fc640 1 -- 172.21.15.37:0/1732376413 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x559e0b852d20 con 0x559e0b96bf80 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.960+0000 7ff2f67fc640 10 client.25840 handle_command_reply: tid=0 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.960+0000 7ff3174ef740 1 client.25840 shutdown 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 msgr2=0x559e0b8f4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f4bb0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8004650 tx=0x7ff2f80b5420 comp rx=0 tx=0).stop 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 shutdown_connections 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559e0b96bf80 0x559e0b96e3a0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559e0b8f0940 0x559e0b8f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559e0b8e6080 0x559e0b8f4bb0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff2f67fc640 0 client.25840 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 --2- 172.21.15.37:0/1732376413 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559e0b8e5760 0x559e0b8f4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 >> 172.21.15.37:0/1732376413 conn(0x559e0b8e1cf0 msgr2=0x559e0b8e0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 shutdown_connections 2024-09-19T06:45:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 1 -- 172.21.15.37:0/1732376413 wait complete. 2024-09-19T06:45:15.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.961+0000 7ff3174ef740 20 client.25840 trim_cache size 0 max 16384 2024-09-19T06:45:15.964 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:45:15.964 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 29, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 14, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "b4359ca8": { 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 14, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 136.6, 2024-09-19T06:45:15.965 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:15.966 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:45:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff2fc0a33b0 msgr2=0x7ff2fc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff2fc0a33b0 0x7ff2fc0a5870 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x559e0b8e2b50 tx=0x7ff300009500 comp rx=0 tx=0).stop 2024-09-19T06:45:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101036e0 msgr2=0x7ff310198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101036e0 0x7ff310198a90 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800e9c0 tx=0x7ff2f800ee80 comp rx=0 tx=0).stop 2024-09-19T06:45:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 shutdown_connections 2024-09-19T06:45:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3101082a0 0x7ff31019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff2fc0a33b0 0x7ff2fc0a5870 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3101036e0 0x7ff310198a90 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 --2- 172.21.15.37:0/1476985207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff310102d30 0x7ff310198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 >> 172.21.15.37:0/1476985207 conn(0x7ff3100fc7d0 msgr2=0x7ff310106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 shutdown_connections 2024-09-19T06:45:15.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:15.962+0000 7ff315116640 1 -- 172.21.15.37:0/1476985207 wait complete. 2024-09-19T06:45:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.087+0000 7f8c9e71a640 1 -- 172.21.15.37:0/3215767474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c98101380 msgr2=0x7f8c9810d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.087+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/3215767474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c98101380 0x7f8c9810d8b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009cd0 tx=0x7f8c9002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.087+0000 7f8c9e71a640 1 -- 172.21.15.37:0/3215767474 shutdown_connections 2024-09-19T06:45:16.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.087+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/3215767474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c98101380 0x7f8c9810d8b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.087+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/3215767474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c981009e0 0x7f8c98100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.087+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/3215767474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c981069f0 0x7f8c98106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 -- 172.21.15.37:0/3215767474 >> 172.21.15.37:0/3215767474 conn(0x7f8c980fc6a0 msgr2=0x7f8c980feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 -- 172.21.15.37:0/3215767474 shutdown_connections 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 -- 172.21.15.37:0/3215767474 wait complete. 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 Processor -- start 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 -- start start 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c981009e0 0x7f8c98199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c98101380 0x7f8c981996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c981069f0 0x7f8c981a41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98110e30 con 0x7f8c981009e0 2024-09-19T06:45:16.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98110cb0 con 0x7f8c981069f0 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c9e71a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c98110fb0 con 0x7f8c98101380 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c981009e0 0x7f8c98199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.088+0000 7f8c97fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c981009e0 0x7f8c98199160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58306/0 (socket says 172.21.15.37:58306) 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c97fff640 1 -- 172.21.15.37:0/2687007917 learned_addr learned my addr 172.21.15.37:0/2687007917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c98101380 0x7f8c981996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c981069f0 0x7f8c981a41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c97fff640 1 -- 172.21.15.37:0/2687007917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c98101380 msgr2=0x7f8c981996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c97fff640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c98101380 0x7f8c981996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c97fff640 1 -- 172.21.15.37:0/2687007917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c981069f0 msgr2=0x7f8c981a41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c97fff640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c981069f0 0x7f8c981a41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c97fff640 1 -- 172.21.15.37:0/2687007917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c981a4720 con 0x7f8c981009e0 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c97fff640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c981009e0 0x7f8c98199160 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800e9f0 tx=0x7f8c8800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:16.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c95ffb640 1 -- 172.21.15.37:0/2687007917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800cd80 con 0x7f8c981009e0 2024-09-19T06:45:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c95ffb640 1 -- 172.21.15.37:0/2687007917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c88010910 con 0x7f8c981009e0 2024-09-19T06:45:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c981a48f0 con 0x7f8c981009e0 2024-09-19T06:45:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c95ffb640 1 -- 172.21.15.37:0/2687007917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c88018990 con 0x7f8c981009e0 2024-09-19T06:45:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.089+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c981a4e60 con 0x7f8c981009e0 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 Processor -- start 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 -- start start 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8c95ffb640 1 -- 172.21.15.37:0/2687007917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8c88010ac0 con 0x7f8c981009e0 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a680af0 0x56481a680ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a68b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a68e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56481a5e52f0 con 0x56481a680af0 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56481a58d270 con 0x56481a68bcb0 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8ca14af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56481a5e12b0 con 0x56481a681410 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8c9ccb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a680af0 0x56481a680ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8c9ccb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a680af0 0x56481a680ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58320/0 (socket says 172.21.15.37:58320) 2024-09-19T06:45:16.100 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8c9ccb7640 1 -- 172.21.15.37:0/3722390378 learned_addr learned my addr 172.21.15.37:0/3722390378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a68e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.092+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a68b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8ffff640 1 -- 172.21.15.37:0/3722390378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 msgr2=0x56481a68b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a68b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8ffff640 1 -- 172.21.15.37:0/3722390378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a680af0 msgr2=0x56481a680ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a680af0 0x56481a680ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8ffff640 1 -- 172.21.15.37:0/3722390378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481a58c340 con 0x56481a68bcb0 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a680af0 0x56481a680ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a68e0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009bf0 tx=0x7f8c80011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64ed5b2ca27cb0f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c95ffb640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8c840a34e0 0x7f8c840a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8e7fc640 1 -- 172.21.15.37:0/3722390378 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80004050 con 0x56481a68bcb0 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8c840a34e0 0x7f8c840a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c95ffb640 1 -- 172.21.15.37:0/2687007917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f8c880cffa0 con 0x7f8c981009e0 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8e7fc640 1 -- 172.21.15.37:0/3722390378 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8001a070 con 0x56481a68bcb0 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.093+0000 7f8c8e7fc640 1 -- 172.21.15.37:0/3722390378 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80004c60 con 0x56481a68bcb0 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 -- 172.21.15.37:0/3722390378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 msgr2=0x56481a68e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8c840a34e0 0x7f8c840a59a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c90005e30 tx=0x7f8c90005c50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a68e0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009bf0 tx=0x7f8c80011810 comp rx=0 tx=0).stop 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 -- 172.21.15.37:0/3722390378 shutdown_connections 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a68e0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a68b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3722390378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a680af0 0x56481a680ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 -- 172.21.15.37:0/3722390378 >> 172.21.15.37:0/3722390378 conn(0x56481a67d060 msgr2=0x56481a67d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 -- 172.21.15.37:0/3722390378 shutdown_connections 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 -- 172.21.15.37:0/3722390378 wait complete. 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.094+0000 7f8ca14af740 1 -- start start 2024-09-19T06:45:16.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 1 Processor -- start 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a680af0 0x56481a688b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a689080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a6895c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56481a68e880 con 0x56481a681410 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56481a56e290 con 0x56481a68bcb0 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56481a68ea00 con 0x56481a680af0 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8ca14af740 10 client.0 _ll_register_callbacks cb 0x56481a59f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a689080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a689080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58332/0 (socket says 172.21.15.37:58332) 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 -- 172.21.15.37:0/3945180144 learned_addr learned my addr 172.21.15.37:0/3945180144 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:16.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a6895c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a680af0 0x56481a688b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 -- 172.21.15.37:0/3945180144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a680af0 msgr2=0x56481a688b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a680af0 0x56481a688b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 -- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 msgr2=0x56481a6895c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a6895c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c8ffff640 1 -- 172.21.15.37:0/3945180144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481a2f8210 con 0x56481a681410 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a680af0 0x56481a688b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.095+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a6895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a689080 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8c80004610 tx=0x7f8c80009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d26078a4314514 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8c777fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8c777fe640 1 -- 172.21.15.37:0/3945180144 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80003a00 con 0x56481a681410 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8c777fe640 1 -- 172.21.15.37:0/3945180144 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f8c80003ba0 con 0x56481a681410 2024-09-19T06:45:16.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8001a070 con 0x56481a681410 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8c777fe640 1 -- 172.21.15.37:0/3945180144 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c80037050 con 0x56481a681410 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 10 client.16440 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56481a3c4620 con 0x56481a681410 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8c777fe640 1 -- 172.21.15.37:0/3945180144 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8001a070 con 0x56481a681410 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8c777fe640 1 -- 172.21.15.37:0/3945180144 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8c80029420 con 0x56481a681410 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 10 client.16440 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 20 client.16440 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a69bb40 0x56481a69df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 4 client.16440 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.096+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56481a5f2a20 con 0x56481a69bb40 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.097+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a69bb40 0x56481a69df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.097+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a69bb40 0x56481a69df30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec6303c87f927611 server_cookie=ebfd8ad9f55b7538 in_seq=0 out_seq=0 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.097+0000 7f8c777fe640 10 client.16440 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8c777fe640 1 -- 172.21.15.37:0/3945180144 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56481a5f2a20 con 0x56481a69bb40 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8c777fe640 10 client.16440 handle_command_reply: tid=0 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 client.16440 shutdown 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 msgr2=0x56481a689080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a689080 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f8c80004610 tx=0x7f8c80009850 comp rx=0 tx=0).stop 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 shutdown_connections 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a69bb40 0x56481a69df30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a68bcb0 0x56481a6895c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a689080 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8c777fe640 0 client.16440 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3945180144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a680af0 0x56481a688b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 >> 172.21.15.37:0/3945180144 conn(0x56481a67d060 msgr2=0x56481a682dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 shutdown_connections 2024-09-19T06:45:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 1 -- 172.21.15.37:0/3945180144 wait complete. 2024-09-19T06:45:16.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.098+0000 7f8ca14af740 20 client.16440 trim_cache size 0 max 16384 2024-09-19T06:45:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.113+0000 7f8ca14af740 1 Processor -- start 2024-09-19T06:45:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.113+0000 7f8ca14af740 1 -- start start 2024-09-19T06:45:16.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a683560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a6893f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a68bcb0 0x56481a68aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8ca14af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981adf70 con 0x56481a681410 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8ca14af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981ab7e0 con 0x56481a680af0 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8ca14af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981a9630 con 0x56481a68bcb0 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a6893f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a6893f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58344/0 (socket says 172.21.15.37:58344) 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 -- 172.21.15.37:0/1578148010 learned_addr learned my addr 172.21.15.37:0/1578148010 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c97fff640 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a683560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a68bcb0 0x56481a68aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 -- 172.21.15.37:0/1578148010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a68bcb0 msgr2=0x56481a68aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a68bcb0 0x56481a68aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 -- 172.21.15.37:0/1578148010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 msgr2=0x56481a683560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a683560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 -- 172.21.15.37:0/1578148010 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56481a67ce90 con 0x56481a681410 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a68bcb0 0x56481a68aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a6893f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c80005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3375cdc6285a8551 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8dffb640 1 -- 172.21.15.37:0/1578148010 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80029420 con 0x56481a681410 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8dffb640 1 -- 172.21.15.37:0/1578148010 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8003f020 con 0x56481a681410 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.114+0000 7f8c8dffb640 1 -- 172.21.15.37:0/1578148010 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c80003800 con 0x56481a681410 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 -- 172.21.15.37:0/1578148010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 msgr2=0x56481a6893f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a6893f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8c80000c00 tx=0x7f8c80005970 comp rx=0 tx=0).stop 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 -- 172.21.15.37:0/1578148010 shutdown_connections 2024-09-19T06:45:16.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a68bcb0 0x56481a68aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a681410 0x56481a6893f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 --2- 172.21.15.37:0/1578148010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a683560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 -- 172.21.15.37:0/1578148010 >> 172.21.15.37:0/1578148010 conn(0x56481a67d060 msgr2=0x56481a68a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 -- 172.21.15.37:0/1578148010 shutdown_connections 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 -- 172.21.15.37:0/1578148010 wait complete. 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 -- start start 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.115+0000 7f8ca14af740 1 Processor -- start 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a686810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a693c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8ca14af740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a68bcb0 0x56481a694420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8ca14af740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981a9ca0 con 0x56481a68bcb0 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8ca14af740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981ab110 con 0x56481a680af0 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8ca14af740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8c981aa370 con 0x56481a681410 2024-09-19T06:45:16.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8ca14af740 10 client.0 _ll_register_callbacks cb 0x7f8c88005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a68bcb0 0x56481a694420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a68bcb0 0x56481a694420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58346/0 (socket says 172.21.15.37:58346) 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 -- 172.21.15.37:0/3746567255 learned_addr learned my addr 172.21.15.37:0/3746567255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a686810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c8ffff640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a693c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 -- 172.21.15.37:0/3746567255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 msgr2=0x56481a693c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a693c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 -- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 msgr2=0x56481a686810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a686810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 -- 172.21.15.37:0/3746567255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56481a684e40 con 0x56481a68bcb0 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a686810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c9ccb7640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a68bcb0 0x56481a694420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8c90003f50 tx=0x7f8c9002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657d52e177a50a08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:16.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c777fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.116+0000 7f8c777fe640 1 -- 172.21.15.37:0/3746567255 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c900044c0 con 0x56481a68bcb0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c8003f020 con 0x56481a68bcb0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8c777fe640 1 -- 172.21.15.37:0/3746567255 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f8c90004660 con 0x56481a68bcb0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8c777fe640 1 -- 172.21.15.37:0/3746567255 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8c90060050 con 0x56481a68bcb0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 10 client.16446 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56481a705890 con 0x56481a68bcb0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8c777fe640 1 -- 172.21.15.37:0/3746567255 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8c8003f020 con 0x56481a68bcb0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8c777fe640 1 -- 172.21.15.37:0/3746567255 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8c90030070 con 0x56481a68bcb0 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 10 client.16446 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 20 client.16446 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a7060f0 0x56481a708510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:16.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 4 client.16446 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:45:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56481a5ecd70 con 0x56481a7060f0 2024-09-19T06:45:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.117+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a7060f0 0x56481a708510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.118+0000 7f8c777fe640 10 client.16446 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:16.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.118+0000 7f8c97fff640 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a7060f0 0x56481a708510 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cba72b9769ad7b6 server_cookie=2b4e2cc485822e05 in_seq=0 out_seq=0 2024-09-19T06:45:16.195 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.90 MB/sec warmup 103 sec latency 234.995 ms 2024-09-19T06:45:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8c777fe640 1 -- 172.21.15.37:0/3746567255 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56481a5ecd70 con 0x56481a7060f0 2024-09-19T06:45:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8c777fe640 10 client.16446 handle_command_reply: tid=0 2024-09-19T06:45:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 client.16446 shutdown 2024-09-19T06:45:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a68bcb0 msgr2=0x56481a694420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a68bcb0 0x56481a694420 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8c90003f50 tx=0x7f8c9002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 shutdown_connections 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56481a7060f0 0x56481a708510 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56481a68bcb0 0x56481a694420 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8c777fe640 0 client.16446 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56481a681410 0x56481a693c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 --2- 172.21.15.37:0/3746567255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56481a680af0 0x56481a686810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 >> 172.21.15.37:0/3746567255 conn(0x56481a67d060 msgr2=0x56481a686fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 shutdown_connections 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 1 -- 172.21.15.37:0/3746567255 wait complete. 2024-09-19T06:45:16.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.232+0000 7f8ca14af740 20 client.16446 trim_cache size 0 max 16384 2024-09-19T06:45:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:45:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:45:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:45:16.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8c840a34e0 msgr2=0x7f8c840a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8c840a34e0 0x7f8c840a59a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f8c90005e30 tx=0x7f8c90005c50 comp rx=0 tx=0).stop 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c981009e0 msgr2=0x7f8c98199160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c981009e0 0x7f8c98199160 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f8c8800e9f0 tx=0x7f8c8800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 shutdown_connections 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c981069f0 0x7f8c981a41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8c98101380 0x7f8c981996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8c840a34e0 0x7f8c840a59a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 --2- 172.21.15.37:0/2687007917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8c981009e0 0x7f8c98199160 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 >> 172.21.15.37:0/2687007917 conn(0x7f8c980fc6a0 msgr2=0x7f8c980fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 shutdown_connections 2024-09-19T06:45:16.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:16.233+0000 7f8c9e71a640 1 -- 172.21.15.37:0/2687007917 wait complete. 2024-09-19T06:45:16.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:16 smithi096 ceph-mon[35004]: pgmap v108: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-19T06:45:16.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:16 smithi037 ceph-mon[31211]: pgmap v108: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-19T06:45:16.819 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'b4359ca8', quiesce took 3.6 seconds. Will release after: 13.000000000000002 2024-09-19T06:45:16.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:16 smithi151 ceph-mon[35236]: pgmap v108: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-19T06:45:17.195 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.79 MB/sec warmup 104 sec latency 1235.091 ms 2024-09-19T06:45:18.195 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.67 MB/sec warmup 105 sec latency 2235.222 ms 2024-09-19T06:45:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:18 smithi096 ceph-mon[35004]: pgmap v109: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-19T06:45:18.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:18 smithi037 ceph-mon[31211]: pgmap v109: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-19T06:45:18.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:18 smithi151 ceph-mon[35236]: pgmap v109: 97 pgs: 97 active+clean; 52 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.8 MiB/s wr, 413 op/s 2024-09-19T06:45:19.195 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.56 MB/sec warmup 106 sec latency 3235.283 ms 2024-09-19T06:45:20.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.45 MB/sec warmup 107 sec latency 4235.377 ms 2024-09-19T06:45:20.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:20 smithi096 ceph-mon[35004]: pgmap v110: 97 pgs: 97 active+clean; 39 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 637 op/s 2024-09-19T06:45:20.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:20 smithi037 ceph-mon[31211]: pgmap v110: 97 pgs: 97 active+clean; 39 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 637 op/s 2024-09-19T06:45:20.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:20 smithi151 ceph-mon[35236]: pgmap v110: 97 pgs: 97 active+clean; 39 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 5.4 MiB/s wr, 637 op/s 2024-09-19T06:45:21.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.35 MB/sec warmup 108 sec latency 5235.477 ms 2024-09-19T06:45:22.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.24 MB/sec warmup 109 sec latency 6235.557 ms 2024-09-19T06:45:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:22 smithi096 ceph-mon[35004]: pgmap v111: 97 pgs: 97 active+clean; 39 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.5 MiB/s wr, 369 op/s 2024-09-19T06:45:22.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:22 smithi037 ceph-mon[31211]: pgmap v111: 97 pgs: 97 active+clean; 39 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.5 MiB/s wr, 369 op/s 2024-09-19T06:45:22.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:22 smithi151 ceph-mon[35236]: pgmap v111: 97 pgs: 97 active+clean; 39 MiB data, 682 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 3.5 MiB/s wr, 369 op/s 2024-09-19T06:45:23.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.14 MB/sec warmup 110 sec latency 7235.664 ms 2024-09-19T06:45:24.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 11.04 MB/sec warmup 111 sec latency 8235.751 ms 2024-09-19T06:45:24.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:24 smithi096 ceph-mon[35004]: pgmap v112: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.2 MiB/s wr, 406 op/s 2024-09-19T06:45:24.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:24 smithi037 ceph-mon[31211]: pgmap v112: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.2 MiB/s wr, 406 op/s 2024-09-19T06:45:24.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:24 smithi151 ceph-mon[35236]: pgmap v112: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 MiB/s rd, 4.2 MiB/s wr, 406 op/s 2024-09-19T06:45:25.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.94 MB/sec warmup 112 sec latency 9235.815 ms 2024-09-19T06:45:26.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.85 MB/sec warmup 113 sec latency 10235.908 ms 2024-09-19T06:45:26.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:26 smithi096 ceph-mon[35004]: pgmap v113: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2024-09-19T06:45:26.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:26 smithi037 ceph-mon[31211]: pgmap v113: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2024-09-19T06:45:26.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:26 smithi151 ceph-mon[35236]: pgmap v113: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2024-09-19T06:45:27.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.75 MB/sec warmup 114 sec latency 11235.957 ms 2024-09-19T06:45:28.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.66 MB/sec warmup 115 sec latency 12236.050 ms 2024-09-19T06:45:28.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:28 smithi096 ceph-mon[35004]: pgmap v114: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2024-09-19T06:45:28.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:28 smithi037 ceph-mon[31211]: pgmap v114: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2024-09-19T06:45:28.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:28 smithi151 ceph-mon[35236]: pgmap v114: 97 pgs: 97 active+clean; 37 MiB data, 688 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.3 MiB/s wr, 261 op/s 2024-09-19T06:45:29.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.57 MB/sec warmup 116 sec latency 13236.131 ms 2024-09-19T06:45:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.327+0000 7f7573f5f640 1 -- 172.21.15.37:0/1794365935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c0745c0 msgr2=0x7f756c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.327+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1794365935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c0745c0 0x7f756c0749a0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f755c009900 tx=0x7f755c02a480 comp rx=0 tx=0).stop 2024-09-19T06:45:29.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.327+0000 7f7573f5f640 1 -- 172.21.15.37:0/1794365935 shutdown_connections 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.327+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1794365935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c10e380 0x7f756c110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.327+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1794365935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c074ee0 0x7f756c10de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.327+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1794365935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c0745c0 0x7f756c0749a0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 -- 172.21.15.37:0/1794365935 >> 172.21.15.37:0/1794365935 conn(0x7f756c0fd7e0 msgr2=0x7f756c0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 -- 172.21.15.37:0/1794365935 shutdown_connections 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 -- 172.21.15.37:0/1794365935 wait complete. 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 Processor -- start 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 -- start start 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c0745c0 0x7f756c19a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c074ee0 0x7f756c19ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c10e380 0x7f756c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f756c114150 con 0x7f756c10e380 2024-09-19T06:45:29.332 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f756c113fd0 con 0x7f756c074ee0 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.328+0000 7f7573f5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1142d0 con 0x7f756c0745c0 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c10e380 0x7f756c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c10e380 0x7f756c19eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35530/0 (socket says 172.21.15.37:35530) 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 -- 172.21.15.37:0/1874737210 learned_addr learned my addr 172.21.15.37:0/1874737210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75714fa640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c074ee0 0x7f756c19ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f7571cfb640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c0745c0 0x7f756c19a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 -- 172.21.15.37:0/1874737210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c0745c0 msgr2=0x7f756c19a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c0745c0 0x7f756c19a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 -- 172.21.15.37:0/1874737210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c074ee0 msgr2=0x7f756c19ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c074ee0 0x7f756c19ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 -- 172.21.15.37:0/1874737210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f756c19f610 con 0x7f756c10e380 2024-09-19T06:45:29.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75714fa640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c074ee0 0x7f756c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f75724fc640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c10e380 0x7f756c19eee0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f756400cce0 tx=0x7f756400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f7562ffd640 1 -- 172.21.15.37:0/1874737210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756400f450 con 0x7f756c10e380 2024-09-19T06:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f7562ffd640 1 -- 172.21.15.37:0/1874737210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756400fad0 con 0x7f756c10e380 2024-09-19T06:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f756c19f900 con 0x7f756c10e380 2024-09-19T06:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f7562ffd640 1 -- 172.21.15.37:0/1874737210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7564018bd0 con 0x7f756c10e380 2024-09-19T06:45:29.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.329+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f756c1a7190 con 0x7f756c10e380 2024-09-19T06:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7562ffd640 1 -- 172.21.15.37:0/1874737210 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f756400fc80 con 0x7f756c10e380 2024-09-19T06:45:29.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 Processor -- start 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 -- start start 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f28af0 0x55ded0f28ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f29410 0x55ded0f33770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f360a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ded0e8d2f0 con 0x55ded0f29410 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ded0e35270 con 0x55ded0f28af0 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.332+0000 7f7576dd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ded0e892b0 con 0x55ded0f33cb0 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f29410 0x55ded0f33770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f29410 0x55ded0f33770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35540/0 (socket says 172.21.15.37:35540) 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 -- 172.21.15.37:0/1197425120 learned_addr learned my addr 172.21.15.37:0/1197425120 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f75714fa640 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f28af0 0x55ded0f28ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7562ffd640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75580a3460 0x7f75580a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f75714fa640 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f360a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f75724fc640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75580a3460 0x7f75580a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 -- 172.21.15.37:0/1197425120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 msgr2=0x55ded0f360a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f360a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 -- 172.21.15.37:0/1197425120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f28af0 msgr2=0x55ded0f28ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f28af0 0x55ded0f28ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 -- 172.21.15.37:0/1197425120 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded0e34340 con 0x55ded0f29410 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f75714fa640 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7562ffd640 1 -- 172.21.15.37:0/1874737210 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f75640d0350 con 0x7f756c10e380 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7571cfb640 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f29410 0x55ded0f33770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f755c009900 tx=0x7f755c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d16ddfe9c35adef4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7557fff640 1 -- 172.21.15.37:0/1197425120 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c035030 con 0x55ded0f29410 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f75724fc640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75580a3460 0x7f75580a5920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f75640cdb50 tx=0x7f7564091ac0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7557fff640 1 -- 172.21.15.37:0/1197425120 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f755c0044b0 con 0x55ded0f29410 2024-09-19T06:45:29.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.333+0000 7f7557fff640 1 -- 172.21.15.37:0/1197425120 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755c003850 con 0x55ded0f29410 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 -- 172.21.15.37:0/1197425120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f29410 msgr2=0x55ded0f33770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f29410 0x55ded0f33770 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f755c009900 tx=0x7f755c0040c0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 -- 172.21.15.37:0/1197425120 shutdown_connections 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f360a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f29410 0x55ded0f33770 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1197425120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f28af0 0x55ded0f28ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 -- 172.21.15.37:0/1197425120 >> 172.21.15.37:0/1197425120 conn(0x55ded0f25060 msgr2=0x55ded0f25440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 -- 172.21.15.37:0/1197425120 shutdown_connections 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.334+0000 7f7576dd5740 1 -- 172.21.15.37:0/1197425120 wait complete. 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 -- start start 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 Processor -- start 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f28af0 0x55ded0f30bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f31630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ded0f36880 con 0x55ded0f28af0 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ded0e16290 con 0x55ded0f29410 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ded0f36a00 con 0x55ded0f33cb0 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7576dd5740 10 client.0 _ll_register_callbacks cb 0x55ded0e47140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f75714fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f28af0 0x55ded0f30bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f75714fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f28af0 0x55ded0f30bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35546/0 (socket says 172.21.15.37:35546) 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f75714fa640 1 -- 172.21.15.37:0/146875850 learned_addr learned my addr 172.21.15.37:0/146875850 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.335+0000 7f7571cfb640 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75714fa640 1 -- 172.21.15.37:0/146875850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 msgr2=0x55ded0f31630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75714fa640 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f31630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75714fa640 1 -- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 msgr2=0x55ded0f310f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75714fa640 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75714fa640 1 -- 172.21.15.37:0/146875850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded0ba0210 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75714fa640 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f28af0 0x55ded0f30bb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f75680039e0 tx=0x7f7568003ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439618e5d821eec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75557fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75557fa640 1 -- 172.21.15.37:0/146875850 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568022060 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75557fa640 1 -- 172.21.15.37:0/146875850 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7568013cf0 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f755c0044b0 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75557fa640 1 -- 172.21.15.37:0/146875850 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f75680130d0 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 10 client.16458 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ded0c6c620 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75557fa640 1 -- 172.21.15.37:0/146875850 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f755c0044b0 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f75557fa640 1 -- 172.21.15.37:0/146875850 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f75680253b0 con 0x55ded0f28af0 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 10 client.16458 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 20 client.16458 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:29.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0f43c80 0x55ded0f46070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 4 client.16458 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.336+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ded0e9aa20 con 0x55ded0f43c80 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.337+0000 7f7571cfb640 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0f43c80 0x55ded0f46070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.337+0000 7f7571cfb640 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0f43c80 0x55ded0f46070 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fffdb8c9d4eaa11c server_cookie=2d3feb3b49d9e1c4 in_seq=0 out_seq=0 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.337+0000 7f75557fa640 10 client.16458 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f75557fa640 1 -- 172.21.15.37:0/146875850 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ded0e9aa20 con 0x55ded0f43c80 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f75557fa640 10 client.16458 handle_command_reply: tid=0 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 client.16458 shutdown 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f28af0 msgr2=0x55ded0f30bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f28af0 0x55ded0f30bb0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f75680039e0 tx=0x7f7568003ea0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 shutdown_connections 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0f43c80 0x55ded0f46070 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f33cb0 0x55ded0f31630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f75557fa640 0 client.16458 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 --2- 172.21.15.37:0/146875850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f28af0 0x55ded0f30bb0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 >> 172.21.15.37:0/146875850 conn(0x55ded0f25060 msgr2=0x55ded0f2adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 shutdown_connections 2024-09-19T06:45:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 1 -- 172.21.15.37:0/146875850 wait complete. 2024-09-19T06:45:29.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.338+0000 7f7576dd5740 20 client.16458 trim_cache size 0 max 16384 2024-09-19T06:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.352+0000 7f7576dd5740 1 Processor -- start 2024-09-19T06:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7576dd5740 1 -- start start 2024-09-19T06:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 0x55ded0f2b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f31620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7576dd5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1b2330 con 0x55ded0f33cb0 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7576dd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1afba0 con 0x55ded0f29410 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7576dd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ad9f0 con 0x55ded0f28af0 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f31620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f31620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35552/0 (socket says 172.21.15.37:35552) 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 -- 172.21.15.37:0/1396424501 learned_addr learned my addr 172.21.15.37:0/1396424501 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f7571cfb640 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 -- 172.21.15.37:0/1396424501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 msgr2=0x55ded0f2b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 0x55ded0f2b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 -- 172.21.15.37:0/1396424501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 msgr2=0x55ded0f310e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 -- 172.21.15.37:0/1396424501 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded0f202d0 con 0x55ded0f33cb0 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75714fa640 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f31620 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f75680039b0 tx=0x7f75680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c83415aa8643d45 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:29.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75577fe640 1 -- 172.21.15.37:0/1396424501 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568000e60 con 0x55ded0f33cb0 2024-09-19T06:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75577fe640 1 -- 172.21.15.37:0/1396424501 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7568013270 con 0x55ded0f33cb0 2024-09-19T06:45:29.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.353+0000 7f75577fe640 1 -- 172.21.15.37:0/1396424501 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568025c20 con 0x55ded0f33cb0 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 -- 172.21.15.37:0/1396424501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 msgr2=0x55ded0f31620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f31620 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f75680039b0 tx=0x7f75680046c0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 -- 172.21.15.37:0/1396424501 shutdown_connections 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f31620 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f310e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 --2- 172.21.15.37:0/1396424501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 0x55ded0f2b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 -- 172.21.15.37:0/1396424501 >> 172.21.15.37:0/1396424501 conn(0x55ded0f25060 msgr2=0x55ded0f32c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 -- 172.21.15.37:0/1396424501 shutdown_connections 2024-09-19T06:45:29.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 -- 172.21.15.37:0/1396424501 wait complete. 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 -- start start 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.354+0000 7f7576dd5740 1 Processor -- start 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 0x55ded0f3bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f3c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7576dd5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f2d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7576dd5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ae060 con 0x55ded0f33cb0 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7576dd5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1af4d0 con 0x55ded0f29410 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7576dd5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f756c1ae730 con 0x55ded0f28af0 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7576dd5740 10 client.0 _ll_register_callbacks cb 0x7f7568004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:29.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f2d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f2d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35562/0 (socket says 172.21.15.37:35562) 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 -- 172.21.15.37:0/2574179207 learned_addr learned my addr 172.21.15.37:0/2574179207 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 0x55ded0f3bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f7571cfb640 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f3c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 -- 172.21.15.37:0/2574179207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 msgr2=0x55ded0f3bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 0x55ded0f3bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 -- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 msgr2=0x55ded0f3c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f3c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 -- 172.21.15.37:0/2574179207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded0f2dd80 con 0x55ded0f33cb0 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75714fa640 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f2d580 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7568009f70 tx=0x7f7568011cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8af8505b3d1505e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.355+0000 7f75557fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f75557fa640 1 -- 172.21.15.37:0/2574179207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7568025bd0 con 0x55ded0f33cb0 2024-09-19T06:45:29.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f75557fa640 1 -- 172.21.15.37:0/2574179207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f75680114d0 con 0x55ded0f33cb0 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7568013270 con 0x55ded0f33cb0 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f75557fa640 1 -- 172.21.15.37:0/2574179207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f75680116d0 con 0x55ded0f33cb0 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 10 client.16464 fetch_fsmap learned FSMap version 29 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ded0fae4d0 con 0x55ded0f33cb0 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f75557fa640 1 -- 172.21.15.37:0/2574179207 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7568013270 con 0x55ded0f33cb0 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f75557fa640 1 -- 172.21.15.37:0/2574179207 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f756803e4d0 con 0x55ded0f33cb0 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 10 client.16464 fetch_fsmap finished waiting for FSMap version 29 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:29.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 20 client.16464 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0faea30 0x55ded0fb0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 4 client.16464 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:45:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.356+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ded0e94d70 con 0x55ded0faea30 2024-09-19T06:45:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.357+0000 7f7571cfb640 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0faea30 0x55ded0fb0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.357+0000 7f7571cfb640 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0faea30 0x55ded0fb0e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c834af64d51e8c8 server_cookie=fa34e0bfe24f0a49 in_seq=0 out_seq=0 2024-09-19T06:45:29.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.357+0000 7f75557fa640 10 client.16464 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:29.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f75557fa640 1 -- 172.21.15.37:0/2574179207 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ded0e94d70 con 0x55ded0faea30 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f75557fa640 10 client.16464 handle_command_reply: tid=0 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 client.16464 shutdown 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 msgr2=0x55ded0f2d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f2d580 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7568009f70 tx=0x7f7568011cc0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 shutdown_connections 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ded0faea30 0x55ded0fb0e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ded0f33cb0 0x55ded0f2d580 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ded0f29410 0x55ded0f3c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f75557fa640 0 client.16464 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 --2- 172.21.15.37:0/2574179207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ded0f28af0 0x55ded0f3bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.424+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 >> 172.21.15.37:0/2574179207 conn(0x55ded0f25060 msgr2=0x55ded0f239a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 shutdown_connections 2024-09-19T06:45:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7576dd5740 1 -- 172.21.15.37:0/2574179207 wait complete. 2024-09-19T06:45:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7576dd5740 20 client.16464 trim_cache size 0 max 16384 2024-09-19T06:45:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:45:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:45:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:45:29.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:45:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75580a3460 msgr2=0x7f75580a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75580a3460 0x7f75580a5920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f75640cdb50 tx=0x7f7564091ac0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c10e380 msgr2=0x7f756c19eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c10e380 0x7f756c19eee0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f756400cce0 tx=0x7f756400a620 comp rx=0 tx=0).stop 2024-09-19T06:45:29.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 shutdown_connections 2024-09-19T06:45:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75580a3460 0x7f75580a5920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f756c10e380 0x7f756c19eee0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f756c074ee0 0x7f756c19ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 --2- 172.21.15.37:0/1874737210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f756c0745c0 0x7f756c19a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 >> 172.21.15.37:0/1874737210 conn(0x7f756c0fd7e0 msgr2=0x7f756c0fe440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 shutdown_connections 2024-09-19T06:45:29.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:29.425+0000 7f7573f5f640 1 -- 172.21.15.37:0/1874737210 wait complete. 2024-09-19T06:45:29.820 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:45:30.100 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:30.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.48 MB/sec warmup 117 sec latency 14236.243 ms 2024-09-19T06:45:30.521 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:30 smithi037 ceph-mon[31211]: pgmap v115: 97 pgs: 97 active+clean; 36 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-09-19T06:45:30.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 -- 172.21.15.37:0/784163224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 msgr2=0x7f0d4810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/784163224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d4810ea90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009ca0 tx=0x7f0d4002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 -- 172.21.15.37:0/784163224 shutdown_connections 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/784163224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d4810ea90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/784163224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d48102790 0x7f0d481067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/784163224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d48101de0 0x7f0d481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 -- 172.21.15.37:0/784163224 >> 172.21.15.37:0/784163224 conn(0x7f0d480fd8c0 msgr2=0x7f0d480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 -- 172.21.15.37:0/784163224 shutdown_connections 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.519+0000 7f0d4e87c640 1 -- 172.21.15.37:0/784163224 wait complete. 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 Processor -- start 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 -- start start 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d48101de0 0x7f0d480744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:30.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d48102790 0x7f0d48074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d48074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111f90 con 0x7f0d48106f10 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48111e10 con 0x7f0d48102790 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4e87c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d48112110 con 0x7f0d48101de0 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4ce19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d48074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4ce19640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d48074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43274/0 (socket says 172.21.15.37:43274) 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d3ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d48102790 0x7f0d48074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d4ce19640 1 -- 172.21.15.37:0/3146419092 learned_addr learned my addr 172.21.15.37:0/3146419092 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.520+0000 7f0d47fff640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d48101de0 0x7f0d480744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d3ffff640 1 -- 172.21.15.37:0/3146419092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d48101de0 msgr2=0x7f0d480744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d3ffff640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d48101de0 0x7f0d480744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d3ffff640 1 -- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 msgr2=0x7f0d48074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:30.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d3ffff640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d48074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d3ffff640 1 -- 172.21.15.37:0/3146419092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d481a6930 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d4ce19640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d48074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d3ffff640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d48102790 0x7f0d48074a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000d6d0 tx=0x7f0d3000dad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d45ffb640 1 -- 172.21.15.37:0/3146419092 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30017070 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d45ffb640 1 -- 172.21.15.37:0/3146419092 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d300047a0 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d481a6c20 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d45ffb640 1 -- 172.21.15.37:0/3146419092 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30009c80 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.521+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d481a71b0 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.522+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d481021c0 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.523+0000 7f0d45ffb640 1 -- 172.21.15.37:0/3146419092 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0d30004950 con 0x7f0d48102790 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.524+0000 7f0d45ffb640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d340a3430 0x7f0d340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.524+0000 7f0d47fff640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d340a3430 0x7f0d340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:30.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.524+0000 7f0d45ffb640 1 -- 172.21.15.37:0/3146419092 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0d300cf470 con 0x7f0d48102790 2024-09-19T06:45:30.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.524+0000 7f0d47fff640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d340a3430 0x7f0d340a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009990 tx=0x7f0d38009110 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:30.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.527+0000 7f0d45ffb640 1 -- 172.21.15.37:0/3146419092 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0d30009e20 con 0x7f0d48102790 2024-09-19T06:45:30.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.739+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0d48075a20 con 0x7f0d48102790 2024-09-19T06:45:30.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.740+0000 7f0d45ffb640 1 -- 172.21.15.37:0/3146419092 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11906 (secure 0 0 0) 0x7f0d48075a20 con 0x7f0d48102790 2024-09-19T06:45:30.741 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:45:30.742 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":29,"btime":"2024-09-19T06:44:24:345285+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:44:24.345273+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:45:30.742 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 29 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d340a3430 msgr2=0x7f0d340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d340a3430 0x7f0d340a58f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0d38009990 tx=0x7f0d38009110 comp rx=0 tx=0).stop 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d48102790 msgr2=0x7f0d48074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d48102790 0x7f0d48074a30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0d3000d6d0 tx=0x7f0d3000dad0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 shutdown_connections 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d340a3430 0x7f0d340a58f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d48106f10 0x7f0d48074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d48102790 0x7f0d48074a30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 --2- 172.21.15.37:0/3146419092 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d48101de0 0x7f0d480744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:30.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 >> 172.21.15.37:0/3146419092 conn(0x7f0d480fd8c0 msgr2=0x7f0d480fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:30.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 shutdown_connections 2024-09-19T06:45:30.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:30.742+0000 7f0d4e87c640 1 -- 172.21.15.37:0/3146419092 wait complete. 2024-09-19T06:45:30.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:30 smithi096 ceph-mon[35004]: pgmap v115: 97 pgs: 97 active+clean; 36 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-09-19T06:45:30.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:30 smithi151 ceph-mon[35236]: pgmap v115: 97 pgs: 97 active+clean; 36 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.3 MiB/s wr, 294 op/s 2024-09-19T06:45:31.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.39 MB/sec warmup 118 sec latency 15236.308 ms 2024-09-19T06:45:31.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:31 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3146419092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:31.688 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id b4359ca8 --release --await' 2024-09-19T06:45:31.688 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id b4359ca8 --release --await 2024-09-19T06:45:31.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:31 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3146419092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:31.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:31 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3146419092' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:31.949 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:32.196 INFO:tasks.workunit.client.0.smithi037.stdout: 1 223877 10.30 MB/sec warmup 119 sec latency 16236.360 ms 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.478+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353698081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd8102d40 msgr2=0x7ffbd8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.478+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353698081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd8102d40 0x7ffbd8103120 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc802a480 comp rx=0 tx=0).stop 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353698081 shutdown_connections 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353698081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd810c170 0x7ffbd810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353698081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd8103660 0x7ffbd810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353698081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd8102d40 0x7ffbd8103120 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353698081 >> 172.21.15.37:0/353698081 conn(0x7ffbd80fc7a0 msgr2=0x7ffbd80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353698081 shutdown_connections 2024-09-19T06:45:32.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353698081 wait complete. 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 Processor -- start 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.479+0000 7ffbdd74f640 1 -- start start 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbdd74f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8102d40 0x7ffbd81984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbdd74f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd8103660 0x7ffbd8198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbdd74f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd810c170 0x7ffbd819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbdd74f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8111fb0 con 0x7ffbd810c170 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbdd74f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8111e30 con 0x7ffbd8102d40 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbdd74f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd8112130 con 0x7ffbd8103660 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd810c170 0x7ffbd819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd810c170 0x7ffbd819cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43292/0 (socket says 172.21.15.37:43292) 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 -- 172.21.15.37:0/353331298 learned_addr learned my addr 172.21.15.37:0/353331298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:32.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8102d40 0x7ffbd81984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd8103660 0x7ffbd8198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 -- 172.21.15.37:0/353331298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd8103660 msgr2=0x7ffbd8198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd8103660 0x7ffbd8198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 -- 172.21.15.37:0/353331298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8102d40 msgr2=0x7ffbd81984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8102d40 0x7ffbd81984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 -- 172.21.15.37:0/353331298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd819d470 con 0x7ffbd810c170 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.480+0000 7ffbd77fe640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd810c170 0x7ffbd819cd90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c970 tx=0x7ffbd000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.481+0000 7ffbbffff640 1 -- 172.21.15.37:0/353331298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0007bb0 con 0x7ffbd810c170 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.481+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd819d760 con 0x7ffbd810c170 2024-09-19T06:45:32.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.481+0000 7ffbbffff640 1 -- 172.21.15.37:0/353331298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbd0007d50 con 0x7ffbd810c170 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.481+0000 7ffbbffff640 1 -- 172.21.15.37:0/353331298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbd0018800 con 0x7ffbd810c170 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.481+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd81a5040 con 0x7ffbd810c170 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.483+0000 7ffbbffff640 1 -- 172.21.15.37:0/353331298 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffbd000fa50 con 0x7ffbd810c170 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.483+0000 7ffbdfb28740 1 Processor -- start 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.483+0000 7ffbdfb28740 1 -- start start 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44470 0x55fc13f44850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44d90 0x55fc13f4f110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f4f650 0x55fc13f51a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fc13ea5620 con 0x55fc13f44d90 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc13e4ce50 con 0x55fc13f44470 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc13ea15e0 con 0x55fc13f4f650 2024-09-19T06:45:32.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44d90 0x55fc13f4f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbbffff640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbc00a33b0 0x7ffbc00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44d90 0x55fc13f4f110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43298/0 (socket says 172.21.15.37:43298) 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 -- 172.21.15.37:0/2855203615 learned_addr learned my addr 172.21.15.37:0/2855203615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd77fe640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbc00a33b0 0x7ffbc00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44470 0x55fc13f44850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbbffff640 1 -- 172.21.15.37:0/353331298 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ffbd00cfde0 con 0x7ffbd810c170 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 -- 172.21.15.37:0/2855203615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f4f650 msgr2=0x55fc13f51a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f4f650 0x55fc13f51a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 -- 172.21.15.37:0/2855203615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44470 msgr2=0x55fc13f44850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44470 0x55fc13f44850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 -- 172.21.15.37:0/2855203615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc13e3dd40 con 0x55fc13f44d90 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44470 0x55fc13f44850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.484+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44d90 0x55fc13f4f110 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc80316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bc076a7225b780b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbd77fe640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbc00a33b0 0x7ffbc00a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c940 tx=0x7ffbd006dde0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbbd7fa640 1 -- 172.21.15.37:0/2855203615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8040070 con 0x55fc13f44d90 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbbd7fa640 1 -- 172.21.15.37:0/2855203615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8005020 con 0x55fc13f44d90 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbbd7fa640 1 -- 172.21.15.37:0/2855203615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8035040 con 0x55fc13f44d90 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 -- 172.21.15.37:0/2855203615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44d90 msgr2=0x55fc13f4f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44d90 0x55fc13f4f110 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8009900 tx=0x7ffbc80316e0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 -- 172.21.15.37:0/2855203615 shutdown_connections 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f4f650 0x55fc13f51a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44d90 0x55fc13f4f110 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/2855203615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44470 0x55fc13f44850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 -- 172.21.15.37:0/2855203615 >> 172.21.15.37:0/2855203615 conn(0x55fc13f40a00 msgr2=0x55fc13f40de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 -- 172.21.15.37:0/2855203615 shutdown_connections 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.485+0000 7ffbdfb28740 1 -- 172.21.15.37:0/2855203615 wait complete. 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 -- start start 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 Processor -- start 2024-09-19T06:45:32.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44470 0x55fc13f4c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44d90 0x55fc13f4ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f4f650 0x55fc13f4cf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fc13f52220 con 0x55fc13f4f650 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc13e2df10 con 0x55fc13f44d90 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fc13f523a0 con 0x55fc13f44470 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbdfb28740 10 client.0 _ll_register_callbacks cb 0x55fc13e51db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbd67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f4f650 0x55fc13f4cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbd67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f4f650 0x55fc13f4cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43312/0 (socket says 172.21.15.37:43312) 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbd6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44d90 0x55fc13f4ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1319600994 learned_addr learned my addr 172.21.15.37:0/1319600994 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.486+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44470 0x55fc13f4c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1319600994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44470 msgr2=0x55fc13f4c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44470 0x55fc13f4c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1319600994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44d90 msgr2=0x55fc13f4ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44d90 0x55fc13f4ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1319600994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc13bb4580 con 0x55fc13f4f650 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f4f650 0x55fc13f4cf60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00afa0 tx=0x7ffbcc00c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3efce2d99d92e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbbaff5640 1 -- 172.21.15.37:0/1319600994 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01f060 con 0x55fc13f4f650 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbbaff5640 1 -- 172.21.15.37:0/1319600994 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ffbcc01a450 con 0x55fc13f4f650 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbc8005020 con 0x55fc13f4f650 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbbaff5640 1 -- 172.21.15.37:0/1319600994 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbcc01a650 con 0x55fc13f4f650 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbdfb28740 10 client.16482 fetch_fsmap learned FSMap version 30 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc13c7e960 con 0x55fc13f4f650 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.487+0000 7ffbbaff5640 1 -- 172.21.15.37:0/1319600994 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc8005020 con 0x55fc13f4f650 2024-09-19T06:45:32.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbbaff5640 1 -- 172.21.15.37:0/1319600994 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ffbcc0227b0 con 0x55fc13f4f650 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbdfb28740 10 client.16482 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbdfb28740 10 client.16482 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbdfb28740 20 client.16482 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13f5f120 0x55fc13f61510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbdfb28740 4 client.16482 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc13eb2b90 con 0x55fc13f5f120 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13f5f120 0x55fc13f61510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13f5f120 0x55fc13f61510 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3cf9ea9183470467 server_cookie=98146b72e8e54fc8 in_seq=0 out_seq=0 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.488+0000 7ffbbaff5640 10 client.16482 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:32.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbbaff5640 1 -- 172.21.15.37:0/1319600994 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fc13eb2b90 con 0x55fc13f5f120 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbbaff5640 10 client.16482 handle_command_reply: tid=0 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 client.16482 shutdown 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f4f650 msgr2=0x55fc13f4cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f4f650 0x55fc13f4cf60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00afa0 tx=0x7ffbcc00c040 comp rx=0 tx=0).stop 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 shutdown_connections 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13f5f120 0x55fc13f61510 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f4f650 0x55fc13f4cf60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f44d90 0x55fc13f4ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbbaff5640 0 client.16482 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:32.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1319600994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44470 0x55fc13f4c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 >> 172.21.15.37:0/1319600994 conn(0x55fc13f40a00 msgr2=0x55fc13f3b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 shutdown_connections 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.489+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1319600994 wait complete. 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.490+0000 7ffbdfb28740 20 client.16482 trim_cache size 0 max 16384 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.506+0000 7ffbdfb28740 1 Processor -- start 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.506+0000 7ffbdfb28740 1 -- start start 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f477f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f3b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 0x55fc13f66db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81adfc0 con 0x55fc13f44470 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81ab860 con 0x55fc13f4f650 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81a96b0 con 0x55fc13f44d90 2024-09-19T06:45:32.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f477f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f477f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43320/0 (socket says 172.21.15.37:43320) 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1421165165 learned_addr learned my addr 172.21.15.37:0/1421165165 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1421165165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 msgr2=0x55fc13f3b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/1421165165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f3b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1421165165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 msgr2=0x55fc13f66db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/1421165165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 0x55fc13f66db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 -- 172.21.15.37:0/1421165165 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc13f4e420 con 0x55fc13f44470 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/1421165165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f477f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc004b90 tx=0x7ffbcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9702d7569b05e8f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbbcff9640 1 -- 172.21.15.37:0/1421165165 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc022da0 con 0x55fc13f44470 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbbcff9640 1 -- 172.21.15.37:0/1421165165 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc030070 con 0x55fc13f44470 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.507+0000 7ffbbcff9640 1 -- 172.21.15.37:0/1421165165 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01ab20 con 0x55fc13f44470 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1421165165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 msgr2=0x55fc13f477f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1421165165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f477f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc004b90 tx=0x7ffbcc004a40 comp rx=0 tx=0).stop 2024-09-19T06:45:32.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1421165165 shutdown_connections 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1421165165 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 0x55fc13f66db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1421165165 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f3b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/1421165165 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f477f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1421165165 >> 172.21.15.37:0/1421165165 conn(0x55fc13f40a00 msgr2=0x55fc13f3af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1421165165 shutdown_connections 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 -- 172.21.15.37:0/1421165165 wait complete. 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 -- start start 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.508+0000 7ffbdfb28740 1 Processor -- start 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f575f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f57b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 0x55fc13f545c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81a9d20 con 0x55fc13f44470 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81ab190 con 0x55fc13f4f650 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd81aa3f0 con 0x55fc13f44d90 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 10 client.0 _ll_register_callbacks cb 0x7ffbcc00abe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43336/0 (socket says 172.21.15.37:43336) 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 -- 172.21.15.37:0/3991207079 learned_addr learned my addr 172.21.15.37:0/3991207079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f57b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 -- 172.21.15.37:0/3991207079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 msgr2=0x55fc13f57b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f57b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 -- 172.21.15.37:0/3991207079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 msgr2=0x55fc13f545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 0x55fc13f545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 -- 172.21.15.37:0/3991207079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc13f44bc0 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f57b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbd67fc640 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f575f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009d90 tx=0x7ffbcc00a490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac797a3804e42b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbbaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbbaff5640 1 -- 172.21.15.37:0/3991207079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01a450 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbbaff5640 1 -- 172.21.15.37:0/3991207079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ffbcc015d90 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.509+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc030070 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbbaff5640 1 -- 172.21.15.37:0/3991207079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7ffbcc01aaa0 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 10 client.16488 fetch_fsmap learned FSMap version 30 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55fc13fca3d0 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbbaff5640 1 -- 172.21.15.37:0/3991207079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc030070 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbbaff5640 1 -- 172.21.15.37:0/3991207079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7ffbcc036460 con 0x55fc13f44470 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 10 client.16488 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 10 client.16488 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:45:32.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 20 client.16488 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13fcaa60 0x55fc13fcce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 4 client.16488 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "b4359ca8", "release": true, "await": true}] 2024-09-19T06:45:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "b4359ca8", "release": true, "await": true}) -- 0x55fc13ead0a0 con 0x55fc13fcaa60 2024-09-19T06:45:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.510+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13fcaa60 0x55fc13fcce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.511+0000 7ffbbaff5640 10 client.16488 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:32.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.511+0000 7ffbd6ffd640 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13fcaa60 0x55fc13fcce80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=663eb3be241be00b server_cookie=9621c0962b51148c in_seq=0 out_seq=0 2024-09-19T06:45:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbbaff5640 1 -- 172.21.15.37:0/3991207079 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55fc13ead0a0 con 0x55fc13fcaa60 2024-09-19T06:45:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbbaff5640 10 client.16488 handle_command_reply: tid=0 2024-09-19T06:45:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 client.16488 shutdown 2024-09-19T06:45:32.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 msgr2=0x55fc13f575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f575f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009d90 tx=0x7ffbcc00a490 comp rx=0 tx=0).stop 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 shutdown_connections 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fc13fcaa60 0x55fc13fcce80 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc13f4f650 0x55fc13f545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fc13f44d90 0x55fc13f57b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 --2- 172.21.15.37:0/3991207079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fc13f44470 0x55fc13f575f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 >> 172.21.15.37:0/3991207079 conn(0x55fc13f40a00 msgr2=0x55fc13f3b750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbbaff5640 0 client.16488 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 shutdown_connections 2024-09-19T06:45:32.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 1 -- 172.21.15.37:0/3991207079 wait complete. 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.524+0000 7ffbdfb28740 20 client.16488 trim_cache size 0 max 16384 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 30, 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 16, 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "b4359ca8": { 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 16, 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:45:32.534 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 136.6, 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:32.535 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:32.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:32.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:45:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbc00a33b0 msgr2=0x7ffbc00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbc00a33b0 0x7ffbc00a5870 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c940 tx=0x7ffbd006dde0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd810c170 msgr2=0x7ffbd819cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:32.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd810c170 0x7ffbd819cd90 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ffbd000c970 tx=0x7ffbd000ce30 comp rx=0 tx=0).stop 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 shutdown_connections 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbc00a33b0 0x7ffbc00a5870 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd810c170 0x7ffbd819cd90 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd8103660 0x7ffbd8198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 --2- 172.21.15.37:0/353331298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd8102d40 0x7ffbd81984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 >> 172.21.15.37:0/353331298 conn(0x7ffbd80fc7a0 msgr2=0x7ffbd80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 shutdown_connections 2024-09-19T06:45:32.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:32.525+0000 7ffbdd74f640 1 -- 172.21.15.37:0/353331298 wait complete. 2024-09-19T06:45:32.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:32 smithi096 ceph-mon[35004]: pgmap v116: 97 pgs: 97 active+clean; 36 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.7 MiB/s wr, 70 op/s 2024-09-19T06:45:32.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:32 smithi037 ceph-mon[31211]: pgmap v116: 97 pgs: 97 active+clean; 36 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.7 MiB/s wr, 70 op/s 2024-09-19T06:45:32.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:32 smithi151 ceph-mon[35236]: pgmap v116: 97 pgs: 97 active+clean; 36 MiB data, 687 MiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 1.7 MiB/s wr, 70 op/s 2024-09-19T06:45:33.358 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'b4359ca8', total seconds elapsed: 20.1 2024-09-19T06:45:33.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:33 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:45:33.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:33 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:45:33.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:33 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:45:34.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 227753 13.96 MB/sec execute 1 sec latency 16568.426 ms 2024-09-19T06:45:34.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:34 smithi096 ceph-mon[35004]: pgmap v117: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.8 MiB/s wr, 132 op/s 2024-09-19T06:45:34.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:34 smithi037 ceph-mon[31211]: pgmap v117: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.8 MiB/s wr, 132 op/s 2024-09-19T06:45:34.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:34 smithi151 ceph-mon[35236]: pgmap v117: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 2.8 MiB/s wr, 132 op/s 2024-09-19T06:45:35.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 230841 16.27 MB/sec execute 2 sec latency 34.212 ms 2024-09-19T06:45:36.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 233214 13.64 MB/sec execute 3 sec latency 13.092 ms 2024-09-19T06:45:36.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:36 smithi096 ceph-mon[35004]: pgmap v118: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:45:36.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:36 smithi037 ceph-mon[31211]: pgmap v118: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:45:36.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:36 smithi151 ceph-mon[35236]: pgmap v118: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:45:36.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:45:37.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 235640 14.27 MB/sec execute 4 sec latency 19.400 ms 2024-09-19T06:45:37.241 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.702+0000 7fb9302bd640 1 -- 172.21.15.37:0/4292920427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928100a60 msgr2=0x7fb928100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.702+0000 7fb9302bd640 1 --2- 172.21.15.37:0/4292920427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928100a60 0x7fb928100ec0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fb918009930 tx=0x7fb91802a420 comp rx=0 tx=0).stop 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.702+0000 7fb9302bd640 1 -- 172.21.15.37:0/4292920427 shutdown_connections 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.702+0000 7fb9302bd640 1 --2- 172.21.15.37:0/4292920427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb928101400 0x7fb92810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.702+0000 7fb9302bd640 1 --2- 172.21.15.37:0/4292920427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928100a60 0x7fb928100ec0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.702+0000 7fb9302bd640 1 --2- 172.21.15.37:0/4292920427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928106a60 0x7fb928106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 -- 172.21.15.37:0/4292920427 >> 172.21.15.37:0/4292920427 conn(0x7fb9280fc780 msgr2=0x7fb9280feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:37.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 -- 172.21.15.37:0/4292920427 shutdown_connections 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 -- 172.21.15.37:0/4292920427 wait complete. 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 Processor -- start 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 -- start start 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100a60 0x7fb928198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb928101400 0x7fb9281989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928106a60 0x7fb92819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb928110e20 con 0x7fb928106a60 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb928110ca0 con 0x7fb928100a60 2024-09-19T06:45:37.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.703+0000 7fb9302bd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb928110fa0 con 0x7fb928101400 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e85a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928106a60 0x7fb92819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e059640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100a60 0x7fb928198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e85a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928106a60 0x7fb92819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43348/0 (socket says 172.21.15.37:43348) 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e85a640 1 -- 172.21.15.37:0/2864122038 learned_addr learned my addr 172.21.15.37:0/2864122038 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92d858640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb928101400 0x7fb9281989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e059640 1 -- 172.21.15.37:0/2864122038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb928101400 msgr2=0x7fb9281989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e059640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb928101400 0x7fb9281989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e059640 1 -- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928106a60 msgr2=0x7fb92819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e059640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928106a60 0x7fb92819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e059640 1 -- 172.21.15.37:0/2864122038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92819d430 con 0x7fb928100a60 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e85a640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928106a60 0x7fb92819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92d858640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb928101400 0x7fb9281989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb92e059640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100a60 0x7fb928198480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb92400e9f0 tx=0x7fb92400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb91f7fe640 1 -- 172.21.15.37:0/2864122038 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92400ccc0 con 0x7fb928100a60 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb91f7fe640 1 -- 172.21.15.37:0/2864122038 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb924010910 con 0x7fb928100a60 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92819d720 con 0x7fb928100a60 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb91f7fe640 1 -- 172.21.15.37:0/2864122038 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb924018960 con 0x7fb928100a60 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.704+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9281a5000 con 0x7fb928100a60 2024-09-19T06:45:37.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb91f7fe640 1 -- 172.21.15.37:0/2864122038 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb924010ac0 con 0x7fb928100a60 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 Processor -- start 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 -- start start 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224360 0x557502224740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224c80 0x557502233550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x557502235e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55750218a2e0 con 0x557502233a90 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557502131ce0 con 0x557502224360 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.707+0000 7fb932696740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5575021862a0 con 0x557502224c80 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb91f7fe640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb9140a33b0 0x7fb9140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92d858640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x557502235e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224360 0x557502224740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224360 0x557502224740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:51442/0 (socket says 172.21.15.37:51442) 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 -- 172.21.15.37:0/3050559842 learned_addr learned my addr 172.21.15.37:0/3050559842 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e059640 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224c80 0x557502233550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb9140a33b0 0x7fb9140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb91f7fe640 1 -- 172.21.15.37:0/2864122038 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb9240d0150 con 0x7fb928100a60 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 -- 172.21.15.37:0/3050559842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224c80 msgr2=0x557502233550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224c80 0x557502233550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 -- 172.21.15.37:0/3050559842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 msgr2=0x557502235e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x557502235e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 -- 172.21.15.37:0/3050559842 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557502116440 con 0x557502224360 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb9140a33b0 0x7fb9140a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e720 tx=0x7fb920018000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.708+0000 7fb92e85a640 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224360 0x557502224740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9200136f0 tx=0x7fb920013bb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57bfa2c0edaf2f65 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb91cdea640 1 -- 172.21.15.37:0/3050559842 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb920025070 con 0x557502224360 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb91cdea640 1 -- 172.21.15.37:0/3050559842 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb920011700 con 0x557502224360 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb91cdea640 1 -- 172.21.15.37:0/3050559842 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92000ce30 con 0x557502224360 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 -- 172.21.15.37:0/3050559842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224360 msgr2=0x557502224740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224360 0x557502224740 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fb9200136f0 tx=0x7fb920013bb0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 -- 172.21.15.37:0/3050559842 shutdown_connections 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x557502235e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224c80 0x557502233550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 --2- 172.21.15.37:0/3050559842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224360 0x557502224740 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 -- 172.21.15.37:0/3050559842 >> 172.21.15.37:0/3050559842 conn(0x557502220a30 msgr2=0x557502220e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 -- 172.21.15.37:0/3050559842 shutdown_connections 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.709+0000 7fb932696740 1 -- 172.21.15.37:0/3050559842 wait complete. 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 -- start start 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 Processor -- start 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502224360 0x557502233400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x5575022306b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502233a90 0x557502230c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557502236600 con 0x557502224360 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557502145050 con 0x557502224c80 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557502236780 con 0x557502233a90 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb92e059640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502224360 0x557502233400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb932696740 10 client.0 _ll_register_callbacks cb 0x557502185160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb92e059640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502224360 0x557502233400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43378/0 (socket says 172.21.15.37:43378) 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb92e059640 1 -- 172.21.15.37:0/2100198448 learned_addr learned my addr 172.21.15.37:0/2100198448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb92d858640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x5575022306b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.710+0000 7fb92e85a640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502233a90 0x557502230c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb92e059640 1 -- 172.21.15.37:0/2100198448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502233a90 msgr2=0x557502230c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb92e059640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502233a90 0x557502230c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb92e059640 1 -- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 msgr2=0x5575022306b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb92e059640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x5575022306b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb92e059640 1 -- 172.21.15.37:0/2100198448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557501efb3c0 con 0x557502224360 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb92d858640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x5575022306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb92e059640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502224360 0x557502233400 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cd8a0 tx=0x7fb924014cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bedb1c974b1481c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb911ffb640 1 -- 172.21.15.37:0/2100198448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9240cdcc0 con 0x557502224360 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb911ffb640 1 -- 172.21.15.37:0/2100198448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb9240d20d0 con 0x557502224360 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb920011700 con 0x557502224360 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb911ffb640 1 -- 172.21.15.37:0/2100198448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb924025170 con 0x557502224360 2024-09-19T06:45:37.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb932696740 10 client.16500 fetch_fsmap learned FSMap version 30 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557502242010 con 0x557502224360 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.711+0000 7fb911ffb640 1 -- 172.21.15.37:0/2100198448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb920011700 con 0x557502224360 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb911ffb640 1 -- 172.21.15.37:0/2100198448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fb9240b5230 con 0x557502224360 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb932696740 10 client.16500 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb932696740 10 client.16500 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb932696740 20 client.16500 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb932696740 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022434d0 0x5575022458e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb932696740 4 client.16500 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557502197850 con 0x5575022434d0 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb92d858640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022434d0 0x5575022458e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb92d858640 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022434d0 0x5575022458e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa782946e3df1694 server_cookie=cf8a79f4da730532 in_seq=0 out_seq=0 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.712+0000 7fb911ffb640 10 client.16500 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb911ffb640 1 -- 172.21.15.37:0/2100198448 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557502197850 con 0x5575022434d0 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb911ffb640 10 client.16500 handle_command_reply: tid=0 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 client.16500 shutdown 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502224360 msgr2=0x557502233400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502224360 0x557502233400 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fb9240cd8a0 tx=0x7fb924014cb0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 shutdown_connections 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022434d0 0x5575022458e0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502233a90 0x557502230c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x5575022306b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 --2- 172.21.15.37:0/2100198448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502224360 0x557502233400 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 >> 172.21.15.37:0/2100198448 conn(0x557502220a30 msgr2=0x557502226790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:37.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb911ffb640 0 client.16500 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:37.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 shutdown_connections 2024-09-19T06:45:37.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 1 -- 172.21.15.37:0/2100198448 wait complete. 2024-09-19T06:45:37.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.714+0000 7fb932696740 20 client.16500 trim_cache size 0 max 16384 2024-09-19T06:45:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.731+0000 7fb932696740 1 Processor -- start 2024-09-19T06:45:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb932696740 1 -- start start 2024-09-19T06:45:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb932696740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x55750223f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb932696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x55750222ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb932696740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x55750224b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb932696740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281ae0a0 con 0x557502233a90 2024-09-19T06:45:37.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb932696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281ab940 con 0x557502224c80 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb932696740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281a9790 con 0x557502224360 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x55750222ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92e059640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x55750223f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x55750222ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:51458/0 (socket says 172.21.15.37:51458) 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 -- 172.21.15.37:0/3094345275 learned_addr learned my addr 172.21.15.37:0/3094345275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92e059640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x55750223f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:36716/0 (socket says 172.21.15.37:36716) 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92e85a640 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x55750224b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 -- 172.21.15.37:0/3094345275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 msgr2=0x55750223f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x55750223f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 -- 172.21.15.37:0/3094345275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 msgr2=0x55750224b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x55750224b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 -- 172.21.15.37:0/3094345275 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55750222f660 con 0x557502224c80 2024-09-19T06:45:37.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.732+0000 7fb92d858640 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x55750222ef80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb918000c00 tx=0x7fb918005b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e56846c1ac5d35e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb913fff640 1 -- 172.21.15.37:0/3094345275 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918035040 con 0x557502224c80 2024-09-19T06:45:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb913fff640 1 -- 172.21.15.37:0/3094345275 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91804a070 con 0x557502224c80 2024-09-19T06:45:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb913fff640 1 -- 172.21.15.37:0/3094345275 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918004ab0 con 0x557502224c80 2024-09-19T06:45:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb932696740 1 -- 172.21.15.37:0/3094345275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 msgr2=0x55750222ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb932696740 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x55750222ef80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb918000c00 tx=0x7fb918005b50 comp rx=0 tx=0).stop 2024-09-19T06:45:37.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb932696740 1 -- 172.21.15.37:0/3094345275 shutdown_connections 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb932696740 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x55750224b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb932696740 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x55750222ef80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb932696740 1 --2- 172.21.15.37:0/3094345275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x55750223f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.733+0000 7fb932696740 1 -- 172.21.15.37:0/3094345275 >> 172.21.15.37:0/3094345275 conn(0x557502220a30 msgr2=0x55750222bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 -- 172.21.15.37:0/3094345275 shutdown_connections 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 -- 172.21.15.37:0/3094345275 wait complete. 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 -- start start 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 Processor -- start 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x557502237b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x557502238060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x5575022ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281a9e00 con 0x557502233a90 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281ab270 con 0x557502224c80 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb9281aa4d0 con 0x557502224360 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb932696740 10 client.0 _ll_register_callbacks cb 0x7fb92400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.734+0000 7fb92e85a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x5575022ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e059640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x557502237b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e85a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x5575022ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43388/0 (socket says 172.21.15.37:43388) 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e85a640 1 -- 172.21.15.37:0/4268860455 learned_addr learned my addr 172.21.15.37:0/4268860455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92d858640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x557502238060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e059640 1 -- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 msgr2=0x557502238060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e059640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x557502238060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e059640 1 -- 172.21.15.37:0/4268860455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 msgr2=0x5575022ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e059640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x5575022ad1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e059640 1 -- 172.21.15.37:0/4268860455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557502224ab0 con 0x557502224360 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e85a640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x5575022ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92d858640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x557502238060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb92e059640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x557502237b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb92409a680 tx=0x7fb924098740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9c18fc0be87231d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb911ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb911ffb640 1 -- 172.21.15.37:0/4268860455 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92409c1a0 con 0x557502224360 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb911ffb640 1 -- 172.21.15.37:0/4268860455 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb924010430 con 0x557502224360 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb91804a070 con 0x557502224360 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb911ffb640 1 -- 172.21.15.37:0/4268860455 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fb9240cdcd0 con 0x557502224360 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb932696740 10 client.25894 fetch_fsmap learned FSMap version 30 2024-09-19T06:45:37.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.735+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x557502230f80 con 0x557502224360 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb911ffb640 1 -- 172.21.15.37:0/4268860455 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb91804a070 con 0x557502224360 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb911ffb640 1 -- 172.21.15.37:0/4268860455 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7fb924098120 con 0x557502224360 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb932696740 10 client.25894 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb932696740 10 client.25894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb932696740 20 client.25894 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb932696740 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022aec40 0x5575022b1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb932696740 4 client.25894 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557502191d60 con 0x5575022aec40 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb92d858640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022aec40 0x5575022b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb92d858640 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022aec40 0x5575022b1080 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4a1d9ec69507edf server_cookie=10efd44e9ef7dd37 in_seq=0 out_seq=0 2024-09-19T06:45:37.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.736+0000 7fb911ffb640 10 client.25894 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb911ffb640 1 -- 172.21.15.37:0/4268860455 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557502191d60 con 0x5575022aec40 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb911ffb640 10 client.25894 handle_command_reply: tid=0 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 client.25894 shutdown 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 msgr2=0x557502237b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x557502237b20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb92409a680 tx=0x7fb924098740 comp rx=0 tx=0).stop 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 shutdown_connections 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5575022aec40 0x5575022b1080 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557502233a90 0x5575022ad1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557502224c80 0x557502238060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb911ffb640 0 client.25894 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:37.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 --2- 172.21.15.37:0/4268860455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557502224360 0x557502237b20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 >> 172.21.15.37:0/4268860455 conn(0x557502220a30 msgr2=0x55750222bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 shutdown_connections 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 1 -- 172.21.15.37:0/4268860455 wait complete. 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb932696740 20 client.25894 trim_cache size 0 max 16384 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:45:37.746 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:45:37.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb9140a33b0 msgr2=0x7fb9140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb9302bd640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb9140a33b0 0x7fb9140a5870 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fb92000e720 tx=0x7fb920018000 comp rx=0 tx=0).stop 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100a60 msgr2=0x7fb928198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.737+0000 7fb9302bd640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100a60 0x7fb928198480 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb92400e9f0 tx=0x7fb92400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 shutdown_connections 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb9140a33b0 0x7fb9140a5870 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb928106a60 0x7fb92819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb928101400 0x7fb9281989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 --2- 172.21.15.37:0/2864122038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb928100a60 0x7fb928198480 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 >> 172.21.15.37:0/2864122038 conn(0x7fb9280fc780 msgr2=0x7fb92806aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 shutdown_connections 2024-09-19T06:45:37.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:37.738+0000 7fb9302bd640 1 -- 172.21.15.37:0/2864122038 wait complete. 2024-09-19T06:45:38.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 238063 14.22 MB/sec execute 5 sec latency 212.770 ms 2024-09-19T06:45:38.550 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:45:38.551 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:45:38.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:38 smithi096 ceph-mon[35004]: pgmap v119: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:45:38.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:38 smithi037 ceph-mon[31211]: pgmap v119: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:45:38.814 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:38.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:38 smithi151 ceph-mon[35236]: pgmap v119: 97 pgs: 97 active+clean; 35 MiB data, 685 MiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 2.0 MiB/s wr, 95 op/s 2024-09-19T06:45:39.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 241192 14.75 MB/sec execute 6 sec latency 11.446 ms 2024-09-19T06:45:39.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.317+0000 7fd733971640 1 -- 172.21.15.37:0/1270076069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 msgr2=0x7fd72c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.317+0000 7fd733971640 1 --2- 172.21.15.37:0/1270076069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 0x7fd72c107b90 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd71c009930 tx=0x7fd71c02a420 comp rx=0 tx=0).stop 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.317+0000 7fd733971640 1 -- 172.21.15.37:0/1270076069 shutdown_connections 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.317+0000 7fd733971640 1 --2- 172.21.15.37:0/1270076069 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd72c1082a0 0x7fd72c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.317+0000 7fd733971640 1 --2- 172.21.15.37:0/1270076069 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 0x7fd72c107b90 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.317+0000 7fd733971640 1 --2- 172.21.15.37:0/1270076069 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c102d30 0x7fd72c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.317+0000 7fd733971640 1 -- 172.21.15.37:0/1270076069 >> 172.21.15.37:0/1270076069 conn(0x7fd72c0fc7d0 msgr2=0x7fd72c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 -- 172.21.15.37:0/1270076069 shutdown_connections 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 -- 172.21.15.37:0/1270076069 wait complete. 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 Processor -- start 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 -- start start 2024-09-19T06:45:39.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c102d30 0x7fd72c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 0x7fd72c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd72c1082a0 0x7fd72c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c112060 con 0x7fd72c1036e0 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c111ee0 con 0x7fd72c1082a0 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.318+0000 7fd733971640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd72c1121e0 con 0x7fd72c102d30 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd730f0c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 0x7fd72c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd730f0c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 0x7fd72c100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43418/0 (socket says 172.21.15.37:43418) 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd730f0c640 1 -- 172.21.15.37:0/3759109455 learned_addr learned my addr 172.21.15.37:0/3759109455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd731f0e640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd72c1082a0 0x7fd72c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd73170d640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c102d30 0x7fd72c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd731f0e640 1 -- 172.21.15.37:0/3759109455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c102d30 msgr2=0x7fd72c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:39.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd731f0e640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c102d30 0x7fd72c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd731f0e640 1 -- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 msgr2=0x7fd72c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd731f0e640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 0x7fd72c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd731f0e640 1 -- 172.21.15.37:0/3759109455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd72c1a6a60 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd731f0e640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd72c1082a0 0x7fd72c101410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd724009fd0 tx=0x7fd72400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd7227fc640 1 -- 172.21.15.37:0/3759109455 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd72400ce20 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd72c1a6d50 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd7227fc640 1 -- 172.21.15.37:0/3759109455 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd724005020 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd7227fc640 1 -- 172.21.15.37:0/3759109455 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd724010720 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.319+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd72c1a7260 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.320+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd72c103110 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.322+0000 7fd7227fc640 1 -- 172.21.15.37:0/3759109455 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd724005540 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.322+0000 7fd7227fc640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7180a33b0 0x7fd7180a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.322+0000 7fd7227fc640 1 -- 172.21.15.37:0/3759109455 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd7240d0390 con 0x7fd72c1082a0 2024-09-19T06:45:39.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.322+0000 7fd73170d640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7180a33b0 0x7fd7180a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.323+0000 7fd73170d640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7180a33b0 0x7fd7180a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7280058c0 tx=0x7fd72800a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:39.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.327+0000 7fd7227fc640 1 -- 172.21.15.37:0/3759109455 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd72409cd70 con 0x7fd72c1082a0 2024-09-19T06:45:39.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.549+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd72c10a2b0 con 0x7fd72c1082a0 2024-09-19T06:45:39.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.550+0000 7fd7227fc640 1 -- 172.21.15.37:0/3759109455 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11904 (secure 0 0 0) 0x7fd72409c670 con 0x7fd72c1082a0 2024-09-19T06:45:39.552 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:45:39.553 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":30,"btime":"2024-09-19T06:45:32:447982+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:31.617019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:45:39.553 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 30 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7180a33b0 msgr2=0x7fd7180a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7180a33b0 0x7fd7180a5870 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd7280058c0 tx=0x7fd72800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd72c1082a0 msgr2=0x7fd72c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd72c1082a0 0x7fd72c101410 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd724009fd0 tx=0x7fd72400ef10 comp rx=0 tx=0).stop 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 shutdown_connections 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd72c1082a0 0x7fd72c101410 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7180a33b0 0x7fd7180a5870 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd72c1036e0 0x7fd72c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 --2- 172.21.15.37:0/3759109455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd72c102d30 0x7fd72c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 >> 172.21.15.37:0/3759109455 conn(0x7fd72c0fc7d0 msgr2=0x7fd72c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:39.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 shutdown_connections 2024-09-19T06:45:39.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:39.553+0000 7fd733971640 1 -- 172.21.15.37:0/3759109455 wait complete. 2024-09-19T06:45:40.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 242907 13.64 MB/sec execute 7 sec latency 101.389 ms 2024-09-19T06:45:40.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:45:40.620 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:40.742 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:40 smithi037 ceph-mon[31211]: pgmap v120: 97 pgs: 97 active+clean; 41 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-09-19T06:45:40.743 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:40 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3759109455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:40.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:40 smithi096 ceph-mon[35004]: pgmap v120: 97 pgs: 97 active+clean; 41 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-09-19T06:45:40.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:40 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3759109455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:40.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:40 smithi151 ceph-mon[35236]: pgmap v120: 97 pgs: 97 active+clean; 41 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 3.6 MiB/s wr, 229 op/s 2024-09-19T06:45:40.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:40 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3759109455' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:45:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.104+0000 7f40cccbf640 1 -- 172.21.15.37:0/921011411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 msgr2=0x7f40c8106840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.104+0000 7f40cccbf640 1 --2- 172.21.15.37:0/921011411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8106840 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009930 tx=0x7f40b002a420 comp rx=0 tx=0).stop 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.104+0000 7f40cccbf640 1 -- 172.21.15.37:0/921011411 shutdown_connections 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.104+0000 7f40cccbf640 1 --2- 172.21.15.37:0/921011411 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40c8106f50 0x7f40c810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.104+0000 7f40cccbf640 1 --2- 172.21.15.37:0/921011411 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8106840 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.105+0000 7f40cccbf640 1 --2- 172.21.15.37:0/921011411 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c81002f0 0x7f40c81006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.105+0000 7f40cccbf640 1 -- 172.21.15.37:0/921011411 >> 172.21.15.37:0/921011411 conn(0x7f40c8077210 msgr2=0x7f40c80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.105+0000 7f40cccbf640 1 -- 172.21.15.37:0/921011411 shutdown_connections 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.105+0000 7f40cccbf640 1 -- 172.21.15.37:0/921011411 wait complete. 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.105+0000 7f40cccbf640 1 Processor -- start 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40cccbf640 1 -- start start 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40cccbf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c81002f0 0x7f40c8198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40cccbf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40cccbf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40c8106f50 0x7f40c819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40cccbf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8111fa0 con 0x7f40c8100ca0 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40cccbf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8111e20 con 0x7f40c8106f50 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40cccbf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8112120 con 0x7f40c81002f0 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42492/0 (socket says 172.21.15.37:42492) 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c5d9b640 1 -- 172.21.15.37:0/1909566486 learned_addr learned my addr 172.21.15.37:0/1909566486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c6d9d640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40c8106f50 0x7f40c819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c659c640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c81002f0 0x7f40c8198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c6d9d640 1 -- 172.21.15.37:0/1909566486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c81002f0 msgr2=0x7f40c8198500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c6d9d640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c81002f0 0x7f40c8198500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c6d9d640 1 -- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 msgr2=0x7f40c8198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c6d9d640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c6d9d640 1 -- 172.21.15.37:0/1909566486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c819d4b0 con 0x7f40c8106f50 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.106+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.107+0000 7f40c6d9d640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40c8106f50 0x7f40c819cdd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f40b800e970 tx=0x7f40b800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.107+0000 7f40bf7fe640 1 -- 172.21.15.37:0/1909566486 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b800ccc0 con 0x7f40c8106f50 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.107+0000 7f40bf7fe640 1 -- 172.21.15.37:0/1909566486 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b8005020 con 0x7f40c8106f50 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.107+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c819d7a0 con 0x7f40c8106f50 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.107+0000 7f40bf7fe640 1 -- 172.21.15.37:0/1909566486 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b8004a00 con 0x7f40c8106f50 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.107+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c81a5080 con 0x7f40c8106f50 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.109+0000 7f40bf7fe640 1 -- 172.21.15.37:0/1909566486 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f40b80051d0 con 0x7f40c8106f50 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 Processor -- start 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 -- start start 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb304a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304a5d0 0x563eb304aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3059780 0x563eb305bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563eb2fb02f0 con 0x563eb304bb40 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563eb2f57cf0 con 0x563eb3059780 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40cf098740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563eb2fac2b0 con 0x563eb304a5d0 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb304a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40bf7fe640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f40b40a33b0 0x7f40b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb304a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42506/0 (socket says 172.21.15.37:42506) 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40c5d9b640 1 -- 172.21.15.37:0/998818265 learned_addr learned my addr 172.21.15.37:0/998818265 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3059780 0x563eb305bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40c659c640 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304a5d0 0x563eb304aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.110+0000 7f40c6d9d640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f40b40a33b0 0x7f40b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40c5d9b640 1 -- 172.21.15.37:0/998818265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304a5d0 msgr2=0x563eb304aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40bf7fe640 1 -- 172.21.15.37:0/1909566486 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f40b80cfcb0 con 0x7f40c8106f50 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304a5d0 0x563eb304aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40c5d9b640 1 -- 172.21.15.37:0/998818265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 msgr2=0x563eb304a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb304a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40c5d9b640 1 -- 172.21.15.37:0/998818265 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb2f3c3a0 con 0x563eb3059780 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3059780 0x563eb305bb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005940 tx=0x7f40b00058d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65111a88778734df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40bcff9640 1 -- 172.21.15.37:0/998818265 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0041070 con 0x563eb3059780 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40bcff9640 1 -- 172.21.15.37:0/998818265 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0005500 con 0x563eb3059780 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40bcff9640 1 -- 172.21.15.37:0/998818265 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0035070 con 0x563eb3059780 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.111+0000 7f40c6d9d640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f40b40a33b0 0x7f40b40a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40b809bc30 tx=0x7f40b8066dc0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 -- 172.21.15.37:0/998818265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3059780 msgr2=0x563eb305bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3059780 0x563eb305bb70 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f40b0005940 tx=0x7f40b00058d0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 -- 172.21.15.37:0/998818265 shutdown_connections 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3059780 0x563eb305bb70 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304a5d0 0x563eb304aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 --2- 172.21.15.37:0/998818265 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb304a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 -- 172.21.15.37:0/998818265 >> 172.21.15.37:0/998818265 conn(0x563eb3013310 msgr2=0x563eb3046c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 -- 172.21.15.37:0/998818265 shutdown_connections 2024-09-19T06:45:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 -- 172.21.15.37:0/998818265 wait complete. 2024-09-19T06:45:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.112+0000 7f40cf098740 1 -- start start 2024-09-19T06:45:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 1 Processor -- start 2024-09-19T06:45:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304a5d0 0x563eb30591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304bb40 0x563eb3056470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3059780 0x563eb3056c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563eb305c420 con 0x563eb3059780 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563eb2f6b060 con 0x563eb304a5d0 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563eb305c5a0 con 0x563eb304bb40 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40cf098740 10 client.0 _ll_register_callbacks cb 0x563eb2cbe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304a5d0 0x563eb30591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304a5d0 0x563eb30591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34944/0 (socket says 172.21.15.37:34944) 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 -- 172.21.15.37:0/3019312946 learned_addr learned my addr 172.21.15.37:0/3019312946 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c659c640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304bb40 0x563eb3056470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3059780 0x563eb3056c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 -- 172.21.15.37:0/3019312946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304bb40 msgr2=0x563eb3056470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304bb40 0x563eb3056470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 -- 172.21.15.37:0/3019312946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3059780 msgr2=0x563eb3056c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3059780 0x563eb3056c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c5d9b640 1 -- 172.21.15.37:0/3019312946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb2d213c0 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.113+0000 7f40c659c640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304bb40 0x563eb3056470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304a5d0 0x563eb30591c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40b0001eb0 tx=0x7f40b0004840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f5a41f15fd6582e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40adffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40adffb640 1 -- 172.21.15.37:0/3019312946 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0053020 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40b0005500 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40adffb640 1 -- 172.21.15.37:0/3019312946 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f40b0004920 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40adffb640 1 -- 172.21.15.37:0/3019312946 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40b0035070 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40cf098740 10 client.26021 fetch_fsmap learned FSMap version 30 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563eb3067f10 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.114+0000 7f40adffb640 1 -- 172.21.15.37:0/3019312946 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40b0005500 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40adffb640 1 -- 172.21.15.37:0/3019312946 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f40b003d9c0 con 0x563eb304a5d0 2024-09-19T06:45:41.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40cf098740 10 client.26021 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40cf098740 10 client.26021 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40cf098740 20 client.26021 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40cf098740 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb3069260 0x563eb306b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40cf098740 4 client.26021 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eb2fbd860 con 0x563eb3069260 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40c659c640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb3069260 0x563eb306b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40c659c640 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb3069260 0x563eb306b650 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6cd1943d3371c87f server_cookie=e6437b149780cfc1 in_seq=0 out_seq=0 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.115+0000 7f40adffb640 10 client.26021 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40adffb640 1 -- 172.21.15.37:0/3019312946 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563eb2fbd860 con 0x563eb3069260 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40adffb640 10 client.26021 handle_command_reply: tid=0 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 client.26021 shutdown 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304a5d0 msgr2=0x563eb30591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304a5d0 0x563eb30591c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f40b0001eb0 tx=0x7f40b0004840 comp rx=0 tx=0).stop 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 shutdown_connections 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb3069260 0x563eb306b650 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3059780 0x563eb3056c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb304bb40 0x563eb3056470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 --2- 172.21.15.37:0/3019312946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304a5d0 0x563eb30591c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40adffb640 0 client.26021 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.116+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 >> 172.21.15.37:0/3019312946 conn(0x563eb3013310 msgr2=0x563eb304c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.117+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 shutdown_connections 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.117+0000 7f40cf098740 1 -- 172.21.15.37:0/3019312946 wait complete. 2024-09-19T06:45:41.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.117+0000 7f40cf098740 20 client.26021 trim_cache size 0 max 16384 2024-09-19T06:45:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 Processor -- start 2024-09-19T06:45:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 -- start start 2024-09-19T06:45:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb3055d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb3055470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3046f20 0x563eb306e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40c81ae0c0 con 0x563eb304bb40 2024-09-19T06:45:41.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40c81ab960 con 0x563eb3046f20 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40cf098740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40c81a97b0 con 0x563eb3059780 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb3055d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb3055d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42524/0 (socket says 172.21.15.37:42524) 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40c5d9b640 1 -- 172.21.15.37:0/2749379955 learned_addr learned my addr 172.21.15.37:0/2749379955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.130+0000 7f40c659c640 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb3055470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3046f20 0x563eb306e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c659c640 1 -- 172.21.15.37:0/2749379955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3046f20 msgr2=0x563eb306e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c659c640 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3046f20 0x563eb306e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c659c640 1 -- 172.21.15.37:0/2749379955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 msgr2=0x563eb3055d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c659c640 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb3055d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c659c640 1 -- 172.21.15.37:0/2749379955 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eb3056f70 con 0x563eb3059780 2024-09-19T06:45:41.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3046f20 0x563eb306e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40c659c640 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb3055470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x563eb305c610 tx=0x7f40c0005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4a90da73ae72e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40affff640 1 -- 172.21.15.37:0/2749379955 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c0004b20 con 0x563eb3059780 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40affff640 1 -- 172.21.15.37:0/2749379955 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c0004cc0 con 0x563eb3059780 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.131+0000 7f40affff640 1 -- 172.21.15.37:0/2749379955 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40c00150d0 con 0x563eb3059780 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 -- 172.21.15.37:0/2749379955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 msgr2=0x563eb3055470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb3055470 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x563eb305c610 tx=0x7f40c0005c80 comp rx=0 tx=0).stop 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 -- 172.21.15.37:0/2749379955 shutdown_connections 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb3046f20 0x563eb306e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb3055470 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 --2- 172.21.15.37:0/2749379955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb304bb40 0x563eb3055d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 -- 172.21.15.37:0/2749379955 >> 172.21.15.37:0/2749379955 conn(0x563eb3013310 msgr2=0x563eb3054d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 -- 172.21.15.37:0/2749379955 shutdown_connections 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.132+0000 7f40cf098740 1 -- 172.21.15.37:0/2749379955 wait complete. 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 -- start start 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 Processor -- start 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3046f20 0x563eb306eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304bb40 0x563eb306bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb306c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40c81a9e20 con 0x563eb3046f20 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40c81ab290 con 0x563eb304bb40 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40c81aa4f0 con 0x563eb3059780 2024-09-19T06:45:41.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40cf098740 10 client.0 _ll_register_callbacks cb 0x7f40b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40c659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304bb40 0x563eb306bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40c5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3046f20 0x563eb306eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40c659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304bb40 0x563eb306bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34970/0 (socket says 172.21.15.37:34970) 2024-09-19T06:45:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.133+0000 7f40c659c640 1 -- 172.21.15.37:0/3055187227 learned_addr learned my addr 172.21.15.37:0/3055187227 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:41.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb306c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40c5d9b640 1 -- 172.21.15.37:0/3055187227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 msgr2=0x563eb306c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb306c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40c5d9b640 1 -- 172.21.15.37:0/3055187227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304bb40 msgr2=0x563eb306bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304bb40 0x563eb306bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40c5d9b640 1 -- 172.21.15.37:0/3055187227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eb306cc20 con 0x563eb3046f20 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40c5d9b640 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3046f20 0x563eb306eca0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009850 tx=0x7f40b0031a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f90d3febc95288 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40adffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40adffb640 1 -- 172.21.15.37:0/3055187227 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40b0053070 con 0x563eb3046f20 2024-09-19T06:45:41.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40adffb640 1 -- 172.21.15.37:0/3055187227 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f40b00048b0 con 0x563eb3046f20 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.134+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40c0004cc0 con 0x563eb3046f20 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40adffb640 1 -- 172.21.15.37:0/3055187227 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f40b006c070 con 0x563eb3046f20 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 10 client.16530 fetch_fsmap learned FSMap version 30 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563eb30d3a60 con 0x563eb3046f20 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40adffb640 1 -- 172.21.15.37:0/3055187227 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40c0004cc0 con 0x563eb3046f20 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40adffb640 1 -- 172.21.15.37:0/3055187227 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 30) ==== 8350+0+0 (secure 0 0 0) 0x7f40b003d450 con 0x563eb3046f20 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 10 client.16530 fetch_fsmap finished waiting for FSMap version 30 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 10 client.16530 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 20 client.16530 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb30d3f40 0x563eb30d6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 4 client.16530 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563eb2fb7d70 con 0x563eb30d3f40 2024-09-19T06:45:41.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.135+0000 7f40c659c640 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb30d3f40 0x563eb30d6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40adffb640 10 client.16530 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40c659c640 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb30d3f40 0x563eb30d6330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a69cc9d3e49c49ba server_cookie=1deb628d197945bf in_seq=0 out_seq=0 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40adffb640 1 -- 172.21.15.37:0/3055187227 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563eb2fb7d70 con 0x563eb30d3f40 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40adffb640 10 client.16530 handle_command_reply: tid=0 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 client.16530 shutdown 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3046f20 msgr2=0x563eb306eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3046f20 0x563eb306eca0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f40b0009850 tx=0x7f40b0031a10 comp rx=0 tx=0).stop 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 shutdown_connections 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x563eb30d3f40 0x563eb30d6330 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563eb3059780 0x563eb306c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563eb304bb40 0x563eb306bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40adffb640 0 client.16530 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 --2- 172.21.15.37:0/3055187227 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563eb3046f20 0x563eb306eca0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.136+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 >> 172.21.15.37:0/3055187227 conn(0x563eb3013310 msgr2=0x563eb304c110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 shutdown_connections 2024-09-19T06:45:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cf098740 1 -- 172.21.15.37:0/3055187227 wait complete. 2024-09-19T06:45:41.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cf098740 20 client.16530 trim_cache size 0 max 16384 2024-09-19T06:45:41.142 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f40b40a33b0 msgr2=0x7f40b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f40b40a33b0 0x7f40b40a5870 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f40b809bc30 tx=0x7f40b8066dc0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40c8106f50 msgr2=0x7f40c819cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40c8106f50 0x7f40c819cdd0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f40b800e970 tx=0x7f40b800ee30 comp rx=0 tx=0).stop 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 shutdown_connections 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f40c8106f50 0x7f40c819cdd0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f40b40a33b0 0x7f40b40a5870 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f40c8100ca0 0x7f40c8198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 --2- 172.21.15.37:0/1909566486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f40c81002f0 0x7f40c8198500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.137+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 >> 172.21.15.37:0/1909566486 conn(0x7f40c8077210 msgr2=0x7f40c8105270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.138+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 shutdown_connections 2024-09-19T06:45:41.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:41.138+0000 7f40cccbf640 1 -- 172.21.15.37:0/1909566486 wait complete. 2024-09-19T06:45:41.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 245589 13.73 MB/sec execute 8 sec latency 14.573 ms 2024-09-19T06:45:42.013 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:45:42.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 248670 14.27 MB/sec execute 9 sec latency 15.032 ms 2024-09-19T06:45:42.278 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2903468648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4101ae0 msgr2=0x7fc2b4101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2903468648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4101ae0 0x7fc2b4101f40 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009930 tx=0x7fc2a802a420 comp rx=0 tx=0).stop 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2903468648 shutdown_connections 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2903468648 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4102480 0x7fc2b410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2903468648 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4101ae0 0x7fc2b4101f40 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2903468648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4107b80 0x7fc2b4107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2903468648 >> 172.21.15.37:0/2903468648 conn(0x7fc2b40fd7e0 msgr2=0x7fc2b40ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2903468648 shutdown_connections 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.518+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2903468648 wait complete. 2024-09-19T06:45:42.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.519+0000 7fc2baa5c640 1 Processor -- start 2024-09-19T06:45:42.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.519+0000 7fc2baa5c640 1 -- start start 2024-09-19T06:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2baa5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4101ae0 0x7fc2b419a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2baa5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4102480 0x7fc2b419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2baa5c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4107b80 0x7fc2b419ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2baa5c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4111f30 con 0x7fc2b4102480 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2baa5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b4111db0 con 0x7fc2b4101ae0 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2baa5c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41120b0 con 0x7fc2b4107b80 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4102480 0x7fc2b419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4101ae0 0x7fc2b419a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4101ae0 0x7fc2b419a570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:34986/0 (socket says 172.21.15.37:34986) 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4102480 0x7fc2b419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42550/0 (socket says 172.21.15.37:42550) 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 -- 172.21.15.37:0/2647409742 learned_addr learned my addr 172.21.15.37:0/2647409742 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4107b80 0x7fc2b419ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 -- 172.21.15.37:0/2647409742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4107b80 msgr2=0x7fc2b419ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4107b80 0x7fc2b419ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 -- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4102480 msgr2=0x7fc2b419aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4102480 0x7fc2b419aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 -- 172.21.15.37:0/2647409742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2b419f560 con 0x7fc2b4101ae0 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.520+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4101ae0 0x7fc2b419a570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400ea60 tx=0x7fc2a400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.521+0000 7fc2b17fa640 1 -- 172.21.15.37:0/2647409742 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a400cde0 con 0x7fc2b4101ae0 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.521+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2b419f850 con 0x7fc2b4101ae0 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.521+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2b41a70e0 con 0x7fc2b4101ae0 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.521+0000 7fc2b17fa640 1 -- 172.21.15.37:0/2647409742 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a40035f0 con 0x7fc2b4101ae0 2024-09-19T06:45:42.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.521+0000 7fc2b17fa640 1 -- 172.21.15.37:0/2647409742 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4010670 con 0x7fc2b4101ae0 2024-09-19T06:45:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2b17fa640 1 -- 172.21.15.37:0/2647409742 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc2a40037a0 con 0x7fc2b4101ae0 2024-09-19T06:45:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 Processor -- start 2024-09-19T06:45:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 -- start start 2024-09-19T06:45:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad662b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6635f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad66388b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fad65872f0 con 0x55fad662b300 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fad652f270 con 0x55fad66364c0 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.524+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fad65832b0 con 0x55fad662bc20 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6635f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b8ff9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad662b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b17fa640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc29c0a3460 0x7fc29c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6635f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:56962/0 (socket says 172.21.15.37:56962) 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 -- 172.21.15.37:0/840762900 learned_addr learned my addr 172.21.15.37:0/840762900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b37fe640 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad66388b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b17fa640 1 -- 172.21.15.37:0/2647409742 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc2a40cff90 con 0x7fc2b4101ae0 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc29c0a3460 0x7fc29c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 -- 172.21.15.37:0/840762900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 msgr2=0x55fad66388b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad66388b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 -- 172.21.15.37:0/840762900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 msgr2=0x55fad662b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad662b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 -- 172.21.15.37:0/840762900 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad652e340 con 0x55fad662bc20 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6635f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4098c40 tx=0x7fc2a400ef90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b64787de4b36e9fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc29c0a3460 0x7fc29c0a5920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005c40 tx=0x7fc2ac004ab0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc29a7fc640 1 -- 172.21.15.37:0/840762900 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a409c970 con 0x55fad662bc20 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc29a7fc640 1 -- 172.21.15.37:0/840762900 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a409c270 con 0x55fad662bc20 2024-09-19T06:45:42.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.525+0000 7fc29a7fc640 1 -- 172.21.15.37:0/840762900 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40cdac0 con 0x55fad662bc20 2024-09-19T06:45:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.526+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/840762900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 msgr2=0x55fad6635f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.526+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6635f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc2a4098c40 tx=0x7fc2a400ef90 comp rx=0 tx=0).stop 2024-09-19T06:45:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/840762900 shutdown_connections 2024-09-19T06:45:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad66388b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6635f80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/840762900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad662b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/840762900 >> 172.21.15.37:0/840762900 conn(0x55fad6627870 msgr2=0x55fad6627c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/840762900 shutdown_connections 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/840762900 wait complete. 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 -- start start 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 Processor -- start 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad6633410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.527+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6633950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad6633e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55fad6639030 con 0x55fad662b300 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fad6510290 con 0x55fad66364c0 2024-09-19T06:45:42.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55fad66391b0 con 0x55fad662bc20 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2bd7f4740 10 client.0 _ll_register_callbacks cb 0x55fad6541140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad6633410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad6633410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42562/0 (socket says 172.21.15.37:42562) 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b3fff640 1 -- 172.21.15.37:0/2932723916 learned_addr learned my addr 172.21.15.37:0/2932723916 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad6633e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b37fe640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6633950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b8ff9640 1 -- 172.21.15.37:0/2932723916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 msgr2=0x55fad6633950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6633950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b8ff9640 1 -- 172.21.15.37:0/2932723916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 msgr2=0x55fad6633410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad6633410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b8ff9640 1 -- 172.21.15.37:0/2932723916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad629a210 con 0x55fad66364c0 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad6633410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad6633e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac013ab0 tx=0x7fc2ac004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b7367765d646a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.528+0000 7fc2937fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2937fe640 1 -- 172.21.15.37:0/2932723916 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac026030 con 0x55fad66364c0 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2937fe640 1 -- 172.21.15.37:0/2932723916 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc2ac01c030 con 0x55fad66364c0 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2a409c270 con 0x55fad66364c0 2024-09-19T06:45:42.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2937fe640 1 -- 172.21.15.37:0/2932723916 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac030050 con 0x55fad66364c0 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 10 client.26039 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fad6366620 con 0x55fad66364c0 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2937fe640 1 -- 172.21.15.37:0/2932723916 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc2a409c270 con 0x55fad66364c0 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2937fe640 1 -- 172.21.15.37:0/2932723916 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc2ac005090 con 0x55fad66364c0 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 10 client.26039 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 10 client.26039 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 20 client.26039 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad6646480 0x55fad66488d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 4 client.26039 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.529+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fad6594a20 con 0x55fad6646480 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.530+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad6646480 0x55fad66488d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.530+0000 7fc2937fe640 10 client.26039 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:42.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.530+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad6646480 0x55fad66488d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b29d7b36b795874e server_cookie=72b345fbe4a3ecbe in_seq=0 out_seq=0 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2937fe640 1 -- 172.21.15.37:0/2932723916 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7fc2a407ae50 con 0x55fad6646480 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2937fe640 10 client.26039 handle_command_reply: tid=0 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 client.26039 shutdown 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 msgr2=0x55fad6633e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad6633e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac013ab0 tx=0x7fc2ac004a40 comp rx=0 tx=0).stop 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 shutdown_connections 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad6646480 0x55fad66488d0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad6633e90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662bc20 0x55fad6633950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2937fe640 0 client.26039 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/2932723916 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662b300 0x55fad6633410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 >> 172.21.15.37:0/2932723916 conn(0x55fad6627870 msgr2=0x55fad662d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 shutdown_connections 2024-09-19T06:45:42.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.531+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/2932723916 wait complete. 2024-09-19T06:45:42.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.532+0000 7fc2bd7f4740 20 client.26039 trim_cache size 0 max 16384 2024-09-19T06:45:42.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 Processor -- start 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 -- start start 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6635380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad664e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad664e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41ae040 con 0x55fad662bc20 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41ab910 con 0x55fad66364c0 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41a97a0 con 0x55fad662b300 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b8ff9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad664e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b8ff9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad664e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35010/0 (socket says 172.21.15.37:35010) 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b3fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6635380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b8ff9640 1 -- 172.21.15.37:0/3035289875 learned_addr learned my addr 172.21.15.37:0/3035289875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b37fe640 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad664e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b3fff640 1 -- 172.21.15.37:0/3035289875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 msgr2=0x55fad664e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad664e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.547+0000 7fc2b3fff640 1 -- 172.21.15.37:0/3035289875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 msgr2=0x55fad664e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad664e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc2b3fff640 1 -- 172.21.15.37:0/3035289875 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fad662b080 con 0x55fad662b300 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc2b37fe640 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad664e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad664e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6635380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2a409c200 tx=0x7fc2a4003f30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e47687b331b78f39 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc299ffb640 1 -- 172.21.15.37:0/3035289875 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a409c040 con 0x55fad662b300 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc299ffb640 1 -- 172.21.15.37:0/3035289875 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fad6594a20 con 0x55fad662b300 2024-09-19T06:45:42.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.548+0000 7fc299ffb640 1 -- 172.21.15.37:0/3035289875 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a40988e0 con 0x55fad662b300 2024-09-19T06:45:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3035289875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 msgr2=0x55fad6635380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6635380 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc2a409c200 tx=0x7fc2a4003f30 comp rx=0 tx=0).stop 2024-09-19T06:45:42.555 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3035289875 shutdown_connections 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad664e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad664e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3035289875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6635380 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3035289875 >> 172.21.15.37:0/3035289875 conn(0x55fad6627870 msgr2=0x55fad6634c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3035289875 shutdown_connections 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3035289875 wait complete. 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.549+0000 7fc2bd7f4740 1 -- start start 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 1 Processor -- start 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6639bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad663a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad663e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.556 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41a9df0 con 0x55fad662bc20 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41ab240 con 0x55fad66364c0 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc2b41aa4a0 con 0x55fad662b300 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2bd7f4740 10 client.0 _ll_register_callbacks cb 0x7fc2ac0139c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad663a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad663a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42580/0 (socket says 172.21.15.37:42580) 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2b37fe640 1 -- 172.21.15.37:0/3480249157 learned_addr learned my addr 172.21.15.37:0/3480249157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad663e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.550+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6639bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2b8ff9640 1 -- 172.21.15.37:0/3480249157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 msgr2=0x55fad6639bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6639bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2b8ff9640 1 -- 172.21.15.37:0/3480249157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 msgr2=0x55fad663a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.557 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad663a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2b8ff9640 1 -- 172.21.15.37:0/3480249157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fad663e8d0 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2b37fe640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad663a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2b8ff9640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad663e260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005480 tx=0x7fc2ac013b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e210873fce26c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2937fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2937fe640 1 -- 172.21.15.37:0/3480249157 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac01c040 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2937fe640 1 -- 172.21.15.37:0/3480249157 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc2ac004420 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fad6594a20 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2937fe640 1 -- 172.21.15.37:0/3480249157 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc2ac026040 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2bd7f4740 10 client.26045 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.551+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55fad66b0d50 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2937fe640 1 -- 172.21.15.37:0/3480249157 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55fad6594a20 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2937fe640 1 -- 172.21.15.37:0/3480249157 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc2ac022480 con 0x55fad66364c0 2024-09-19T06:45:42.558 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2bd7f4740 10 client.26045 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2bd7f4740 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2bd7f4740 20 client.26045 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad66b14d0 0x55fad66b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2bd7f4740 4 client.26045 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fad658ed70 con 0x55fad66b14d0 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad66b14d0 0x55fad66b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2937fe640 10 client.26045 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:42.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.552+0000 7fc2b3fff640 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad66b14d0 0x55fad66b38f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=168316a4167ad026 server_cookie=7c95de67b4234a1a in_seq=0 out_seq=0 2024-09-19T06:45:42.665 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:42 smithi037 ceph-mon[31211]: pgmap v121: 97 pgs: 97 active+clean; 41 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.7 MiB/s wr, 196 op/s 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2937fe640 1 -- 172.21.15.37:0/3480249157 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fad658ed70 con 0x55fad66b14d0 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2937fe640 10 client.26045 handle_command_reply: tid=0 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 client.26045 shutdown 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 msgr2=0x55fad663e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad663e260 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005480 tx=0x7fc2ac013b80 comp rx=0 tx=0).stop 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 shutdown_connections 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55fad66b14d0 0x55fad66b38f0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fad66364c0 0x55fad663e260 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55fad662bc20 0x55fad663a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 --2- 172.21.15.37:0/3480249157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55fad662b300 0x55fad6639bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2937fe640 0 client.26045 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 >> 172.21.15.37:0/3480249157 conn(0x55fad6627870 msgr2=0x55fad662d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 shutdown_connections 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 1 -- 172.21.15.37:0/3480249157 wait complete. 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.676+0000 7fc2bd7f4740 20 client.26045 trim_cache size 0 max 16384 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:45:42.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:45:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc29c0a3460 msgr2=0x7fc29c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc29c0a3460 0x7fc29c0a5920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2ac005c40 tx=0x7fc2ac004ab0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4101ae0 msgr2=0x7fc2b419a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4101ae0 0x7fc2b419a570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400ea60 tx=0x7fc2a400ef20 comp rx=0 tx=0).stop 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 shutdown_connections 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc2b4107b80 0x7fc2b419ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc29c0a3460 0x7fc29c0a5920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc2b4102480 0x7fc2b419aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 --2- 172.21.15.37:0/2647409742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2b4101ae0 0x7fc2b419a570 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 >> 172.21.15.37:0/2647409742 conn(0x7fc2b40fd7e0 msgr2=0x7fc2b410d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 shutdown_connections 2024-09-19T06:45:42.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:42.677+0000 7fc2baa5c640 1 -- 172.21.15.37:0/2647409742 wait complete. 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 -- 172.21.15.37:0/522547237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f181410e450 msgr2=0x7f18141108a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 --2- 172.21.15.37:0/522547237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f181410e450 0x7f18141108a0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f180c009ca0 tx=0x7f180c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 -- 172.21.15.37:0/522547237 shutdown_connections 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 --2- 172.21.15.37:0/522547237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f181410e450 0x7f18141108a0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 --2- 172.21.15.37:0/522547237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1814074f70 0x7f181410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 --2- 172.21.15.37:0/522547237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18140745c0 0x7f18140749a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 -- 172.21.15.37:0/522547237 >> 172.21.15.37:0/522547237 conn(0x7f18140fd8d0 msgr2=0x7f18140ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 -- 172.21.15.37:0/522547237 shutdown_connections 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.757+0000 7f1819222640 1 -- 172.21.15.37:0/522547237 wait complete. 2024-09-19T06:45:42.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.758+0000 7f1819222640 1 Processor -- start 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.758+0000 7f1819222640 1 -- start start 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1819222640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18140745c0 0x7f181419a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1819222640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1814074f70 0x7f181419abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1819222640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f181410e450 0x7f181419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1819222640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1814114200 con 0x7f1814074f70 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1819222640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1814114080 con 0x7f18140745c0 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1819222640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1814114380 con 0x7f181410e450 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1814074f70 0x7f181419abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1814074f70 0x7f181419abd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42598/0 (socket says 172.21.15.37:42598) 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 -- 172.21.15.37:0/1331233854 learned_addr learned my addr 172.21.15.37:0/1331233854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1812d9d640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18140745c0 0x7f181419a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f181359e640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f181410e450 0x7f181419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 -- 172.21.15.37:0/1331233854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f181410e450 msgr2=0x7f181419ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f181410e450 0x7f181419ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 -- 172.21.15.37:0/1331233854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18140745c0 msgr2=0x7f181419a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18140745c0 0x7f181419a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 -- 172.21.15.37:0/1331233854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f181419f640 con 0x7f1814074f70 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1812d9d640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18140745c0 0x7f181419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f180bfff640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1814074f70 0x7f181419abd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e9f0 tx=0x7f17fc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1810d99640 1 -- 172.21.15.37:0/1331233854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc00cd80 con 0x7f1814074f70 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f181419f930 con 0x7f1814074f70 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.759+0000 7f1810d99640 1 -- 172.21.15.37:0/1331233854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17fc005500 con 0x7f1814074f70 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.760+0000 7f1810d99640 1 -- 172.21.15.37:0/1331233854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc010690 con 0x7f1814074f70 2024-09-19T06:45:42.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.760+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18141a7210 con 0x7f1814074f70 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.762+0000 7f1810d99640 1 -- 172.21.15.37:0/1331233854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f17fc0046c0 con 0x7f1814074f70 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.762+0000 7f181b5fb740 1 Processor -- start 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.762+0000 7f181b5fb740 1 -- start start 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d795090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f1810d99640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18000a33b0 0x7f18000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d795a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d7a2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181b5fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560c4d6fb2f0 con 0x560c4d796b40 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181b5fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560c4d6a2cf0 con 0x560c4d7a0370 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181b5fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560c4d6f72b0 con 0x560c4d7955d0 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d795090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d795090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42602/0 (socket says 172.21.15.37:42602) 2024-09-19T06:45:42.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f1812d9d640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18000a33b0 0x7f18000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 -- 172.21.15.37:0/2772511613 learned_addr learned my addr 172.21.15.37:0/2772511613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d7a2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f1810d99640 1 -- 172.21.15.37:0/1331233854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f17fc0d0120 con 0x7f1814074f70 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f180bfff640 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d795a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 -- 172.21.15.37:0/2772511613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 msgr2=0x560c4d795a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d795a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 -- 172.21.15.37:0/2772511613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 msgr2=0x560c4d7a2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d7a2760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f181359e640 1 -- 172.21.15.37:0/2772511613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c4d6873a0 con 0x560c4d796b40 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.763+0000 7f1812d9d640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18000a33b0 0x7f18000a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1804009fc0 tx=0x7f1804005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.764+0000 7f181359e640 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d795090 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x560c4d796070 tx=0x7f180c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d292900dde3b9a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.764+0000 7f18097fa640 1 -- 172.21.15.37:0/2772511613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c031db0 con 0x560c4d796b40 2024-09-19T06:45:42.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.764+0000 7f18097fa640 1 -- 172.21.15.37:0/2772511613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c036040 con 0x560c4d796b40 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.764+0000 7f18097fa640 1 -- 172.21.15.37:0/2772511613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f180c004370 con 0x560c4d796b40 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.764+0000 7f181b5fb740 1 -- 172.21.15.37:0/2772511613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 msgr2=0x560c4d795090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.764+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d795090 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x560c4d796070 tx=0x7f180c009360 comp rx=0 tx=0).stop 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- 172.21.15.37:0/2772511613 shutdown_connections 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d7a2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d795a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2772511613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d795090 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- 172.21.15.37:0/2772511613 >> 172.21.15.37:0/2772511613 conn(0x560c4d75e310 msgr2=0x560c4d791c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- 172.21.15.37:0/2772511613 shutdown_connections 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- 172.21.15.37:0/2772511613 wait complete. 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- start start 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 Processor -- start 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79fd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7a0370 0x560c4d79d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560c4d7a3010 con 0x560c4d7a0370 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560c4d6b6060 con 0x560c4d796b40 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560c4d7a3190 con 0x560c4d7955d0 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181b5fb740 10 client.0 _ll_register_callbacks cb 0x560c4d4098c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7a0370 0x560c4d79d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f180bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f180bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35036/0 (socket says 172.21.15.37:35036) 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f180bfff640 1 -- 172.21.15.37:0/780653919 learned_addr learned my addr 172.21.15.37:0/780653919 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.765+0000 7f181359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7a0370 0x560c4d79d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42604/0 (socket says 172.21.15.37:42604) 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f1812d9d640 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f180bfff640 1 -- 172.21.15.37:0/780653919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 msgr2=0x560c4d79fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f180bfff640 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f180bfff640 1 -- 172.21.15.37:0/780653919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7a0370 msgr2=0x560c4d79d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f180bfff640 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7a0370 0x560c4d79d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f180bfff640 1 -- 172.21.15.37:0/780653919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c4d46c3c0 con 0x560c4d796b40 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f181359e640 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7a0370 0x560c4d79d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f180bfff640 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79cfd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e9c0 tx=0x7f17fc09cb60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9051d93719d24e7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f17fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f17fa7fc640 1 -- 172.21.15.37:0/780653919 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc0a5010 con 0x560c4d796b40 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180c036040 con 0x560c4d796b40 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f17fa7fc640 1 -- 172.21.15.37:0/780653919 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f17fc0cda90 con 0x560c4d796b40 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f17fa7fc640 1 -- 172.21.15.37:0/780653919 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f17fc0cdc90 con 0x560c4d796b40 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f181b5fb740 10 client.26054 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.766+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c4d7aeac0 con 0x560c4d796b40 2024-09-19T06:45:42.781 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f17fa7fc640 1 -- 172.21.15.37:0/780653919 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180c036040 con 0x560c4d796b40 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f17fa7fc640 1 -- 172.21.15.37:0/780653919 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f17fc09b7f0 con 0x560c4d796b40 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f181b5fb740 10 client.26054 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f181b5fb740 10 client.26054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f181b5fb740 20 client.26054 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f181b5fb740 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d7aff20 0x560c4d7b2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f181b5fb740 4 client.26054 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c4d708860 con 0x560c4d7aff20 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f1812d9d640 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d7aff20 0x560c4d7b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f1812d9d640 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d7aff20 0x560c4d7b2330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6438ebfb9937983f server_cookie=9825984a37ce8ca5 in_seq=0 out_seq=0 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.767+0000 7f17fa7fc640 10 client.26054 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f17fa7fc640 1 -- 172.21.15.37:0/780653919 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c4d708860 con 0x560c4d7aff20 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f17fa7fc640 10 client.26054 handle_command_reply: tid=0 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 client.26054 shutdown 2024-09-19T06:45:42.782 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 msgr2=0x560c4d79cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79cfd0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e9c0 tx=0x7f17fc09cb60 comp rx=0 tx=0).stop 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 shutdown_connections 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d7aff20 0x560c4d7b2330 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7a0370 0x560c4d79d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79cfd0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 --2- 172.21.15.37:0/780653919 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f17fa7fc640 0 client.26054 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 >> 172.21.15.37:0/780653919 conn(0x560c4d75e310 msgr2=0x560c4d797960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 shutdown_connections 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 1 -- 172.21.15.37:0/780653919 wait complete. 2024-09-19T06:45:42.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.768+0000 7f181b5fb740 20 client.26054 trim_cache size 0 max 16384 2024-09-19T06:45:42.784 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:42 smithi096 ceph-mon[35004]: pgmap v121: 97 pgs: 97 active+clean; 41 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.7 MiB/s wr, 196 op/s 2024-09-19T06:45:42.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 Processor -- start 2024-09-19T06:45:42.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 -- start start 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7955d0 0x560c4d7ac2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7a0370 0x560c4d7b7c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18141b2380 con 0x560c4d7955d0 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18141afc50 con 0x560c4d796b40 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.783+0000 7f181b5fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18141adaa0 con 0x560c4d7a0370 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7955d0 0x560c4d7ac2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7955d0 0x560c4d7ac2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42612/0 (socket says 172.21.15.37:42612) 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 -- 172.21.15.37:0/2600138335 learned_addr learned my addr 172.21.15.37:0/2600138335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f180bfff640 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f181359e640 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7a0370 0x560c4d7b7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 -- 172.21.15.37:0/2600138335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7a0370 msgr2=0x560c4d7b7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7a0370 0x560c4d7b7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 -- 172.21.15.37:0/2600138335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 msgr2=0x560c4d79c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 -- 172.21.15.37:0/2600138335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c4d798120 con 0x560c4d7955d0 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1812d9d640 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7955d0 0x560c4d7ac2b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x560c4d7b2ba0 tx=0x7f1804014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb547c0141e41ed0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1808ff9640 1 -- 172.21.15.37:0/2600138335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040040f0 con 0x560c4d7955d0 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1808ff9640 1 -- 172.21.15.37:0/2600138335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f180400f960 con 0x560c4d7955d0 2024-09-19T06:45:42.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.784+0000 7f1808ff9640 1 -- 172.21.15.37:0/2600138335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18040288d0 con 0x560c4d7955d0 2024-09-19T06:45:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.785+0000 7f181b5fb740 1 -- 172.21.15.37:0/2600138335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7955d0 msgr2=0x560c4d7ac2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.785+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7955d0 0x560c4d7ac2b0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x560c4d7b2ba0 tx=0x7f1804014000 comp rx=0 tx=0).stop 2024-09-19T06:45:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.785+0000 7f181b5fb740 1 -- 172.21.15.37:0/2600138335 shutdown_connections 2024-09-19T06:45:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.785+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7a0370 0x560c4d7b7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.785+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d796b40 0x560c4d79c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.785+0000 7f181b5fb740 1 --2- 172.21.15.37:0/2600138335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d7955d0 0x560c4d7ac2b0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.785+0000 7f181b5fb740 1 -- 172.21.15.37:0/2600138335 >> 172.21.15.37:0/2600138335 conn(0x560c4d75e310 msgr2=0x560c4d79c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.786+0000 7f181b5fb740 1 -- 172.21.15.37:0/2600138335 shutdown_connections 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.786+0000 7f181b5fb740 1 -- 172.21.15.37:0/2600138335 wait complete. 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.786+0000 7f181b5fb740 1 -- start start 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 1 Processor -- start 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d79a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d79abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18141ae110 con 0x560c4d796b40 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18141af580 con 0x560c4d7a0370 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18141ae7e0 con 0x560c4d7955d0 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181b5fb740 10 client.0 _ll_register_callbacks cb 0x7f17fc098640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d79a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d79a690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42614/0 (socket says 172.21.15.37:42614) 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 -- 172.21.15.37:0/3893987043 learned_addr learned my addr 172.21.15.37:0/3893987043 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f181359e640 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d79abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f1812d9d640 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 -- 172.21.15.37:0/3893987043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 msgr2=0x560c4d79a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 -- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 msgr2=0x560c4d79abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d79abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.787+0000 7f180bfff640 1 -- 172.21.15.37:0/3893987043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c4d819d70 con 0x560c4d796b40 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f180bfff640 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d79a690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0097d0 tx=0x7f17fc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7385ffeaf8237b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f17fa7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f17fa7fc640 1 -- 172.21.15.37:0/3893987043 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17fc09bee0 con 0x560c4d796b40 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f17fa7fc640 1 -- 172.21.15.37:0/3893987043 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f17fc0b5210 con 0x560c4d796b40 2024-09-19T06:45:42.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f180400f960 con 0x560c4d796b40 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f17fa7fc640 1 -- 172.21.15.37:0/3893987043 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f17fc0cdc90 con 0x560c4d796b40 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f181b5fb740 10 client.16566 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560c4d81b030 con 0x560c4d796b40 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.788+0000 7f17fa7fc640 1 -- 172.21.15.37:0/3893987043 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17fc0bc760 con 0x560c4d796b40 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f17fa7fc640 1 -- 172.21.15.37:0/3893987043 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f17fc0a18e0 con 0x560c4d796b40 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f181b5fb740 10 client.16566 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f181b5fb740 10 client.16566 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f181b5fb740 20 client.16566 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f181b5fb740 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d81b7e0 0x560c4d81dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f181b5fb740 4 client.16566 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f180400f960 con 0x560c4d81b7e0 2024-09-19T06:45:42.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f1812d9d640 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d81b7e0 0x560c4d81dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:42.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f1812d9d640 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d81b7e0 0x560c4d81dc00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9d4c83621f5bcede server_cookie=ceb078d00cda4a9 in_seq=0 out_seq=0 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f17fa7fc640 10 client.16566 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f17fa7fc640 1 -- 172.21.15.37:0/3893987043 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f180400f960 con 0x560c4d81b7e0 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f17fa7fc640 10 client.16566 handle_command_reply: tid=0 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.789+0000 7f181b5fb740 1 client.16566 shutdown 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 msgr2=0x560c4d79a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d79a690 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0097d0 tx=0x7f17fc009310 comp rx=0 tx=0).stop 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 shutdown_connections 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x560c4d81b7e0 0x560c4d81dc00 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560c4d7a0370 0x560c4d79abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560c4d796b40 0x560c4d79a690 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 --2- 172.21.15.37:0/3893987043 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560c4d7955d0 0x560c4d79a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 >> 172.21.15.37:0/3893987043 conn(0x560c4d75e310 msgr2=0x560c4d797110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f17fa7fc640 0 client.16566 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:45:42.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 shutdown_connections 2024-09-19T06:45:42.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 1 -- 172.21.15.37:0/3893987043 wait complete. 2024-09-19T06:45:42.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f181b5fb740 20 client.16566 trim_cache size 0 max 16384 2024-09-19T06:45:42.796 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18000a33b0 msgr2=0x7f18000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f1819222640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18000a33b0 0x7f18000a5870 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1804009fc0 tx=0x7f1804005f90 comp rx=0 tx=0).stop 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1814074f70 msgr2=0x7f181419abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.790+0000 7f1819222640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1814074f70 0x7f181419abd0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f17fc00e9f0 tx=0x7f17fc00edf0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 shutdown_connections 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f181410e450 0x7f181419ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18000a33b0 0x7f18000a5870 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1814074f70 0x7f181419abd0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 --2- 172.21.15.37:0/1331233854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18140745c0 0x7f181419a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 >> 172.21.15.37:0/1331233854 conn(0x7f18140fd8d0 msgr2=0x7f18140fe570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 shutdown_connections 2024-09-19T06:45:42.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:42.791+0000 7f1819222640 1 -- 172.21.15.37:0/1331233854 wait complete. 2024-09-19T06:45:42.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:42 smithi151 ceph-mon[35236]: pgmap v121: 97 pgs: 97 active+clean; 41 MiB data, 735 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.7 MiB/s wr, 196 op/s 2024-09-19T06:45:43.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 250952 13.66 MB/sec execute 10 sec latency 42.381 ms 2024-09-19T06:45:43.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:45:43.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:43 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:45:43.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:43 smithi096 ceph-mon[35004]: pgmap v122: 97 pgs: 97 active+clean; 47 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-09-19T06:45:43.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:43 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:45:43.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:43 smithi037 ceph-mon[31211]: pgmap v122: 97 pgs: 97 active+clean; 47 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-09-19T06:45:43.823 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:43.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:43 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:45:43.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:43 smithi151 ceph-mon[35236]: pgmap v122: 97 pgs: 97 active+clean; 47 MiB data, 743 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 4.2 MiB/s wr, 338 op/s 2024-09-19T06:45:44.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 253117 13.61 MB/sec execute 11 sec latency 37.788 ms 2024-09-19T06:45:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 -- 172.21.15.37:0/4214899499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48108290 msgr2=0x7feb4810ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/4214899499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48108290 0x7feb4810ec30 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7feb40009cd0 tx=0x7feb4002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 -- 172.21.15.37:0/4214899499 shutdown_connections 2024-09-19T06:45:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/4214899499 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48108290 0x7feb4810ec30 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/4214899499 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb48103740 0x7feb48107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/4214899499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb48102d90 0x7feb48103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 -- 172.21.15.37:0/4214899499 >> 172.21.15.37:0/4214899499 conn(0x7feb480fc7d0 msgr2=0x7feb480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 -- 172.21.15.37:0/4214899499 shutdown_connections 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.271+0000 7feb4dbbb640 1 -- 172.21.15.37:0/4214899499 wait complete. 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.272+0000 7feb4dbbb640 1 Processor -- start 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.272+0000 7feb4dbbb640 1 -- start start 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb4dbbb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48102d90 0x7feb481009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb4dbbb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb48103740 0x7feb48100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb4dbbb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb48108290 0x7feb48101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb4dbbb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb48112090 con 0x7feb48102d90 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48102d90 0x7feb481009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb4dbbb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb48111f10 con 0x7feb48108290 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb4dbbb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb48112210 con 0x7feb48103740 2024-09-19T06:45:44.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48102d90 0x7feb481009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42624/0 (socket says 172.21.15.37:42624) 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 -- 172.21.15.37:0/3495028141 learned_addr learned my addr 172.21.15.37:0/3495028141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb47fff640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb48108290 0x7feb48101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb46ffd640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb48103740 0x7feb48100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 -- 172.21.15.37:0/3495028141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb48103740 msgr2=0x7feb48100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb48103740 0x7feb48100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 -- 172.21.15.37:0/3495028141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb48108290 msgr2=0x7feb48101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb48108290 0x7feb48101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 -- 172.21.15.37:0/3495028141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb481a69a0 con 0x7feb48102d90 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.273+0000 7feb477fe640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48102d90 0x7feb481009f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feb3800e9f0 tx=0x7feb3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.274+0000 7feb44ff9640 1 -- 172.21.15.37:0/3495028141 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3800ccc0 con 0x7feb48102d90 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.274+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb481a6c90 con 0x7feb48102d90 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.274+0000 7feb44ff9640 1 -- 172.21.15.37:0/3495028141 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb38010910 con 0x7feb48102d90 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.274+0000 7feb44ff9640 1 -- 172.21.15.37:0/3495028141 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380188a0 con 0x7feb48102d90 2024-09-19T06:45:44.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.274+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb481a71c0 con 0x7feb48102d90 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.276+0000 7feb44ff9640 1 -- 172.21.15.37:0/3495028141 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7feb38018a40 con 0x7feb48102d90 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.276+0000 7feb4ff94740 1 Processor -- start 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb4ff94740 1 -- start start 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbaa3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaa3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbab0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb4ff94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5581bba092f0 con 0x5581bbaae370 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb4ff94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581bb9b0cf0 con 0x5581bbaa4b40 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb4ff94740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5581bba052b0 con 0x5581bbaa35d0 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbaa3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb44ff9640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feb300a33b0 0x7feb300a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb477fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaa3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbaa3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35078/0 (socket says 172.21.15.37:35078) 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb47fff640 1 -- 172.21.15.37:0/531523161 learned_addr learned my addr 172.21.15.37:0/531523161 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb46ffd640 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbab0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb47fff640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feb300a33b0 0x7feb300a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.277+0000 7feb44ff9640 1 -- 172.21.15.37:0/3495028141 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7feb380d01c0 con 0x7feb48102d90 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb477fe640 1 -- 172.21.15.37:0/531523161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 msgr2=0x5581bbaa3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb477fe640 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbaa3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb477fe640 1 -- 172.21.15.37:0/531523161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 msgr2=0x5581bbab0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb477fe640 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbab0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb477fe640 1 -- 172.21.15.37:0/531523161 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581bb9953a0 con 0x5581bbaa35d0 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb47fff640 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbaa3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:44.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb46ffd640 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbab0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb477fe640 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaa3a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb380cd910 tx=0x7feb3800cef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3be3bb5f58b6c8d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb47fff640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feb300a33b0 0x7feb300a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb40005fe0 tx=0x7feb4002a8e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb2dffb640 1 -- 172.21.15.37:0/531523161 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb380857a0 con 0x5581bbaa35d0 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb2dffb640 1 -- 172.21.15.37:0/531523161 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3809cba0 con 0x5581bbaa35d0 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.278+0000 7feb2dffb640 1 -- 172.21.15.37:0/531523161 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb38010ac0 con 0x5581bbaa35d0 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 -- 172.21.15.37:0/531523161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 msgr2=0x5581bbaa3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaa3a10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7feb380cd910 tx=0x7feb3800cef0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 -- 172.21.15.37:0/531523161 shutdown_connections 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbab0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaa3a10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 --2- 172.21.15.37:0/531523161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbaa3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 -- 172.21.15.37:0/531523161 >> 172.21.15.37:0/531523161 conn(0x5581bba6c310 msgr2=0x5581bba9fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:44.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 -- 172.21.15.37:0/531523161 shutdown_connections 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 -- 172.21.15.37:0/531523161 wait complete. 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.279+0000 7feb4ff94740 1 -- start start 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 1 Processor -- start 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaaddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaa4b40 0x5581bbaab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaae370 0x5581bbaab800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5581bbab1010 con 0x5581bbaa4b40 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5581bb9c4060 con 0x5581bbaae370 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5581bbab1190 con 0x5581bbaa35d0 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb4ff94740 10 client.0 _ll_register_callbacks cb 0x5581bb7178c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaa4b40 0x5581bbaab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaae370 0x5581bbaab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaa4b40 0x5581bbaab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42650/0 (socket says 172.21.15.37:42650) 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb47fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaae370 0x5581bbaab800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35086/0 (socket says 172.21.15.37:35086) 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 -- 172.21.15.37:0/1215308675 learned_addr learned my addr 172.21.15.37:0/1215308675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb477fe640 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaaddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 -- 172.21.15.37:0/1215308675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 msgr2=0x5581bbaaddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaaddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 -- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaae370 msgr2=0x5581bbaab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaae370 0x5581bbaab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb46ffd640 1 -- 172.21.15.37:0/1215308675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581bb77a3c0 con 0x5581bbaa4b40 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.280+0000 7feb47fff640 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaae370 0x5581bbaab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb46ffd640 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaa4b40 0x5581bbaab070 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0107f0 tx=0x7feb3c010bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8876467bf8442b5d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb26ffd640 1 -- 172.21.15.37:0/1215308675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3c01e060 con 0x5581bbaa4b40 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb26ffd640 1 -- 172.21.15.37:0/1215308675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7feb3c00fb10 con 0x5581bbaa4b40 2024-09-19T06:45:44.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb3809cba0 con 0x5581bbaa4b40 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb26ffd640 1 -- 172.21.15.37:0/1215308675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb3c00fd10 con 0x5581bbaa4b40 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb4ff94740 10 client.16578 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5581bbabcb80 con 0x5581bbaa4b40 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb26ffd640 1 -- 172.21.15.37:0/1215308675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb3809cba0 con 0x5581bbaa4b40 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb26ffd640 1 -- 172.21.15.37:0/1215308675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7feb3c00e4d0 con 0x5581bbaa4b40 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb4ff94740 10 client.16578 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb4ff94740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.281+0000 7feb4ff94740 20 client.16578 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.282+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbabe090 0x5581bbac04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.282+0000 7feb4ff94740 4 client.16578 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.282+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581bba16860 con 0x5581bbabe090 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.282+0000 7feb477fe640 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbabe090 0x5581bbac04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.282+0000 7feb26ffd640 10 client.16578 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.282+0000 7feb477fe640 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbabe090 0x5581bbac04b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76da9667b6287439 server_cookie=98bdc392c1f129b6 in_seq=0 out_seq=0 2024-09-19T06:45:44.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb26ffd640 1 -- 172.21.15.37:0/1215308675 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5581bba16860 con 0x5581bbabe090 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb26ffd640 10 client.16578 handle_command_reply: tid=0 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 client.16578 shutdown 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaa4b40 msgr2=0x5581bbaab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaa4b40 0x5581bbaab070 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7feb3c0107f0 tx=0x7feb3c010bf0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 shutdown_connections 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbabe090 0x5581bbac04b0 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaae370 0x5581bbaab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaa4b40 0x5581bbaab070 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1215308675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbaaddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 >> 172.21.15.37:0/1215308675 conn(0x5581bba6c310 msgr2=0x5581bbaa5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:44.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb26ffd640 0 client.16578 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 shutdown_connections 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 1 -- 172.21.15.37:0/1215308675 wait complete. 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.283+0000 7feb4ff94740 20 client.16578 trim_cache size 0 max 16384 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 Processor -- start 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 -- start start 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa35d0 0x5581bba9fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa4b40 0x5581bbaa4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaaa050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb481b2420 con 0x5581bbaae370 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb481afcf0 con 0x5581bbaa35d0 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb4ff94740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb481adb40 con 0x5581bbaa4b40 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaaa050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.316 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaaa050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42658/0 (socket says 172.21.15.37:42658) 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa35d0 0x5581bba9fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb47fff640 1 -- 172.21.15.37:0/1348910460 learned_addr learned my addr 172.21.15.37:0/1348910460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.298+0000 7feb46ffd640 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa4b40 0x5581bbaa4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb47fff640 1 -- 172.21.15.37:0/1348910460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa4b40 msgr2=0x5581bbaa4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb47fff640 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa4b40 0x5581bbaa4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb47fff640 1 -- 172.21.15.37:0/1348910460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa35d0 msgr2=0x5581bba9fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb47fff640 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa35d0 0x5581bba9fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb47fff640 1 -- 172.21.15.37:0/1348910460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581bbaae130 con 0x5581bbaae370 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb46ffd640 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa4b40 0x5581bbaa4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb477fe640 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa35d0 0x5581bba9fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb47fff640 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaaa050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb40004ab0 tx=0x7feb40005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e815fc46590115 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb2d7fa640 1 -- 172.21.15.37:0/1348910460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40050070 con 0x5581bbaae370 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb2d7fa640 1 -- 172.21.15.37:0/1348910460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb40036050 con 0x5581bbaae370 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.299+0000 7feb2d7fa640 1 -- 172.21.15.37:0/1348910460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4004a690 con 0x5581bbaae370 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 -- 172.21.15.37:0/1348910460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 msgr2=0x5581bbaaa050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaaa050 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7feb40004ab0 tx=0x7feb40005ef0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 -- 172.21.15.37:0/1348910460 shutdown_connections 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaaa050 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa4b40 0x5581bbaa4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 --2- 172.21.15.37:0/1348910460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa35d0 0x5581bba9fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 -- 172.21.15.37:0/1348910460 >> 172.21.15.37:0/1348910460 conn(0x5581bba6c310 msgr2=0x5581bbaa9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 -- 172.21.15.37:0/1348910460 shutdown_connections 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 -- 172.21.15.37:0/1348910460 wait complete. 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.300+0000 7feb4ff94740 1 -- start start 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 1 Processor -- start 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbab6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbab6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaa78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feb481ae1b0 con 0x5581bbaae370 2024-09-19T06:45:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb481af620 con 0x5581bbaa4b40 2024-09-19T06:45:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feb481ae880 con 0x5581bbaa35d0 2024-09-19T06:45:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb4ff94740 10 client.0 _ll_register_callbacks cb 0x7feb3c010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:44.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbab6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb46ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbab6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35098/0 (socket says 172.21.15.37:35098) 2024-09-19T06:45:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaa78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.301+0000 7feb46ffd640 1 -- 172.21.15.37:0/444354206 learned_addr learned my addr 172.21.15.37:0/444354206 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb477fe640 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbab6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb477fe640 1 -- 172.21.15.37:0/444354206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 msgr2=0x5581bbab6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb477fe640 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbab6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb477fe640 1 -- 172.21.15.37:0/444354206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 msgr2=0x5581bbaa78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb477fe640 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaa78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb477fe640 1 -- 172.21.15.37:0/444354206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581bbaa8080 con 0x5581bbaa35d0 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb477fe640 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbab6670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb3809cd50 tx=0x7feb3800ce60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ba7174aff387ab2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb26ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb26ffd640 1 -- 172.21.15.37:0/444354206 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb3809cba0 con 0x5581bbaa35d0 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb26ffd640 1 -- 172.21.15.37:0/444354206 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7feb38098b10 con 0x5581bbaa35d0 2024-09-19T06:45:44.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.302+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feb40036050 con 0x5581bbaa35d0 2024-09-19T06:45:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb26ffd640 1 -- 172.21.15.37:0/444354206 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7feb380a3bd0 con 0x5581bbaa35d0 2024-09-19T06:45:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 10 client.25963 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5581bbb310a0 con 0x5581bbaa35d0 2024-09-19T06:45:44.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb26ffd640 1 -- 172.21.15.37:0/444354206 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feb40036050 con 0x5581bbaa35d0 2024-09-19T06:45:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb26ffd640 1 -- 172.21.15.37:0/444354206 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7feb380b52c0 con 0x5581bbaa35d0 2024-09-19T06:45:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 10 client.25963 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 10 client.25963 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:44.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 20 client.25963 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:44.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbb313e0 0x5581bbb337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:44.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 4 client.25963 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:45:44.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5581bba10d70 con 0x5581bbb313e0 2024-09-19T06:45:44.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.303+0000 7feb46ffd640 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbb313e0 0x5581bbb337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:44.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.304+0000 7feb26ffd640 10 client.25963 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:44.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.304+0000 7feb46ffd640 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbb313e0 0x5581bbb337d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f85939bd727dddb server_cookie=2c42e8d2887bb50c in_seq=0 out_seq=0 2024-09-19T06:45:44.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb26ffd640 1 -- 172.21.15.37:0/444354206 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5581bba10d70 con 0x5581bbb313e0 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb26ffd640 10 client.25963 handle_command_reply: tid=0 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 client.25963 shutdown 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 msgr2=0x5581bbab6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbab6670 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7feb3809cd50 tx=0x7feb3800ce60 comp rx=0 tx=0).stop 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 shutdown_connections 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5581bbb313e0 0x5581bbb337d0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5581bbaae370 0x5581bbaa78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb26ffd640 0 client.25963 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:44.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5581bbaa4b40 0x5581bbab6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 --2- 172.21.15.37:0/444354206 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5581bbaa35d0 0x5581bbab6670 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 >> 172.21.15.37:0/444354206 conn(0x5581bba6c310 msgr2=0x5581bbaaa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:44.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 shutdown_connections 2024-09-19T06:45:44.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.305+0000 7feb4ff94740 1 -- 172.21.15.37:0/444354206 wait complete. 2024-09-19T06:45:44.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4ff94740 20 client.25963 trim_cache size 0 max 16384 2024-09-19T06:45:44.329 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:45:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feb300a33b0 msgr2=0x7feb300a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feb300a33b0 0x7feb300a5870 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb40005fe0 tx=0x7feb4002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48102d90 msgr2=0x7feb481009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48102d90 0x7feb481009f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7feb3800e9f0 tx=0x7feb3800edf0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 shutdown_connections 2024-09-19T06:45:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb48108290 0x7feb48101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb48103740 0x7feb48100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feb300a33b0 0x7feb300a5870 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 --2- 172.21.15.37:0/3495028141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feb48102d90 0x7feb481009f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 >> 172.21.15.37:0/3495028141 conn(0x7feb480fc7d0 msgr2=0x7feb48106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 shutdown_connections 2024-09-19T06:45:44.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:44.306+0000 7feb4dbbb640 1 -- 172.21.15.37:0/3495028141 wait complete. 2024-09-19T06:45:45.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 255812 13.89 MB/sec execute 12 sec latency 51.667 ms 2024-09-19T06:45:46.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:45 smithi096 ceph-mon[35004]: pgmap v123: 97 pgs: 97 active+clean; 47 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-19T06:45:46.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:45 smithi037 ceph-mon[31211]: pgmap v123: 97 pgs: 97 active+clean; 47 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-19T06:45:46.091 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:45:46.092 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:45:46.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:45 smithi151 ceph-mon[35236]: pgmap v123: 97 pgs: 97 active+clean; 47 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-19T06:45:46.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 258967 14.17 MB/sec execute 13 sec latency 40.816 ms 2024-09-19T06:45:46.373 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 -- 172.21.15.37:0/209858825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890100b20 msgr2=0x7f8890100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 --2- 172.21.15.37:0/209858825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890100b20 0x7f8890100f00 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f888c009900 tx=0x7f888c02a410 comp rx=0 tx=0).stop 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 -- 172.21.15.37:0/209858825 shutdown_connections 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 --2- 172.21.15.37:0/209858825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f889010e500 0x7f8890110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 --2- 172.21.15.37:0/209858825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8890101440 0x7f889010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 --2- 172.21.15.37:0/209858825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890100b20 0x7f8890100f00 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 -- 172.21.15.37:0/209858825 >> 172.21.15.37:0/209858825 conn(0x7f88900fc7d0 msgr2=0x7f88900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 -- 172.21.15.37:0/209858825 shutdown_connections 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.848+0000 7f8898151640 1 -- 172.21.15.37:0/209858825 wait complete. 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 Processor -- start 2024-09-19T06:45:46.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 -- start start 2024-09-19T06:45:46.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8890100b20 0x7f88901a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890101440 0x7f88901a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f889010e500 0x7f889019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8890114280 con 0x7f8890101440 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8890114100 con 0x7f889010e500 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8898151640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8890114400 con 0x7f8890100b20 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f88956ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890101440 0x7f88901a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f88956ec640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890101440 0x7f88901a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42686/0 (socket says 172.21.15.37:42686) 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f88956ec640 1 -- 172.21.15.37:0/2843248534 learned_addr learned my addr 172.21.15.37:0/2843248534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f88966ee640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f889010e500 0x7f889019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.849+0000 7f8895eed640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8890100b20 0x7f88901a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f88956ec640 1 -- 172.21.15.37:0/2843248534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8890100b20 msgr2=0x7f88901a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f88956ec640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8890100b20 0x7f88901a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f88956ec640 1 -- 172.21.15.37:0/2843248534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f889010e500 msgr2=0x7f889019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f88956ec640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f889010e500 0x7f889019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f88956ec640 1 -- 172.21.15.37:0/2843248534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f889019ace0 con 0x7f8890101440 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f88966ee640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f889010e500 0x7f889019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f88956ec640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890101440 0x7f88901a0930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f888000d950 tx=0x7f888000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f8886ffd640 1 -- 172.21.15.37:0/2843248534 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880017070 con 0x7f8890101440 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f889019afd0 con 0x7f8890101440 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f8886ffd640 1 -- 172.21.15.37:0/2843248534 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8880009c30 con 0x7f8890101440 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88901a72c0 con 0x7f8890101440 2024-09-19T06:45:46.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.850+0000 7f8886ffd640 1 -- 172.21.15.37:0/2843248534 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888000f600 con 0x7f8890101440 2024-09-19T06:45:46.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.853+0000 7f8886ffd640 1 -- 172.21.15.37:0/2843248534 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f888001a070 con 0x7f8890101440 2024-09-19T06:45:46.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.853+0000 7f889a52a740 1 Processor -- start 2024-09-19T06:45:46.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.853+0000 7f889a52a740 1 -- start start 2024-09-19T06:45:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f889a52a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4adf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f889a52a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f889a52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4b8d40 0x56407e4bb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f889a52a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56407e413280 con 0x56407e4adb60 2024-09-19T06:45:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f889a52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56407e3bab10 con 0x56407e4b8d40 2024-09-19T06:45:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f889a52a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56407e40f240 con 0x56407e4ae480 2024-09-19T06:45:46.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4adf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42696/0 (socket says 172.21.15.37:42696) 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 -- 172.21.15.37:0/351627282 learned_addr learned my addr 172.21.15.37:0/351627282 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f8886ffd640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f887c0a33b0 0x7f887c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f8895eed640 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88956ec640 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4b8d40 0x56407e4bb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 -- 172.21.15.37:0/351627282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 msgr2=0x56407e4b8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 -- 172.21.15.37:0/351627282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4b8d40 msgr2=0x56407e4bb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4b8d40 0x56407e4bb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 -- 172.21.15.37:0/351627282 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56407e3c71f0 con 0x56407e4adb60 2024-09-19T06:45:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f8895eed640 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f8886ffd640 1 -- 172.21.15.37:0/2843248534 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f88800cf710 con 0x7f8890101440 2024-09-19T06:45:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.854+0000 7f88966ee640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f887c0a33b0 0x7f887c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.855+0000 7f88966ee640 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4adf40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f88880118c0 tx=0x7f8888011d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75f2d31fda3c287b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.855+0000 7f887bfff640 1 -- 172.21.15.37:0/351627282 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888017070 con 0x56407e4adb60 2024-09-19T06:45:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.855+0000 7f887bfff640 1 -- 172.21.15.37:0/351627282 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888801c430 con 0x56407e4adb60 2024-09-19T06:45:46.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.855+0000 7f887bfff640 1 -- 172.21.15.37:0/351627282 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888800e610 con 0x56407e4adb60 2024-09-19T06:45:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 -- 172.21.15.37:0/351627282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 msgr2=0x56407e4adf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f88966ee640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f887c0a33b0 0x7f887c0a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8888023000 tx=0x7f8888024000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:46.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4adf40 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f88880118c0 tx=0x7f8888011d80 comp rx=0 tx=0).stop 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 -- 172.21.15.37:0/351627282 shutdown_connections 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4b8d40 0x56407e4bb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 --2- 172.21.15.37:0/351627282 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4adf40 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 -- 172.21.15.37:0/351627282 >> 172.21.15.37:0/351627282 conn(0x56407e4aa0f0 msgr2=0x56407e4aa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 -- 172.21.15.37:0/351627282 shutdown_connections 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.856+0000 7f889a52a740 1 -- 172.21.15.37:0/351627282 wait complete. 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 -- start start 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 Processor -- start 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4adb60 0x56407e4b5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4b8d40 0x56407e4b6680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56407e4bb910 con 0x56407e4b8d40 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56407e3cdf10 con 0x56407e4adb60 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56407e4bba90 con 0x56407e4ae480 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f889a52a740 10 client.0 _ll_register_callbacks cb 0x56407e396a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f88966ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4b8d40 0x56407e4b6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f8895eed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4adb60 0x56407e4b5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f8895eed640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4adb60 0x56407e4b5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35136/0 (socket says 172.21.15.37:35136) 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f88966ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4b8d40 0x56407e4b6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42698/0 (socket says 172.21.15.37:42698) 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.857+0000 7f8895eed640 1 -- 172.21.15.37:0/2474160133 learned_addr learned my addr 172.21.15.37:0/2474160133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:46.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f88956ec640 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f8895eed640 1 -- 172.21.15.37:0/2474160133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 msgr2=0x56407e4b6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f8895eed640 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f8895eed640 1 -- 172.21.15.37:0/2474160133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4b8d40 msgr2=0x56407e4b6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f8895eed640 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4b8d40 0x56407e4b6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f8895eed640 1 -- 172.21.15.37:0/2474160133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56407e1846a0 con 0x56407e4adb60 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f8895eed640 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4adb60 0x56407e4b5c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f888c009900 tx=0x7f888c031850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f74065ed6d228a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f88797fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f88797fa640 1 -- 172.21.15.37:0/2474160133 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888c035040 con 0x56407e4adb60 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f888801c430 con 0x56407e4adb60 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f88797fa640 1 -- 172.21.15.37:0/2474160133 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f888c004240 con 0x56407e4adb60 2024-09-19T06:45:46.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f88797fa640 1 -- 172.21.15.37:0/2474160133 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f888c004440 con 0x56407e4adb60 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f889a52a740 10 client.26090 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.858+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56407e4c74f0 con 0x56407e4adb60 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f88797fa640 1 -- 172.21.15.37:0/2474160133 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f888801c430 con 0x56407e4adb60 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f88797fa640 1 -- 172.21.15.37:0/2474160133 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f888c004c90 con 0x56407e4adb60 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f889a52a740 10 client.26090 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f889a52a740 10 client.26090 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f889a52a740 20 client.26090 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f889a52a740 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e4c8950 0x56407e4cad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f889a52a740 4 client.26090 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56407e4207f0 con 0x56407e4c8950 2024-09-19T06:45:46.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f88956ec640 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e4c8950 0x56407e4cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.859+0000 7f88797fa640 10 client.26090 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:46.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.860+0000 7f88956ec640 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e4c8950 0x56407e4cad40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e53658e71137a506 server_cookie=d9c7a74d1e5674c8 in_seq=0 out_seq=0 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.861+0000 7f88797fa640 1 -- 172.21.15.37:0/2474160133 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56407e4207f0 con 0x56407e4c8950 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.861+0000 7f88797fa640 10 client.26090 handle_command_reply: tid=0 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.861+0000 7f889a52a740 1 client.26090 shutdown 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.861+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4adb60 msgr2=0x56407e4b5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.861+0000 7f889a52a740 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4adb60 0x56407e4b5c00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f888c009900 tx=0x7f888c031850 comp rx=0 tx=0).stop 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 shutdown_connections 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e4c8950 0x56407e4cad40 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4b8d40 0x56407e4b6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4ae480 0x56407e4b6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 --2- 172.21.15.37:0/2474160133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4adb60 0x56407e4b5c00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 >> 172.21.15.37:0/2474160133 conn(0x56407e4aa0f0 msgr2=0x56407e4a9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f88797fa640 0 client.26090 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 shutdown_connections 2024-09-19T06:45:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 1 -- 172.21.15.37:0/2474160133 wait complete. 2024-09-19T06:45:46.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.862+0000 7f889a52a740 20 client.26090 trim_cache size 0 max 16384 2024-09-19T06:45:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.883+0000 7f889a52a740 1 Processor -- start 2024-09-19T06:45:46.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.883+0000 7f889a52a740 1 -- start start 2024-09-19T06:45:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.883+0000 7f889a52a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f889a52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4b7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f889a52a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f889a52a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88901b2410 con 0x56407e4adb60 2024-09-19T06:45:46.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f889a52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88901afce0 con 0x56407e4ae480 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f889a52a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88901adb30 con 0x56407e4b8d40 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f8895eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f8895eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4b05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42708/0 (socket says 172.21.15.37:42708) 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f8895eed640 1 -- 172.21.15.37:0/2937513059 learned_addr learned my addr 172.21.15.37:0/2937513059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f88966ee640 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.884+0000 7f88956ec640 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4b7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.885+0000 7f88966ee640 1 -- 172.21.15.37:0/2937513059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 msgr2=0x56407e4b7a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.885+0000 7f88966ee640 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4b7a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.885+0000 7f88966ee640 1 -- 172.21.15.37:0/2937513059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 msgr2=0x56407e4b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.885+0000 7f88966ee640 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.885+0000 7f88966ee640 1 -- 172.21.15.37:0/2937513059 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56407e4ae2b0 con 0x56407e4b8d40 2024-09-19T06:45:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.885+0000 7f88966ee640 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4d0610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f888801cf00 tx=0x7f8888009520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=442f53eaeda08dd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.886+0000 7f887b7fe640 1 -- 172.21.15.37:0/2937513059 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8888017070 con 0x56407e4b8d40 2024-09-19T06:45:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.886+0000 7f887b7fe640 1 -- 172.21.15.37:0/2937513059 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8888011490 con 0x56407e4b8d40 2024-09-19T06:45:46.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.886+0000 7f887b7fe640 1 -- 172.21.15.37:0/2937513059 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888801c690 con 0x56407e4b8d40 2024-09-19T06:45:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.887+0000 7f889a52a740 1 -- 172.21.15.37:0/2937513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 msgr2=0x56407e4d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.887+0000 7f889a52a740 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4d0610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f888801cf00 tx=0x7f8888009520 comp rx=0 tx=0).stop 2024-09-19T06:45:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.887+0000 7f889a52a740 1 -- 172.21.15.37:0/2937513059 shutdown_connections 2024-09-19T06:45:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.887+0000 7f889a52a740 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4d0610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.887+0000 7f889a52a740 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4b7a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.887+0000 7f889a52a740 1 --2- 172.21.15.37:0/2937513059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.887+0000 7f889a52a740 1 -- 172.21.15.37:0/2937513059 >> 172.21.15.37:0/2937513059 conn(0x56407e4aa0f0 msgr2=0x56407e4b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.888+0000 7f889a52a740 1 -- 172.21.15.37:0/2937513059 shutdown_connections 2024-09-19T06:45:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.888+0000 7f889a52a740 1 -- 172.21.15.37:0/2937513059 wait complete. 2024-09-19T06:45:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.888+0000 7f889a52a740 1 -- start start 2024-09-19T06:45:46.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.888+0000 7f889a52a740 1 Processor -- start 2024-09-19T06:45:46.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f889a52a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4c1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f889a52a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4bc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f889a52a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4bce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f889a52a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f88901ae1a0 con 0x56407e4adb60 2024-09-19T06:45:46.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f889a52a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88901af610 con 0x56407e4ae480 2024-09-19T06:45:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f889a52a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f88901ae870 con 0x56407e4b8d40 2024-09-19T06:45:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f889a52a740 10 client.0 _ll_register_callbacks cb 0x7f888c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f8895eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4c1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f8895eed640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4c1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42718/0 (socket says 172.21.15.37:42718) 2024-09-19T06:45:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f8895eed640 1 -- 172.21.15.37:0/2284348844 learned_addr learned my addr 172.21.15.37:0/2284348844 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.889+0000 7f88956ec640 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4bc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.890+0000 7f88966ee640 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4bce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.890+0000 7f88956ec640 1 -- 172.21.15.37:0/2284348844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 msgr2=0x56407e4bce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.890+0000 7f88956ec640 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4bce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.890+0000 7f88956ec640 1 -- 172.21.15.37:0/2284348844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 msgr2=0x56407e4c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.890+0000 7f88956ec640 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4c1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.890+0000 7f88956ec640 1 -- 172.21.15.37:0/2284348844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56407e4bd690 con 0x56407e4ae480 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.890+0000 7f88956ec640 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4bc870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88800cce60 tx=0x7f8880086040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fea07873b824f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.891+0000 7f88797fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:46.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.891+0000 7f88797fa640 1 -- 172.21.15.37:0/2284348844 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8880021040 con 0x56407e4ae480 2024-09-19T06:45:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.891+0000 7f88797fa640 1 -- 172.21.15.37:0/2284348844 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f8880098110 con 0x56407e4ae480 2024-09-19T06:45:46.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.891+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8888011490 con 0x56407e4ae480 2024-09-19T06:45:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f88797fa640 1 -- 172.21.15.37:0/2284348844 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f888009baa0 con 0x56407e4ae480 2024-09-19T06:45:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f889a52a740 10 client.26096 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56407e53bf40 con 0x56407e4ae480 2024-09-19T06:45:46.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f88797fa640 1 -- 172.21.15.37:0/2284348844 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8888011490 con 0x56407e4ae480 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f88797fa640 1 -- 172.21.15.37:0/2284348844 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f88800cfbc0 con 0x56407e4ae480 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f889a52a740 10 client.26096 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f889a52a740 10 client.26096 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.892+0000 7f889a52a740 20 client.26096 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.893+0000 7f889a52a740 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e53c620 0x56407e53ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.893+0000 7f889a52a740 4 client.26096 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.893+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56407e41ad00 con 0x56407e53c620 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.893+0000 7f8895eed640 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e53c620 0x56407e53ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.894+0000 7f8895eed640 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e53c620 0x56407e53ea40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cedbbe35bdddc0a3 server_cookie=af3001ef34e09cf9 in_seq=0 out_seq=0 2024-09-19T06:45:46.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.894+0000 7f88797fa640 10 client.26096 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f88797fa640 1 -- 172.21.15.37:0/2284348844 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56407e41ad00 con 0x56407e53c620 2024-09-19T06:45:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f88797fa640 10 client.26096 handle_command_reply: tid=0 2024-09-19T06:45:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 client.26096 shutdown 2024-09-19T06:45:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 msgr2=0x56407e4bc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4bc870 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f88800cce60 tx=0x7f8880086040 comp rx=0 tx=0).stop 2024-09-19T06:45:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 shutdown_connections 2024-09-19T06:45:46.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56407e53c620 0x56407e53ea40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56407e4b8d40 0x56407e4bce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f88797fa640 0 client.26096 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56407e4ae480 0x56407e4bc870 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 --2- 172.21.15.37:0/2284348844 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56407e4adb60 0x56407e4c1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.895+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 >> 172.21.15.37:0/2284348844 conn(0x56407e4aa0f0 msgr2=0x56407e4a9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:46.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.896+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 shutdown_connections 2024-09-19T06:45:46.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.896+0000 7f889a52a740 1 -- 172.21.15.37:0/2284348844 wait complete. 2024-09-19T06:45:46.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.896+0000 7f889a52a740 20 client.26096 trim_cache size 0 max 16384 2024-09-19T06:45:46.899 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:45:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:45:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "e9f8aecb-470e-4d02-93fb-a0cd6278e9e0", 2024-09-19T06:45:46.899 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:45:46.899 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f887c0a33b0 msgr2=0x7f887c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f887c0a33b0 0x7f887c0a5870 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8888023000 tx=0x7f8888024000 comp rx=0 tx=0).stop 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890101440 msgr2=0x7f88901a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890101440 0x7f88901a0930 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f888000d950 tx=0x7f888000de10 comp rx=0 tx=0).stop 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 shutdown_connections 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f889010e500 0x7f889019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f887c0a33b0 0x7f887c0a5870 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8890101440 0x7f88901a0930 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 --2- 172.21.15.37:0/2843248534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8890100b20 0x7f88901a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:46.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 >> 172.21.15.37:0/2843248534 conn(0x7f88900fc7d0 msgr2=0x7f88900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:46.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 shutdown_connections 2024-09-19T06:45:46.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:46.897+0000 7f8898151640 1 -- 172.21.15.37:0/2843248534 wait complete. 2024-09-19T06:45:47.197 INFO:tasks.workunit.client.0.smithi037.stdout: 1 261317 13.77 MB/sec execute 14 sec latency 42.137 ms 2024-09-19T06:45:47.964 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:47 smithi037 ceph-mon[31211]: pgmap v124: 97 pgs: 97 active+clean; 47 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-19T06:45:47.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:47 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:45:47.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:47 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:47.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:47 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:45:47.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:47 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:45:47.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:47 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:47.965 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:47 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:47.992 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:45:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:47 smithi096 ceph-mon[35004]: pgmap v124: 97 pgs: 97 active+clean; 47 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-19T06:45:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:47 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:45:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:47 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:47 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:45:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:47 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:45:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:47 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:48.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:47 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:47 smithi151 ceph-mon[35236]: pgmap v124: 97 pgs: 97 active+clean; 47 MiB data, 769 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.5 MiB/s wr, 324 op/s 2024-09-19T06:45:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:47 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:45:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:47 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:47 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:45:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:47 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:45:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:47 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:48.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:47 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:45:48.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 263719 13.80 MB/sec execute 15 sec latency 21.076 ms 2024-09-19T06:45:48.256 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:45:48.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 -- 172.21.15.37:0/4286062119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6298109120 msgr2=0x7f6298110ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 --2- 172.21.15.37:0/4286062119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6298109120 0x7f6298110ca0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f6290009ca0 tx=0x7f629002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 -- 172.21.15.37:0/4286062119 shutdown_connections 2024-09-19T06:45:48.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 --2- 172.21.15.37:0/4286062119 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6298109120 0x7f6298110ca0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 --2- 172.21.15.37:0/4286062119 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62981049a0 0x7f62981089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 --2- 172.21.15.37:0/4286062119 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6298103ff0 0x7f62981043d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 -- 172.21.15.37:0/4286062119 >> 172.21.15.37:0/4286062119 conn(0x7f62980ffab0 msgr2=0x7f6298101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 -- 172.21.15.37:0/4286062119 shutdown_connections 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 -- 172.21.15.37:0/4286062119 wait complete. 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 Processor -- start 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.727+0000 7f629d7e9640 1 -- start start 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f629d7e9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6298103ff0 0x7f629819a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f629d7e9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62981049a0 0x7f629819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f629d7e9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6298109120 0x7f629819eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f629d7e9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6298114150 con 0x7f62981049a0 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f629d7e9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6298113fd0 con 0x7f6298109120 2024-09-19T06:45:48.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f629d7e9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62981142d0 con 0x7f6298103ff0 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62981049a0 0x7f629819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62981049a0 0x7f629819ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42732/0 (socket says 172.21.15.37:42732) 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 -- 172.21.15.37:0/660690480 learned_addr learned my addr 172.21.15.37:0/660690480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f62977fe640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6298109120 0x7f629819eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f6296ffd640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6298103ff0 0x7f629819a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 -- 172.21.15.37:0/660690480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6298103ff0 msgr2=0x7f629819a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6298103ff0 0x7f629819a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 -- 172.21.15.37:0/660690480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6298109120 msgr2=0x7f629819eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6298109120 0x7f629819eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 -- 172.21.15.37:0/660690480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629819f5c0 con 0x7f62981049a0 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.728+0000 7f628ffff640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62981049a0 0x7f629819ab50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f628000e9f0 tx=0x7f628000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.729+0000 7f6294ff9640 1 -- 172.21.15.37:0/660690480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628000cd80 con 0x7f62981049a0 2024-09-19T06:45:48.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.729+0000 7f6294ff9640 1 -- 172.21.15.37:0/660690480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6280010910 con 0x7f62981049a0 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.729+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629819f8b0 con 0x7f62981049a0 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.729+0000 7f6294ff9640 1 -- 172.21.15.37:0/660690480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6280018990 con 0x7f62981049a0 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.729+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62981a7190 con 0x7f62981049a0 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.731+0000 7f6294ff9640 1 -- 172.21.15.37:0/660690480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6280010ac0 con 0x7f62981049a0 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.731+0000 7f629fbc2740 1 Processor -- start 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.731+0000 7f629fbc2740 1 -- start start 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.731+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c05880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.731+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c10140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.731+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c12a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f629fbc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f6c7b6b2e0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f629fbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6c7b12ce0 con 0x55f6c7c054a0 2024-09-19T06:45:48.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f629fbc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f6c7b672a0 con 0x55f6c7c10680 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c10140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f62977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c05880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c10140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42746/0 (socket says 172.21.15.37:42746) 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 -- 172.21.15.37:0/3190784474 learned_addr learned my addr 172.21.15.37:0/3190784474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f628ffff640 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c12a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 -- 172.21.15.37:0/3190784474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 msgr2=0x55f6c7c12a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c12a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 -- 172.21.15.37:0/3190784474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 msgr2=0x55f6c7c05880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c05880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 -- 172.21.15.37:0/3190784474 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c7af7440 con 0x55f6c7c05dc0 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6294ff9640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f62840a33b0 0x7f62840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f62977fe640 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c05880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f62977fe640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f62840a33b0 0x7f62840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6296ffd640 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c10140 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6288009d20 tx=0x7f6288011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3a2644f465210c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f6294ff9640 1 -- 172.21.15.37:0/660690480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f62800cff80 con 0x7f62981049a0 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.732+0000 7f628d7fa640 1 -- 172.21.15.37:0/3190784474 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628801e070 con 0x55f6c7c05dc0 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f628d7fa640 1 -- 172.21.15.37:0/3190784474 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6288019b10 con 0x55f6c7c05dc0 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f628d7fa640 1 -- 172.21.15.37:0/3190784474 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f628800ebf0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 -- 172.21.15.37:0/3190784474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 msgr2=0x55f6c7c10140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f62977fe640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f62840a33b0 0x7f62840a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6290004af0 tx=0x7f6290004a80 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c10140 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f6288009d20 tx=0x7f6288011810 comp rx=0 tx=0).stop 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 -- 172.21.15.37:0/3190784474 shutdown_connections 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c12a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c10140 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 --2- 172.21.15.37:0/3190784474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c05880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 -- 172.21.15.37:0/3190784474 >> 172.21.15.37:0/3190784474 conn(0x55f6c7c01a30 msgr2=0x55f6c7c01e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:48.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 -- 172.21.15.37:0/3190784474 shutdown_connections 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.733+0000 7f629fbc2740 1 -- 172.21.15.37:0/3190784474 wait complete. 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 -- start start 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 Processor -- start 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c054a0 0x55f6c7c10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c10680 0x55f6c7c0d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f6c7c13250 con 0x55f6c7c054a0 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f6c7b26050 con 0x55f6c7c10680 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f6c7c133d0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f629fbc2740 10 client.0 _ll_register_callbacks cb 0x55f6c7b66160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c054a0 0x55f6c7c10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f62977fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c10680 0x55f6c7c0d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c054a0 0x55f6c7c10030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42754/0 (socket says 172.21.15.37:42754) 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f6296ffd640 1 -- 172.21.15.37:0/4194476042 learned_addr learned my addr 172.21.15.37:0/4194476042 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.734+0000 7f628ffff640 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f62977fe640 1 -- 172.21.15.37:0/4194476042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c05dc0 msgr2=0x55f6c7c0d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f62977fe640 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f62977fe640 1 -- 172.21.15.37:0/4194476042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c054a0 msgr2=0x55f6c7c10030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f62977fe640 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c054a0 0x55f6c7c10030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f62977fe640 1 -- 172.21.15.37:0/4194476042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c78dc3c0 con 0x55f6c7c10680 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f6296ffd640 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c054a0 0x55f6c7c10030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f62977fe640 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c10680 0x55f6c7c0d880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6290006010 tx=0x7f629002a8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1036bc16b24f3ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f627e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f627e7fc640 1 -- 172.21.15.37:0/4194476042 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6290031db0 con 0x55f6c7c10680 2024-09-19T06:45:48.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f627e7fc640 1 -- 172.21.15.37:0/4194476042 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f62900318e0 con 0x55f6c7c10680 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6288019b10 con 0x55f6c7c10680 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f627e7fc640 1 -- 172.21.15.37:0/4194476042 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6290004590 con 0x55f6c7c10680 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f629fbc2740 10 client.26108 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.735+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f6c7c1ec60 con 0x55f6c7c10680 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f627e7fc640 1 -- 172.21.15.37:0/4194476042 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6288019b10 con 0x55f6c7c10680 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f627e7fc640 1 -- 172.21.15.37:0/4194476042 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f62900305c0 con 0x55f6c7c10680 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f629fbc2740 10 client.26108 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f629fbc2740 10 client.26108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f629fbc2740 20 client.26108 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f629fbc2740 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c20070 0x55f6c7c22480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f629fbc2740 4 client.26108 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6c7b78850 con 0x55f6c7c20070 2024-09-19T06:45:48.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f6296ffd640 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c20070 0x55f6c7c22480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f6296ffd640 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c20070 0x55f6c7c22480 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75390ff815ac462c server_cookie=35934a7edbb33ff7 in_seq=0 out_seq=0 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.736+0000 7f627e7fc640 10 client.26108 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f627e7fc640 1 -- 172.21.15.37:0/4194476042 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f6c7b78850 con 0x55f6c7c20070 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f627e7fc640 10 client.26108 handle_command_reply: tid=0 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 client.26108 shutdown 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c10680 msgr2=0x55f6c7c0d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c10680 0x55f6c7c0d880 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6290006010 tx=0x7f629002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 shutdown_connections 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c20070 0x55f6c7c22480 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c10680 0x55f6c7c0d880 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f627e7fc640 0 client.26108 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 --2- 172.21.15.37:0/4194476042 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c054a0 0x55f6c7c10030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 >> 172.21.15.37:0/4194476042 conn(0x55f6c7c01a30 msgr2=0x55f6c7c07760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 shutdown_connections 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 1 -- 172.21.15.37:0/4194476042 wait complete. 2024-09-19T06:45:48.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.738+0000 7f629fbc2740 20 client.26108 trim_cache size 0 max 16384 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 Processor -- start 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 -- start start 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c0cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c27d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62981b23e0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62981afcb0 con 0x55f6c7c054a0 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f629fbc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62981adb00 con 0x55f6c7c10680 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.758+0000 7f628ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42766/0 (socket says 172.21.15.37:42766) 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c0cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 -- 172.21.15.37:0/2383524630 learned_addr learned my addr 172.21.15.37:0/2383524630 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f62977fe640 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c27d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 -- 172.21.15.37:0/2383524630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 msgr2=0x55f6c7c27d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c27d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 -- 172.21.15.37:0/2383524630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 msgr2=0x55f6c7c0cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c0cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 -- 172.21.15.37:0/2383524630 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6c7c0c290 con 0x55f6c7c05dc0 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f6296ffd640 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c0cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628ffff640 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0f250 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f628009c010 tx=0x7f6280018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3bf6f118b3b765 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:48.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628cff9640 1 -- 172.21.15.37:0/2383524630 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800a0610 con 0x55f6c7c05dc0 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628cff9640 1 -- 172.21.15.37:0/2383524630 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62800a4ed0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.759+0000 7f628cff9640 1 -- 172.21.15.37:0/2383524630 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800bc0b0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 -- 172.21.15.37:0/2383524630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 msgr2=0x55f6c7c0f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0f250 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f628009c010 tx=0x7f6280018420 comp rx=0 tx=0).stop 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 -- 172.21.15.37:0/2383524630 shutdown_connections 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c27d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c0f250 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2383524630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c0cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 -- 172.21.15.37:0/2383524630 >> 172.21.15.37:0/2383524630 conn(0x55f6c7c01a30 msgr2=0x55f6c7c0eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 -- 172.21.15.37:0/2383524630 shutdown_connections 2024-09-19T06:45:48.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 -- 172.21.15.37:0/2383524630 wait complete. 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.760+0000 7f629fbc2740 1 -- start start 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 1 Processor -- start 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c18d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c160d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c16610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f62981ae170 con 0x55f6c7c05dc0 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62981af5e0 con 0x55f6c7c054a0 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f62981ae840 con 0x55f6c7c10680 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c160d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f6296ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c18d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c160d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42782/0 (socket says 172.21.15.37:42782) 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 -- 172.21.15.37:0/2646471611 learned_addr learned my addr 172.21.15.37:0/2646471611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:48.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f629fbc2740 10 client.0 _ll_register_callbacks cb 0x7f6290009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f62977fe640 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c16610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 -- 172.21.15.37:0/2646471611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 msgr2=0x55f6c7c16610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c16610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 -- 172.21.15.37:0/2646471611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 msgr2=0x55f6c7c18d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c18d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.761+0000 7f628ffff640 1 -- 172.21.15.37:0/2646471611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6c7c05bf0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f628ffff640 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c160d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f62800cdcd0 tx=0x7f6280005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c02c63b443e808e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f627e7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f627e7fc640 1 -- 172.21.15.37:0/2646471611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62800b5060 con 0x55f6c7c05dc0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f627e7fc640 1 -- 172.21.15.37:0/2646471611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6280098980 con 0x55f6c7c05dc0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62800a4ed0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f627e7fc640 1 -- 172.21.15.37:0/2646471611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f62800a0170 con 0x55f6c7c05dc0 2024-09-19T06:45:48.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 10 client.16620 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f6c7c8ab90 con 0x55f6c7c05dc0 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f627e7fc640 1 -- 172.21.15.37:0/2646471611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f62800a4ed0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f627e7fc640 1 -- 172.21.15.37:0/2646471611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6280018ba0 con 0x55f6c7c05dc0 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 10 client.16620 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 10 client.16620 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 20 client.16620 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c8b1e0 0x55f6c7c8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 4 client.16620 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.762+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f6c7b72d60 con 0x55f6c7c8b1e0 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.763+0000 7f6296ffd640 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c8b1e0 0x55f6c7c8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.763+0000 7f627e7fc640 10 client.16620 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:48.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.763+0000 7f6296ffd640 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c8b1e0 0x55f6c7c8d600 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4946b8665ecf12e4 server_cookie=89e37f4945e77f15 in_seq=0 out_seq=0 2024-09-19T06:45:48.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.764+0000 7f627e7fc640 1 -- 172.21.15.37:0/2646471611 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f6c7b72d60 con 0x55f6c7c8b1e0 2024-09-19T06:45:48.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.764+0000 7f627e7fc640 10 client.16620 handle_command_reply: tid=0 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.764+0000 7f629fbc2740 1 client.16620 shutdown 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.764+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 msgr2=0x55f6c7c160d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.764+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c160d0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f62800cdcd0 tx=0x7f6280005c70 comp rx=0 tx=0).stop 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 shutdown_connections 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f6c7c8b1e0 0x55f6c7c8d600 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f6c7c10680 0x55f6c7c16610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f6c7c05dc0 0x55f6c7c160d0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 --2- 172.21.15.37:0/2646471611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f6c7c054a0 0x55f6c7c18d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f627e7fc640 0 client.16620 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 >> 172.21.15.37:0/2646471611 conn(0x55f6c7c01a30 msgr2=0x55f6c7c0f360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 shutdown_connections 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 1 -- 172.21.15.37:0/2646471611 wait complete. 2024-09-19T06:45:48.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629fbc2740 20 client.16620 trim_cache size 0 max 16384 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: "e9f8aecb-470e-4d02-93fb-a0cd6278e9e0": { 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "e9f8aecb-470e-4d02-93fb-a0cd6278e9e0", 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:48.773 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:45:48.774 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:45:48.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f62840a33b0 msgr2=0x7f62840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f62840a33b0 0x7f62840a5870 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6290004af0 tx=0x7f6290004a80 comp rx=0 tx=0).stop 2024-09-19T06:45:48.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62981049a0 msgr2=0x7f629819ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:48.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62981049a0 0x7f629819ab50 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f628000e9f0 tx=0x7f628000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 shutdown_connections 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6298109120 0x7f629819eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f62840a33b0 0x7f62840a5870 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f62981049a0 0x7f629819ab50 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 --2- 172.21.15.37:0/660690480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6298103ff0 0x7f629819a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.765+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 >> 172.21.15.37:0/660690480 conn(0x7f62980ffab0 msgr2=0x7f6298100d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.766+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 shutdown_connections 2024-09-19T06:45:48.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:45:48.766+0000 7f629d7e9640 1 -- 172.21.15.37:0/660690480 wait complete. 2024-09-19T06:45:49.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 266514 14.00 MB/sec execute 16 sec latency 40.605 ms 2024-09-19T06:45:49.617 INFO:tasks.cephfs.filesystem:scrub status for tag:e9f8aecb-470e-4d02-93fb-a0cd6278e9e0 - {'path': '/', 'tag': 'e9f8aecb-470e-4d02-93fb-a0cd6278e9e0', 'options': 'recursive,force'} 2024-09-19T06:45:50.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:49 smithi096 ceph-mon[35004]: pgmap v125: 97 pgs: 97 active+clean; 40 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2024-09-19T06:45:50.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:49 smithi037 ceph-mon[31211]: pgmap v125: 97 pgs: 97 active+clean; 40 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2024-09-19T06:45:50.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:49 smithi151 ceph-mon[35236]: pgmap v125: 97 pgs: 97 active+clean; 40 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 5.4 MiB/s wr, 541 op/s 2024-09-19T06:45:50.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 269033 13.92 MB/sec execute 17 sec latency 173.058 ms 2024-09-19T06:45:51.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 271412 13.81 MB/sec execute 18 sec latency 35.101 ms 2024-09-19T06:45:52.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:51 smithi096 ceph-mon[35004]: pgmap v126: 97 pgs: 97 active+clean; 40 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.7 MiB/s wr, 407 op/s 2024-09-19T06:45:52.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:51 smithi037 ceph-mon[31211]: pgmap v126: 97 pgs: 97 active+clean; 40 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.7 MiB/s wr, 407 op/s 2024-09-19T06:45:52.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:51 smithi151 ceph-mon[35236]: pgmap v126: 97 pgs: 97 active+clean; 40 MiB data, 850 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 MiB/s rd, 3.7 MiB/s wr, 407 op/s 2024-09-19T06:45:52.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 274075 13.85 MB/sec execute 19 sec latency 18.647 ms 2024-09-19T06:45:53.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 277069 14.07 MB/sec execute 20 sec latency 10.263 ms 2024-09-19T06:45:54.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:53 smithi096 ceph-mon[35004]: pgmap v127: 97 pgs: 97 active+clean; 44 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.4 MiB/s wr, 462 op/s 2024-09-19T06:45:54.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:53 smithi037 ceph-mon[31211]: pgmap v127: 97 pgs: 97 active+clean; 44 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.4 MiB/s wr, 462 op/s 2024-09-19T06:45:54.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:53 smithi151 ceph-mon[35236]: pgmap v127: 97 pgs: 97 active+clean; 44 MiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 4.4 MiB/s wr, 462 op/s 2024-09-19T06:45:54.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 280230 14.23 MB/sec execute 21 sec latency 11.524 ms 2024-09-19T06:45:55.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 282803 14.02 MB/sec execute 22 sec latency 13.236 ms 2024-09-19T06:45:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 -- 172.21.15.37:0/1971910558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab4102cc0 msgr2=0x7f7ab41030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 --2- 172.21.15.37:0/1971910558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41030a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8009900 tx=0x7f7aa802a480 comp rx=0 tx=0).stop 2024-09-19T06:45:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 -- 172.21.15.37:0/1971910558 shutdown_connections 2024-09-19T06:45:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 --2- 172.21.15.37:0/1971910558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab410c110 0x7f7ab410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 --2- 172.21.15.37:0/1971910558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41035e0 0x7f7ab410bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 --2- 172.21.15.37:0/1971910558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41030a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 -- 172.21.15.37:0/1971910558 >> 172.21.15.37:0/1971910558 conn(0x7f7ab40fc6b0 msgr2=0x7f7ab40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 -- 172.21.15.37:0/1971910558 shutdown_connections 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.772+0000 7f7abb21e640 1 -- 172.21.15.37:0/1971910558 wait complete. 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 Processor -- start 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 -- start start 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41035e0 0x7f7ab4198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab410c110 0x7f7ab419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111ef0 con 0x7f7ab410c110 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111d70 con 0x7f7ab4102cc0 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7abb21e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4112070 con 0x7f7ab41035e0 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7ab97bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab410c110 0x7f7ab419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.776 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7ab8fba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7ab97bb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab410c110 0x7f7ab419cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46582/0 (socket says 172.21.15.37:46582) 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7ab8fba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47664/0 (socket says 172.21.15.37:47664) 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7ab97bb640 1 -- 172.21.15.37:0/2589262304 learned_addr learned my addr 172.21.15.37:0/2589262304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.773+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41035e0 0x7f7ab4198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab97bb640 1 -- 172.21.15.37:0/2589262304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41035e0 msgr2=0x7f7ab4198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab97bb640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41035e0 0x7f7ab4198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab97bb640 1 -- 172.21.15.37:0/2589262304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102cc0 msgr2=0x7f7ab41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab97bb640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab97bb640 1 -- 172.21.15.37:0/2589262304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab419d3d0 con 0x7f7ab410c110 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab97bb640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab410c110 0x7f7ab419cca0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00efd0 tx=0x7f7aac00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:55.777 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab1ffb640 1 -- 172.21.15.37:0/2589262304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac003b70 con 0x7f7ab410c110 2024-09-19T06:45:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab1ffb640 1 -- 172.21.15.37:0/2589262304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aac003d10 con 0x7f7ab410c110 2024-09-19T06:45:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab419d6c0 con 0x7f7ab410c110 2024-09-19T06:45:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7ab1ffb640 1 -- 172.21.15.37:0/2589262304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0109f0 con 0x7f7ab410c110 2024-09-19T06:45:55.778 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.774+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab41a4f50 con 0x7f7ab410c110 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7ab1ffb640 1 -- 172.21.15.37:0/2589262304 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7aac004020 con 0x7f7ab410c110 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 Processor -- start 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 -- start start 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a7ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7e310 0x55c740a8cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740a8f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c7409e22f0 con 0x55c740a7e310 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c74098a270 con 0x55c740a7d9f0 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7abe08c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c7409de2b0 con 0x55c740a8d0c0 2024-09-19T06:45:55.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7ab1ffb640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7aa40a3460 0x7f7aa40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7ab8fba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7e310 0x55c740a8cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a7ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7ab8fba640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7e310 0x55c740a8cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46592/0 (socket says 172.21.15.37:46592) 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7ab8fba640 1 -- 172.21.15.37:0/3589086379 learned_addr learned my addr 172.21.15.37:0/3589086379 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.777+0000 7f7ab97bb640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7aa40a3460 0x7f7aa40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740a8f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab1ffb640 1 -- 172.21.15.37:0/2589262304 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7aac0d0140 con 0x7f7ab410c110 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab3fff640 1 -- 172.21.15.37:0/3589086379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 msgr2=0x55c740a8f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740a8f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab3fff640 1 -- 172.21.15.37:0/3589086379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7e310 msgr2=0x55c740a8cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7e310 0x55c740a8cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab3fff640 1 -- 172.21.15.37:0/3589086379 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c740989340 con 0x55c740a7d9f0 2024-09-19T06:45:55.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7e310 0x55c740a8cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a7ddd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ab410d4c0 tx=0x7f7aa0009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=763c4bea0848712a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7ab97bb640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7aa40a3460 0x7f7aa40a5920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7aac09c0c0 tx=0x7f7aac004920 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7a9effd640 1 -- 172.21.15.37:0/3589086379 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0004050 con 0x55c740a7d9f0 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7a9effd640 1 -- 172.21.15.37:0/3589086379 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0005500 con 0x55c740a7d9f0 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.778+0000 7f7a9effd640 1 -- 172.21.15.37:0/3589086379 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00035f0 con 0x55c740a7d9f0 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 -- 172.21.15.37:0/3589086379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 msgr2=0x55c740a7ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a7ddd0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ab410d4c0 tx=0x7f7aa0009930 comp rx=0 tx=0).stop 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 -- 172.21.15.37:0/3589086379 shutdown_connections 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740a8f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7e310 0x55c740a8cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3589086379 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a7ddd0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 -- 172.21.15.37:0/3589086379 >> 172.21.15.37:0/3589086379 conn(0x55c740a7a060 msgr2=0x55c740a7a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:55.786 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 -- 172.21.15.37:0/3589086379 shutdown_connections 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 -- 172.21.15.37:0/3589086379 wait complete. 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 -- start start 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.779+0000 7f7abe08c740 1 Processor -- start 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a8a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a8d0c0 0x55c740a8aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7abe08c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c740a8fd20 con 0x55c740a8d0c0 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7abe08c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c74096b290 con 0x55c740a7d9f0 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7abe08c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c740a8fea0 con 0x55c740a7e310 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7abe08c740 10 client.0 _ll_register_callbacks cb 0x55c74099c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:55.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a8d0c0 0x55c740a8aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a8d0c0 0x55c740a8aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46608/0 (socket says 172.21.15.37:46608) 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 -- 172.21.15.37:0/3675070079 learned_addr learned my addr 172.21.15.37:0/3675070079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a8a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 -- 172.21.15.37:0/3675070079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 msgr2=0x55c740a8a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a8a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 -- 172.21.15.37:0/3675070079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a8d0c0 msgr2=0x55c740a8aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a8d0c0 0x55c740a8aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 -- 172.21.15.37:0/3675070079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7406f5210 con 0x55c740a7d9f0 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a8a030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0004980 tx=0x7f7aa001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=390c74cc5d5f266e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.780+0000 7f7a93fff640 1 -- 172.21.15.37:0/3675070079 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0028040 con 0x55c740a7d9f0 2024-09-19T06:45:55.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7a93fff640 1 -- 172.21.15.37:0/3675070079 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7aa001e460 con 0x55c740a7d9f0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa0005500 con 0x55c740a7d9f0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7a93fff640 1 -- 172.21.15.37:0/3675070079 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7aa001e660 con 0x55c740a7d9f0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 10 client.26126 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c7407c1620 con 0x55c740a7d9f0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7a93fff640 1 -- 172.21.15.37:0/3675070079 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0005500 con 0x55c740a7d9f0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7a93fff640 1 -- 172.21.15.37:0/3675070079 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7aa000f9f0 con 0x55c740a7d9f0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 10 client.26126 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 10 client.26126 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 20 client.26126 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740a9cfa0 0x55c740a9f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 4 client.26126 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7409efa20 con 0x55c740a9cfa0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.781+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740a9cfa0 0x55c740a9f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.789 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.782+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740a9cfa0 0x55c740a9f390 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=852eca5f7117b243 server_cookie=933e4ac170450303 in_seq=0 out_seq=0 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.782+0000 7f7a93fff640 10 client.26126 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.782+0000 7f7a93fff640 1 -- 172.21.15.37:0/3675070079 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7409efa20 con 0x55c740a9cfa0 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7a93fff640 10 client.26126 handle_command_reply: tid=0 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 client.26126 shutdown 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 msgr2=0x55c740a8a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a8a030 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0004980 tx=0x7f7aa001a040 comp rx=0 tx=0).stop 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 shutdown_connections 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740a9cfa0 0x55c740a9f390 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a8d0c0 0x55c740a8aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a8a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7a93fff640 0 client.26126 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:55.790 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 --2- 172.21.15.37:0/3675070079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7d9f0 0x55c740a8a030 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 >> 172.21.15.37:0/3675070079 conn(0x55c740a7a060 msgr2=0x55c740a7fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 shutdown_connections 2024-09-19T06:45:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 1 -- 172.21.15.37:0/3675070079 wait complete. 2024-09-19T06:45:55.791 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.783+0000 7f7abe08c740 20 client.26126 trim_cache size 0 max 16384 2024-09-19T06:45:55.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.799+0000 7f7abe08c740 1 Processor -- start 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7abe08c740 1 -- start start 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a87270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a877b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a8d0c0 0x55c740a8b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7abe08c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41adf60 con 0x55c740a7d9f0 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7abe08c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41ab830 con 0x55c740a8d0c0 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7abe08c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41a9680 con 0x55c740a7e310 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a87270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a87270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46612/0 (socket says 172.21.15.37:46612) 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 -- 172.21.15.37:0/2286763758 learned_addr learned my addr 172.21.15.37:0/2286763758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:55.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a877b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a8d0c0 0x55c740a8b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 -- 172.21.15.37:0/2286763758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 msgr2=0x55c740a877b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a877b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 -- 172.21.15.37:0/2286763758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a8d0c0 msgr2=0x55c740a8b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a8d0c0 0x55c740a8b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 -- 172.21.15.37:0/2286763758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c740aa5570 con 0x55c740a7d9f0 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.800+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a87270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e690 tx=0x7f7aa0005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f08107298bfa1754 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.801+0000 7f7a9e7fc640 1 -- 172.21.15.37:0/2286763758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000e3b0 con 0x55c740a7d9f0 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.801+0000 7f7a9e7fc640 1 -- 172.21.15.37:0/2286763758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0023030 con 0x55c740a7d9f0 2024-09-19T06:45:55.803 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.801+0000 7f7a9e7fc640 1 -- 172.21.15.37:0/2286763758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000e7b0 con 0x55c740a7d9f0 2024-09-19T06:45:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.801+0000 7f7abe08c740 1 -- 172.21.15.37:0/2286763758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 msgr2=0x55c740a87270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.801+0000 7f7abe08c740 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a87270 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000e690 tx=0x7f7aa0005970 comp rx=0 tx=0).stop 2024-09-19T06:45:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 -- 172.21.15.37:0/2286763758 shutdown_connections 2024-09-19T06:45:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a8d0c0 0x55c740a8b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a7e310 0x55c740a877b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.807 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 --2- 172.21.15.37:0/2286763758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a87270 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 -- 172.21.15.37:0/2286763758 >> 172.21.15.37:0/2286763758 conn(0x55c740a7a060 msgr2=0x55c740a89100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 -- 172.21.15.37:0/2286763758 shutdown_connections 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 -- 172.21.15.37:0/2286763758 wait complete. 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 -- start start 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.802+0000 7f7abe08c740 1 Processor -- start 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a9fc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7e310 0x55c740aa0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7abe08c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740afe890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7abe08c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41a9cf0 con 0x55c740a7d9f0 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7abe08c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41ab160 con 0x55c740a7e310 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7abe08c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41aa3c0 con 0x55c740a8d0c0 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7abe08c740 10 client.0 _ll_register_callbacks cb 0x7f7aa0009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:45:55.808 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a9fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab8fba640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7e310 0x55c740aa0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a9fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46626/0 (socket says 172.21.15.37:46626) 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 -- 172.21.15.37:0/1084885040 learned_addr learned my addr 172.21.15.37:0/1084885040 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740afe890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 -- 172.21.15.37:0/1084885040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 msgr2=0x55c740afe890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740afe890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 -- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7e310 msgr2=0x55c740aa0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7e310 0x55c740aa0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab3fff640 1 -- 172.21.15.37:0/1084885040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c740afef30 con 0x55c740a7d9f0 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.803+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7e310 0x55c740aa0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7ab3fff640 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a9fc40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009380 tx=0x7f7aa0009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395d213b3cef977 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:45:55.809 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7a93fff640 1 -- 172.21.15.37:0/1084885040 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0042040 con 0x55c740a7d9f0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7a93fff640 1 -- 172.21.15.37:0/1084885040 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7aa001e800 con 0x55c740a7d9f0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa0023030 con 0x55c740a7d9f0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7a93fff640 1 -- 172.21.15.37:0/1084885040 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7aa000e3b0 con 0x55c740a7d9f0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 10 client.16638 fetch_fsmap learned FSMap version 31 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c740b074f0 con 0x55c740a7d9f0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7a93fff640 1 -- 172.21.15.37:0/1084885040 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7aa0023030 con 0x55c740a7d9f0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7a93fff640 1 -- 172.21.15.37:0/1084885040 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7aa000f3c0 con 0x55c740a7d9f0 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 10 client.16638 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 10 client.16638 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 20 client.16638 populate_metadata read hostname 'smithi037' 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740b07aa0 0x55c740b09e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 4 client.16638 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:45:55.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.804+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7409e9d70 con 0x55c740b07aa0 2024-09-19T06:45:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.805+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740b07aa0 0x55c740b09e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:45:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.805+0000 7f7a93fff640 10 client.16638 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:55.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:55.805+0000 7f7ab8fba640 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740b07aa0 0x55c740b09e90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9461c96e6e5a2930 server_cookie=ac1ebef07c9b00c3 in_seq=0 out_seq=0 2024-09-19T06:45:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.003+0000 7f7a93fff640 1 -- 172.21.15.37:0/1084885040 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7409e9d70 con 0x55c740b07aa0 2024-09-19T06:45:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.003+0000 7f7a93fff640 10 client.16638 handle_command_reply: tid=0 2024-09-19T06:45:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.003+0000 7f7abe08c740 1 client.16638 shutdown 2024-09-19T06:45:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 msgr2=0x55c740a9fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a9fc40 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009380 tx=0x7f7aa0009d80 comp rx=0 tx=0).stop 2024-09-19T06:45:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 shutdown_connections 2024-09-19T06:45:56.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c740b07aa0 0x55c740b09e90 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c740a8d0c0 0x55c740afe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7a93fff640 0 client.16638 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c740a7e310 0x55c740aa0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 --2- 172.21.15.37:0/1084885040 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c740a7d9f0 0x55c740a9fc40 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 >> 172.21.15.37:0/1084885040 conn(0x55c740a7a060 msgr2=0x55c740a98130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 shutdown_connections 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 1 -- 172.21.15.37:0/1084885040 wait complete. 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.004+0000 7f7abe08c740 20 client.16638 trim_cache size 0 max 16384 2024-09-19T06:45:56.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:45:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:45:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:45:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:45:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7aa40a3460 msgr2=0x7f7aa40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7aa40a3460 0x7f7aa40a5920 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f7aac09c0c0 tx=0x7f7aac004920 comp rx=0 tx=0).stop 2024-09-19T06:45:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab410c110 msgr2=0x7f7ab419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:45:56.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab410c110 0x7f7ab419cca0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f7aac00efd0 tx=0x7f7aac00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 shutdown_connections 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7aa40a3460 0x7f7aa40a5920 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7ab410c110 0x7f7ab419cca0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7ab41035e0 0x7f7ab4198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 --2- 172.21.15.37:0/2589262304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ab4102cc0 0x7f7ab41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 >> 172.21.15.37:0/2589262304 conn(0x7f7ab40fc6b0 msgr2=0x7f7ab40fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 shutdown_connections 2024-09-19T06:45:56.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:45:56.005+0000 7f7abb21e640 1 -- 172.21.15.37:0/2589262304 wait complete. 2024-09-19T06:45:56.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:55 smithi096 ceph-mon[35004]: pgmap v128: 97 pgs: 97 active+clean; 48 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-19T06:45:56.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:55 smithi037 ceph-mon[31211]: pgmap v128: 97 pgs: 97 active+clean; 48 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-19T06:45:56.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:55 smithi151 ceph-mon[35236]: pgmap v128: 97 pgs: 97 active+clean; 48 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 3.4 MiB/s wr, 378 op/s 2024-09-19T06:45:56.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 284765 13.94 MB/sec execute 23 sec latency 30.281 ms 2024-09-19T06:45:57.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:56 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:45:57.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:56 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:45:57.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:56 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:45:57.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 287725 14.11 MB/sec execute 24 sec latency 37.262 ms 2024-09-19T06:45:58.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:57 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:45:58.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:57 smithi096 ceph-mon[35004]: pgmap v129: 97 pgs: 97 active+clean; 48 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2024-09-19T06:45:58.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:57 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:45:58.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:57 smithi037 ceph-mon[31211]: pgmap v129: 97 pgs: 97 active+clean; 48 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2024-09-19T06:45:58.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:57 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:45:58.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:57 smithi151 ceph-mon[35236]: pgmap v129: 97 pgs: 97 active+clean; 48 MiB data, 873 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 3.0 MiB/s wr, 330 op/s 2024-09-19T06:45:58.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 290829 14.25 MB/sec execute 25 sec latency 20.520 ms 2024-09-19T06:45:59.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 293270 14.03 MB/sec execute 26 sec latency 26.175 ms 2024-09-19T06:46:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:59 smithi096 ceph-mon[35004]: pgmap v130: 97 pgs: 97 active+clean; 39 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 571 op/s 2024-09-19T06:46:00.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:45:59 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:00.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:59 smithi037 ceph-mon[31211]: pgmap v130: 97 pgs: 97 active+clean; 39 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 571 op/s 2024-09-19T06:46:00.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:45:59 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:00.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:59 smithi151 ceph-mon[35236]: pgmap v130: 97 pgs: 97 active+clean; 39 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 4.0 MiB/s wr, 571 op/s 2024-09-19T06:46:00.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:45:59 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:46:00.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 295493 14.03 MB/sec execute 27 sec latency 14.383 ms 2024-09-19T06:46:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:00 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:00 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:01.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:00 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:46:01.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:00 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:01.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:00 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:01.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:00 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:46:01.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:00 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:46:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:00 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:46:01.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:00 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:46:01.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 298107 14.10 MB/sec execute 28 sec latency 22.572 ms 2024-09-19T06:46:02.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 300533 13.93 MB/sec execute 29 sec latency 13.450 ms 2024-09-19T06:46:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:01 smithi096 ceph-mon[35004]: pgmap v131: 97 pgs: 97 active+clean; 39 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.2 MiB/s wr, 353 op/s 2024-09-19T06:46:02.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:01 smithi037 ceph-mon[31211]: pgmap v131: 97 pgs: 97 active+clean; 39 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.2 MiB/s wr, 353 op/s 2024-09-19T06:46:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:01 smithi151 ceph-mon[35236]: pgmap v131: 97 pgs: 97 active+clean; 39 MiB data, 905 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 2.2 MiB/s wr, 353 op/s 2024-09-19T06:46:03.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 302932 14.01 MB/sec execute 30 sec latency 15.286 ms 2024-09-19T06:46:04.188 INFO:tasks.quiescer.fs.[cephfs]:Total duration (59.6) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:46:04.188 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T06:46:04.188 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:04.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 305732 14.02 MB/sec execute 31 sec latency 11.364 ms 2024-09-19T06:46:04.213 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:03 smithi037 ceph-mon[31211]: pgmap v132: 97 pgs: 97 active+clean; 46 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.0 MiB/s wr, 468 op/s 2024-09-19T06:46:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:03 smithi096 ceph-mon[35004]: pgmap v132: 97 pgs: 97 active+clean; 46 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.0 MiB/s wr, 468 op/s 2024-09-19T06:46:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:03 smithi151 ceph-mon[35236]: pgmap v132: 97 pgs: 97 active+clean; 46 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.0 MiB/s wr, 468 op/s 2024-09-19T06:46:04.466 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 -- 172.21.15.37:0/4244209137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54101400 msgr2=0x7f9c5410d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/4244209137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54101400 0x7f9c5410d930 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009cd0 tx=0x7f9c4c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 -- 172.21.15.37:0/4244209137 shutdown_connections 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/4244209137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54101400 0x7f9c5410d930 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/4244209137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c54100a60 0x7f9c54100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/4244209137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54106a60 0x7f9c54106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 -- 172.21.15.37:0/4244209137 >> 172.21.15.37:0/4244209137 conn(0x7f9c540fc780 msgr2=0x7f9c540feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 -- 172.21.15.37:0/4244209137 shutdown_connections 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.018+0000 7f9c58bd0640 1 -- 172.21.15.37:0/4244209137 wait complete. 2024-09-19T06:46:05.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 Processor -- start 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 -- start start 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54100a60 0x7f9c540fce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c54101400 0x7f9c540fd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54106a60 0x7f9c540fd890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54110e60 con 0x7f9c54100a60 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54110ce0 con 0x7f9c54101400 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c58bd0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54110fe0 con 0x7f9c54106a60 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.019+0000 7f9c5259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54100a60 0x7f9c540fce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54100a60 0x7f9c540fce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48794/0 (socket says 172.21.15.37:48794) 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 -- 172.21.15.37:0/2441155611 learned_addr learned my addr 172.21.15.37:0/2441155611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:05.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c51d9b640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c54101400 0x7f9c540fd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c52d9d640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54106a60 0x7f9c540fd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 -- 172.21.15.37:0/2441155611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54106a60 msgr2=0x7f9c540fd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54106a60 0x7f9c540fd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 -- 172.21.15.37:0/2441155611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c54101400 msgr2=0x7f9c540fd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c54101400 0x7f9c540fd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 -- 172.21.15.37:0/2441155611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c541a68d0 con 0x7f9c54100a60 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c5259c640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54100a60 0x7f9c540fce10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d770 tx=0x7f9c4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c437fe640 1 -- 172.21.15.37:0/2441155611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44017070 con 0x7f9c54100a60 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c437fe640 1 -- 172.21.15.37:0/2441155611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c44005500 con 0x7f9c54100a60 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c541a6bc0 con 0x7f9c54100a60 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c437fe640 1 -- 172.21.15.37:0/2441155611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004e00 con 0x7f9c54100a60 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.020+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541a70b0 con 0x7f9c54100a60 2024-09-19T06:46:05.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.021+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c54100ec0 con 0x7f9c54100a60 2024-09-19T06:46:05.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.023+0000 7f9c437fe640 1 -- 172.21.15.37:0/2441155611 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9c44004fa0 con 0x7f9c54100a60 2024-09-19T06:46:05.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.023+0000 7f9c437fe640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c3c0a33b0 0x7f9c3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:05.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.023+0000 7f9c437fe640 1 -- 172.21.15.37:0/2441155611 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9c440cf6a0 con 0x7f9c54100a60 2024-09-19T06:46:05.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.023+0000 7f9c51d9b640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c3c0a33b0 0x7f9c3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:05.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.024+0000 7f9c51d9b640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c3c0a33b0 0x7f9c3c0a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c540fe8a0 tx=0x7f9c48005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:05.105 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.028+0000 7f9c437fe640 1 -- 172.21.15.37:0/2441155611 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9c4409b9d0 con 0x7f9c54100a60 2024-09-19T06:46:05.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 308414 14.03 MB/sec execute 32 sec latency 51.685 ms 2024-09-19T06:46:05.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.250+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9c54109a50 con 0x7f9c54100a60 2024-09-19T06:46:05.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.250+0000 7f9c437fe640 1 -- 172.21.15.37:0/2441155611 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f9c4409fcd0 con 0x7f9c54100a60 2024-09-19T06:46:05.251 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:46:05.252 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-19T06:45:42:465959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:41.617479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:46:05.252 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-19T06:46:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.253+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c3c0a33b0 msgr2=0x7f9c3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.253+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c3c0a33b0 0x7f9c3c0a5870 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9c540fe8a0 tx=0x7f9c48005f90 comp rx=0 tx=0).stop 2024-09-19T06:46:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.253+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54100a60 msgr2=0x7f9c540fce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.253+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54100a60 0x7f9c540fce10 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9c4400d770 tx=0x7f9c4400dc30 comp rx=0 tx=0).stop 2024-09-19T06:46:05.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.253+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 shutdown_connections 2024-09-19T06:46:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.253+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54106a60 0x7f9c540fd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.253+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c54101400 0x7f9c540fd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.254+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c3c0a33b0 0x7f9c3c0a5870 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.254+0000 7f9c58bd0640 1 --2- 172.21.15.37:0/2441155611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54100a60 0x7f9c540fce10 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.254+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 >> 172.21.15.37:0/2441155611 conn(0x7f9c540fc780 msgr2=0x7f9c5406aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.254+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 shutdown_connections 2024-09-19T06:46:05.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:05.254+0000 7f9c58bd0640 1 -- 172.21.15.37:0/2441155611 wait complete. 2024-09-19T06:46:06.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 310877 13.92 MB/sec execute 33 sec latency 37.284 ms 2024-09-19T06:46:06.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:05 smithi096 ceph-mon[35004]: pgmap v133: 97 pgs: 97 active+clean; 47 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-09-19T06:46:06.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:05 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2441155611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:06.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:05 smithi037 ceph-mon[31211]: pgmap v133: 97 pgs: 97 active+clean; 47 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-09-19T06:46:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:05 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2441155611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:05 smithi151 ceph-mon[35236]: pgmap v133: 97 pgs: 97 active+clean; 47 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 MiB/s rd, 2.5 MiB/s wr, 434 op/s 2024-09-19T06:46:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:05 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2441155611' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:06.479 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T06:46:06.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T06:46:06.792 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:07.198 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313030 13.92 MB/sec execute 34 sec latency 20.282 ms 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 -- 172.21.15.37:0/1241073340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff301036e0 msgr2=0x7eff30107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 --2- 172.21.15.37:0/1241073340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff301036e0 0x7eff30107b90 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009890 tx=0x7eff1c02a380 comp rx=0 tx=0).stop 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 -- 172.21.15.37:0/1241073340 shutdown_connections 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 --2- 172.21.15.37:0/1241073340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff301082a0 0x7eff3010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 --2- 172.21.15.37:0/1241073340 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff301036e0 0x7eff30107b90 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 --2- 172.21.15.37:0/1241073340 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff30102d30 0x7eff30103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 -- 172.21.15.37:0/1241073340 >> 172.21.15.37:0/1241073340 conn(0x7eff300fc7d0 msgr2=0x7eff300febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 -- 172.21.15.37:0/1241073340 shutdown_connections 2024-09-19T06:46:07.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 -- 172.21.15.37:0/1241073340 wait complete. 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.262+0000 7eff36c9e640 1 Processor -- start 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff36c9e640 1 -- start start 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff36c9e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff30102d30 0x7eff301a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff36c9e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff301036e0 0x7eff301a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff36c9e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff301082a0 0x7eff3019a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff36c9e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff301120a0 con 0x7eff30102d30 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff36c9e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff30111f20 con 0x7eff301036e0 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff36c9e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff30112220 con 0x7eff301082a0 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff30102d30 0x7eff301a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff30102d30 0x7eff301a0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48816/0 (socket says 172.21.15.37:48816) 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 -- 172.21.15.37:0/3589626406 learned_addr learned my addr 172.21.15.37:0/3589626406 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff2ffff640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff301036e0 0x7eff301a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff3523b640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff301082a0 0x7eff3019a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 -- 172.21.15.37:0/3589626406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff301082a0 msgr2=0x7eff3019a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff301082a0 0x7eff3019a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 -- 172.21.15.37:0/3589626406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff301036e0 msgr2=0x7eff301a0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff301036e0 0x7eff301a0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff34a3a640 1 -- 172.21.15.37:0/3589626406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff3019ad40 con 0x7eff30102d30 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.263+0000 7eff2ffff640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff301036e0 0x7eff301a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.264+0000 7eff34a3a640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff30102d30 0x7eff301a0450 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e9f0 tx=0x7eff2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.264+0000 7eff2dffb640 1 -- 172.21.15.37:0/3589626406 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2400cd80 con 0x7eff30102d30 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.264+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff3019b030 con 0x7eff30102d30 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.264+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff301a7320 con 0x7eff30102d30 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.264+0000 7eff2dffb640 1 -- 172.21.15.37:0/3589626406 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff24005500 con 0x7eff30102d30 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.264+0000 7eff2dffb640 1 -- 172.21.15.37:0/3589626406 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff24010690 con 0x7eff30102d30 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff2dffb640 1 -- 172.21.15.37:0/3589626406 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7eff240046c0 con 0x7eff30102d30 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 Processor -- start 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 -- start start 2024-09-19T06:46:07.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00d2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00dfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570f00382a0 con 0x5570f00dd940 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570effdfb10 con 0x5570f00d2760 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff39077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570f0034260 con 0x5570f00d3080 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.267+0000 7eff3523b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00d2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00d2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:46154/0 (socket says 172.21.15.37:46154) 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 -- 172.21.15.37:0/506648121 learned_addr learned my addr 172.21.15.37:0/506648121 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff2ffff640 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00dfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff34a3a640 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 -- 172.21.15.37:0/506648121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 msgr2=0x5570f00dd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 -- 172.21.15.37:0/506648121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 msgr2=0x5570f00dfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00dfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 -- 172.21.15.37:0/506648121 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570efda9b90 con 0x5570f00d2760 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff2dffb640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7eff200a3430 0x7eff200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00d2b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff2800f400 tx=0x7eff2800f8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdc400dfa71db098 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff3523b640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7eff200a3430 0x7eff200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff2dffb640 1 -- 172.21.15.37:0/3589626406 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7eff240d0080 con 0x7eff30102d30 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.268+0000 7eff1affd640 1 -- 172.21.15.37:0/506648121 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2800c920 con 0x5570f00d2760 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff1affd640 1 -- 172.21.15.37:0/506648121 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2801bac0 con 0x5570f00d2760 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff1affd640 1 -- 172.21.15.37:0/506648121 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28019c20 con 0x5570f00d2760 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 -- 172.21.15.37:0/506648121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 msgr2=0x5570f00d2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00d2b40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7eff2800f400 tx=0x7eff2800f8c0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff3523b640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7eff200a3430 0x7eff200a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff28020000 tx=0x7eff28023000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 -- 172.21.15.37:0/506648121 shutdown_connections 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00dfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 --2- 172.21.15.37:0/506648121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00d2b40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 -- 172.21.15.37:0/506648121 >> 172.21.15.37:0/506648121 conn(0x5570f00cecf0 msgr2=0x5570f00cf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 -- 172.21.15.37:0/506648121 shutdown_connections 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.269+0000 7eff39077740 1 -- 172.21.15.37:0/506648121 wait complete. 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.270+0000 7eff39077740 1 -- start start 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.270+0000 7eff39077740 1 Processor -- start 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.270+0000 7eff39077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d2760 0x5570f00da7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.270+0000 7eff39077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff39077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00db220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff39077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570f00e0510 con 0x5570f00d2760 2024-09-19T06:46:07.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff39077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570effc06e0 con 0x5570f00dd940 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff39077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570f00e0690 con 0x5570f00d3080 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff39077740 10 client.0 _ll_register_callbacks cb 0x5570effe4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff34a3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d2760 0x5570f00da7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00db220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00db220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:46160/0 (socket says 172.21.15.37:46160) 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff34a3a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d2760 0x5570f00da7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48840/0 (socket says 172.21.15.37:48840) 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 -- 172.21.15.37:0/2521320281 learned_addr learned my addr 172.21.15.37:0/2521320281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff2ffff640 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 -- 172.21.15.37:0/2521320281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 msgr2=0x5570f00dace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 -- 172.21.15.37:0/2521320281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d2760 msgr2=0x5570f00da7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d2760 0x5570f00da7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 -- 172.21.15.37:0/2521320281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f00e99b0 con 0x5570f00dd940 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff2ffff640 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff3523b640 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00db220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff28005fd0 tx=0x7eff2801b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88c7e3d85361bc48 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff0ffff640 1 -- 172.21.15.37:0/2521320281 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28029040 con 0x5570f00dd940 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff0ffff640 1 -- 172.21.15.37:0/2521320281 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7eff28004a00 con 0x5570f00dd940 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.271+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2801bac0 con 0x5570f00dd940 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff0ffff640 1 -- 172.21.15.37:0/2521320281 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7eff2803b050 con 0x5570f00dd940 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 10 client.26150 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5570f00ec340 con 0x5570f00dd940 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff0ffff640 1 -- 172.21.15.37:0/2521320281 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2801bac0 con 0x5570f00dd940 2024-09-19T06:46:07.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff0ffff640 1 -- 172.21.15.37:0/2521320281 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7eff2802c070 con 0x5570f00dd940 2024-09-19T06:46:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 10 client.26150 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 10 client.26150 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:46:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 20 client.26150 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f00ed740 0x5570f00efb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 4 client.26150 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570f00459d0 con 0x5570f00ed740 2024-09-19T06:46:07.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.272+0000 7eff34a3a640 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f00ed740 0x5570f00efb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.273+0000 7eff34a3a640 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f00ed740 0x5570f00efb30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cfd046f3a8457c server_cookie=bc9d6f9d95a2ec56 in_seq=0 out_seq=0 2024-09-19T06:46:07.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.273+0000 7eff0ffff640 10 client.26150 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:07.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.273+0000 7eff0ffff640 1 -- 172.21.15.37:0/2521320281 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570f00459d0 con 0x5570f00ed740 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.273+0000 7eff0ffff640 10 client.26150 handle_command_reply: tid=0 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.273+0000 7eff39077740 1 client.26150 shutdown 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 msgr2=0x5570f00db220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00db220 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7eff28005fd0 tx=0x7eff2801b870 comp rx=0 tx=0).stop 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 shutdown_connections 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f00ed740 0x5570f00efb30 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00db220 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00dace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 --2- 172.21.15.37:0/2521320281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d2760 0x5570f00da7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 >> 172.21.15.37:0/2521320281 conn(0x5570f00cecf0 msgr2=0x5570f00cde70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff0ffff640 0 client.26150 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 shutdown_connections 2024-09-19T06:46:07.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 1 -- 172.21.15.37:0/2521320281 wait complete. 2024-09-19T06:46:07.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.274+0000 7eff39077740 20 client.26150 trim_cache size 0 max 16384 2024-09-19T06:46:07.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.291+0000 7eff39077740 1 Processor -- start 2024-09-19T06:46:07.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.291+0000 7eff39077740 1 -- start start 2024-09-19T06:46:07.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.291+0000 7eff39077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d2760 0x5570f00d51c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.291+0000 7eff39077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d3080 0x5570f00f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff39077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff39077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff14011090 con 0x5570f00d3080 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff39077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400e960 con 0x5570f00dd940 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff39077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400c7b0 con 0x5570f00d2760 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff2ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d3080 0x5570f00f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d2760 0x5570f00d51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d2760 0x5570f00d51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:55384/0 (socket says 172.21.15.37:55384) 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 -- 172.21.15.37:0/303530716 learned_addr learned my addr 172.21.15.37:0/303530716 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff2ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d3080 0x5570f00f5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48852/0 (socket says 172.21.15.37:48852) 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff3523b640 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 -- 172.21.15.37:0/303530716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 msgr2=0x5570f00f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 -- 172.21.15.37:0/303530716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d3080 msgr2=0x5570f00f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d3080 0x5570f00f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 -- 172.21.15.37:0/303530716 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570f00ceb20 con 0x5570f00d2760 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff2ffff640 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d3080 0x5570f00f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:07.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff34a3a640 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d2760 0x5570f00d51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7eff2409c060 tx=0x7eff240d2590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd83af792adffd8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:07.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff1a7fc640 1 -- 172.21.15.37:0/303530716 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff2409bcc0 con 0x5570f00d2760 2024-09-19T06:46:07.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff1a7fc640 1 -- 172.21.15.37:0/303530716 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2409ca60 con 0x5570f00d2760 2024-09-19T06:46:07.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.292+0000 7eff1a7fc640 1 -- 172.21.15.37:0/303530716 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff240989d0 con 0x5570f00d2760 2024-09-19T06:46:07.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 -- 172.21.15.37:0/303530716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d2760 msgr2=0x5570f00d51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d2760 0x5570f00d51c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7eff2409c060 tx=0x7eff240d2590 comp rx=0 tx=0).stop 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 -- 172.21.15.37:0/303530716 shutdown_connections 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00dd940 0x5570f00f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00d3080 0x5570f00f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 --2- 172.21.15.37:0/303530716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d2760 0x5570f00d51c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 -- 172.21.15.37:0/303530716 >> 172.21.15.37:0/303530716 conn(0x5570f00cecf0 msgr2=0x5570f00dc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 -- 172.21.15.37:0/303530716 shutdown_connections 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.293+0000 7eff39077740 1 -- 172.21.15.37:0/303530716 wait complete. 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 -- start start 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 Processor -- start 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00e5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00e61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00e2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400ce20 con 0x5570f00dd940 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400e290 con 0x5570f00d2760 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7eff1400d4f0 con 0x5570f00d3080 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff39077740 10 client.0 _ll_register_callbacks cb 0x7eff28019cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff3523b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00e2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff34a3a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00e5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff2ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00e61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff3523b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00e2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48868/0 (socket says 172.21.15.37:48868) 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.294+0000 7eff3523b640 1 -- 172.21.15.37:0/1414807823 learned_addr learned my addr 172.21.15.37:0/1414807823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff3523b640 1 -- 172.21.15.37:0/1414807823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 msgr2=0x5570f00e61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff3523b640 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00e61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff3523b640 1 -- 172.21.15.37:0/1414807823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 msgr2=0x5570f00e5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff3523b640 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00e5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff3523b640 1 -- 172.21.15.37:0/1414807823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570f00ca260 con 0x5570f00dd940 2024-09-19T06:46:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff2ffff640 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff3523b640 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00e2250 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7eff28004fb0 tx=0x7eff28009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e26a99feffa1bb18 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff0ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff0ffff640 1 -- 172.21.15.37:0/1414807823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff28029040 con 0x5570f00dd940 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff0ffff640 1 -- 172.21.15.37:0/1414807823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7eff28015040 con 0x5570f00dd940 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff2409ca60 con 0x5570f00dd940 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff0ffff640 1 -- 172.21.15.37:0/1414807823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7eff2800c640 con 0x5570f00dd940 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff39077740 10 client.16662 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.295+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5570f0158c90 con 0x5570f00dd940 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff0ffff640 1 -- 172.21.15.37:0/1414807823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7eff2409ca60 con 0x5570f00dd940 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff0ffff640 1 -- 172.21.15.37:0/1414807823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7eff2802c070 con 0x5570f00dd940 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff39077740 10 client.16662 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff39077740 10 client.16662 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff39077740 20 client.16662 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff39077740 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f01593e0 0x5570f015b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff39077740 4 client.16662 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T06:46:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x5570f003fd20 con 0x5570f01593e0 2024-09-19T06:46:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff34a3a640 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f01593e0 0x5570f015b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff34a3a640 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f01593e0 0x5570f015b800 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=278d3106aee72b3f server_cookie=3c733d5071b4019 in_seq=0 out_seq=0 2024-09-19T06:46:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.296+0000 7eff0ffff640 10 client.16662 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff0ffff640 1 -- 172.21.15.37:0/1414807823 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5570f003fd20 con 0x5570f01593e0 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff0ffff640 10 client.16662 handle_command_reply: tid=0 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 client.16662 shutdown 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 msgr2=0x5570f00e2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00e2250 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7eff28004fb0 tx=0x7eff28009cf0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 shutdown_connections 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570f01593e0 0x5570f015b800 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570f00dd940 0x5570f00e2250 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff0ffff640 0 client.16662 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570f00d3080 0x5570f00e61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 --2- 172.21.15.37:0/1414807823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570f00d2760 0x5570f00e5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.383+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 >> 172.21.15.37:0/1414807823 conn(0x5570f00cecf0 msgr2=0x5570f00cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.384+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 shutdown_connections 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.384+0000 7eff39077740 1 -- 172.21.15.37:0/1414807823 wait complete. 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.384+0000 7eff39077740 20 client.16662 trim_cache size 0 max 16384 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 31, 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:46:07.386 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 18, 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "7518637c": { 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 18, 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.8, 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:46:07.387 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:07.388 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.384+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7eff200a3430 msgr2=0x7eff200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7eff200a3430 0x7eff200a58f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7eff28020000 tx=0x7eff28023000 comp rx=0 tx=0).stop 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff30102d30 msgr2=0x7eff301a0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff30102d30 0x7eff301a0450 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7eff2400e9f0 tx=0x7eff2400edf0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 shutdown_connections 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7eff301082a0 0x7eff3019a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff301036e0 0x7eff301a0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7eff200a3430 0x7eff200a58f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 --2- 172.21.15.37:0/3589626406 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7eff30102d30 0x7eff301a0450 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 >> 172.21.15.37:0/3589626406 conn(0x7eff300fc7d0 msgr2=0x7eff3010cfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 shutdown_connections 2024-09-19T06:46:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:07.385+0000 7eff36c9e640 1 -- 172.21.15.37:0/3589626406 wait complete. 2024-09-19T06:46:08.199 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 13.56 MB/sec execute 35 sec latency 891.930 ms 2024-09-19T06:46:08.251 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:07 smithi037 ceph-mon[31211]: pgmap v134: 97 pgs: 97 active+clean; 47 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 375 op/s 2024-09-19T06:46:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:07 smithi096 ceph-mon[35004]: pgmap v134: 97 pgs: 97 active+clean; 47 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 375 op/s 2024-09-19T06:46:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:07 smithi151 ceph-mon[35236]: pgmap v134: 97 pgs: 97 active+clean; 47 MiB data, 906 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.0 MiB/s wr, 375 op/s 2024-09-19T06:46:08.672 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '7518637c', quiesce took 4.5 seconds. Will release after: 25.3 2024-09-19T06:46:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 -- 172.21.15.37:0/2948185596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 msgr2=0x7f6d18103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/2948185596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18103060 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009900 tx=0x7f6d0802a480 comp rx=0 tx=0).stop 2024-09-19T06:46:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 -- 172.21.15.37:0/2948185596 shutdown_connections 2024-09-19T06:46:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/2948185596 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d1810c0d0 0x7f6d1810e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/2948185596 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d181035a0 0x7f6d1810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/2948185596 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18103060 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 -- 172.21.15.37:0/2948185596 >> 172.21.15.37:0/2948185596 conn(0x7f6d18077990 msgr2=0x7f6d180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 -- 172.21.15.37:0/2948185596 shutdown_connections 2024-09-19T06:46:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.108+0000 7f6d1d05e640 1 -- 172.21.15.37:0/2948185596 wait complete. 2024-09-19T06:46:09.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 Processor -- start 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 -- start start 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d181035a0 0x7f6d18198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d1810c0d0 0x7f6d1819cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18111eb0 con 0x7f6d18102c80 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18111d30 con 0x7f6d181035a0 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.109+0000 7f6d1d05e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d18112030 con 0x7f6d1810c0d0 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48884/0 (socket says 172.21.15.37:48884) 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 -- 172.21.15.37:0/4057713515 learned_addr learned my addr 172.21.15.37:0/4057713515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d1759e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d1810c0d0 0x7f6d1819cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d1659c640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d181035a0 0x7f6d18198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 -- 172.21.15.37:0/4057713515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d1810c0d0 msgr2=0x7f6d1819cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d1810c0d0 0x7f6d1819cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 -- 172.21.15.37:0/4057713515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d181035a0 msgr2=0x7f6d18198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d181035a0 0x7f6d18198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 -- 172.21.15.37:0/4057713515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d1819d330 con 0x7f6d18102c80 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18198340 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6d08006010 tx=0x7f6d080042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6cfffff640 1 -- 172.21.15.37:0/4057713515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d0803f030 con 0x7f6d18102c80 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d1819d5c0 con 0x7f6d18102c80 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6cfffff640 1 -- 172.21.15.37:0/4057713515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d08003ce0 con 0x7f6d18102c80 2024-09-19T06:46:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6cfffff640 1 -- 172.21.15.37:0/4057713515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d08035070 con 0x7f6d18102c80 2024-09-19T06:46:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.110+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d181a4eb0 con 0x7f6d18102c80 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.113+0000 7f6cfffff640 1 -- 172.21.15.37:0/4057713515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6d08042030 con 0x7f6d18102c80 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.113+0000 7f6d1fdfd740 1 Processor -- start 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.113+0000 7f6d1fdfd740 1 -- start start 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1f97af0 0x5584a1f97ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5584a1efc2f0 con 0x5584a1f97af0 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5584a1ea4270 con 0x5584a1fa2cb0 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5584a1ef82b0 con 0x5584a1f98410 2024-09-19T06:46:09.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1f97af0 0x5584a1f97ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6cfffff640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d000a3460 0x7f6d000a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1f97af0 0x5584a1f97ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48898/0 (socket says 172.21.15.37:48898) 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 -- 172.21.15.37:0/2310883961 learned_addr learned my addr 172.21.15.37:0/2310883961 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1659c640 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d000a3460 0x7f6d000a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 -- 172.21.15.37:0/2310883961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 msgr2=0x5584a1fa2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6cfffff640 1 -- 172.21.15.37:0/4057713515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6d080f0290 con 0x7f6d18102c80 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 -- 172.21.15.37:0/2310883961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1fa2cb0 msgr2=0x5584a1fa50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 -- 172.21.15.37:0/2310883961 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a1ea3340 con 0x5584a1f97af0 2024-09-19T06:46:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1659c640 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.114+0000 7f6d1759e640 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1f97af0 0x5584a1f97ed0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6d10014a60 tx=0x7f6d10014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad43023e5f87607b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6cfd7fa640 1 -- 172.21.15.37:0/2310883961 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1001ec50 con 0x5584a1f97af0 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6cfd7fa640 1 -- 172.21.15.37:0/2310883961 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d10023070 con 0x5584a1f97af0 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6cfd7fa640 1 -- 172.21.15.37:0/2310883961 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10005690 con 0x5584a1f97af0 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/2310883961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1f97af0 msgr2=0x5584a1f97ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1759e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d000a3460 0x7f6d000a5920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d10022000 tx=0x7f6d1001e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1f97af0 0x5584a1f97ed0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f6d10014a60 tx=0x7f6d10014f20 comp rx=0 tx=0).stop 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/2310883961 shutdown_connections 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/2310883961 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1f97af0 0x5584a1f97ed0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/2310883961 >> 172.21.15.37:0/2310883961 conn(0x5584a1f94060 msgr2=0x5584a1f94440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.115+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/2310883961 shutdown_connections 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/2310883961 wait complete. 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 -- start start 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 Processor -- start 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1f9fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5584a1fa5880 con 0x5584a1fa2cb0 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5584a1e85290 con 0x5584a1f97af0 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5584a1fa5a00 con 0x5584a1f98410 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1fdfd740 10 client.0 _ll_register_callbacks cb 0x5584a1eb6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48914/0 (socket says 172.21.15.37:48914) 2024-09-19T06:46:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1759e640 1 -- 172.21.15.37:0/801242624 learned_addr learned my addr 172.21.15.37:0/801242624 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1f9fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.116+0000 7f6d1659c640 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1759e640 1 -- 172.21.15.37:0/801242624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 msgr2=0x5584a1fa0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1759e640 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1759e640 1 -- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 msgr2=0x5584a1f9fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1759e640 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1f9fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1759e640 1 -- 172.21.15.37:0/801242624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a1c0f210 con 0x5584a1fa2cb0 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1759e640 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa0670 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009640 tx=0x7f6d100128d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65c554b8e5669777 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6cfaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6cfaff5640 1 -- 172.21.15.37:0/801242624 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1002d040 con 0x5584a1fa2cb0 2024-09-19T06:46:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6cfaff5640 1 -- 172.21.15.37:0/801242624 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6d10005970 con 0x5584a1fa2cb0 2024-09-19T06:46:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d10023070 con 0x5584a1fa2cb0 2024-09-19T06:46:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6cfaff5640 1 -- 172.21.15.37:0/801242624 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6d1003f050 con 0x5584a1fa2cb0 2024-09-19T06:46:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1fdfd740 10 client.16674 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5584a1cdb620 con 0x5584a1fa2cb0 2024-09-19T06:46:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.117+0000 7f6cfaff5640 1 -- 172.21.15.37:0/801242624 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d10023070 con 0x5584a1fa2cb0 2024-09-19T06:46:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6cfaff5640 1 -- 172.21.15.37:0/801242624 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6d1000f040 con 0x5584a1fa2cb0 2024-09-19T06:46:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6d1fdfd740 10 client.16674 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6d1fdfd740 10 client.16674 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6d1fdfd740 20 client.16674 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a1fb2b10 0x5584a1fb4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6d1fdfd740 4 client.16674 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584a1f09a20 con 0x5584a1fb2b10 2024-09-19T06:46:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.118+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a1fb2b10 0x5584a1fb4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a1fb2b10 0x5584a1fb4f00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81872e6a1dd5f90c server_cookie=ab27b4bacaf86a29 in_seq=0 out_seq=0 2024-09-19T06:46:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6cfaff5640 10 client.16674 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6cfaff5640 1 -- 172.21.15.37:0/801242624 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5584a1f09a20 con 0x5584a1fb2b10 2024-09-19T06:46:09.128 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6cfaff5640 10 client.16674 handle_command_reply: tid=0 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 client.16674 shutdown 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 msgr2=0x5584a1fa0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa0670 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009640 tx=0x7f6d100128d0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 shutdown_connections 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a1fb2b10 0x5584a1fb4f00 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fa0670 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fa0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6cfaff5640 0 client.16674 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.119+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/801242624 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1f9fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.120+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 >> 172.21.15.37:0/801242624 conn(0x5584a1f94060 msgr2=0x5584a1f99d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.120+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 shutdown_connections 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.120+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/801242624 wait complete. 2024-09-19T06:46:09.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.120+0000 7f6d1fdfd740 20 client.16674 trim_cache size 0 max 16384 2024-09-19T06:46:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.134+0000 7f6d1fdfd740 1 Processor -- start 2024-09-19T06:46:09.139 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.134+0000 7f6d1fdfd740 1 -- start start 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fa04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fbaad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181b0070 con 0x5584a1fa2cb0 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ad940 con 0x5584a1f97af0 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ab790 con 0x5584a1f98410 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fbaad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fbaad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48918/0 (socket says 172.21.15.37:48918) 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 -- 172.21.15.37:0/1042473085 learned_addr learned my addr 172.21.15.37:0/1042473085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:09.140 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fa04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 -- 172.21.15.37:0/1042473085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 msgr2=0x5584a1fba590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1659c640 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fba590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 -- 172.21.15.37:0/1042473085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 msgr2=0x5584a1fa04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fa04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 -- 172.21.15.37:0/1042473085 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584a1fa2740 con 0x5584a1fa2cb0 2024-09-19T06:46:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fa04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1659c640 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6d1759e640 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fbaad0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005410 tx=0x7f6d100051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52fc0dd177ef8ec7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6cfcff9640 1 -- 172.21.15.37:0/1042473085 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10047020 con 0x5584a1fa2cb0 2024-09-19T06:46:09.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6cfcff9640 1 -- 172.21.15.37:0/1042473085 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d10023070 con 0x5584a1fa2cb0 2024-09-19T06:46:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.135+0000 7f6cfcff9640 1 -- 172.21.15.37:0/1042473085 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1002d040 con 0x5584a1fa2cb0 2024-09-19T06:46:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/1042473085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 msgr2=0x5584a1fbaad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fbaad0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f6d10005410 tx=0x7f6d100051b0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/1042473085 shutdown_connections 2024-09-19T06:46:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1fbaad0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1fba590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/1042473085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fa04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/1042473085 >> 172.21.15.37:0/1042473085 conn(0x5584a1f94060 msgr2=0x5584a1fa1c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/1042473085 shutdown_connections 2024-09-19T06:46:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/1042473085 wait complete. 2024-09-19T06:46:09.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 -- start start 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.136+0000 7f6d1fdfd740 1 Processor -- start 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fab270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1f9c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1fdfd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1f9cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181abe00 con 0x5584a1fa2cb0 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ad270 con 0x5584a1f97af0 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1fdfd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d181ac4d0 con 0x5584a1f98410 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1fdfd740 10 client.0 _ll_register_callbacks cb 0x7f6d1002a040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1f9cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1f9cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:48930/0 (socket says 172.21.15.37:48930) 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 -- 172.21.15.37:0/4159666019 learned_addr learned my addr 172.21.15.37:0/4159666019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1659c640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1f9c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.145 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 -- 172.21.15.37:0/4159666019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 msgr2=0x5584a1f9c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1f9c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 -- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 msgr2=0x5584a1fab270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fab270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fab270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 -- 172.21.15.37:0/4159666019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584a1f9d450 con 0x5584a1fa2cb0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6d1759e640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1f9cd60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006010 tx=0x7f6d10012c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fd17ce62579d97c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6cfaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6cfaff5640 1 -- 172.21.15.37:0/4159666019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d10050020 con 0x5584a1fa2cb0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.137+0000 7f6cfaff5640 1 -- 172.21.15.37:0/4159666019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6d100146f0 con 0x5584a1fa2cb0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d10023070 con 0x5584a1fa2cb0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6cfaff5640 1 -- 172.21.15.37:0/4159666019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6d1005b050 con 0x5584a1fa2cb0 2024-09-19T06:46:09.146 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 10 client.16680 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5584a201d3a0 con 0x5584a1fa2cb0 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6cfaff5640 1 -- 172.21.15.37:0/4159666019 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d10023070 con 0x5584a1fa2cb0 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6cfaff5640 1 -- 172.21.15.37:0/4159666019 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6d1004c930 con 0x5584a1fa2cb0 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 10 client.16680 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 10 client.16680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 20 client.16680 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a201d9f0 0x5584a201fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 4 client.16680 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5584a1f03d70 con 0x5584a201d9f0 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.138+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a201d9f0 0x5584a201fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.139+0000 7f6d16d9d640 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a201d9f0 0x5584a201fe10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edadd1e4a6b5a9 server_cookie=da0408d015b06c3e in_seq=0 out_seq=0 2024-09-19T06:46:09.147 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.139+0000 7f6cfaff5640 10 client.16680 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:09.201 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 13.18 MB/sec execute 36 sec latency 1894.256 ms 2024-09-19T06:46:09.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.250+0000 7f6cfaff5640 1 -- 172.21.15.37:0/4159666019 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5584a1f03d70 con 0x5584a201d9f0 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.250+0000 7f6cfaff5640 10 client.16680 handle_command_reply: tid=0 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.250+0000 7f6d1fdfd740 1 client.16680 shutdown 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 msgr2=0x5584a1f9cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1f9cd60 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006010 tx=0x7f6d10012c80 comp rx=0 tx=0).stop 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 shutdown_connections 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5584a201d9f0 0x5584a201fe10 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5584a1fa2cb0 0x5584a1f9cd60 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5584a1f98410 0x5584a1f9c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6cfaff5640 0 client.16680 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 --2- 172.21.15.37:0/4159666019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5584a1f97af0 0x5584a1fab270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 >> 172.21.15.37:0/4159666019 conn(0x5584a1f94060 msgr2=0x5584a1fa1450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 shutdown_connections 2024-09-19T06:46:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 1 -- 172.21.15.37:0/4159666019 wait complete. 2024-09-19T06:46:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.251+0000 7f6d1fdfd740 20 client.16680 trim_cache size 0 max 16384 2024-09-19T06:46:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:46:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:46:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:46:09.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d000a3460 msgr2=0x7f6d000a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d000a3460 0x7f6d000a5920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6d10022000 tx=0x7f6d1001e000 comp rx=0 tx=0).stop 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 msgr2=0x7f6d18198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18198340 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f6d08006010 tx=0x7f6d080042f0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 shutdown_connections 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d1810c0d0 0x7f6d1819cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d181035a0 0x7f6d18198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d000a3460 0x7f6d000a5920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 --2- 172.21.15.37:0/4057713515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d18102c80 0x7f6d18198340 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:09.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 >> 172.21.15.37:0/4057713515 conn(0x7f6d18077990 msgr2=0x7f6d180fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 shutdown_connections 2024-09-19T06:46:09.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:09.252+0000 7f6d1d05e640 1 -- 172.21.15.37:0/4057713515 wait complete. 2024-09-19T06:46:10.201 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 12.83 MB/sec execute 37 sec latency 2894.335 ms 2024-09-19T06:46:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:09 smithi096 ceph-mon[35004]: pgmap v135: 97 pgs: 97 active+clean; 55 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 496 op/s 2024-09-19T06:46:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:09 smithi037 ceph-mon[31211]: pgmap v135: 97 pgs: 97 active+clean; 55 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 496 op/s 2024-09-19T06:46:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:09 smithi151 ceph-mon[35236]: pgmap v135: 97 pgs: 97 active+clean; 55 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 3.4 MiB/s wr, 496 op/s 2024-09-19T06:46:11.201 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 12.49 MB/sec execute 38 sec latency 3894.447 ms 2024-09-19T06:46:12.201 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 12.17 MB/sec execute 39 sec latency 4894.548 ms 2024-09-19T06:46:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:11 smithi096 ceph-mon[35004]: pgmap v136: 97 pgs: 97 active+clean; 55 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.4 MiB/s wr, 255 op/s 2024-09-19T06:46:12.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:11 smithi037 ceph-mon[31211]: pgmap v136: 97 pgs: 97 active+clean; 55 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.4 MiB/s wr, 255 op/s 2024-09-19T06:46:12.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:11 smithi151 ceph-mon[35236]: pgmap v136: 97 pgs: 97 active+clean; 55 MiB data, 958 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 2.4 MiB/s wr, 255 op/s 2024-09-19T06:46:13.201 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 11.87 MB/sec execute 40 sec latency 5894.629 ms 2024-09-19T06:46:14.201 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 11.58 MB/sec execute 41 sec latency 6894.728 ms 2024-09-19T06:46:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:13 smithi096 ceph-mon[35004]: pgmap v137: 97 pgs: 97 active+clean; 43 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-09-19T06:46:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:13 smithi037 ceph-mon[31211]: pgmap v137: 97 pgs: 97 active+clean; 43 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-09-19T06:46:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:13 smithi151 ceph-mon[35236]: pgmap v137: 97 pgs: 97 active+clean; 43 MiB data, 947 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 MiB/s rd, 3.2 MiB/s wr, 361 op/s 2024-09-19T06:46:15.201 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 11.30 MB/sec execute 42 sec latency 7894.775 ms 2024-09-19T06:46:16.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 11.04 MB/sec execute 43 sec latency 8894.836 ms 2024-09-19T06:46:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:15 smithi096 ceph-mon[35004]: pgmap v138: 97 pgs: 97 active+clean; 43 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-19T06:46:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:15 smithi037 ceph-mon[31211]: pgmap v138: 97 pgs: 97 active+clean; 43 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-19T06:46:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:15 smithi151 ceph-mon[35236]: pgmap v138: 97 pgs: 97 active+clean; 43 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 MiB/s rd, 2.6 MiB/s wr, 270 op/s 2024-09-19T06:46:17.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 10.79 MB/sec execute 44 sec latency 9894.916 ms 2024-09-19T06:46:18.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 10.55 MB/sec execute 45 sec latency 10895.012 ms 2024-09-19T06:46:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:17 smithi096 ceph-mon[35004]: pgmap v139: 97 pgs: 97 active+clean; 43 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-09-19T06:46:18.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:17 smithi037 ceph-mon[31211]: pgmap v139: 97 pgs: 97 active+clean; 43 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-09-19T06:46:18.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:17 smithi151 ceph-mon[35236]: pgmap v139: 97 pgs: 97 active+clean; 43 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.4 MiB/s wr, 250 op/s 2024-09-19T06:46:19.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 10.32 MB/sec execute 46 sec latency 11895.095 ms 2024-09-19T06:46:19.618 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:46:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:20.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 10.10 MB/sec execute 47 sec latency 12895.137 ms 2024-09-19T06:46:20.273 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:19 smithi037 ceph-mon[31211]: pgmap v140: 97 pgs: 97 active+clean; 42 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-19T06:46:20.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:19 smithi096 ceph-mon[35004]: pgmap v140: 97 pgs: 97 active+clean; 42 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-19T06:46:20.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:19 smithi151 ceph-mon[35236]: pgmap v140: 97 pgs: 97 active+clean; 42 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 3.4 MiB/s wr, 290 op/s 2024-09-19T06:46:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 -- 172.21.15.37:0/2391033903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c106f10 msgr2=0x7f909c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 --2- 172.21.15.37:0/2391033903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c106f10 0x7f909c10ea90 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9094009ca0 tx=0x7f909402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 -- 172.21.15.37:0/2391033903 shutdown_connections 2024-09-19T06:46:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 --2- 172.21.15.37:0/2391033903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c106f10 0x7f909c10ea90 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 --2- 172.21.15.37:0/2391033903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c102790 0x7f909c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 --2- 172.21.15.37:0/2391033903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c101de0 0x7f909c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 -- 172.21.15.37:0/2391033903 >> 172.21.15.37:0/2391033903 conn(0x7f909c0fd8c0 msgr2=0x7f909c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.357+0000 7f90a2e44640 1 -- 172.21.15.37:0/2391033903 shutdown_connections 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.358+0000 7f90a2e44640 1 -- 172.21.15.37:0/2391033903 wait complete. 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.358+0000 7f90a2e44640 1 Processor -- start 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.358+0000 7f90a2e44640 1 -- start start 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.358+0000 7f90a2e44640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c101de0 0x7f909c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a2e44640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c102790 0x7f909c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a2e44640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c106f10 0x7f909c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a2e44640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f909c111f70 con 0x7f909c101de0 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a2e44640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f909c111df0 con 0x7f909c102790 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a2e44640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1120f0 con 0x7f909c106f10 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c101de0 0x7f909c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c101de0 0x7f909c1983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53850/0 (socket says 172.21.15.37:53850) 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 -- 172.21.15.37:0/834827535 learned_addr learned my addr 172.21.15.37:0/834827535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f9093fff640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c102790 0x7f909c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 -- 172.21.15.37:0/834827535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c106f10 msgr2=0x7f909c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a13e1640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c106f10 0x7f909c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c106f10 0x7f909c19cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 -- 172.21.15.37:0/834827535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c102790 msgr2=0x7f909c198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c102790 0x7f909c198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 -- 172.21.15.37:0/834827535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c19d380 con 0x7f909c101de0 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f9093fff640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c102790 0x7f909c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a0be0640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c101de0 0x7f909c1983d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f909800e9f0 tx=0x7f909800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f9091ffb640 1 -- 172.21.15.37:0/834827535 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909800cd80 con 0x7f909c101de0 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f9091ffb640 1 -- 172.21.15.37:0/834827535 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9098005500 con 0x7f909c101de0 2024-09-19T06:46:20.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c19d670 con 0x7f909c101de0 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.359+0000 7f9091ffb640 1 -- 172.21.15.37:0/834827535 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098010690 con 0x7f909c101de0 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.360+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c1a4f50 con 0x7f909c101de0 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f9091ffb640 1 -- 172.21.15.37:0/834827535 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f90980046c0 con 0x7f909c101de0 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 Processor -- start 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 -- start start 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696f4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696ff140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x555669701a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55566965a2e0 con 0x5556696ff680 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555669601ce0 con 0x5556696f4dc0 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f90a521d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5556696562a0 con 0x5556696f44a0 2024-09-19T06:46:20.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.362+0000 7f9091ffb640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90840a33b0 0x7f90840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x555669701a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x555669701a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53860/0 (socket says 172.21.15.37:53860) 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 -- 172.21.15.37:0/179111020 learned_addr learned my addr 172.21.15.37:0/179111020 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f90a13e1640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90840a33b0 0x7f90840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9091ffb640 1 -- 172.21.15.37:0/834827535 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f90980d02e0 con 0x7f909c101de0 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f90a13e1640 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696f4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f90a0be0640 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696ff140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 -- 172.21.15.37:0/179111020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 msgr2=0x5556696f4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696f4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 -- 172.21.15.37:0/179111020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 msgr2=0x5556696ff140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696ff140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 -- 172.21.15.37:0/179111020 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556695e6440 con 0x5556696ff680 2024-09-19T06:46:20.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f90a0be0640 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9093fff640 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x555669701a70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f908c009b90 tx=0x7f908c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53db6fe407b16085 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f90a13e1640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90840a33b0 0x7f90840a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f909402f540 tx=0x7f9094009360 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9082ffd640 1 -- 172.21.15.37:0/179111020 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c016070 con 0x5556696ff680 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9082ffd640 1 -- 172.21.15.37:0/179111020 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c005500 con 0x5556696ff680 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.363+0000 7f9082ffd640 1 -- 172.21.15.37:0/179111020 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c011470 con 0x5556696ff680 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 -- 172.21.15.37:0/179111020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 msgr2=0x555669701a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x555669701a70 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f908c009b90 tx=0x7f908c00e6c0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 -- 172.21.15.37:0/179111020 shutdown_connections 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x555669701a70 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696ff140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 --2- 172.21.15.37:0/179111020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696f4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 -- 172.21.15.37:0/179111020 >> 172.21.15.37:0/179111020 conn(0x5556696f0a30 msgr2=0x5556696f0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 -- 172.21.15.37:0/179111020 shutdown_connections 2024-09-19T06:46:20.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 -- 172.21.15.37:0/179111020 wait complete. 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.364+0000 7f90a521d740 1 -- start start 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 1 Processor -- start 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696fef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696fc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x5556696fc740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555669702250 con 0x5556696ff680 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555669615050 con 0x5556696f4dc0 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5556697023d0 con 0x5556696f44a0 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a521d740 10 client.0 _ll_register_callbacks cb 0x555669655160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a13e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x5556696fc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a13e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x5556696fc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53868/0 (socket says 172.21.15.37:53868) 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a13e1640 1 -- 172.21.15.37:0/3145140985 learned_addr learned my addr 172.21.15.37:0/3145140985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f9093fff640 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696fc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.365+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696fef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a13e1640 1 -- 172.21.15.37:0/3145140985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 msgr2=0x5556696fef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a13e1640 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696fef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a13e1640 1 -- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 msgr2=0x5556696fc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a13e1640 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696fc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a13e1640 1 -- 172.21.15.37:0/3145140985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556693cb3c0 con 0x5556696ff680 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a13e1640 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x5556696fc740 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9094031700 tx=0x7f909402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2010b25e501a4d5c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f907bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f907bfff640 1 -- 172.21.15.37:0/3145140985 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909402f810 con 0x5556696ff680 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f907bfff640 1 -- 172.21.15.37:0/3145140985 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f909402f9b0 con 0x5556696ff680 2024-09-19T06:46:20.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f908c005500 con 0x5556696ff680 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f907bfff640 1 -- 172.21.15.37:0/3145140985 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f909402fbf0 con 0x5556696ff680 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a521d740 10 client.16692 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55566970dba0 con 0x5556696ff680 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f907bfff640 1 -- 172.21.15.37:0/3145140985 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c005500 con 0x5556696ff680 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.366+0000 7f907bfff640 1 -- 172.21.15.37:0/3145140985 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f9094030cd0 con 0x5556696ff680 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a521d740 10 client.16692 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a521d740 10 client.16692 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a521d740 20 client.16692 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a521d740 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55566970f060 0x5556697114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a521d740 4 client.16692 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555669667850 con 0x55566970f060 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55566970f060 0x5556697114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f907bfff640 10 client.16692 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.367+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55566970f060 0x5556697114b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a3fe9700aee896c server_cookie=914891f221ab6685 in_seq=0 out_seq=0 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f907bfff640 1 -- 172.21.15.37:0/3145140985 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555669667850 con 0x55566970f060 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f907bfff640 10 client.16692 handle_command_reply: tid=0 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 client.16692 shutdown 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 msgr2=0x5556696fc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x5556696fc740 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f9094031700 tx=0x7f909402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 shutdown_connections 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55566970f060 0x5556697114b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696ff680 0x5556696fc740 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556696fc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 --2- 172.21.15.37:0/3145140985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x5556696fef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f907bfff640 0 client.16692 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 >> 172.21.15.37:0/3145140985 conn(0x5556696f0a30 msgr2=0x5556696f6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:20.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 shutdown_connections 2024-09-19T06:46:20.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 1 -- 172.21.15.37:0/3145140985 wait complete. 2024-09-19T06:46:20.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.368+0000 7f90a521d740 20 client.16692 trim_cache size 0 max 16384 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.385+0000 7f90a521d740 1 Processor -- start 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.385+0000 7f90a521d740 1 -- start start 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a521d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x5556696fe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a521d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x555669716d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a521d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556697172c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a521d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1adf80 con 0x5556696f0a30 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a521d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1ab850 con 0x5556696f4dc0 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a521d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1a96a0 con 0x5556696f44a0 2024-09-19T06:46:20.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x5556696fe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x5556696fe170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53882/0 (socket says 172.21.15.37:53882) 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 -- 172.21.15.37:0/3787465260 learned_addr learned my addr 172.21.15.37:0/3787465260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 -- 172.21.15.37:0/3787465260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 msgr2=0x555669716d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3787465260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x555669716d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 -- 172.21.15.37:0/3787465260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 msgr2=0x5556697172c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3787465260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556697172c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 -- 172.21.15.37:0/3787465260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556696fa750 con 0x5556696f0a30 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3787465260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x5556696fe170 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f909809c4f0 tx=0x7f90980d0140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=501fb3e4e23987fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90827fc640 1 -- 172.21.15.37:0/3787465260 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909809c560 con 0x5556696f0a30 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.386+0000 7f90827fc640 1 -- 172.21.15.37:0/3787465260 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90980a0910 con 0x5556696f0a30 2024-09-19T06:46:20.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90827fc640 1 -- 172.21.15.37:0/3787465260 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90980cdc50 con 0x5556696f0a30 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 -- 172.21.15.37:0/3787465260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 msgr2=0x5556696fe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 --2- 172.21.15.37:0/3787465260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x5556696fe170 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f909809c4f0 tx=0x7f90980d0140 comp rx=0 tx=0).stop 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 -- 172.21.15.37:0/3787465260 shutdown_connections 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 --2- 172.21.15.37:0/3787465260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f4dc0 0x5556697172c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 --2- 172.21.15.37:0/3787465260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f44a0 0x555669716d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 --2- 172.21.15.37:0/3787465260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x5556696fe170 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 -- 172.21.15.37:0/3787465260 >> 172.21.15.37:0/3787465260 conn(0x5556696fc8a0 msgr2=0x5556696fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 -- 172.21.15.37:0/3787465260 shutdown_connections 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 -- 172.21.15.37:0/3787465260 wait complete. 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.387+0000 7f90a521d740 1 -- start start 2024-09-19T06:46:20.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 1 Processor -- start 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x555669705300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f44a0 0x555669705840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f4dc0 0x555669705d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1a9d10 con 0x5556696f0a30 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1ab180 con 0x5556696f44a0 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f909c1aa3e0 con 0x5556696f4dc0 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a521d740 10 client.0 _ll_register_callbacks cb 0x7f9098005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x555669705300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x555669705300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53888/0 (socket says 172.21.15.37:53888) 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 -- 172.21.15.37:0/3807782178 learned_addr learned my addr 172.21.15.37:0/3807782178 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a13e1640 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f4dc0 0x555669705d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f9093fff640 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f44a0 0x555669705840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 -- 172.21.15.37:0/3807782178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f4dc0 msgr2=0x555669705d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f4dc0 0x555669705d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 -- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f44a0 msgr2=0x555669705840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f44a0 0x555669705840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.388+0000 7f90a0be0640 1 -- 172.21.15.37:0/3807782178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555669778d60 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a0be0640 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x555669705300 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f909809c110 tx=0x7f90980a51d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9acf4a05d2cee213 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f907bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f907bfff640 1 -- 172.21.15.37:0/3807782178 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9098065020 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f907bfff640 1 -- 172.21.15.37:0/3807782178 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f90980672c0 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90980a0910 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f907bfff640 1 -- 172.21.15.37:0/3807782178 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f90980cdc50 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 10 client.16698 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555669779f70 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f907bfff640 1 -- 172.21.15.37:0/3807782178 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f90980a0910 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f907bfff640 1 -- 172.21.15.37:0/3807782178 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f909809b9b0 con 0x5556696f0a30 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 10 client.16698 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:20.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 10 client.16698 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 20 client.16698 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556696ff680 0x55566977c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 4 client.16698 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.389+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555669661d60 con 0x5556696ff680 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f9093fff640 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556696ff680 0x55566977c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f907bfff640 10 client.16698 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f9093fff640 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556696ff680 0x55566977c5f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba79c5c2b768bfd6 server_cookie=bcda127d2e825faf in_seq=0 out_seq=0 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f907bfff640 1 -- 172.21.15.37:0/3807782178 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555669661d60 con 0x5556696ff680 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f907bfff640 10 client.16698 handle_command_reply: tid=0 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 client.16698 shutdown 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 msgr2=0x555669705300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x555669705300 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f909809c110 tx=0x7f90980a51d0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 shutdown_connections 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556696ff680 0x55566977c5f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556696f4dc0 0x555669705d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556696f44a0 0x555669705840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 --2- 172.21.15.37:0/3807782178 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556696f0a30 0x555669705300 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 >> 172.21.15.37:0/3807782178 conn(0x5556696fc8a0 msgr2=0x5556696fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.390+0000 7f907bfff640 0 client.16698 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 shutdown_connections 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a521d740 1 -- 172.21.15.37:0/3807782178 wait complete. 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a521d740 20 client.16698 trim_cache size 0 max 16384 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:46:20.397 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90840a33b0 msgr2=0x7f90840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90840a33b0 0x7f90840a5870 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f909402f540 tx=0x7f9094009360 comp rx=0 tx=0).stop 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c101de0 msgr2=0x7f909c1983d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c101de0 0x7f909c1983d0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f909800e9f0 tx=0x7f909800edf0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 shutdown_connections 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f909c106f10 0x7f909c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f909c102790 0x7f909c198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90840a33b0 0x7f90840a5870 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 --2- 172.21.15.37:0/834827535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f909c101de0 0x7f909c1983d0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 >> 172.21.15.37:0/834827535 conn(0x7f909c0fd8c0 msgr2=0x7f909c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 shutdown_connections 2024-09-19T06:46:20.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:20.391+0000 7f90a2e44640 1 -- 172.21.15.37:0/834827535 wait complete. 2024-09-19T06:46:21.102 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:46:21.102 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:21.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 9.89 MB/sec execute 48 sec latency 13895.197 ms 2024-09-19T06:46:21.359 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 -- 172.21.15.37:0/3426531367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101480 msgr2=0x7fa56c101860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 --2- 172.21.15.37:0/3426531367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101480 0x7fa56c101860 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009900 tx=0x7fa55c02a410 comp rx=0 tx=0).stop 2024-09-19T06:46:21.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 -- 172.21.15.37:0/3426531367 shutdown_connections 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 --2- 172.21.15.37:0/3426531367 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c105e80 0x7fa56c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 --2- 172.21.15.37:0/3426531367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa56c101e30 0x7fa56c105750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 --2- 172.21.15.37:0/3426531367 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101480 0x7fa56c101860 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 -- 172.21.15.37:0/3426531367 >> 172.21.15.37:0/3426531367 conn(0x7fa56c077210 msgr2=0x7fa56c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 -- 172.21.15.37:0/3426531367 shutdown_connections 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.790+0000 7fa57443d640 1 -- 172.21.15.37:0/3426531367 wait complete. 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 Processor -- start 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 -- start start 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa56c101480 0x7fa56c0fc690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101e30 0x7fa56c0fcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c105e80 0x7fa56c0fd110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c110ee0 con 0x7fa56c101e30 2024-09-19T06:46:21.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c110d60 con 0x7fa56c101480 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa57443d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa56c111060 con 0x7fa56c105e80 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa5719d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101e30 0x7fa56c0fcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa5719d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101e30 0x7fa56c0fcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53914/0 (socket says 172.21.15.37:53914) 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa5719d8640 1 -- 172.21.15.37:0/562134135 learned_addr learned my addr 172.21.15.37:0/562134135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.791+0000 7fa5721d9640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa56c101480 0x7fa56c0fc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa5729da640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c105e80 0x7fa56c0fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa5719d8640 1 -- 172.21.15.37:0/562134135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c105e80 msgr2=0x7fa56c0fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa5719d8640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c105e80 0x7fa56c0fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa5719d8640 1 -- 172.21.15.37:0/562134135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa56c101480 msgr2=0x7fa56c0fc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa5719d8640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa56c101480 0x7fa56c0fc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa5719d8640 1 -- 172.21.15.37:0/562134135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa56c1a6980 con 0x7fa56c101e30 2024-09-19T06:46:21.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa5719d8640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101e30 0x7fa56c0fcbd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa56000d950 tx=0x7fa56000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa56b7fe640 1 -- 172.21.15.37:0/562134135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa560017030 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa56c1a6c70 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa56b7fe640 1 -- 172.21.15.37:0/562134135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa560004020 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa56b7fe640 1 -- 172.21.15.37:0/562134135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5600047a0 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.792+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa56c1a71b0 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.793+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa56c101860 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.795+0000 7fa56b7fe640 1 -- 172.21.15.37:0/562134135 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa5600041d0 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.796+0000 7fa56b7fe640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5540a33b0 0x7fa5540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.796+0000 7fa56b7fe640 1 -- 172.21.15.37:0/562134135 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fa5600cfe00 con 0x7fa56c101e30 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.796+0000 7fa5721d9640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5540a33b0 0x7fa5540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.796+0000 7fa5721d9640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5540a33b0 0x7fa5540a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009230 tx=0x7fa55c0058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:21.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:21.799+0000 7fa56b7fe640 1 -- 172.21.15.37:0/562134135 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa56009c780 con 0x7fa56c101e30 2024-09-19T06:46:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.039+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa56c1a7440 con 0x7fa56c101e30 2024-09-19T06:46:22.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.039+0000 7fa56b7fe640 1 -- 172.21.15.37:0/562134135 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fa56009c080 con 0x7fa56c101e30 2024-09-19T06:46:22.041 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:46:22.042 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-19T06:45:42:465959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:41.617479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:46:22.042 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-19T06:46:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5540a33b0 msgr2=0x7fa5540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5540a33b0 0x7fa5540a5870 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009230 tx=0x7fa55c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101e30 msgr2=0x7fa56c0fcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101e30 0x7fa56c0fcbd0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fa56000d950 tx=0x7fa56000de10 comp rx=0 tx=0).stop 2024-09-19T06:46:22.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 shutdown_connections 2024-09-19T06:46:22.044 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:21 smithi037 ceph-mon[31211]: pgmap v141: 97 pgs: 97 active+clean; 42 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-19T06:46:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa56c105e80 0x7fa56c0fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5540a33b0 0x7fa5540a5870 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa56c101e30 0x7fa56c0fcbd0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 --2- 172.21.15.37:0/562134135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa56c101480 0x7fa56c0fc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.041+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 >> 172.21.15.37:0/562134135 conn(0x7fa56c077210 msgr2=0x7fa56c100250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.042+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 shutdown_connections 2024-09-19T06:46:22.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:22.042+0000 7fa57443d640 1 -- 172.21.15.37:0/562134135 wait complete. 2024-09-19T06:46:22.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 9.69 MB/sec execute 49 sec latency 14895.232 ms 2024-09-19T06:46:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:21 smithi096 ceph-mon[35004]: pgmap v141: 97 pgs: 97 active+clean; 42 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-19T06:46:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 -- 172.21.15.37:0/654208742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5934102d20 msgr2=0x7f5934103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 --2- 172.21.15.37:0/654208742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5934102d20 0x7f5934103100 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f5924009900 tx=0x7f592402a480 comp rx=0 tx=0).stop 2024-09-19T06:46:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 -- 172.21.15.37:0/654208742 shutdown_connections 2024-09-19T06:46:22.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 --2- 172.21.15.37:0/654208742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f593410c0f0 0x7f593410e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 --2- 172.21.15.37:0/654208742 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5934103640 0x7f593410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 --2- 172.21.15.37:0/654208742 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5934102d20 0x7f5934103100 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 -- 172.21.15.37:0/654208742 >> 172.21.15.37:0/654208742 conn(0x7f5934077990 msgr2=0x7f59340fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 -- 172.21.15.37:0/654208742 shutdown_connections 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.344+0000 7f5939b86640 1 -- 172.21.15.37:0/654208742 wait complete. 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 Processor -- start 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 -- start start 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5934102d20 0x7f5934198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5934103640 0x7f5934198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593410c0f0 0x7f593419cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5934111ec0 con 0x7f593410c0f0 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5934111d40 con 0x7f5934103640 2024-09-19T06:46:22.349 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5939b86640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5934112040 con 0x7f5934102d20 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5933fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593410c0f0 0x7f593419cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5933fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593410c0f0 0x7f593419cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53934/0 (socket says 172.21.15.37:53934) 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.345+0000 7f5933fff640 1 -- 172.21.15.37:0/1081667667 learned_addr learned my addr 172.21.15.37:0/1081667667 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f592affd640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5934103640 0x7f5934198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f59337fe640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5934102d20 0x7f5934198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5933fff640 1 -- 172.21.15.37:0/1081667667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5934102d20 msgr2=0x7f5934198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5933fff640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5934102d20 0x7f5934198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5933fff640 1 -- 172.21.15.37:0/1081667667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5934103640 msgr2=0x7f5934198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5933fff640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5934103640 0x7f5934198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5933fff640 1 -- 172.21.15.37:0/1081667667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593419d340 con 0x7f593410c0f0 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5933fff640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593410c0f0 0x7f593419cc10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f592c009fd0 tx=0x7f592c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f59317fa640 1 -- 172.21.15.37:0/1081667667 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c017070 con 0x7f593410c0f0 2024-09-19T06:46:22.350 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f59317fa640 1 -- 172.21.15.37:0/1081667667 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f592c003e50 con 0x7f593410c0f0 2024-09-19T06:46:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f59317fa640 1 -- 172.21.15.37:0/1081667667 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f592c0106c0 con 0x7f593410c0f0 2024-09-19T06:46:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593419d630 con 0x7f593410c0f0 2024-09-19T06:46:22.351 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.346+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59341a4ec0 con 0x7f593410c0f0 2024-09-19T06:46:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.348+0000 7f59317fa640 1 -- 172.21.15.37:0/1081667667 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f592c003970 con 0x7f593410c0f0 2024-09-19T06:46:22.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 Processor -- start 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 -- start start 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d071caf0 0x55e5d071ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d0727770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d0727cb0 0x55e5d072a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e5d06812f0 con 0x55e5d071d410 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5d0629270 con 0x55e5d071caf0 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f593c9fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5d067d2b0 con 0x55e5d0727cb0 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d0727770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d0727770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53938/0 (socket says 172.21.15.37:53938) 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 -- 172.21.15.37:0/2866753142 learned_addr learned my addr 172.21.15.37:0/2866753142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:22.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59317fa640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59200a34e0 0x7f59200a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f592affd640 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d0727cb0 0x55e5d072a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f592affd640 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d071caf0 0x55e5d071ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f5933fff640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59200a34e0 0x7f59200a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 -- 172.21.15.37:0/2866753142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d0727cb0 msgr2=0x55e5d072a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d0727cb0 0x55e5d072a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 -- 172.21.15.37:0/2866753142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d071caf0 msgr2=0x55e5d071ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d071caf0 0x55e5d071ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.349+0000 7f59337fe640 1 -- 172.21.15.37:0/2866753142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d0628340 con 0x55e5d071d410 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.350+0000 7f59317fa640 1 -- 172.21.15.37:0/1081667667 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f592c0cf910 con 0x7f593410c0f0 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.350+0000 7f59337fe640 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d0727770 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5924009900 tx=0x7f5924005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60d7900a4d39749 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.350+0000 7f5929ffb640 1 -- 172.21.15.37:0/2866753142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924040070 con 0x55e5d071d410 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.350+0000 7f5933fff640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59200a34e0 0x7f59200a59a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f592c09bc30 tx=0x7f592c0044b0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:22.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.350+0000 7f5929ffb640 1 -- 172.21.15.37:0/2866753142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5924031e20 con 0x55e5d071d410 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.350+0000 7f5929ffb640 1 -- 172.21.15.37:0/2866753142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924035040 con 0x55e5d071d410 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 -- 172.21.15.37:0/2866753142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 msgr2=0x55e5d0727770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d0727770 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f5924009900 tx=0x7f5924005e80 comp rx=0 tx=0).stop 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 -- 172.21.15.37:0/2866753142 shutdown_connections 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d0727cb0 0x55e5d072a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d0727770 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 --2- 172.21.15.37:0/2866753142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d071caf0 0x55e5d071ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 -- 172.21.15.37:0/2866753142 >> 172.21.15.37:0/2866753142 conn(0x55e5d0719060 msgr2=0x55e5d0719440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 -- 172.21.15.37:0/2866753142 shutdown_connections 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 -- 172.21.15.37:0/2866753142 wait complete. 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 -- start start 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.351+0000 7f593c9fb740 1 Processor -- start 2024-09-19T06:46:22.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0724b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d0725060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d07255a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e5d072a880 con 0x55e5d071caf0 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e5d060a290 con 0x55e5d0727cb0 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e5d072aa00 con 0x55e5d071d410 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 10 client.0 _ll_register_callbacks cb 0x55e5d063b140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0724b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0724b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53952/0 (socket says 172.21.15.37:53952) 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 -- 172.21.15.37:0/3896582248 learned_addr learned my addr 172.21.15.37:0/3896582248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f59337fe640 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d0725060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 -- 172.21.15.37:0/3896582248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 msgr2=0x55e5d0725060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d0725060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 -- 172.21.15.37:0/3896582248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 msgr2=0x55e5d07255a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d07255a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 -- 172.21.15.37:0/3896582248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d0394210 con 0x55e5d071caf0 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f59337fe640 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d0725060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f592affd640 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0724b20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f591c004440 tx=0x7f591c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3d7aa6dd8c4e57 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f5912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f5912ffd640 1 -- 172.21.15.37:0/3896582248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c005220 con 0x55e5d071caf0 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f5912ffd640 1 -- 172.21.15.37:0/3896582248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f591c0053c0 con 0x55e5d071caf0 2024-09-19T06:46:22.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5924031e20 con 0x55e5d071caf0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f5912ffd640 1 -- 172.21.15.37:0/3896582248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f591c0055c0 con 0x55e5d071caf0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.352+0000 7f593c9fb740 10 client.16716 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e5d0460620 con 0x55e5d071caf0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f5912ffd640 1 -- 172.21.15.37:0/3896582248 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5924031e20 con 0x55e5d071caf0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f5912ffd640 1 -- 172.21.15.37:0/3896582248 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f591c01c030 con 0x55e5d071caf0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f593c9fb740 10 client.16716 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f593c9fb740 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f593c9fb740 20 client.16716 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d0737ce0 0x55e5d073a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f593c9fb740 4 client.16716 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e5d068ea20 con 0x55e5d0737ce0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.353+0000 7f59337fe640 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d0737ce0 0x55e5d073a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f59337fe640 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d0737ce0 0x55e5d073a110 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9750151eb7f75eb7 server_cookie=b28dbe6e256e1a23 in_seq=0 out_seq=0 2024-09-19T06:46:22.361 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f5912ffd640 10 client.16716 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:22.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f5912ffd640 1 -- 172.21.15.37:0/3896582248 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e5d068ea20 con 0x55e5d0737ce0 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f5912ffd640 10 client.16716 handle_command_reply: tid=0 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 client.16716 shutdown 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 msgr2=0x55e5d0724b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0724b20 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f591c004440 tx=0x7f591c0039c0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 shutdown_connections 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d0737ce0 0x55e5d073a110 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d07255a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d0725060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3896582248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0724b20 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f5912ffd640 0 client.16716 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.354+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 >> 172.21.15.37:0/3896582248 conn(0x55e5d0719060 msgr2=0x55e5d071edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.355+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 shutdown_connections 2024-09-19T06:46:22.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.355+0000 7f593c9fb740 1 -- 172.21.15.37:0/3896582248 wait complete. 2024-09-19T06:46:22.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.355+0000 7f593c9fb740 20 client.16716 trim_cache size 0 max 16384 2024-09-19T06:46:22.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:21 smithi151 ceph-mon[35236]: pgmap v141: 97 pgs: 97 active+clean; 42 MiB data, 941 MiB used, 1.0 TiB / 1.0 TiB avail; 183 KiB/s rd, 2.0 MiB/s wr, 170 op/s 2024-09-19T06:46:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 Processor -- start 2024-09-19T06:46:22.375 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 -- start start 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071caf0 0x55e5d071f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d07253d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d073f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59341b0050 con 0x55e5d071d410 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f59341ad920 con 0x55e5d0727cb0 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f593c9fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59341ab770 con 0x55e5d071caf0 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f59337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d07253d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f59337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d07253d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53954/0 (socket says 172.21.15.37:53954) 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f59337fe640 1 -- 172.21.15.37:0/3953096941 learned_addr learned my addr 172.21.15.37:0/3953096941 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.369+0000 7f592affd640 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d073f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.376 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f592affd640 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071caf0 0x55e5d071f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59337fe640 1 -- 172.21.15.37:0/3953096941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071caf0 msgr2=0x55e5d071f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59337fe640 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071caf0 0x55e5d071f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59337fe640 1 -- 172.21.15.37:0/3953096941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 msgr2=0x55e5d073f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59337fe640 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d073f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59337fe640 1 -- 172.21.15.37:0/3953096941 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5d0736830 con 0x55e5d071d410 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f592affd640 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d073f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59337fe640 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d07253d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f592402a9a0 tx=0x7f592402aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57c83e01d79e5338 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59297fa640 1 -- 172.21.15.37:0/3953096941 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924040030 con 0x55e5d071d410 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59297fa640 1 -- 172.21.15.37:0/3953096941 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5924005140 con 0x55e5d071d410 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f59297fa640 1 -- 172.21.15.37:0/3953096941 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5924035040 con 0x55e5d071d410 2024-09-19T06:46:22.377 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f593c9fb740 1 -- 172.21.15.37:0/3953096941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 msgr2=0x55e5d07253d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.370+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d07253d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f592402a9a0 tx=0x7f592402aba0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- 172.21.15.37:0/3953096941 shutdown_connections 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d073f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071d410 0x55e5d07253d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 --2- 172.21.15.37:0/3953096941 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071caf0 0x55e5d071f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- 172.21.15.37:0/3953096941 >> 172.21.15.37:0/3953096941 conn(0x55e5d0719060 msgr2=0x55e5d0726ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- 172.21.15.37:0/3953096941 shutdown_connections 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- 172.21.15.37:0/3953096941 wait complete. 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- start start 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 Processor -- start 2024-09-19T06:46:22.378 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0730520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d072b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d072be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f59341abde0 con 0x55e5d071caf0 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f59341ad250 con 0x55e5d0727cb0 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f59341ac4b0 con 0x55e5d071d410 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f593c9fb740 10 client.0 _ll_register_callbacks cb 0x7f591c00bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f592affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0730520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.371+0000 7f592affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0730520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53960/0 (socket says 172.21.15.37:53960) 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f592affd640 1 -- 172.21.15.37:0/941511432 learned_addr learned my addr 172.21.15.37:0/941511432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f59337fe640 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d072b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f592affd640 1 -- 172.21.15.37:0/941511432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 msgr2=0x55e5d072b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f592affd640 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d072b840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.379 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f592affd640 1 -- 172.21.15.37:0/941511432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 msgr2=0x55e5d072be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f592affd640 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d072be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f592affd640 1 -- 172.21.15.37:0/941511432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5d072c660 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f592affd640 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0730520 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f591c0057c0 tx=0x7f591c00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fe6da4f5a722bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f5912ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f5912ffd640 1 -- 172.21.15.37:0/941511432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f591c02a040 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f5912ffd640 1 -- 172.21.15.37:0/941511432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f591c032670 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5924005140 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f5912ffd640 1 -- 172.21.15.37:0/941511432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f591c032870 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f593c9fb740 10 client.16722 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.372+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e5d07a6e80 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f5912ffd640 1 -- 172.21.15.37:0/941511432 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5924005140 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f5912ffd640 1 -- 172.21.15.37:0/941511432 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f591c01c040 con 0x55e5d071caf0 2024-09-19T06:46:22.380 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f593c9fb740 10 client.16722 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f593c9fb740 10 client.16722 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f593c9fb740 20 client.16722 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f593c9fb740 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d07a7580 0x55e5d07a99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f593c9fb740 4 client.16722 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e5d0688d70 con 0x55e5d07a7580 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f59337fe640 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d07a7580 0x55e5d07a99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f5912ffd640 10 client.16722 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:22.381 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.373+0000 7f59337fe640 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d07a7580 0x55e5d07a99a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50905c1ba8de0baf server_cookie=80838d37f74c87e8 in_seq=0 out_seq=0 2024-09-19T06:46:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f5912ffd640 1 -- 172.21.15.37:0/941511432 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e5d0688d70 con 0x55e5d07a7580 2024-09-19T06:46:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f5912ffd640 10 client.16722 handle_command_reply: tid=0 2024-09-19T06:46:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 client.16722 shutdown 2024-09-19T06:46:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 msgr2=0x55e5d0730520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0730520 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f591c0057c0 tx=0x7f591c00be90 comp rx=0 tx=0).stop 2024-09-19T06:46:22.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 shutdown_connections 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e5d07a7580 0x55e5d07a99a0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e5d0727cb0 0x55e5d072be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e5d071d410 0x55e5d072b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 --2- 172.21.15.37:0/941511432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e5d071caf0 0x55e5d0730520 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f5912ffd640 0 client.16722 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 >> 172.21.15.37:0/941511432 conn(0x55e5d0719060 msgr2=0x55e5d0726380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 shutdown_connections 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 1 -- 172.21.15.37:0/941511432 wait complete. 2024-09-19T06:46:22.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.417+0000 7f593c9fb740 20 client.16722 trim_cache size 0 max 16384 2024-09-19T06:46:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:46:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:46:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:46:22.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59200a34e0 msgr2=0x7f59200a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59200a34e0 0x7f59200a59a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f592c09bc30 tx=0x7f592c0044b0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593410c0f0 msgr2=0x7f593419cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593410c0f0 0x7f593419cc10 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f592c009fd0 tx=0x7f592c00b260 comp rx=0 tx=0).stop 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 shutdown_connections 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f59200a34e0 0x7f59200a59a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f593410c0f0 0x7f593419cc10 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5934103640 0x7f5934198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 --2- 172.21.15.37:0/1081667667 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5934102d20 0x7f5934198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 >> 172.21.15.37:0/1081667667 conn(0x7f5934077990 msgr2=0x7f59340fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 shutdown_connections 2024-09-19T06:46:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:22.418+0000 7f5939b86640 1 -- 172.21.15.37:0/1081667667 wait complete. 2024-09-19T06:46:22.785 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:46:23.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:22 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/562134135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:23.080 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:23.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 9.49 MB/sec execute 50 sec latency 15895.259 ms 2024-09-19T06:46:23.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:22 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/562134135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:23.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:22 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/562134135' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:23.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 -- 172.21.15.37:0/3839032334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 msgr2=0x7faf3410ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/3839032334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf3410ec50 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009cd0 tx=0x7faf2c02a910 comp rx=0 tx=0).stop 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 -- 172.21.15.37:0/3839032334 shutdown_connections 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/3839032334 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf3410ec50 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/3839032334 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf34103760 0x7faf34107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/3839032334 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34102db0 0x7faf34103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 -- 172.21.15.37:0/3839032334 >> 172.21.15.37:0/3839032334 conn(0x7faf340fc7d0 msgr2=0x7faf340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 -- 172.21.15.37:0/3839032334 shutdown_connections 2024-09-19T06:46:23.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 -- 172.21.15.37:0/3839032334 wait complete. 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.526+0000 7faf3b5fe640 1 Processor -- start 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3b5fe640 1 -- start start 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3b5fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34102db0 0x7faf34100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3b5fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf34103760 0x7faf34100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3b5fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf34101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3b5fe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf34112170 con 0x7faf341082e0 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3b5fe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf34111ff0 con 0x7faf34103760 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3b5fe640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf341122f0 con 0x7faf34102db0 2024-09-19T06:46:23.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf34101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf34101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53980/0 (socket says 172.21.15.37:53980) 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 -- 172.21.15.37:0/852328380 learned_addr learned my addr 172.21.15.37:0/852328380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf38b99640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf34103760 0x7faf34100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf3939a640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34102db0 0x7faf34100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 -- 172.21.15.37:0/852328380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34102db0 msgr2=0x7faf34100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34102db0 0x7faf34100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 -- 172.21.15.37:0/852328380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf34103760 msgr2=0x7faf34100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf34103760 0x7faf34100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf39b9b640 1 -- 172.21.15.37:0/852328380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf341a6b60 con 0x7faf341082e0 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.527+0000 7faf38b99640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf34103760 0x7faf34100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.528+0000 7faf39b9b640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf34101490 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009ca0 tx=0x7faf2c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.528+0000 7faf2a7fc640 1 -- 172.21.15.37:0/852328380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0317e0 con 0x7faf341082e0 2024-09-19T06:46:23.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.528+0000 7faf2a7fc640 1 -- 172.21.15.37:0/852328380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf2c031980 con 0x7faf341082e0 2024-09-19T06:46:23.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.528+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf341a6df0 con 0x7faf341082e0 2024-09-19T06:46:23.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.528+0000 7faf2a7fc640 1 -- 172.21.15.37:0/852328380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf2c02fd90 con 0x7faf341082e0 2024-09-19T06:46:23.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.528+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf341a7350 con 0x7faf341082e0 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.530+0000 7faf2a7fc640 1 -- 172.21.15.37:0/852328380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7faf2c004040 con 0x7faf341082e0 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.530+0000 7faf3d9d7740 1 Processor -- start 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3d9d7740 1 -- start start 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0ee090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0ee5d0 0x559d1f0eea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559d1f0542f0 con 0x559d1f0ee5d0 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d1effbcf0 con 0x559d1f0efb40 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559d1f0502b0 con 0x559d1f0f9370 2024-09-19T06:46:23.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0ee5d0 0x559d1f0eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0ee5d0 0x559d1f0eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53982/0 (socket says 172.21.15.37:53982) 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 -- 172.21.15.37:0/2149429493 learned_addr learned my addr 172.21.15.37:0/2149429493 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf2a7fc640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faf200a3430 0x7faf200a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf38b99640 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf38b99640 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0ee090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf39b9b640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faf200a3430 0x7faf200a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf2a7fc640 1 -- 172.21.15.37:0/852328380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7faf2c0f10f0 con 0x7faf341082e0 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 -- 172.21.15.37:0/2149429493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 msgr2=0x559d1f0fb760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0fb760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 -- 172.21.15.37:0/2149429493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 msgr2=0x559d1f0ee090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0ee090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.531+0000 7faf3939a640 1 -- 172.21.15.37:0/2149429493 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d1efe03a0 con 0x559d1f0ee5d0 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.532+0000 7faf3939a640 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0ee5d0 0x559d1f0eea10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faf3000b820 tx=0x7faf3000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c072f5a4f4f184f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:23.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.532+0000 7faf1f7fe640 1 -- 172.21.15.37:0/2149429493 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf3000d950 con 0x559d1f0ee5d0 2024-09-19T06:46:23.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.532+0000 7faf39b9b640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faf200a3430 0x7faf200a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0eeaa0 tx=0x7faf2c08d1a0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:23.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.532+0000 7faf1f7fe640 1 -- 172.21.15.37:0/2149429493 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf30019070 con 0x559d1f0ee5d0 2024-09-19T06:46:23.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.532+0000 7faf1f7fe640 1 -- 172.21.15.37:0/2149429493 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300123e0 con 0x559d1f0ee5d0 2024-09-19T06:46:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 -- 172.21.15.37:0/2149429493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0ee5d0 msgr2=0x559d1f0eea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0ee5d0 0x559d1f0eea10 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7faf3000b820 tx=0x7faf3000bce0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 -- 172.21.15.37:0/2149429493 shutdown_connections 2024-09-19T06:46:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0fb760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0ee5d0 0x559d1f0eea10 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/2149429493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0ee090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 -- 172.21.15.37:0/2149429493 >> 172.21.15.37:0/2149429493 conn(0x559d1f0b7310 msgr2=0x559d1f0eac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 -- 172.21.15.37:0/2149429493 shutdown_connections 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 -- 172.21.15.37:0/2149429493 wait complete. 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 -- start start 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.533+0000 7faf3d9d7740 1 Processor -- start 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559d1f0fc070 con 0x559d1f0f9370 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d1f00f060 con 0x559d1f0efb40 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559d1f0fc1f0 con 0x559d1f0ee5d0 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3d9d7740 10 client.0 _ll_register_callbacks cb 0x559d1ed628c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:23.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53996/0 (socket says 172.21.15.37:53996) 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 -- 172.21.15.37:0/635686936 learned_addr learned my addr 172.21.15.37:0/635686936 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf3939a640 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 -- 172.21.15.37:0/635686936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0ee5d0 msgr2=0x559d1f0f8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 -- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 msgr2=0x559d1f0f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 -- 172.21.15.37:0/635686936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d1edc53c0 con 0x559d1f0f9370 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf38b99640 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f64d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7faf24019e40 tx=0x7faf240047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e021e4d021e64c2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.534+0000 7faf1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf1cff9640 1 -- 172.21.15.37:0/635686936 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24014df0 con 0x559d1f0f9370 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf30019070 con 0x559d1f0f9370 2024-09-19T06:46:23.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf1cff9640 1 -- 172.21.15.37:0/635686936 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7faf24016400 con 0x559d1f0f9370 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf1cff9640 1 -- 172.21.15.37:0/635686936 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faf24029050 con 0x559d1f0f9370 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 10 client.16734 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d1f107b10 con 0x559d1f0f9370 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf1cff9640 1 -- 172.21.15.37:0/635686936 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf30019070 con 0x559d1f0f9370 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf1cff9640 1 -- 172.21.15.37:0/635686936 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7faf24015040 con 0x559d1f0f9370 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 10 client.16734 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 10 client.16734 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 20 client.16734 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f108f70 0x559d1f10b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 4 client.16734 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d1f061860 con 0x559d1f108f70 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.535+0000 7faf3939a640 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f108f70 0x559d1f10b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.536+0000 7faf3939a640 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f108f70 0x559d1f10b380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b51eb77812c637a9 server_cookie=cdb6b35ff49ee73 in_seq=0 out_seq=0 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.536+0000 7faf1cff9640 10 client.16734 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf1cff9640 1 -- 172.21.15.37:0/635686936 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559d1f061860 con 0x559d1f108f70 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf1cff9640 10 client.16734 handle_command_reply: tid=0 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 client.16734 shutdown 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 msgr2=0x559d1f0f64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f64d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7faf24019e40 tx=0x7faf240047d0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 shutdown_connections 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f108f70 0x559d1f10b380 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f64d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0efb40 0x559d1f0f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/635686936 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf1cff9640 0 client.16734 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 >> 172.21.15.37:0/635686936 conn(0x559d1f0b7310 msgr2=0x559d1f0f0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:23.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 shutdown_connections 2024-09-19T06:46:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 1 -- 172.21.15.37:0/635686936 wait complete. 2024-09-19T06:46:23.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.537+0000 7faf3d9d7740 20 client.16734 trim_cache size 0 max 16384 2024-09-19T06:46:23.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 Processor -- start 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 -- start start 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 0x559d1f0e9c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0efb40 0x559d1f0f1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0f1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf341b2550 con 0x559d1f0efb40 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf341afe20 con 0x559d1f0ee5d0 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf341adc70 con 0x559d1f0f9370 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3939a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0efb40 0x559d1f0f1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3939a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0efb40 0x559d1f0f1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54002/0 (socket says 172.21.15.37:54002) 2024-09-19T06:46:23.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3939a640 1 -- 172.21.15.37:0/3918379205 learned_addr learned my addr 172.21.15.37:0/3918379205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf38b99640 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0f1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.550+0000 7faf3939a640 1 -- 172.21.15.37:0/3918379205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 msgr2=0x559d1f0f1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf3939a640 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0f1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf3939a640 1 -- 172.21.15.37:0/3918379205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 msgr2=0x559d1f0e9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf3939a640 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 0x559d1f0e9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf3939a640 1 -- 172.21.15.37:0/3918379205 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d1f0f1ba0 con 0x559d1f0efb40 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf38b99640 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf3939a640 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0efb40 0x559d1f0f1120 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x559d1f0ebac0 tx=0x7faf30008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=accf5c0210085d07 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf1effd640 1 -- 172.21.15.37:0/3918379205 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf300149b0 con 0x559d1f0efb40 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf1effd640 1 -- 172.21.15.37:0/3918379205 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf30019040 con 0x559d1f0efb40 2024-09-19T06:46:23.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf1effd640 1 -- 172.21.15.37:0/3918379205 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf30027670 con 0x559d1f0efb40 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3918379205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0efb40 msgr2=0x559d1f0f1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.551+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0efb40 0x559d1f0f1120 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x559d1f0ebac0 tx=0x7faf30008040 comp rx=0 tx=0).stop 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3918379205 shutdown_connections 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0f9370 0x559d1f0f1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0efb40 0x559d1f0f1120 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3918379205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 0x559d1f0e9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3918379205 >> 172.21.15.37:0/3918379205 conn(0x559d1f0b7310 msgr2=0x559d1f0e9500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3918379205 shutdown_connections 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3918379205 wait complete. 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- start start 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 Processor -- start 2024-09-19T06:46:23.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0efb40 0x559d1f0f3660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7faf341ae2e0 con 0x559d1f0f9370 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf341af750 con 0x559d1f0ee5d0 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.552+0000 7faf3d9d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7faf341ae9b0 con 0x559d1f0efb40 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf3d9d7740 10 client.0 _ll_register_callbacks cb 0x7faf2400ac00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54014/0 (socket says 172.21.15.37:54014) 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 -- 172.21.15.37:0/3455471455 learned_addr learned my addr 172.21.15.37:0/3455471455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf3939a640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0efb40 0x559d1f0f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 -- 172.21.15.37:0/3455471455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0efb40 msgr2=0x559d1f0f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0efb40 0x559d1f0f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 -- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 msgr2=0x559d1f0f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f3120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 -- 172.21.15.37:0/3455471455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d1f172d50 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf3939a640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0efb40 0x559d1f0f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf38b99640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f3ba0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faf2400a970 tx=0x7faf2400a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cb2f380de702f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf1cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf1cff9640 1 -- 172.21.15.37:0/3455471455 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf24019910 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf30019040 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf1cff9640 1 -- 172.21.15.37:0/3455471455 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7faf240059c0 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf1cff9640 1 -- 172.21.15.37:0/3455471455 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7faf24005bc0 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf3d9d7740 10 client.16740 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.553+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x559d1f173fa0 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf1cff9640 1 -- 172.21.15.37:0/3455471455 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faf30019040 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf1cff9640 1 -- 172.21.15.37:0/3455471455 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7faf24006830 con 0x559d1f0f9370 2024-09-19T06:46:23.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf3d9d7740 10 client.16740 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf3d9d7740 10 client.16740 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf3d9d7740 20 client.16740 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f1746c0 0x559d1f176ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf3d9d7740 4 client.16740 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559d1f05bd70 con 0x559d1f1746c0 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.554+0000 7faf3939a640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f1746c0 0x559d1f176ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3939a640 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f1746c0 0x559d1f176ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=587687e9b9d168fb server_cookie=afb302a5e41c7766 in_seq=0 out_seq=0 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf1cff9640 10 client.16740 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf1cff9640 1 -- 172.21.15.37:0/3455471455 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559d1f05bd70 con 0x559d1f1746c0 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf1cff9640 10 client.16740 handle_command_reply: tid=0 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 client.16740 shutdown 2024-09-19T06:46:23.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 msgr2=0x559d1f0f3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f3ba0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faf2400a970 tx=0x7faf2400a2b0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 shutdown_connections 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x559d1f1746c0 0x559d1f176ab0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559d1f0f9370 0x559d1f0f3ba0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559d1f0efb40 0x559d1f0f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 --2- 172.21.15.37:0/3455471455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d1f0ee5d0 0x559d1f0f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 >> 172.21.15.37:0/3455471455 conn(0x559d1f0b7310 msgr2=0x559d1f0f7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf1cff9640 0 client.16740 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 shutdown_connections 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 1 -- 172.21.15.37:0/3455471455 wait complete. 2024-09-19T06:46:23.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.555+0000 7faf3d9d7740 20 client.16740 trim_cache size 0 max 16384 2024-09-19T06:46:23.563 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:46:23.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faf200a3430 msgr2=0x7faf200a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faf200a3430 0x7faf200a58f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf2c0eeaa0 tx=0x7faf2c08d1a0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 msgr2=0x7faf34101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf34101490 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7faf2c009ca0 tx=0x7faf2c003f90 comp rx=0 tx=0).stop 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 shutdown_connections 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7faf200a3430 0x7faf200a58f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7faf341082e0 0x7faf34101490 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf34103760 0x7faf34100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 --2- 172.21.15.37:0/852328380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faf34102db0 0x7faf34100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:23.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 >> 172.21.15.37:0/852328380 conn(0x7faf340fc7d0 msgr2=0x7faf34105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 shutdown_connections 2024-09-19T06:46:23.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:23.556+0000 7faf3b5fe640 1 -- 172.21.15.37:0/852328380 wait complete. 2024-09-19T06:46:24.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 9.31 MB/sec execute 51 sec latency 16895.351 ms 2024-09-19T06:46:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:23 smithi096 ceph-mon[35004]: pgmap v142: 97 pgs: 97 active+clean; 37 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-19T06:46:24.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:23 smithi037 ceph-mon[31211]: pgmap v142: 97 pgs: 97 active+clean; 37 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-19T06:46:24.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:23 smithi151 ceph-mon[35236]: pgmap v142: 97 pgs: 97 active+clean; 37 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 2.7 MiB/s wr, 216 op/s 2024-09-19T06:46:24.479 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:46:24.731 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 -- 172.21.15.37:0/46454768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 msgr2=0x7f74d01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 --2- 172.21.15.37:0/46454768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d01067f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f74c4009900 tx=0x7f74c402a410 comp rx=0 tx=0).stop 2024-09-19T06:46:25.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 -- 172.21.15.37:0/46454768 shutdown_connections 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 --2- 172.21.15.37:0/46454768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74d0106f20 0x7f74d010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 --2- 172.21.15.37:0/46454768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d01067f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 --2- 172.21.15.37:0/46454768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74d0101df0 0x7f74d01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 -- 172.21.15.37:0/46454768 >> 172.21.15.37:0/46454768 conn(0x7f74d00fd8f0 msgr2=0x7f74d00ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 -- 172.21.15.37:0/46454768 shutdown_connections 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.170+0000 7f74d6e53640 1 -- 172.21.15.37:0/46454768 wait complete. 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 Processor -- start 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 -- start start 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74d0101df0 0x7f74d00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74d0106f20 0x7f74d0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0111fa0 con 0x7f74d01027a0 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0111e20 con 0x7f74d0106f20 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d6e53640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74d0112120 con 0x7f74d0101df0 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54024/0 (socket says 172.21.15.37:54024) 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74cffff640 1 -- 172.21.15.37:0/181056210 learned_addr learned my addr 172.21.15.37:0/181056210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.171+0000 7f74d53f0640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74d0106f20 0x7f74d0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74d4bef640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74d0101df0 0x7f74d00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cffff640 1 -- 172.21.15.37:0/181056210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74d0101df0 msgr2=0x7f74d00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cffff640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74d0101df0 0x7f74d00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cffff640 1 -- 172.21.15.37:0/181056210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74d0106f20 msgr2=0x7f74d0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cffff640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74d0106f20 0x7f74d0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cffff640 1 -- 172.21.15.37:0/181056210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74d01a69b0 con 0x7f74d01027a0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cffff640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d0074a30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006010 tx=0x7f74c40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cdffb640 1 -- 172.21.15.37:0/181056210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c403f030 con 0x7f74d01027a0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cdffb640 1 -- 172.21.15.37:0/181056210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c4003ce0 con 0x7f74d01027a0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74d01a6c40 con 0x7f74d01027a0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.172+0000 7f74cdffb640 1 -- 172.21.15.37:0/181056210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4035070 con 0x7f74d01027a0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.173+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74d01a7120 con 0x7f74d01027a0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74cdffb640 1 -- 172.21.15.37:0/181056210 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f74c4042030 con 0x7f74d01027a0 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 Processor -- start 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 -- start start 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2ec090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2ec5d0 0x55e76f2eca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2f7370 0x55e76f2f9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e76f2522f0 con 0x55e76f2f7370 2024-09-19T06:46:25.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e76f1f9cf0 con 0x55e76f2ec5d0 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.175+0000 7f74d922c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e76f24e2b0 con 0x55e76f2edb40 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cdffb640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f74b80a33b0 0x7f74b80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2f7370 0x55e76f2f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2f7370 0x55e76f2f9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54032/0 (socket says 172.21.15.37:54032) 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 -- 172.21.15.37:0/1203525459 learned_addr learned my addr 172.21.15.37:0/1203525459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74d53f0640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f74b80a33b0 0x7f74b80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74d4bef640 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2ec5d0 0x55e76f2eca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cdffb640 1 -- 172.21.15.37:0/181056210 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f74c40f0230 con 0x7f74d01027a0 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74d53f0640 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2ec090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 -- 172.21.15.37:0/1203525459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 msgr2=0x55e76f2ec090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2ec090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 -- 172.21.15.37:0/1203525459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2ec5d0 msgr2=0x55e76f2eca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2ec5d0 0x55e76f2eca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 -- 172.21.15.37:0/1203525459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76f1de3a0 con 0x55e76f2f7370 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74d53f0640 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74cffff640 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2f7370 0x55e76f2f9760 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f74c40c4d10 tx=0x7f74c4003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a54b07232c8a6a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74d53f0640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f74b80a33b0 0x7f74b80a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009fd0 tx=0x7f74c8009390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74b6ffd640 1 -- 172.21.15.37:0/1203525459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c4049030 con 0x55e76f2f7370 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74b6ffd640 1 -- 172.21.15.37:0/1203525459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c403b580 con 0x55e76f2f7370 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.176+0000 7f74b6ffd640 1 -- 172.21.15.37:0/1203525459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40c0980 con 0x55e76f2f7370 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 -- 172.21.15.37:0/1203525459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2f7370 msgr2=0x55e76f2f9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2f7370 0x55e76f2f9760 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f74c40c4d10 tx=0x7f74c4003f90 comp rx=0 tx=0).stop 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 -- 172.21.15.37:0/1203525459 shutdown_connections 2024-09-19T06:46:25.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2f7370 0x55e76f2f9760 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2ec5d0 0x55e76f2eca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 --2- 172.21.15.37:0/1203525459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2ec090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 -- 172.21.15.37:0/1203525459 >> 172.21.15.37:0/1203525459 conn(0x55e76f2b5310 msgr2=0x55e76f2e8c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 -- 172.21.15.37:0/1203525459 shutdown_connections 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.177+0000 7f74d922c740 1 -- 172.21.15.37:0/1203525459 wait complete. 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 -- start start 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 Processor -- start 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2f4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e76f2f9e30 con 0x55e76f2edb40 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e76f20d060 con 0x55e76f2f7370 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e76f2f9fb0 con 0x55e76f2ec5d0 2024-09-19T06:46:25.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d922c740 10 client.0 _ll_register_callbacks cb 0x55e76ef608c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54046/0 (socket says 172.21.15.37:54046) 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74cffff640 1 -- 172.21.15.37:0/895870818 learned_addr learned my addr 172.21.15.37:0/895870818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d53f0640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.178+0000 7f74d4bef640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74cffff640 1 -- 172.21.15.37:0/895870818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 msgr2=0x55e76f2f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74cffff640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74cffff640 1 -- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 msgr2=0x55e76f2f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74cffff640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74cffff640 1 -- 172.21.15.37:0/895870818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76efc33c0 con 0x55e76f2edb40 2024-09-19T06:46:25.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74d53f0640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74cffff640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2f4010 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f74c403d040 tx=0x7f74c4032ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0005b3b0ecc51c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74affff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74affff640 1 -- 172.21.15.37:0/895870818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40f0030 con 0x55e76f2edb40 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74affff640 1 -- 172.21.15.37:0/895870818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f74c40b8c40 con 0x55e76f2edb40 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e76f25f860 con 0x55e76f2edb40 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74affff640 1 -- 172.21.15.37:0/895870818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f74c4049030 con 0x55e76f2edb40 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74d922c740 10 client.16752 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e76f2f9e30 con 0x55e76f2edb40 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.179+0000 7f74affff640 1 -- 172.21.15.37:0/895870818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e76f25f860 con 0x55e76f2edb40 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74affff640 1 -- 172.21.15.37:0/895870818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f74c40bc830 con 0x55e76f2edb40 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d922c740 10 client.16752 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d922c740 10 client.16752 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:46:25.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d922c740 20 client.16752 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d922c740 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f308e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d922c740 4 client.16752 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e76f259d70 con 0x55e76f306a50 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d4bef640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f308e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74affff640 10 client.16752 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.180+0000 7f74d4bef640 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f308e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3b847dc200e1550 server_cookie=5c35c80f2ced282a in_seq=0 out_seq=0 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74affff640 1 -- 172.21.15.37:0/895870818 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e76f259d70 con 0x55e76f306a50 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74affff640 10 client.16752 handle_command_reply: tid=0 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 client.16752 shutdown 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 msgr2=0x55e76f2f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2f4010 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f74c403d040 tx=0x7f74c4032ae0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 shutdown_connections 2024-09-19T06:46:25.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f308e40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74affff640 0 client.16752 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2f4010 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 --2- 172.21.15.37:0/895870818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.181+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 >> 172.21.15.37:0/895870818 conn(0x55e76f2b5310 msgr2=0x55e76f2ee990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.182+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 shutdown_connections 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.182+0000 7f74d922c740 1 -- 172.21.15.37:0/895870818 wait complete. 2024-09-19T06:46:25.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.182+0000 7f74d922c740 20 client.16752 trim_cache size 0 max 16384 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 Processor -- start 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 -- start start 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2ec5d0 0x55e76f303000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2f2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01b2340 con 0x55e76f2ec5d0 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01afc10 con 0x55e76f2f7370 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d922c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ada60 con 0x55e76f2edb40 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d4bef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2ec5d0 0x55e76f303000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.206 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d4bef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2ec5d0 0x55e76f303000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54048/0 (socket says 172.21.15.37:54048) 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d4bef640 1 -- 172.21.15.37:0/552222810 learned_addr learned my addr 172.21.15.37:0/552222810 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.195+0000 7f74d53f0640 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74cffff640 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2f2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74d4bef640 1 -- 172.21.15.37:0/552222810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 msgr2=0x55e76f2f2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74d4bef640 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2f2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74d4bef640 1 -- 172.21.15.37:0/552222810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 msgr2=0x55e76f2f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74d4bef640 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74d4bef640 1 -- 172.21.15.37:0/552222810 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e76f2e8de0 con 0x55e76f2ec5d0 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74d53f0640 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74d4bef640 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2ec5d0 0x55e76f303000 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f74c000b370 tx=0x7f74c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2053b0c1aba4c0e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74b67fc640 1 -- 172.21.15.37:0/552222810 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c0004230 con 0x55e76f2ec5d0 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74b67fc640 1 -- 172.21.15.37:0/552222810 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c00043d0 con 0x55e76f2ec5d0 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.196+0000 7f74b67fc640 1 -- 172.21.15.37:0/552222810 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c001d980 con 0x55e76f2ec5d0 2024-09-19T06:46:25.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 -- 172.21.15.37:0/552222810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2ec5d0 msgr2=0x55e76f303000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2ec5d0 0x55e76f303000 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f74c000b370 tx=0x7f74c0011810 comp rx=0 tx=0).stop 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 -- 172.21.15.37:0/552222810 shutdown_connections 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2f8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2edb40 0x55e76f2f2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 --2- 172.21.15.37:0/552222810 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2ec5d0 0x55e76f303000 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 -- 172.21.15.37:0/552222810 >> 172.21.15.37:0/552222810 conn(0x55e76f2b5310 msgr2=0x55e76f2f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 -- 172.21.15.37:0/552222810 shutdown_connections 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 -- 172.21.15.37:0/552222810 wait complete. 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 -- start start 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.197+0000 7f74d922c740 1 Processor -- start 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2ff100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2ff640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ae0d0 con 0x55e76f2edb40 2024-09-19T06:46:25.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01af540 con 0x55e76f2f7370 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74d01ae7a0 con 0x55e76f2ec5d0 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 10 client.0 _ll_register_callbacks cb 0x7f74c000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2ff100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2ff100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54058/0 (socket says 172.21.15.37:54058) 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 -- 172.21.15.37:0/3259646735 learned_addr learned my addr 172.21.15.37:0/3259646735 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d53f0640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2ff640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d4bef640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 -- 172.21.15.37:0/3259646735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 msgr2=0x55e76f2f1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 -- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 msgr2=0x55e76f2ff640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2ff640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 -- 172.21.15.37:0/3259646735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e76f3091b0 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d4bef640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d53f0640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74cffff640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2ff100 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f74c40bc2d0 tx=0x7f74c4003f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfe6ffe8adf6faf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74affff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74affff640 1 -- 172.21.15.37:0/3259646735 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c40f0040 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74affff640 1 -- 172.21.15.37:0/3259646735 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f74c40bbc20 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.198+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74c00043d0 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74affff640 1 -- 172.21.15.37:0/3259646735 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f74c40bc110 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 10 client.16758 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e76f3091b0 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74affff640 1 -- 172.21.15.37:0/3259646735 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74c00043d0 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74affff640 1 -- 172.21.15.37:0/3259646735 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f74c40cc300 con 0x55e76f2edb40 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 10 client.16758 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:25.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 10 client.16758 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 20 client.16758 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f3737a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 4 client.16758 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e76f1d6170 con 0x55e76f306a50 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.199+0000 7f74d4bef640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f3737a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d4bef640 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f3737a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=452a2abc120fdbe3 server_cookie=a6318046def5df8a in_seq=0 out_seq=0 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74affff640 10 client.16758 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74affff640 1 -- 172.21.15.37:0/3259646735 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e76f1d6170 con 0x55e76f306a50 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74affff640 10 client.16758 handle_command_reply: tid=0 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 client.16758 shutdown 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 msgr2=0x55e76f2ff100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2ff100 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f74c40bc2d0 tx=0x7f74c4003f80 comp rx=0 tx=0).stop 2024-09-19T06:46:25.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 shutdown_connections 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e76f306a50 0x55e76f3737a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e76f2f7370 0x55e76f2ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e76f2edb40 0x55e76f2ff100 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74affff640 0 client.16758 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 --2- 172.21.15.37:0/3259646735 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e76f2ec5d0 0x55e76f2f1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 >> 172.21.15.37:0/3259646735 conn(0x55e76f2b5310 msgr2=0x55e76f2f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 shutdown_connections 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 1 -- 172.21.15.37:0/3259646735 wait complete. 2024-09-19T06:46:25.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.200+0000 7f74d922c740 20 client.16758 trim_cache size 0 max 16384 2024-09-19T06:46:25.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 9.13 MB/sec execute 52 sec latency 17895.394 ms 2024-09-19T06:46:25.213 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:46:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f74b80a33b0 msgr2=0x7f74b80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f74b80a33b0 0x7f74b80a5870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f74c8009fd0 tx=0x7f74c8009390 comp rx=0 tx=0).stop 2024-09-19T06:46:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 msgr2=0x7f74d0074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d0074a30 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f74c4006010 tx=0x7f74c40042f0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 shutdown_connections 2024-09-19T06:46:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f74d0106f20 0x7f74d0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f74b80a33b0 0x7f74b80a5870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f74d01027a0 0x7f74d0074a30 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 --2- 172.21.15.37:0/181056210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f74d0101df0 0x7f74d00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 >> 172.21.15.37:0/181056210 conn(0x7f74d00fd8f0 msgr2=0x7f74d00feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 shutdown_connections 2024-09-19T06:46:25.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:25.201+0000 7f74d6e53640 1 -- 172.21.15.37:0/181056210 wait complete. 2024-09-19T06:46:25.998 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:46:26.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 8.96 MB/sec execute 53 sec latency 18895.422 ms 2024-09-19T06:46:26.255 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:26 smithi096 ceph-mon[35004]: pgmap v143: 97 pgs: 97 active+clean; 36 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-09-19T06:46:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:26 smithi037 ceph-mon[31211]: pgmap v143: 97 pgs: 97 active+clean; 36 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-09-19T06:46:26.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:26 smithi151 ceph-mon[35236]: pgmap v143: 97 pgs: 97 active+clean; 36 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 2.1 MiB/s wr, 119 op/s 2024-09-19T06:46:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 -- 172.21.15.37:0/2406151444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b28102df0 msgr2=0x7f7b281031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 --2- 172.21.15.37:0/2406151444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b28102df0 0x7f7b281031d0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f7b24009ca0 tx=0x7f7b2402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 -- 172.21.15.37:0/2406151444 shutdown_connections 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 --2- 172.21.15.37:0/2406151444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b2810c2f0 0x7f7b2810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 --2- 172.21.15.37:0/2406151444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b281037a0 0x7f7b2810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 --2- 172.21.15.37:0/2406151444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b28102df0 0x7f7b281031d0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 -- 172.21.15.37:0/2406151444 >> 172.21.15.37:0/2406151444 conn(0x7f7b280fc7d0 msgr2=0x7f7b280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 -- 172.21.15.37:0/2406151444 shutdown_connections 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 -- 172.21.15.37:0/2406151444 wait complete. 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 Processor -- start 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.714+0000 7f7b30428640 1 -- start start 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b30428640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b28102df0 0x7f7b28198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b30428640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b281037a0 0x7f7b28198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b30428640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b2810c2f0 0x7f7b2819cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b30428640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281120f0 con 0x7f7b2810c2f0 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b30428640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b28111f70 con 0x7f7b281037a0 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b30428640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b28112270 con 0x7f7b28102df0 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b2810c2f0 0x7f7b2819cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b2810c2f0 0x7f7b2819cf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54084/0 (socket says 172.21.15.37:54084) 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 -- 172.21.15.37:0/1727494245 learned_addr learned my addr 172.21.15.37:0/1727494245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b28102df0 0x7f7b28198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b281037a0 0x7f7b28198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 -- 172.21.15.37:0/1727494245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b28102df0 msgr2=0x7f7b28198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b28102df0 0x7f7b28198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 -- 172.21.15.37:0/1727494245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b281037a0 msgr2=0x7f7b28198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b281037a0 0x7f7b28198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 -- 172.21.15.37:0/1727494245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b2819d600 con 0x7f7b2810c2f0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b281037a0 0x7f7b28198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b2e9c5640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b2810c2f0 0x7f7b2819cf20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c970 tx=0x7f7b2000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.715+0000 7f7b1f7fe640 1 -- 172.21.15.37:0/1727494245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b20007bb0 con 0x7f7b2810c2f0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.716+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b2819d8f0 con 0x7f7b2810c2f0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.716+0000 7f7b1f7fe640 1 -- 172.21.15.37:0/1727494245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b20007d50 con 0x7f7b2810c2f0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.716+0000 7f7b1f7fe640 1 -- 172.21.15.37:0/1727494245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b2000f450 con 0x7f7b2810c2f0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.716+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b281a51d0 con 0x7f7b2810c2f0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.718+0000 7f7b1f7fe640 1 -- 172.21.15.37:0/1727494245 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7b20004f60 con 0x7f7b2810c2f0 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.718+0000 7f7b32801740 1 Processor -- start 2024-09-19T06:46:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.718+0000 7f7b32801740 1 -- start start 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b32801740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a84b40 0x564fa1a83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b32801740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 0x564fa1a83a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b32801740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a90760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b32801740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fa19e92f0 con 0x564fa1a84b40 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b32801740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564fa1990cf0 con 0x564fa1a8e370 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b32801740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564fa19e52b0 con 0x564fa1a835d0 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a84b40 0x564fa1a83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a84b40 0x564fa1a83090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54096/0 (socket says 172.21.15.37:54096) 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/1756344211 learned_addr learned my addr 172.21.15.37:0/1756344211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b1f7fe640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b140a33b0 0x7f7b140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a90760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/1756344211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 msgr2=0x564fa1a83a10 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 0x564fa1a83a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/1756344211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 msgr2=0x564fa1a90760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a90760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/1756344211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa19753a0 con 0x564fa1a84b40 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2e9c5640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b140a33b0 0x7f7b140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b1f7fe640 1 -- 172.21.15.37:0/1727494245 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7b200cf9d0 con 0x7f7b2810c2f0 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.719+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a84b40 0x564fa1a83090 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005fd0 tx=0x7f7b180095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6b06217916705 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b1cdea640 1 -- 172.21.15.37:0/1756344211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1801f070 con 0x564fa1a84b40 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b1cdea640 1 -- 172.21.15.37:0/1756344211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b1801a930 con 0x564fa1a84b40 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b1cdea640 1 -- 172.21.15.37:0/1756344211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18010820 con 0x564fa1a84b40 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 -- 172.21.15.37:0/1756344211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a84b40 msgr2=0x564fa1a83090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b2e9c5640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b140a33b0 0x7f7b140a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b200cd1d0 tx=0x7f7b20091140 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a84b40 0x564fa1a83090 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f7b18005fd0 tx=0x7f7b180095d0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 -- 172.21.15.37:0/1756344211 shutdown_connections 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a90760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 0x564fa1a83a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 --2- 172.21.15.37:0/1756344211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a84b40 0x564fa1a83090 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 -- 172.21.15.37:0/1756344211 >> 172.21.15.37:0/1756344211 conn(0x564fa1a4c310 msgr2=0x564fa1a7fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.720+0000 7f7b32801740 1 -- 172.21.15.37:0/1756344211 shutdown_connections 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 -- 172.21.15.37:0/1756344211 wait complete. 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 -- start start 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 Processor -- start 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a8dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a84b40 0x564fa1a8b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a8b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564fa1a91070 con 0x564fa1a835d0 2024-09-19T06:46:26.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564fa19a4060 con 0x564fa1a8e370 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564fa1a911f0 con 0x564fa1a84b40 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b32801740 10 client.0 _ll_register_callbacks cb 0x564fa16f78c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a8dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a8dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54108/0 (socket says 172.21.15.37:54108) 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/2418878079 learned_addr learned my addr 172.21.15.37:0/2418878079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.721+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a84b40 0x564fa1a8b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a8b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/2418878079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a84b40 msgr2=0x564fa1a8b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a84b40 0x564fa1a8b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 msgr2=0x564fa1a8b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a8b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/2418878079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa175a3c0 con 0x564fa1a835d0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a8dd80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7b18015320 tx=0x7f7b18009380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2cd29fe8c19fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b11ffb640 1 -- 172.21.15.37:0/2418878079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1801f070 con 0x564fa1a835d0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b1801a930 con 0x564fa1a835d0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b11ffb640 1 -- 172.21.15.37:0/2418878079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7b18005050 con 0x564fa1a835d0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b11ffb640 1 -- 172.21.15.37:0/2418878079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7b18005250 con 0x564fa1a835d0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b32801740 10 client.16770 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564fa1a9cb40 con 0x564fa1a835d0 2024-09-19T06:46:26.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.722+0000 7f7b11ffb640 1 -- 172.21.15.37:0/2418878079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b1801a930 con 0x564fa1a835d0 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b11ffb640 1 -- 172.21.15.37:0/2418878079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7b1800c030 con 0x564fa1a835d0 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b32801740 10 client.16770 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b32801740 10 client.16770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b32801740 20 client.16770 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b32801740 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1a9de90 0x564fa1aa0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b32801740 4 client.16770 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fa19f6860 con 0x564fa1a9de90 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1a9de90 0x564fa1aa0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b11ffb640 10 client.16770 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.723+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1a9de90 0x564fa1aa0280 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e245aa958e7189f3 server_cookie=88ae6e8a44fee9a2 in_seq=0 out_seq=0 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b11ffb640 1 -- 172.21.15.37:0/2418878079 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fa19f6860 con 0x564fa1a9de90 2024-09-19T06:46:26.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b11ffb640 10 client.16770 handle_command_reply: tid=0 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 client.16770 shutdown 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 msgr2=0x564fa1a8dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a8dd80 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f7b18015320 tx=0x7f7b18009380 comp rx=0 tx=0).stop 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 shutdown_connections 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1a9de90 0x564fa1aa0280 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a8e370 0x564fa1a8b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a84b40 0x564fa1a8b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b11ffb640 0 client.16770 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 --2- 172.21.15.37:0/2418878079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a8dd80 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.724+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 >> 172.21.15.37:0/2418878079 conn(0x564fa1a4c310 msgr2=0x564fa1a85960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:26.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.725+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 shutdown_connections 2024-09-19T06:46:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.725+0000 7f7b32801740 1 -- 172.21.15.37:0/2418878079 wait complete. 2024-09-19T06:46:26.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.725+0000 7f7b32801740 20 client.16770 trim_cache size 0 max 16384 2024-09-19T06:46:26.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 Processor -- start 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 -- start start 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a9a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1a8a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a8e370 0x564fa1aa32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281ae060 con 0x564fa1a835d0 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281ab930 con 0x564fa1a84b40 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b32801740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281a9780 con 0x564fa1a8e370 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a9a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a9a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54110/0 (socket says 172.21.15.37:54110) 2024-09-19T06:46:26.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/120933729 learned_addr learned my addr 172.21.15.37:0/120933729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1a8a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/120933729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a8e370 msgr2=0x564fa1aa32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a8e370 0x564fa1aa32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/120933729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 msgr2=0x564fa1a8a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1a8a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/120933729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fa1a89580 con 0x564fa1a835d0 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.738+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1a8a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a9a490 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009ac0 tx=0x7f7b18009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a54353133911810 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b13fff640 1 -- 172.21.15.37:0/120933729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b1801f030 con 0x564fa1a835d0 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b13fff640 1 -- 172.21.15.37:0/120933729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b18020040 con 0x564fa1a835d0 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b13fff640 1 -- 172.21.15.37:0/120933729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b18005650 con 0x564fa1a835d0 2024-09-19T06:46:26.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 -- 172.21.15.37:0/120933729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 msgr2=0x564fa1a9a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a9a490 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009ac0 tx=0x7f7b18009f20 comp rx=0 tx=0).stop 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 -- 172.21.15.37:0/120933729 shutdown_connections 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a8e370 0x564fa1aa32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1a8a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 --2- 172.21.15.37:0/120933729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a835d0 0x564fa1a9a490 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 -- 172.21.15.37:0/120933729 >> 172.21.15.37:0/120933729 conn(0x564fa1a4c310 msgr2=0x564fa1a89900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 -- 172.21.15.37:0/120933729 shutdown_connections 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.739+0000 7f7b32801740 1 -- 172.21.15.37:0/120933729 wait complete. 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 -- start start 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 Processor -- start 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 0x564fa1aa0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1aa0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a8e370 0x564fa1b00790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281a9df0 con 0x564fa1a8e370 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281ab260 con 0x564fa1a84b40 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b281aa4c0 con 0x564fa1a835d0 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b32801740 10 client.0 _ll_register_callbacks cb 0x7f7b1801d070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a8e370 0x564fa1b00790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b2e1c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1aa0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b2d9c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a8e370 0x564fa1b00790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54126/0 (socket says 172.21.15.37:54126) 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b2d9c3640 1 -- 172.21.15.37:0/3193877752 learned_addr learned my addr 172.21.15.37:0/3193877752 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.740+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 0x564fa1aa0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b2e1c4640 1 -- 172.21.15.37:0/3193877752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 msgr2=0x564fa1aa0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:26.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 0x564fa1aa0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b2e1c4640 1 -- 172.21.15.37:0/3193877752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a8e370 msgr2=0x564fa1b00790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a8e370 0x564fa1b00790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b2e1c4640 1 -- 172.21.15.37:0/3193877752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fa1b00e30 con 0x564fa1a84b40 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b2e1c4640 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1aa0ed0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b24000f80 tx=0x7f7b24004aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6169b4cd41f526 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b11ffb640 1 -- 172.21.15.37:0/3193877752 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b24003c40 con 0x564fa1a84b40 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b18020040 con 0x564fa1a84b40 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b11ffb640 1 -- 172.21.15.37:0/3193877752 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7b24003de0 con 0x564fa1a84b40 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b11ffb640 1 -- 172.21.15.37:0/3193877752 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7b24058050 con 0x564fa1a84b40 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b32801740 10 client.26240 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564fa1b081f0 con 0x564fa1a84b40 2024-09-19T06:46:26.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.741+0000 7f7b11ffb640 1 -- 172.21.15.37:0/3193877752 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b18020040 con 0x564fa1a84b40 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b11ffb640 1 -- 172.21.15.37:0/3193877752 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7b24049450 con 0x564fa1a84b40 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 10 client.26240 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 10 client.26240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 20 client.26240 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1b08950 0x564fa1b0ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 4 client.26240 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fa19f0d70 con 0x564fa1b08950 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1b08950 0x564fa1b0ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b2d9c3640 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1b08950 0x564fa1b0ad70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7227591721f842fc server_cookie=c903ccff0d052aa in_seq=0 out_seq=0 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b11ffb640 10 client.26240 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b11ffb640 1 -- 172.21.15.37:0/3193877752 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fa19f0d70 con 0x564fa1b08950 2024-09-19T06:46:26.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b11ffb640 10 client.26240 handle_command_reply: tid=0 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 1 client.26240 shutdown 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 msgr2=0x564fa1aa0ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.742+0000 7f7b32801740 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1aa0ed0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7b24000f80 tx=0x7f7b24004aa0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 shutdown_connections 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564fa1b08950 0x564fa1b0ad70 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564fa1a8e370 0x564fa1b00790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564fa1a84b40 0x564fa1aa0ed0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b11ffb640 0 client.26240 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 --2- 172.21.15.37:0/3193877752 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564fa1a835d0 0x564fa1aa0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 >> 172.21.15.37:0/3193877752 conn(0x564fa1a4c310 msgr2=0x564fa1a85110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 shutdown_connections 2024-09-19T06:46:26.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 1 -- 172.21.15.37:0/3193877752 wait complete. 2024-09-19T06:46:26.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b32801740 20 client.26240 trim_cache size 0 max 16384 2024-09-19T06:46:26.756 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:46:26.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b140a33b0 msgr2=0x7f7b140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b140a33b0 0x7f7b140a5870 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7b200cd1d0 tx=0x7f7b20091140 comp rx=0 tx=0).stop 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b2810c2f0 msgr2=0x7f7b2819cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b2810c2f0 0x7f7b2819cf20 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000c970 tx=0x7f7b2000ce30 comp rx=0 tx=0).stop 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 shutdown_connections 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b140a33b0 0x7f7b140a5870 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b2810c2f0 0x7f7b2819cf20 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b281037a0 0x7f7b28198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 --2- 172.21.15.37:0/1727494245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b28102df0 0x7f7b28198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 >> 172.21.15.37:0/1727494245 conn(0x7f7b280fc7d0 msgr2=0x7f7b280fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.743+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 shutdown_connections 2024-09-19T06:46:26.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:26.744+0000 7f7b30428640 1 -- 172.21.15.37:0/1727494245 wait complete. 2024-09-19T06:46:27.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 8.79 MB/sec execute 54 sec latency 19895.488 ms 2024-09-19T06:46:28.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 8.63 MB/sec execute 55 sec latency 20895.602 ms 2024-09-19T06:46:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:28 smithi096 ceph-mon[35004]: pgmap v144: 97 pgs: 97 active+clean; 36 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-19T06:46:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:28 smithi037 ceph-mon[31211]: pgmap v144: 97 pgs: 97 active+clean; 36 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-19T06:46:28.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:28 smithi151 ceph-mon[35236]: pgmap v144: 97 pgs: 97 active+clean; 36 MiB data, 945 MiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 1.9 MiB/s wr, 95 op/s 2024-09-19T06:46:28.592 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:46:28.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:46:28.863 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:29.202 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 8.48 MB/sec execute 56 sec latency 21895.691 ms 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 -- 172.21.15.37:0/2718906580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b41036e0 msgr2=0x7ff2b4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2718906580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b41036e0 0x7ff2b4107b90 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8009930 tx=0x7ff2a802a420 comp rx=0 tx=0).stop 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 -- 172.21.15.37:0/2718906580 shutdown_connections 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2718906580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2b41082a0 0x7ff2b410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2718906580 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b41036e0 0x7ff2b4107b90 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2718906580 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b4102d30 0x7ff2b4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 -- 172.21.15.37:0/2718906580 >> 172.21.15.37:0/2718906580 conn(0x7ff2b40fc7d0 msgr2=0x7ff2b40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 -- 172.21.15.37:0/2718906580 shutdown_connections 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 -- 172.21.15.37:0/2718906580 wait complete. 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.326+0000 7ff2b940b640 1 Processor -- start 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b940b640 1 -- start start 2024-09-19T06:46:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b940b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102d30 0x7ff2b4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b940b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2b41036e0 0x7ff2b4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b940b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b41082a0 0x7ff2b4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b940b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41120e0 con 0x7ff2b4102d30 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b940b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4111f60 con 0x7ff2b41036e0 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b940b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b4112260 con 0x7ff2b41082a0 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102d30 0x7ff2b4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102d30 0x7ff2b4100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54146/0 (socket says 172.21.15.37:54146) 2024-09-19T06:46:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 -- 172.21.15.37:0/2383403260 learned_addr learned my addr 172.21.15.37:0/2383403260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b41082a0 0x7ff2b4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 -- 172.21.15.37:0/2383403260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b41082a0 msgr2=0x7ff2b4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2b41036e0 0x7ff2b4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b41082a0 0x7ff2b4101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 -- 172.21.15.37:0/2383403260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2b41036e0 msgr2=0x7ff2b4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2b41036e0 0x7ff2b4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 -- 172.21.15.37:0/2383403260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2b41a6b20 con 0x7ff2b4102d30 2024-09-19T06:46:29.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2b41036e0 0x7ff2b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.327+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102d30 0x7ff2b4100990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b4a0 tx=0x7ff2a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.328+0000 7ff29bfff640 1 -- 172.21.15.37:0/2383403260 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4014070 con 0x7ff2b4102d30 2024-09-19T06:46:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.328+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2b41a6e10 con 0x7ff2b4102d30 2024-09-19T06:46:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.328+0000 7ff29bfff640 1 -- 172.21.15.37:0/2383403260 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2a4010af0 con 0x7ff2b4102d30 2024-09-19T06:46:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.328+0000 7ff29bfff640 1 -- 172.21.15.37:0/2383403260 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a4017d10 con 0x7ff2b4102d30 2024-09-19T06:46:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.328+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2b41a7270 con 0x7ff2b4102d30 2024-09-19T06:46:29.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.330+0000 7ff29bfff640 1 -- 172.21.15.37:0/2383403260 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff2a4017070 con 0x7ff2b4102d30 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.330+0000 7ff2bb7e4740 1 Processor -- start 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.330+0000 7ff2bb7e4740 1 -- start start 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917588b60 0x560917588f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x560917593800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917596130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5609174ee280 con 0x560917588b60 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560917495b10 con 0x560917589480 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5609174ea240 con 0x560917593d40 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff29bfff640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff29c0a33b0 0x7ff29c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917588b60 0x560917588f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917588b60 0x560917588f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54150/0 (socket says 172.21.15.37:54150) 2024-09-19T06:46:29.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3945610007 learned_addr learned my addr 172.21.15.37:0/3945610007 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff29c0a33b0 0x7ff29c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917596130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x560917593800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff29bfff640 1 -- 172.21.15.37:0/2383403260 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff2a40cf0f0 con 0x7ff2b4102d30 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3945610007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 msgr2=0x560917596130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917596130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3945610007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 msgr2=0x560917593800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x560917593800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3945610007 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609174a21f0 con 0x560917588b60 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917596130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.331+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917588b60 0x560917588f40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012cf0 tx=0x7ff2ac00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ef4ee3748b9484 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff29c0a33b0 0x7ff29c0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00afd0 tx=0x7ff2ac01a000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2997fa640 1 -- 172.21.15.37:0/3945610007 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac020070 con 0x560917588b60 2024-09-19T06:46:29.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2997fa640 1 -- 172.21.15.37:0/3945610007 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac00aad0 con 0x560917588b60 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2997fa640 1 -- 172.21.15.37:0/3945610007 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac01cbb0 con 0x560917588b60 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3945610007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917588b60 msgr2=0x560917588f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917588b60 0x560917588f40 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012cf0 tx=0x7ff2ac00f730 comp rx=0 tx=0).stop 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3945610007 shutdown_connections 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917596130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x560917593800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3945610007 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917588b60 0x560917588f40 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3945610007 >> 172.21.15.37:0/3945610007 conn(0x5609175850f0 msgr2=0x5609175854d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3945610007 shutdown_connections 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.332+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3945610007 wait complete. 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 -- start start 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 Processor -- start 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917590ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x5609175911e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917593d40 0x560917591720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560917596910 con 0x560917589480 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5609174a8f10 con 0x560917593d40 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560917596a90 con 0x560917588b60 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2bb7e4740 10 client.0 _ll_register_callbacks cb 0x560917471a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x5609175911e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x5609175911e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54162/0 (socket says 172.21.15.37:54162) 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2b2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917590ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2b27fc640 1 -- 172.21.15.37:0/3448491521 learned_addr learned my addr 172.21.15.37:0/3448491521 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.333+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917593d40 0x560917591720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2b27fc640 1 -- 172.21.15.37:0/3448491521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 msgr2=0x560917590ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917590ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2b27fc640 1 -- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917593d40 msgr2=0x560917591720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917593d40 0x560917591720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2b27fc640 1 -- 172.21.15.37:0/3448491521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56091725f6a0 con 0x560917589480 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x5609175911e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40697d0 tx=0x7ff2a8031920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe06e6bcf64013a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff296ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff296ff5640 1 -- 172.21.15.37:0/3448491521 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8035030 con 0x560917589480 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff296ff5640 1 -- 172.21.15.37:0/3448491521 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff2a8004460 con 0x560917589480 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ac00aad0 con 0x560917589480 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff296ff5640 1 -- 172.21.15.37:0/3448491521 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2a8005030 con 0x560917589480 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2bb7e4740 10 client.16788 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5609175a2590 con 0x560917589480 2024-09-19T06:46:29.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.334+0000 7ff296ff5640 1 -- 172.21.15.37:0/3448491521 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac00aad0 con 0x560917589480 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff296ff5640 1 -- 172.21.15.37:0/3448491521 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff2a80035f0 con 0x560917589480 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff2bb7e4740 10 client.16788 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff2bb7e4740 10 client.16788 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff2bb7e4740 20 client.16788 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5609175a39f0 0x5609175a5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff2bb7e4740 4 client.16788 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609174fb7f0 con 0x5609175a39f0 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.335+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5609175a39f0 0x5609175a5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.336+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5609175a39f0 0x5609175a5e30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0cbae61d54793b server_cookie=81a95782a2bd63c4 in_seq=0 out_seq=0 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.336+0000 7ff296ff5640 10 client.16788 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.336+0000 7ff296ff5640 1 -- 172.21.15.37:0/3448491521 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5609174fb7f0 con 0x5609175a39f0 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.336+0000 7ff296ff5640 10 client.16788 handle_command_reply: tid=0 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.336+0000 7ff2bb7e4740 1 client.16788 shutdown 2024-09-19T06:46:29.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 msgr2=0x5609175911e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x5609175911e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff2b40697d0 tx=0x7ff2a8031920 comp rx=0 tx=0).stop 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 shutdown_connections 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5609175a39f0 0x5609175a5e30 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917593d40 0x560917591720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff296ff5640 0 client.16788 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x5609175911e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3448491521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917590ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 >> 172.21.15.37:0/3448491521 conn(0x5609175850f0 msgr2=0x56091758ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 shutdown_connections 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3448491521 wait complete. 2024-09-19T06:46:29.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.337+0000 7ff2bb7e4740 20 client.16788 trim_cache size 0 max 16384 2024-09-19T06:46:29.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.354+0000 7ff2bb7e4740 1 Processor -- start 2024-09-19T06:46:29.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.354+0000 7ff2bb7e4740 1 -- start start 2024-09-19T06:46:29.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.354+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917591570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.354+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x5609175ab700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917593d40 0x5609175abc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41b24e0 con 0x560917593d40 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41afdb0 con 0x560917589480 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41adc00 con 0x560917588b60 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917593d40 0x5609175abc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917593d40 0x5609175abc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54166/0 (socket says 172.21.15.37:54166) 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3270205157 learned_addr learned my addr 172.21.15.37:0/3270205157 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917591570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x5609175ab700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3270205157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 msgr2=0x560917591570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917591570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3270205157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 msgr2=0x5609175ab700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x5609175ab700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 -- 172.21.15.37:0/3270205157 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56091757f680 con 0x560917593d40 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x5609175ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917593d40 0x5609175abc40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012cc0 tx=0x7ff2ac01c670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2786798b3d4c66e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff298ff9640 1 -- 172.21.15.37:0/3270205157 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac020040 con 0x560917593d40 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff298ff9640 1 -- 172.21.15.37:0/3270205157 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac012050 con 0x560917593d40 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.355+0000 7ff298ff9640 1 -- 172.21.15.37:0/3270205157 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ac01c840 con 0x560917593d40 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3270205157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917593d40 msgr2=0x5609175abc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917593d40 0x5609175abc40 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac012cc0 tx=0x7ff2ac01c670 comp rx=0 tx=0).stop 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3270205157 shutdown_connections 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917593d40 0x5609175abc40 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917589480 0x5609175ab700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/3270205157 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917588b60 0x560917591570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3270205157 >> 172.21.15.37:0/3270205157 conn(0x5609175850f0 msgr2=0x560917592c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3270205157 shutdown_connections 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/3270205157 wait complete. 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.356+0000 7ff2bb7e4740 1 -- start start 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 1 Processor -- start 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917588b60 0x56091759bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x56091759c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917598cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41ae270 con 0x560917589480 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41af6e0 con 0x560917588b60 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff2b41ae940 con 0x560917593d40 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2bb7e4740 10 client.0 _ll_register_callbacks cb 0x7ff2a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x56091759c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x56091759c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54174/0 (socket says 172.21.15.37:54174) 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 -- 172.21.15.37:0/2071815754 learned_addr learned my addr 172.21.15.37:0/2071815754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b37fe640 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917598cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917588b60 0x56091759bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 -- 172.21.15.37:0/2071815754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 msgr2=0x560917598cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917598cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 -- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917588b60 msgr2=0x56091759bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917588b60 0x56091759bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.357+0000 7ff2b27fc640 1 -- 172.21.15.37:0/2071815754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609175991f0 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2b27fc640 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x56091759c220 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8032c00 tx=0x7ff2a8003fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c605061dcf9838a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff296ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff296ff5640 1 -- 172.21.15.37:0/2071815754 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8035070 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff296ff5640 1 -- 172.21.15.37:0/2071815754 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff2a80051d0 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ac012050 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff296ff5640 1 -- 172.21.15.37:0/2071815754 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff2a805c050 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 10 client.16794 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56091760ed50 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff296ff5640 1 -- 172.21.15.37:0/2071815754 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff2ac012050 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff296ff5640 1 -- 172.21.15.37:0/2071815754 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff2a8004ba0 con 0x560917589480 2024-09-19T06:46:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 10 client.16794 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 10 client.16794 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 20 client.16794 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56091760f4a0 0x5609176118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 4 client.16794 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.358+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5609174f5d00 con 0x56091760f4a0 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.359+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56091760f4a0 0x5609176118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.359+0000 7ff2b2ffd640 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56091760f4a0 0x5609176118c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ed737df6bd9a1d server_cookie=221c496cd5c06526 in_seq=0 out_seq=0 2024-09-19T06:46:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.359+0000 7ff296ff5640 10 client.16794 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff296ff5640 1 -- 172.21.15.37:0/2071815754 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5609174f5d00 con 0x56091760f4a0 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff296ff5640 10 client.16794 handle_command_reply: tid=0 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 client.16794 shutdown 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 msgr2=0x56091759c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x56091759c220 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8032c00 tx=0x7ff2a8003fe0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 shutdown_connections 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56091760f4a0 0x5609176118c0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560917593d40 0x560917598cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560917589480 0x56091759c220 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 --2- 172.21.15.37:0/2071815754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560917588b60 0x56091759bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff296ff5640 0 client.16794 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 >> 172.21.15.37:0/2071815754 conn(0x5609175850f0 msgr2=0x560917592310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 shutdown_connections 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.360+0000 7ff2bb7e4740 1 -- 172.21.15.37:0/2071815754 wait complete. 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2bb7e4740 20 client.16794 trim_cache size 0 max 16384 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:46:29.368 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "910880f4-7a87-40bd-a15b-9c6df92e7312", 2024-09-19T06:46:29.369 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:46:29.369 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:46:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff29c0a33b0 msgr2=0x7ff29c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff29c0a33b0 0x7ff29c0a5870 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00afd0 tx=0x7ff2ac01a000 comp rx=0 tx=0).stop 2024-09-19T06:46:29.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102d30 msgr2=0x7ff2b4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102d30 0x7ff2b4100990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff2a400b4a0 tx=0x7ff2a400b960 comp rx=0 tx=0).stop 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 shutdown_connections 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff2b41082a0 0x7ff2b4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2b41036e0 0x7ff2b4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff29c0a33b0 0x7ff29c0a5870 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 --2- 172.21.15.37:0/2383403260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff2b4102d30 0x7ff2b4100990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 >> 172.21.15.37:0/2383403260 conn(0x7ff2b40fc7d0 msgr2=0x7ff2b4105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 shutdown_connections 2024-09-19T06:46:29.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:29.361+0000 7ff2b940b640 1 -- 172.21.15.37:0/2383403260 wait complete. 2024-09-19T06:46:30.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 8.33 MB/sec execute 57 sec latency 22895.740 ms 2024-09-19T06:46:30.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:46:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:30 smithi096 ceph-mon[35004]: pgmap v145: 97 pgs: 97 active+clean; 34 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.9 MiB/s wr, 132 op/s 2024-09-19T06:46:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:30 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:46:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:30 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:30 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:46:30.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:30 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:30 smithi037 ceph-mon[31211]: pgmap v145: 97 pgs: 97 active+clean; 34 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.9 MiB/s wr, 132 op/s 2024-09-19T06:46:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:30 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:46:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:30 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:30 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:46:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:30 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:30.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:30 smithi151 ceph-mon[35236]: pgmap v145: 97 pgs: 97 active+clean; 34 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 2.9 MiB/s wr, 132 op/s 2024-09-19T06:46:30.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:30 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:46:30.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:30 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:30.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:30 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:46:30.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:30 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:30.513 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1717437979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c1082e0 msgr2=0x7fd09c10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1717437979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c1082e0 0x7fd09c10ec50 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fd094009cd0 tx=0x7fd09402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1717437979 shutdown_connections 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1717437979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c1082e0 0x7fd09c10ec50 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1717437979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd09c103760 0x7fd09c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1717437979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd09c102db0 0x7fd09c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1717437979 >> 172.21.15.37:0/1717437979 conn(0x7fd09c0fc7d0 msgr2=0x7fd09c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1717437979 shutdown_connections 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1717437979 wait complete. 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 Processor -- start 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.967+0000 7fd0a41c6640 1 -- start start 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a41c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c102db0 0x7fd09c1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a41c6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd09c103760 0x7fd09c1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a41c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd09c1082e0 0x7fd09c19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a41c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1120b0 con 0x7fd09c102db0 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a41c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c111f30 con 0x7fd09c1082e0 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a41c6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c112230 con 0x7fd09c103760 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c102db0 0x7fd09c1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c102db0 0x7fd09c1a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58402/0 (socket says 172.21.15.37:58402) 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1088904221 learned_addr learned my addr 172.21.15.37:0/1088904221 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a2763640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd09c1082e0 0x7fd09c19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1761640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd09c103760 0x7fd09c1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1088904221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd09c103760 msgr2=0x7fd09c1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd09c103760 0x7fd09c1a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1088904221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd09c1082e0 msgr2=0x7fd09c19a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:30.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd09c1082e0 0x7fd09c19a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1088904221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd09c19acc0 con 0x7fd09c102db0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a2763640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd09c1082e0 0x7fd09c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c102db0 0x7fd09c1a03d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd09800d950 tx=0x7fd09800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd08effd640 1 -- 172.21.15.37:0/1088904221 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd098017030 con 0x7fd09c102db0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd09c19afb0 con 0x7fd09c102db0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd08effd640 1 -- 172.21.15.37:0/1088904221 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd098004020 con 0x7fd09c102db0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.968+0000 7fd08effd640 1 -- 172.21.15.37:0/1088904221 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980047a0 con 0x7fd09c102db0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.969+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd09c1a72a0 con 0x7fd09c102db0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.971+0000 7fd08effd640 1 -- 172.21.15.37:0/1088904221 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd0980041d0 con 0x7fd09c102db0 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.971+0000 7fd0a659f740 1 Processor -- start 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.971+0000 7fd0a659f740 1 -- start start 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.971+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c10880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.971+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c1b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c1da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a659f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b11b762e0 con 0x560b11c104a0 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a659f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560b11b1dce0 con 0x560b11c10dc0 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a659f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b11b722a0 con 0x560b11c1b680 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c10880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c10880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58414/0 (socket says 172.21.15.37:58414) 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 -- 172.21.15.37:0/23007555 learned_addr learned my addr 172.21.15.37:0/23007555 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd08effd640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd0880a33b0 0x7fd0880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c1b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a1761640 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c1da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd08effd640 1 -- 172.21.15.37:0/1088904221 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd0980cf7e0 con 0x7fd09c102db0 2024-09-19T06:46:30.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 -- 172.21.15.37:0/23007555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 msgr2=0x560b11c1da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c1da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 -- 172.21.15.37:0/23007555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 msgr2=0x560b11c1b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c1b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 -- 172.21.15.37:0/23007555 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b11b02440 con 0x560b11c104a0 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd0880a33b0 0x7fd0880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.972+0000 7fd0a2763640 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c10880 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd094005e30 tx=0x7fd094005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5876f160701c9e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd087fff640 1 -- 172.21.15.37:0/23007555 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09404d070 con 0x560b11c104a0 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd087fff640 1 -- 172.21.15.37:0/23007555 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd094043070 con 0x560b11c104a0 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd087fff640 1 -- 172.21.15.37:0/23007555 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd094030b80 con 0x560b11c104a0 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a659f740 1 -- 172.21.15.37:0/23007555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 msgr2=0x560b11c10880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:30.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a2763640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd0880a33b0 0x7fd0880a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd09404a000 tx=0x7fd09404b000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a659f740 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c10880 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fd094005e30 tx=0x7fd094005f70 comp rx=0 tx=0).stop 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a659f740 1 -- 172.21.15.37:0/23007555 shutdown_connections 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a659f740 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c1da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a659f740 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c1b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a659f740 1 --2- 172.21.15.37:0/23007555 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c10880 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.973+0000 7fd0a659f740 1 -- 172.21.15.37:0/23007555 >> 172.21.15.37:0/23007555 conn(0x560b11c0ca30 msgr2=0x560b11c0ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 -- 172.21.15.37:0/23007555 shutdown_connections 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 -- 172.21.15.37:0/23007555 wait complete. 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 -- start start 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 Processor -- start 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c1afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c10dc0 0x560b11c18280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c1b680 0x560b11c187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560b11c1e250 con 0x560b11c1b680 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560b11b31050 con 0x560b11c104a0 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560b11c1e3d0 con 0x560b11c10dc0 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a659f740 10 client.0 _ll_register_callbacks cb 0x560b11b71160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.974+0000 7fd0a2763640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c1b680 0x560b11c187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c1b680 0x560b11c187e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58416/0 (socket says 172.21.15.37:58416) 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 -- 172.21.15.37:0/447207133 learned_addr learned my addr 172.21.15.37:0/447207133 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c1afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a1761640 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c10dc0 0x560b11c18280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 -- 172.21.15.37:0/447207133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c10dc0 msgr2=0x560b11c18280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:30.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c10dc0 0x560b11c18280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 -- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 msgr2=0x560b11c1afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c1afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 -- 172.21.15.37:0/447207133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b118e73c0 con 0x560b11c1b680 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a2763640 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c1b680 0x560b11c187e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd094004af0 tx=0x7fd0940307c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cedab2b3b74450 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0857fa640 1 -- 172.21.15.37:0/447207133 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09404d050 con 0x560b11c1b680 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0857fa640 1 -- 172.21.15.37:0/447207133 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd094030940 con 0x560b11c1b680 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.975+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd094043070 con 0x560b11c1b680 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0857fa640 1 -- 172.21.15.37:0/447207133 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd094004860 con 0x560b11c1b680 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 10 client.16806 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b11c29c50 con 0x560b11c1b680 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0857fa640 1 -- 172.21.15.37:0/447207133 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd094043070 con 0x560b11c1b680 2024-09-19T06:46:30.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0857fa640 1 -- 172.21.15.37:0/447207133 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd094030070 con 0x560b11c1b680 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 10 client.16806 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 20 client.16806 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c2afa0 0x560b11c2d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 4 client.16806 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b11b83850 con 0x560b11c2afa0 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.976+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c2afa0 0x560b11c2d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.977+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c2afa0 0x560b11c2d390 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8658cc384662978 server_cookie=125c796420d99764 in_seq=0 out_seq=0 2024-09-19T06:46:30.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.977+0000 7fd0857fa640 10 client.16806 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.977+0000 7fd0857fa640 1 -- 172.21.15.37:0/447207133 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560b11b83850 con 0x560b11c2afa0 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.977+0000 7fd0857fa640 10 client.16806 handle_command_reply: tid=0 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 client.16806 shutdown 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c1b680 msgr2=0x560b11c187e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c1b680 0x560b11c187e0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fd094004af0 tx=0x7fd0940307c0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 shutdown_connections 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c2afa0 0x560b11c2d390 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c1b680 0x560b11c187e0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c10dc0 0x560b11c18280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0857fa640 0 client.16806 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:30.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 --2- 172.21.15.37:0/447207133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c1afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:30.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 >> 172.21.15.37:0/447207133 conn(0x560b11c0ca30 msgr2=0x560b11c0bb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:30.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 shutdown_connections 2024-09-19T06:46:30.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 1 -- 172.21.15.37:0/447207133 wait complete. 2024-09-19T06:46:30.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.978+0000 7fd0a659f740 20 client.16806 trim_cache size 0 max 16384 2024-09-19T06:46:31.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 Processor -- start 2024-09-19T06:46:31.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 -- start start 2024-09-19T06:46:31.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c303b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c154e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1b24f0 con 0x560b11c104a0 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1afdc0 con 0x560b11c10dc0 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.995+0000 7fd0a659f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1adc10 con 0x560b11c1b680 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c303b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c303b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58426/0 (socket says 172.21.15.37:58426) 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1184479202 learned_addr learned my addr 172.21.15.37:0/1184479202 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1761640 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a2763640 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c154e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1184479202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 msgr2=0x560b11c154e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:31.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c154e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1184479202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 msgr2=0x560b11c308f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c308f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 -- 172.21.15.37:0/1184479202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b11c18b50 con 0x560b11c104a0 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1761640 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a2763640 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c303b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd0980ccfe0 tx=0x7fd098086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b34d870c80873439 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0877fe640 1 -- 172.21.15.37:0/1184479202 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0980a4730 con 0x560b11c104a0 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0877fe640 1 -- 172.21.15.37:0/1184479202 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd098099dc0 con 0x560b11c104a0 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.996+0000 7fd0877fe640 1 -- 172.21.15.37:0/1184479202 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd09809af10 con 0x560b11c104a0 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 -- 172.21.15.37:0/1184479202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 msgr2=0x560b11c303b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c303b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd0980ccfe0 tx=0x7fd098086040 comp rx=0 tx=0).stop 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 -- 172.21.15.37:0/1184479202 shutdown_connections 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c154e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c10dc0 0x560b11c308f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 --2- 172.21.15.37:0/1184479202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c104a0 0x560b11c303b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 -- 172.21.15.37:0/1184479202 >> 172.21.15.37:0/1184479202 conn(0x560b11c0ca30 msgr2=0x560b11c138c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 -- 172.21.15.37:0/1184479202 shutdown_connections 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 -- 172.21.15.37:0/1184479202 wait complete. 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 -- start start 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.997+0000 7fd0a659f740 1 Processor -- start 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c30c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c10dc0 0x560b11c2dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a659f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c2e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a659f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1ae280 con 0x560b11c10dc0 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a659f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1af6f0 con 0x560b11c104a0 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a659f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd09c1ae950 con 0x560b11c1b680 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a659f740 10 client.0 _ll_register_callbacks cb 0x7fd09404b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c10dc0 0x560b11c2dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c10dc0 0x560b11c2dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58430/0 (socket says 172.21.15.37:58430) 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 -- 172.21.15.37:0/654811851 learned_addr learned my addr 172.21.15.37:0/654811851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c30c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a2763640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c2e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 -- 172.21.15.37:0/654811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 msgr2=0x560b11c2e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c2e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 -- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 msgr2=0x560b11c30c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c30c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 -- 172.21.15.37:0/654811851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b11c2ede0 con 0x560b11c10dc0 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a2763640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0a1761640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c10dc0 0x560b11c2dfa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd09c19b6d0 tx=0x7fd090013890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32dfcaa639647f1d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.998+0000 7fd0857fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0857fa640 1 -- 172.21.15.37:0/654811851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd090005020 con 0x560b11c10dc0 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0857fa640 1 -- 172.21.15.37:0/654811851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fd0900051c0 con 0x560b11c10dc0 2024-09-19T06:46:31.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd098099dc0 con 0x560b11c10dc0 2024-09-19T06:46:31.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0857fa640 1 -- 172.21.15.37:0/654811851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fd0900053c0 con 0x560b11c10dc0 2024-09-19T06:46:31.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 10 client.16812 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:31.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560b11c959f0 con 0x560b11c10dc0 2024-09-19T06:46:31.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0857fa640 1 -- 172.21.15.37:0/654811851 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd098099dc0 con 0x560b11c10dc0 2024-09-19T06:46:31.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0857fa640 1 -- 172.21.15.37:0/654811851 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fd09000f040 con 0x560b11c10dc0 2024-09-19T06:46:31.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 10 client.16812 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:31.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:31.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 20 client.16812 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:31.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c96190 0x560b11c985b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:31.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 4 client.16812 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b11b7dd60 con 0x560b11c96190 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:30.999+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c96190 0x560b11c985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a1f62640 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c96190 0x560b11c985b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=793b2186cd418a9f server_cookie=36a9a03b9a077f11 in_seq=0 out_seq=0 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0857fa640 10 client.16812 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0857fa640 1 -- 172.21.15.37:0/654811851 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560b11b7dd60 con 0x560b11c96190 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0857fa640 10 client.16812 handle_command_reply: tid=0 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 client.16812 shutdown 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c10dc0 msgr2=0x560b11c2dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c10dc0 0x560b11c2dfa0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd09c19b6d0 tx=0x7fd090013890 comp rx=0 tx=0).stop 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 shutdown_connections 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560b11c96190 0x560b11c985b0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560b11c1b680 0x560b11c2e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0857fa640 0 client.16812 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560b11c10dc0 0x560b11c2dfa0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 --2- 172.21.15.37:0/654811851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560b11c104a0 0x560b11c30c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 >> 172.21.15.37:0/654811851 conn(0x560b11c0ca30 msgr2=0x560b11c19ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 shutdown_connections 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 1 -- 172.21.15.37:0/654811851 wait complete. 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.000+0000 7fd0a659f740 20 client.16812 trim_cache size 0 max 16384 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stdout: "910880f4-7a87-40bd-a15b-9c6df92e7312": { 2024-09-19T06:46:31.008 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:46:31.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "910880f4-7a87-40bd-a15b-9c6df92e7312", 2024-09-19T06:46:31.009 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:46:31.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:31.009 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:31.009 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:46:31.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd0880a33b0 msgr2=0x7fd0880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd0880a33b0 0x7fd0880a5870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fd09404a000 tx=0x7fd09404b000 comp rx=0 tx=0).stop 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c102db0 msgr2=0x7fd09c1a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c102db0 0x7fd09c1a03d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fd09800d950 tx=0x7fd09800de10 comp rx=0 tx=0).stop 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 shutdown_connections 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd09c1082e0 0x7fd09c19a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd09c103760 0x7fd09c1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd0880a33b0 0x7fd0880a5870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 --2- 172.21.15.37:0/1088904221 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd09c102db0 0x7fd09c1a03d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 >> 172.21.15.37:0/1088904221 conn(0x7fd09c0fc7d0 msgr2=0x7fd09c101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 shutdown_connections 2024-09-19T06:46:31.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:31.001+0000 7fd0a41c6640 1 -- 172.21.15.37:0/1088904221 wait complete. 2024-09-19T06:46:31.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 8.18 MB/sec execute 58 sec latency 23896.156 ms 2024-09-19T06:46:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:31 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:46:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:31 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:31 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:46:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:31 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:31.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:31 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:46:31.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:31 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:46:31.788 INFO:tasks.cephfs.filesystem:scrub status for tag:910880f4-7a87-40bd-a15b-9c6df92e7312 - {'path': '/', 'tag': '910880f4-7a87-40bd-a15b-9c6df92e7312', 'options': 'recursive,force'} 2024-09-19T06:46:32.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 8.04 MB/sec execute 59 sec latency 24896.206 ms 2024-09-19T06:46:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:32 smithi096 ceph-mon[35004]: pgmap v146: 97 pgs: 97 active+clean; 34 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-09-19T06:46:32.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:32 smithi037 ceph-mon[31211]: pgmap v146: 97 pgs: 97 active+clean; 34 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-09-19T06:46:32.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:32 smithi151 ceph-mon[35236]: pgmap v146: 97 pgs: 97 active+clean; 34 MiB data, 937 MiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 1.9 MiB/s wr, 91 op/s 2024-09-19T06:46:33.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 7.91 MB/sec execute 60 sec latency 25896.314 ms 2024-09-19T06:46:33.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:34.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 7.78 MB/sec execute 61 sec latency 26896.358 ms 2024-09-19T06:46:34.245 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:34 smithi096 ceph-mon[35004]: pgmap v147: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-09-19T06:46:34.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:34 smithi037 ceph-mon[31211]: pgmap v147: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-09-19T06:46:34.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:34 smithi151 ceph-mon[35236]: pgmap v147: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 2.5 MiB/s wr, 125 op/s 2024-09-19T06:46:34.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.703+0000 7fc1ac663640 1 -- 172.21.15.37:0/3572026931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4103740 msgr2=0x7fc1a4107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:34.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.703+0000 7fc1ac663640 1 --2- 172.21.15.37:0/3572026931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4103740 0x7fc1a4107b80 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc194009930 tx=0x7fc19402a420 comp rx=0 tx=0).stop 2024-09-19T06:46:34.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.703+0000 7fc1ac663640 1 -- 172.21.15.37:0/3572026931 shutdown_connections 2024-09-19T06:46:34.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.703+0000 7fc1ac663640 1 --2- 172.21.15.37:0/3572026931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4108290 0x7fc1a410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.703+0000 7fc1ac663640 1 --2- 172.21.15.37:0/3572026931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4103740 0x7fc1a4107b80 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.703+0000 7fc1ac663640 1 --2- 172.21.15.37:0/3572026931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4102d90 0x7fc1a4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.703+0000 7fc1ac663640 1 -- 172.21.15.37:0/3572026931 >> 172.21.15.37:0/3572026931 conn(0x7fc1a40fc7d0 msgr2=0x7fc1a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 -- 172.21.15.37:0/3572026931 shutdown_connections 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 -- 172.21.15.37:0/3572026931 wait complete. 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 Processor -- start 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 -- start start 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102d90 0x7fc1a41009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4103740 0x7fc1a4100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4108290 0x7fc1a4101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4112050 con 0x7fc1a4102d90 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a4111ed0 con 0x7fc1a4108290 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.704+0000 7fc1ac663640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a41121d0 con 0x7fc1a4103740 2024-09-19T06:46:34.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102d90 0x7fc1a41009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102d90 0x7fc1a41009f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58462/0 (socket says 172.21.15.37:58462) 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 -- 172.21.15.37:0/139950305 learned_addr learned my addr 172.21.15.37:0/139950305 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aac00640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4108290 0x7fc1a4101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1a9bfe640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4103740 0x7fc1a4100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 -- 172.21.15.37:0/139950305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4103740 msgr2=0x7fc1a4100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4103740 0x7fc1a4100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 -- 172.21.15.37:0/139950305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4108290 msgr2=0x7fc1a4101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4108290 0x7fc1a4101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 -- 172.21.15.37:0/139950305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a41a6a60 con 0x7fc1a4102d90 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc1aa3ff640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102d90 0x7fc1a41009f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000d950 tx=0x7fc1a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.705+0000 7fc19b7fe640 1 -- 172.21.15.37:0/139950305 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a0017070 con 0x7fc1a4102d90 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.706+0000 7fc19b7fe640 1 -- 172.21.15.37:0/139950305 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1a0009c30 con 0x7fc1a4102d90 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.706+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a41a6d50 con 0x7fc1a4102d90 2024-09-19T06:46:34.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.706+0000 7fc19b7fe640 1 -- 172.21.15.37:0/139950305 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a000f6b0 con 0x7fc1a4102d90 2024-09-19T06:46:34.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.706+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a41a7200 con 0x7fc1a4102d90 2024-09-19T06:46:34.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.707+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1a4103170 con 0x7fc1a4102d90 2024-09-19T06:46:34.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.708+0000 7fc19b7fe640 1 -- 172.21.15.37:0/139950305 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc1a00109d0 con 0x7fc1a4102d90 2024-09-19T06:46:34.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.708+0000 7fc19b7fe640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc1900a33b0 0x7fc1900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:34.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.709+0000 7fc1a9bfe640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc1900a33b0 0x7fc1900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:34.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.709+0000 7fc19b7fe640 1 -- 172.21.15.37:0/139950305 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc1a00cf920 con 0x7fc1a4102d90 2024-09-19T06:46:34.715 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.709+0000 7fc1a9bfe640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc1900a33b0 0x7fc1900a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc194004770 tx=0x7fc1940058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:34.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.717+0000 7fc19b7fe640 1 -- 172.21.15.37:0/139950305 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc1a0009de0 con 0x7fc1a4102d90 2024-09-19T06:46:34.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.931+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1a410a2a0 con 0x7fc1a4102d90 2024-09-19T06:46:34.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.932+0000 7fc19b7fe640 1 -- 172.21.15.37:0/139950305 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fc1a009bc50 con 0x7fc1a4102d90 2024-09-19T06:46:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:46:34.935 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-19T06:45:42:465959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:41.617479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:46:34.935 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-19T06:46:34.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc1900a33b0 msgr2=0x7fc1900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:34.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc1900a33b0 0x7fc1900a5870 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fc194004770 tx=0x7fc1940058d0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102d90 msgr2=0x7fc1a41009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:34.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102d90 0x7fc1a41009f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000d950 tx=0x7fc1a000de10 comp rx=0 tx=0).stop 2024-09-19T06:46:34.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 shutdown_connections 2024-09-19T06:46:34.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1a4108290 0x7fc1a4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc1a4103740 0x7fc1a4100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc1900a33b0 0x7fc1900a5870 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 --2- 172.21.15.37:0/139950305 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc1a4102d90 0x7fc1a41009f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:34.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 >> 172.21.15.37:0/139950305 conn(0x7fc1a40fc7d0 msgr2=0x7fc1a4105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:34.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 shutdown_connections 2024-09-19T06:46:34.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:34.934+0000 7fc1ac663640 1 -- 172.21.15.37:0/139950305 wait complete. 2024-09-19T06:46:35.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 7.66 MB/sec execute 62 sec latency 27896.410 ms 2024-09-19T06:46:35.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:35 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/139950305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:35.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:35 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/139950305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:35.511 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 -- 172.21.15.37:0/3370374437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102c80 msgr2=0x7ff3cc103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 --2- 172.21.15.37:0/3370374437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102c80 0x7ff3cc103060 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009900 tx=0x7ff3c002a480 comp rx=0 tx=0).stop 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 -- 172.21.15.37:0/3370374437 shutdown_connections 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 --2- 172.21.15.37:0/3370374437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc10c0d0 0x7ff3cc10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 --2- 172.21.15.37:0/3370374437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc1035a0 0x7ff3cc10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 --2- 172.21.15.37:0/3370374437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102c80 0x7ff3cc103060 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 -- 172.21.15.37:0/3370374437 >> 172.21.15.37:0/3370374437 conn(0x7ff3cc077990 msgr2=0x7ff3cc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 -- 172.21.15.37:0/3370374437 shutdown_connections 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.508+0000 7ff3d2add640 1 -- 172.21.15.37:0/3370374437 wait complete. 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 Processor -- start 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 -- start start 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc102c80 0x7ff3cc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc1035a0 0x7ff3cc1988e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc10c0d0 0x7ff3cc19cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc111eb0 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc111d30 con 0x7ff3cc102c80 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d2add640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc112030 con 0x7ff3cc1035a0 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc10c0d0 0x7ff3cc19cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc10c0d0 0x7ff3cc19cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58486/0 (socket says 172.21.15.37:58486) 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 -- 172.21.15.37:0/4261993893 learned_addr learned my addr 172.21.15.37:0/4261993893 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc1035a0 0x7ff3cc1988e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d0879640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc102c80 0x7ff3cc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 -- 172.21.15.37:0/4261993893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc1035a0 msgr2=0x7ff3cc1988e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc1035a0 0x7ff3cc1988e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 -- 172.21.15.37:0/4261993893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc102c80 msgr2=0x7ff3cc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc102c80 0x7ff3cc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d107a640 1 -- 172.21.15.37:0/4261993893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc19d390 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.509+0000 7ff3d0879640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc102c80 0x7ff3cc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.510+0000 7ff3d107a640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc10c0d0 0x7ff3cc19cc60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400eaa0 tx=0x7ff3c400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.510+0000 7ff3c9ffb640 1 -- 172.21.15.37:0/4261993893 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c400cdb0 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.510+0000 7ff3c9ffb640 1 -- 172.21.15.37:0/4261993893 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c4005020 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.510+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc19d680 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.510+0000 7ff3c9ffb640 1 -- 172.21.15.37:0/4261993893 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c40116b0 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.510+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc1a4f10 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 Processor -- start 2024-09-19T06:46:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3c9ffb640 1 -- 172.21.15.37:0/4261993893 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff3c4005560 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 -- start start 2024-09-19T06:46:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x557619677ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619682770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196850a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5576195dc2f0 con 0x557619678410 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557619584270 con 0x557619682cb0 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d5875740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5576195d82b0 con 0x557619677af0 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.513+0000 7ff3d0879640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619682770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619682770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58502/0 (socket says 172.21.15.37:58502) 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 -- 172.21.15.37:0/1842396304 learned_addr learned my addr 172.21.15.37:0/1842396304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x557619677ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196850a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3c9ffb640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3bc0a34e0 0x7ff3bc0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 -- 172.21.15.37:0/1842396304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 msgr2=0x557619677ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x557619677ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 -- 172.21.15.37:0/1842396304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 msgr2=0x5576196850a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196850a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 -- 172.21.15.37:0/1842396304 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557619583340 con 0x557619678410 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d107a640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3bc0a34e0 0x7ff3bc0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3c9ffb640 1 -- 172.21.15.37:0/4261993893 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff3c40cfe60 con 0x7ff3cc10c0d0 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3d0879640 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619682770 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0033000 tx=0x7ff3c0032cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ed145c85f49a0d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3b6ffd640 1 -- 172.21.15.37:0/1842396304 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c003f070 con 0x557619678410 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3b6ffd640 1 -- 172.21.15.37:0/1842396304 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c00317b0 con 0x557619678410 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.514+0000 7ff3b6ffd640 1 -- 172.21.15.37:0/1842396304 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0035070 con 0x557619678410 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 -- 172.21.15.37:0/1842396304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 msgr2=0x557619682770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d107a640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3bc0a34e0 0x7ff3bc0a59a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40cd660 tx=0x7ff3c401eb40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619682770 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0033000 tx=0x7ff3c0032cb0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 -- 172.21.15.37:0/1842396304 shutdown_connections 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196850a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619682770 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1842396304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x557619677ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 -- 172.21.15.37:0/1842396304 >> 172.21.15.37:0/1842396304 conn(0x557619674060 msgr2=0x557619674440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 -- 172.21.15.37:0/1842396304 shutdown_connections 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.515+0000 7ff3d5875740 1 -- 172.21.15.37:0/1842396304 wait complete. 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 -- start start 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 Processor -- start 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x55761967fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619680110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x557619680650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557619685880 con 0x557619677af0 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557619565290 con 0x557619682cb0 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557619685a00 con 0x557619678410 2024-09-19T06:46:35.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d5875740 10 client.0 _ll_register_callbacks cb 0x557619596140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x55761967fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x55761967fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58512/0 (socket says 172.21.15.37:58512) 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 -- 172.21.15.37:0/3525587856 learned_addr learned my addr 172.21.15.37:0/3525587856 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3d0879640 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619680110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x557619680650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 -- 172.21.15.37:0/3525587856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 msgr2=0x557619680110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619680110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 -- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 msgr2=0x557619680650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x557619680650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.516+0000 7ff3cbfff640 1 -- 172.21.15.37:0/3525587856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576192ef210 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x55761967fbd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800bf90 tx=0x7ff3b800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a66a99272731f22a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3abfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3abfff640 1 -- 172.21.15.37:0/3525587856 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8004220 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c00317b0 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3abfff640 1 -- 172.21.15.37:0/3525587856 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff3b80043c0 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3abfff640 1 -- 172.21.15.37:0/3525587856 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff3b8010880 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3d5875740 10 client.16830 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5576193bb620 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3abfff640 1 -- 172.21.15.37:0/3525587856 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c00317b0 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.517+0000 7ff3abfff640 1 -- 172.21.15.37:0/3525587856 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff3b801c070 con 0x557619677af0 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d5875740 10 client.16830 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d5875740 10 client.16830 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:35.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d5875740 20 client.16830 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d5875740 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557619692bf0 0x557619694fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d5875740 4 client.16830 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576195e9a20 con 0x557619692bf0 2024-09-19T06:46:35.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d0879640 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557619692bf0 0x557619694fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:35 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/139950305' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3abfff640 10 client.16830 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.518+0000 7ff3d0879640 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557619692bf0 0x557619694fe0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4040026253ba1ed4 server_cookie=b1fea1c440651412 in_seq=0 out_seq=0 2024-09-19T06:46:35.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3abfff640 1 -- 172.21.15.37:0/3525587856 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576195e9a20 con 0x557619692bf0 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3abfff640 10 client.16830 handle_command_reply: tid=0 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 client.16830 shutdown 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 msgr2=0x55761967fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x55761967fbd0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7ff3b800bf90 tx=0x7ff3b800b930 comp rx=0 tx=0).stop 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 shutdown_connections 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557619692bf0 0x557619694fe0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x557619680650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619680110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 --2- 172.21.15.37:0/3525587856 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x55761967fbd0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3abfff640 0 client.16830 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 >> 172.21.15.37:0/3525587856 conn(0x557619674060 msgr2=0x557619679d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 shutdown_connections 2024-09-19T06:46:35.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 1 -- 172.21.15.37:0/3525587856 wait complete. 2024-09-19T06:46:35.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.519+0000 7ff3d5875740 20 client.16830 trim_cache size 0 max 16384 2024-09-19T06:46:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 Processor -- start 2024-09-19T06:46:35.536 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 -- start start 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x55761967a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619680480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x55761969a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1b00d0 con 0x557619678410 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ad9a0 con 0x557619682cb0 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d5875740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ab7f0 con 0x557619677af0 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619680480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619680480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58520/0 (socket says 172.21.15.37:58520) 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 -- 172.21.15.37:0/1653758380 learned_addr learned my addr 172.21.15.37:0/1653758380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x55761967a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x55761969a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 -- 172.21.15.37:0/1653758380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 msgr2=0x55761967a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x55761967a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 -- 172.21.15.37:0/1653758380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 msgr2=0x55761969a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x55761969a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.534+0000 7ff3d0879640 1 -- 172.21.15.37:0/1653758380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55761966f2d0 con 0x557619678410 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d0879640 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619680480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009900 tx=0x7ff3c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7477773accaf84 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3b67fc640 1 -- 172.21.15.37:0/1653758380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c0035070 con 0x557619678410 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3b67fc640 1 -- 172.21.15.37:0/1653758380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c0004040 con 0x557619678410 2024-09-19T06:46:35.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3b67fc640 1 -- 172.21.15.37:0/1653758380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c003f040 con 0x557619678410 2024-09-19T06:46:35.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d5875740 1 -- 172.21.15.37:0/1653758380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 msgr2=0x557619680480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619680480 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009900 tx=0x7ff3c0000bc0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d5875740 1 -- 172.21.15.37:0/1653758380 shutdown_connections 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x55761969a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619678410 0x557619680480 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d5875740 1 --2- 172.21.15.37:0/1653758380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619677af0 0x55761967a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.535+0000 7ff3d5875740 1 -- 172.21.15.37:0/1653758380 >> 172.21.15.37:0/1653758380 conn(0x557619674060 msgr2=0x557619681c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 -- 172.21.15.37:0/1653758380 shutdown_connections 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 -- 172.21.15.37:0/1653758380 wait complete. 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 -- start start 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 Processor -- start 2024-09-19T06:46:35.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x557619685de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619686320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196fb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1abe60 con 0x557619677af0 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ad2d0 con 0x557619682cb0 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1ac530 con 0x557619678410 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3d5875740 10 client.0 _ll_register_callbacks cb 0x7ff3b800be20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.536+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x557619685de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x557619685de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58532/0 (socket says 172.21.15.37:58532) 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 -- 172.21.15.37:0/2705177139 learned_addr learned my addr 172.21.15.37:0/2705177139 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3d0879640 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619686320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 -- 172.21.15.37:0/2705177139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 msgr2=0x557619686320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619686320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 -- 172.21.15.37:0/2705177139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 msgr2=0x5576196fb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196fb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 -- 172.21.15.37:0/2705177139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576196fbdd0 con 0x557619677af0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x557619685de0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005fd0 tx=0x7ff3b800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26e1c5fac5931517 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3abfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3abfff640 1 -- 172.21.15.37:0/2705177139 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b8010e60 con 0x557619677af0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3abfff640 1 -- 172.21.15.37:0/2705177139 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7ff3b80054d0 con 0x557619677af0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3c0004040 con 0x557619677af0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3abfff640 1 -- 172.21.15.37:0/2705177139 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7ff3b80103d0 con 0x557619677af0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3d5875740 10 client.16836 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.537+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5576196fcfd0 con 0x557619677af0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3abfff640 1 -- 172.21.15.37:0/2705177139 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3c0004040 con 0x557619677af0 2024-09-19T06:46:35.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3abfff640 1 -- 172.21.15.37:0/2705177139 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7ff3b801c040 con 0x557619677af0 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3d5875740 10 client.16836 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3d5875740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3d5875740 20 client.16836 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3d5875740 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576196fd6c0 0x5576196ffae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3d5875740 4 client.16836 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5576195e3d70 con 0x5576196fd6c0 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.538+0000 7ff3d0879640 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576196fd6c0 0x5576196ffae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.539+0000 7ff3abfff640 10 client.16836 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:35.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.539+0000 7ff3d0879640 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576196fd6c0 0x5576196ffae0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679f98b79f49be14 server_cookie=925bbcae4014379d in_seq=0 out_seq=0 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.616+0000 7ff3abfff640 1 -- 172.21.15.37:0/2705177139 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5576195e3d70 con 0x5576196fd6c0 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.616+0000 7ff3abfff640 10 client.16836 handle_command_reply: tid=0 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.616+0000 7ff3d5875740 1 client.16836 shutdown 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 msgr2=0x557619685de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x557619685de0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7ff3b8005fd0 tx=0x7ff3b800bc60 comp rx=0 tx=0).stop 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 shutdown_connections 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576196fd6c0 0x5576196ffae0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557619682cb0 0x5576196fb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3abfff640 0 client.16836 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557619678410 0x557619686320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.619 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 --2- 172.21.15.37:0/2705177139 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557619677af0 0x557619685de0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 >> 172.21.15.37:0/2705177139 conn(0x557619674060 msgr2=0x5576196729a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 shutdown_connections 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 1 -- 172.21.15.37:0/2705177139 wait complete. 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.617+0000 7ff3d5875740 20 client.16836 trim_cache size 0 max 16384 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:46:35.620 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:46:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3bc0a34e0 msgr2=0x7ff3bc0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3bc0a34e0 0x7ff3bc0a59a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff3c40cd660 tx=0x7ff3c401eb40 comp rx=0 tx=0).stop 2024-09-19T06:46:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc10c0d0 msgr2=0x7ff3cc19cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc10c0d0 0x7ff3cc19cc60 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff3c400eaa0 tx=0x7ff3c400ef60 comp rx=0 tx=0).stop 2024-09-19T06:46:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 shutdown_connections 2024-09-19T06:46:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3bc0a34e0 0x7ff3bc0a59a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.621 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc10c0d0 0x7ff3cc19cc60 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc1035a0 0x7ff3cc1988e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 --2- 172.21.15.37:0/4261993893 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc102c80 0x7ff3cc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 >> 172.21.15.37:0/4261993893 conn(0x7ff3cc077990 msgr2=0x7ff3cc0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 shutdown_connections 2024-09-19T06:46:35.622 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:35.618+0000 7ff3d2add640 1 -- 172.21.15.37:0/4261993893 wait complete. 2024-09-19T06:46:35.779 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 7518637c --release --await' 2024-09-19T06:46:35.780 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 7518637c --release --await 2024-09-19T06:46:36.040 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:36.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 313310 7.53 MB/sec execute 63 sec latency 28896.508 ms 2024-09-19T06:46:36.320 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:36 smithi037 ceph-mon[31211]: pgmap v148: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-09-19T06:46:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:36 smithi151 ceph-mon[35236]: pgmap v148: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-09-19T06:46:36.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:36 smithi096 ceph-mon[35004]: pgmap v148: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 2.0 MiB/s wr, 86 op/s 2024-09-19T06:46:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 -- 172.21.15.37:0/3560401390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c101de0 msgr2=0x7f6a5c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 --2- 172.21.15.37:0/3560401390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c101de0 0x7f6a5c1021c0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6a50009900 tx=0x7f6a5002a410 comp rx=0 tx=0).stop 2024-09-19T06:46:36.555 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 -- 172.21.15.37:0/3560401390 shutdown_connections 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 --2- 172.21.15.37:0/3560401390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5c106f10 0x7f6a5c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 --2- 172.21.15.37:0/3560401390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a5c102790 0x7f6a5c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 --2- 172.21.15.37:0/3560401390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c101de0 0x7f6a5c1021c0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 -- 172.21.15.37:0/3560401390 >> 172.21.15.37:0/3560401390 conn(0x7f6a5c0fd8c0 msgr2=0x7f6a5c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 -- 172.21.15.37:0/3560401390 shutdown_connections 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.546+0000 7f6a62c26640 1 -- 172.21.15.37:0/3560401390 wait complete. 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 Processor -- start 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 -- start start 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a5c101de0 0x7f6a5c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5c102790 0x7f6a5c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c106f10 0x7f6a5c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.556 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c111f90 con 0x7f6a5c106f10 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c111e10 con 0x7f6a5c101de0 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.547+0000 7f6a62c26640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c112110 con 0x7f6a5c102790 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c106f10 0x7f6a5c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c106f10 0x7f6a5c19cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58556/0 (socket says 172.21.15.37:58556) 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 -- 172.21.15.37:0/2459635211 learned_addr learned my addr 172.21.15.37:0/2459635211 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5c102790 0x7f6a5c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a609c2640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a5c101de0 0x7f6a5c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 -- 172.21.15.37:0/2459635211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5c102790 msgr2=0x7f6a5c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5c102790 0x7f6a5c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 -- 172.21.15.37:0/2459635211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a5c101de0 msgr2=0x7f6a5c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a5c101de0 0x7f6a5c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 -- 172.21.15.37:0/2459635211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a5c19d4c0 con 0x7f6a5c106f10 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a611c3640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c106f10 0x7f6a5c19cde0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400b970 tx=0x7f6a5400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a59ffb640 1 -- 172.21.15.37:0/2459635211 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54019070 con 0x7f6a5c106f10 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a59ffb640 1 -- 172.21.15.37:0/2459635211 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a5400cb40 con 0x7f6a5c106f10 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a5c19d7b0 con 0x7f6a5c106f10 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.548+0000 7f6a59ffb640 1 -- 172.21.15.37:0/2459635211 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54007650 con 0x7f6a5c106f10 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.549+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a5c1a5090 con 0x7f6a5c106f10 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a59ffb640 1 -- 172.21.15.37:0/2459635211 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6a5401c070 con 0x7f6a5c106f10 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 Processor -- start 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 -- start start 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a59ffb640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a4c0a33b0 0x7f6a4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a1d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a1ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e14a138620 con 0x55e14a1d3980 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e14a0dfe50 con 0x55e14a1d3060 2024-09-19T06:46:36.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.551+0000 7f6a64fff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e14a1345e0 con 0x55e14a1de240 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a4c0a33b0 0x7f6a4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a59ffb640 1 -- 172.21.15.37:0/2459635211 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6a540cf170 con 0x7f6a5c106f10 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a1ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a1ddd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58572/0 (socket says 172.21.15.37:58572) 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 -- 172.21.15.37:0/3602862935 learned_addr learned my addr 172.21.15.37:0/3602862935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a609c2640 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a1d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a611c3640 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 -- 172.21.15.37:0/3602862935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 msgr2=0x55e14a1e0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 -- 172.21.15.37:0/3602862935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 msgr2=0x55e14a1d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a1d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 -- 172.21.15.37:0/3602862935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14a0d0d40 con 0x55e14a1d3980 2024-09-19T06:46:36.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a609c2640 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a1d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a1ddd00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6a48012a80 tx=0x7f6a48012f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486484d45d083222 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a4c0a33b0 0x7f6a4c0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800f7f0 tx=0x7f6a4801d000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a46ffd640 1 -- 172.21.15.37:0/3602862935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a48022070 con 0x55e14a1d3980 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a46ffd640 1 -- 172.21.15.37:0/3602862935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a48012590 con 0x55e14a1d3980 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.552+0000 7f6a46ffd640 1 -- 172.21.15.37:0/3602862935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801e820 con 0x55e14a1d3980 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 -- 172.21.15.37:0/3602862935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 msgr2=0x55e14a1ddd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a1ddd00 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f6a48012a80 tx=0x7f6a48012f40 comp rx=0 tx=0).stop 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 -- 172.21.15.37:0/3602862935 shutdown_connections 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a1ddd00 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 --2- 172.21.15.37:0/3602862935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a1d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 -- 172.21.15.37:0/3602862935 >> 172.21.15.37:0/3602862935 conn(0x55e14a1cf5f0 msgr2=0x55e14a1cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 -- 172.21.15.37:0/3602862935 shutdown_connections 2024-09-19T06:46:36.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.553+0000 7f6a64fff740 1 -- 172.21.15.37:0/3602862935 wait complete. 2024-09-19T06:46:36.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 -- start start 2024-09-19T06:46:36.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 Processor -- start 2024-09-19T06:46:36.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3060 0x55e14a1db1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3980 0x55e14a1db720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1de240 0x55e14a1dbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e14a1e0db0 con 0x55e14a1d3060 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e14a0c0f10 con 0x55e14a1de240 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e14a1e0f30 con 0x55e14a1d3980 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a64fff740 10 client.0 _ll_register_callbacks cb 0x55e14a0e4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.554+0000 7f6a609c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3060 0x55e14a1db1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3060 0x55e14a1db1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58576/0 (socket says 172.21.15.37:58576) 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 -- 172.21.15.37:0/1199159368 learned_addr learned my addr 172.21.15.37:0/1199159368 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3980 0x55e14a1db720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 -- 172.21.15.37:0/1199159368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3980 msgr2=0x55e14a1db720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3980 0x55e14a1db720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 -- 172.21.15.37:0/1199159368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1de240 msgr2=0x55e14a1dbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1de240 0x55e14a1dbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 -- 172.21.15.37:0/1199159368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e149e47580 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3980 0x55e14a1db720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a609c2640 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3060 0x55e14a1db1e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6a5002a930 tx=0x7f6a50031980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb158444b6eaaa1a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a3bfff640 1 -- 172.21.15.37:0/1199159368 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a50035040 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a3bfff640 1 -- 172.21.15.37:0/1199159368 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6a50004300 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a48012590 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a3bfff640 1 -- 172.21.15.37:0/1199159368 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6a50005020 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a64fff740 10 client.16848 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.555+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e149f11960 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a3bfff640 1 -- 172.21.15.37:0/1199159368 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a48012590 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a3bfff640 1 -- 172.21.15.37:0/1199159368 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6a500035f0 con 0x55e14a1d3060 2024-09-19T06:46:36.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a64fff740 10 client.16848 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a64fff740 10 client.16848 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a64fff740 20 client.16848 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a64fff740 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a1edd40 0x55e14a1f0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a64fff740 4 client.16848 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e14a145b90 con 0x55e14a1edd40 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.556+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a1edd40 0x55e14a1f0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.557+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a1edd40 0x55e14a1f0170 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c83e0affa19c43f server_cookie=45f5d3310424fe1 in_seq=0 out_seq=0 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.557+0000 7f6a3bfff640 10 client.16848 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.557+0000 7f6a3bfff640 1 -- 172.21.15.37:0/1199159368 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e14a145b90 con 0x55e14a1edd40 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.557+0000 7f6a3bfff640 10 client.16848 handle_command_reply: tid=0 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.557+0000 7f6a64fff740 1 client.16848 shutdown 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3060 msgr2=0x55e14a1db1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3060 0x55e14a1db1e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f6a5002a930 tx=0x7f6a50031980 comp rx=0 tx=0).stop 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 shutdown_connections 2024-09-19T06:46:36.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a1edd40 0x55e14a1f0170 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1de240 0x55e14a1dbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3980 0x55e14a1db720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 --2- 172.21.15.37:0/1199159368 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3060 0x55e14a1db1e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 >> 172.21.15.37:0/1199159368 conn(0x55e14a1cf5f0 msgr2=0x55e14a1ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a3bfff640 0 client.16848 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 shutdown_connections 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 1 -- 172.21.15.37:0/1199159368 wait complete. 2024-09-19T06:46:36.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.558+0000 7f6a64fff740 20 client.16848 trim_cache size 0 max 16384 2024-09-19T06:46:36.577 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.575+0000 7f6a64fff740 1 Processor -- start 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.575+0000 7f6a64fff740 1 -- start start 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3060 0x55e14a1ce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3980 0x55e14a1f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1de240 0x55e14a1f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a64fff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c1b0270 con 0x55e14a1de240 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a64fff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c1adb40 con 0x55e14a1d3980 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a64fff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c1ab990 con 0x55e14a1d3060 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1de240 0x55e14a1f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a609c2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3060 0x55e14a1ce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1de240 0x55e14a1f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58580/0 (socket says 172.21.15.37:58580) 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 -- 172.21.15.37:0/2679074930 learned_addr learned my addr 172.21.15.37:0/2679074930 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:36.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3980 0x55e14a1f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 -- 172.21.15.37:0/2679074930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3060 msgr2=0x55e14a1ce640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3060 0x55e14a1ce640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 -- 172.21.15.37:0/2679074930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3980 msgr2=0x55e14a1f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3980 0x55e14a1f59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 -- 172.21.15.37:0/2679074930 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e14a1da190 con 0x55e14a1de240 2024-09-19T06:46:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.576+0000 7f6a611c3640 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1de240 0x55e14a1f5f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6a5409b310 tx=0x7f6a5409b150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83e10a68479d466 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.577+0000 7f6a467fc640 1 -- 172.21.15.37:0/2679074930 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a5409b610 con 0x55e14a1de240 2024-09-19T06:46:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.577+0000 7f6a467fc640 1 -- 172.21.15.37:0/2679074930 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a54097040 con 0x55e14a1de240 2024-09-19T06:46:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.577+0000 7f6a467fc640 1 -- 172.21.15.37:0/2679074930 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a54097c80 con 0x55e14a1de240 2024-09-19T06:46:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 -- 172.21.15.37:0/2679074930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1de240 msgr2=0x55e14a1f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1de240 0x55e14a1f5f30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f6a5409b310 tx=0x7f6a5409b150 comp rx=0 tx=0).stop 2024-09-19T06:46:36.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 -- 172.21.15.37:0/2679074930 shutdown_connections 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1de240 0x55e14a1f5f30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3980 0x55e14a1f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2679074930 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1d3060 0x55e14a1ce640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 -- 172.21.15.37:0/2679074930 >> 172.21.15.37:0/2679074930 conn(0x55e14a1cf5f0 msgr2=0x55e14a1cdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 -- 172.21.15.37:0/2679074930 shutdown_connections 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 -- 172.21.15.37:0/2679074930 wait complete. 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 -- start start 2024-09-19T06:46:36.581 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.578+0000 7f6a64fff740 1 Processor -- start 2024-09-19T06:46:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a2520a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a2525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a64fff740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a64fff740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c1ac000 con 0x55e14a1d3980 2024-09-19T06:46:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a64fff740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c1ad470 con 0x55e14a1d3060 2024-09-19T06:46:36.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a64fff740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a5c1ac6d0 con 0x55e14a1de240 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a2525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a5bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a2525e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58588/0 (socket says 172.21.15.37:58588) 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a5bfff640 1 -- 172.21.15.37:0/2863038650 learned_addr learned my addr 172.21.15.37:0/2863038650 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a64fff740 10 client.0 _ll_register_callbacks cb 0x7f6a50005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a609c2640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a2520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.579+0000 7f6a611c3640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a5bfff640 1 -- 172.21.15.37:0/2863038650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 msgr2=0x55e14a1e6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a5bfff640 1 -- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 msgr2=0x55e14a2520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a2520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a5bfff640 1 -- 172.21.15.37:0/2863038650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e14a1e67c0 con 0x55e14a1d3980 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a609c2640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a2520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:36.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a5bfff640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a2525e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6a48012a80 tx=0x7f6a4801e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e2e0fd98b77a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a3bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a3bfff640 1 -- 172.21.15.37:0/2863038650 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a4801b040 con 0x55e14a1d3980 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a3bfff640 1 -- 172.21.15.37:0/2863038650 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6a4801e530 con 0x55e14a1d3980 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a54097040 con 0x55e14a1d3980 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a3bfff640 1 -- 172.21.15.37:0/2863038650 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6a48022040 con 0x55e14a1d3980 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a64fff740 10 client.16854 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e14a25d8e0 con 0x55e14a1d3980 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.580+0000 7f6a3bfff640 1 -- 172.21.15.37:0/2863038650 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a54097040 con 0x55e14a1d3980 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a3bfff640 1 -- 172.21.15.37:0/2863038650 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6a4801c870 con 0x55e14a1d3980 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a64fff740 10 client.16854 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a64fff740 10 client.16854 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a64fff740 20 client.16854 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a25df50 0x55e14a260370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:36.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a64fff740 4 client.16854 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "7518637c", "release": true, "await": true}] 2024-09-19T06:46:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "7518637c", "release": true, "await": true}) -- 0x55e14a1d01a0 con 0x55e14a25df50 2024-09-19T06:46:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a609c2640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a25df50 0x55e14a260370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a609c2640 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a25df50 0x55e14a260370 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a3e8710b54b531 server_cookie=ba92e6c71a9198bd in_seq=0 out_seq=0 2024-09-19T06:46:36.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.581+0000 7f6a3bfff640 10 client.16854 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a3bfff640 1 -- 172.21.15.37:0/2863038650 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55e14a1d01a0 con 0x55e14a25df50 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a3bfff640 10 client.16854 handle_command_reply: tid=0 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 client.16854 shutdown 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 msgr2=0x55e14a2525e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a2525e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f6a48012a80 tx=0x7f6a4801e420 comp rx=0 tx=0).stop 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 shutdown_connections 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e14a25df50 0x55e14a260370 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a3bfff640 0 client.16854 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e14a1de240 0x55e14a1e6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e14a1d3980 0x55e14a2525e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 --2- 172.21.15.37:0/2863038650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e14a1d3060 0x55e14a2520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 >> 172.21.15.37:0/2863038650 conn(0x55e14a1cf5f0 msgr2=0x55e14a1ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 shutdown_connections 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 1 -- 172.21.15.37:0/2863038650 wait complete. 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.594+0000 7f6a64fff740 20 client.16854 trim_cache size 0 max 16384 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 31, 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 20, 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stdout: "7518637c": { 2024-09-19T06:46:36.597 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 20, 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.8, 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:36.598 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a4c0a33b0 msgr2=0x7f6a4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a4c0a33b0 0x7f6a4c0a5870 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800f7f0 tx=0x7f6a4801d000 comp rx=0 tx=0).stop 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c106f10 msgr2=0x7f6a5c19cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:36.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c106f10 0x7f6a5c19cde0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400b970 tx=0x7f6a5400be30 comp rx=0 tx=0).stop 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 shutdown_connections 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a4c0a33b0 0x7f6a4c0a5870 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a5c106f10 0x7f6a5c19cde0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a5c102790 0x7f6a5c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 --2- 172.21.15.37:0/2459635211 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a5c101de0 0x7f6a5c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 >> 172.21.15.37:0/2459635211 conn(0x7f6a5c0fd8c0 msgr2=0x7f6a5c0fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 shutdown_connections 2024-09-19T06:46:36.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:36.595+0000 7f6a62c26640 1 -- 172.21.15.37:0/2459635211 wait complete. 2024-09-19T06:46:37.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 315153 7.58 MB/sec execute 64 sec latency 29307.096 ms 2024-09-19T06:46:37.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:37 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:46:37.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:37 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:46:37.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:37 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:46:37.566 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '7518637c', total seconds elapsed: 33.4 2024-09-19T06:46:37.566 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.8 2024-09-19T06:46:37.566 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:46:37.835 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:38.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 317599 7.65 MB/sec execute 65 sec latency 12.709 ms 2024-09-19T06:46:38.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:38 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:46:38.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:38 smithi037 ceph-mon[31211]: pgmap v149: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 -- 172.21.15.37:0/2034926285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094102570 msgr2=0x7f90941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 --2- 172.21.15.37:0/2034926285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094102570 0x7f90941029b0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f907c009900 tx=0x7f907c02a480 comp rx=0 tx=0).stop 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 -- 172.21.15.37:0/2034926285 shutdown_connections 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 --2- 172.21.15.37:0/2034926285 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9094102ef0 0x7f909410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 --2- 172.21.15.37:0/2034926285 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094102570 0x7f90941029b0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 --2- 172.21.15.37:0/2034926285 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9094100e20 0x7f9094101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 -- 172.21.15.37:0/2034926285 >> 172.21.15.37:0/2034926285 conn(0x7f90940fc7d0 msgr2=0x7f90940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 -- 172.21.15.37:0/2034926285 shutdown_connections 2024-09-19T06:46:38.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.280+0000 7f9098fe9640 1 -- 172.21.15.37:0/2034926285 wait complete. 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 Processor -- start 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 -- start start 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094100e20 0x7f90941973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9094102570 0x7f9094197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9094102ef0 0x7f909419bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9094110f80 con 0x7f9094100e20 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9094110e00 con 0x7f9094102570 2024-09-19T06:46:38.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9098fe9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9094111100 con 0x7f9094102ef0 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f909259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094100e20 0x7f90941973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9091d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9094102570 0x7f9094197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f909259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094100e20 0x7f90941973c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58608/0 (socket says 172.21.15.37:58608) 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f909259c640 1 -- 172.21.15.37:0/4099349731 learned_addr learned my addr 172.21.15.37:0/4099349731 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.281+0000 7f9092d9d640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9094102ef0 0x7f909419bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f909259c640 1 -- 172.21.15.37:0/4099349731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9094102ef0 msgr2=0x7f909419bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f909259c640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9094102ef0 0x7f909419bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f909259c640 1 -- 172.21.15.37:0/4099349731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9094102570 msgr2=0x7f9094197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f909259c640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9094102570 0x7f9094197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f909259c640 1 -- 172.21.15.37:0/4099349731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909419c370 con 0x7f9094100e20 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f9091d9b640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9094102570 0x7f9094197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f909259c640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094100e20 0x7f90941973c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f908c00ebc0 tx=0x7f908c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:38.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f908b7fe640 1 -- 172.21.15.37:0/4099349731 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c010630 con 0x7f9094100e20 2024-09-19T06:46:38.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909419c660 con 0x7f9094100e20 2024-09-19T06:46:38.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f908b7fe640 1 -- 172.21.15.37:0/4099349731 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f908c010cb0 con 0x7f9094100e20 2024-09-19T06:46:38.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f908b7fe640 1 -- 172.21.15.37:0/4099349731 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f908c018bd0 con 0x7f9094100e20 2024-09-19T06:46:38.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.282+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9094069f70 con 0x7f9094100e20 2024-09-19T06:46:38.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.283+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90941021b0 con 0x7f9094100e20 2024-09-19T06:46:38.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.285+0000 7f908b7fe640 1 -- 172.21.15.37:0/4099349731 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f908c0107d0 con 0x7f9094100e20 2024-09-19T06:46:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.286+0000 7f908b7fe640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90800a33b0 0x7f90800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.286+0000 7f9091d9b640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90800a33b0 0x7f90800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:38.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.286+0000 7f908b7fe640 1 -- 172.21.15.37:0/4099349731 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f908c0d0510 con 0x7f9094100e20 2024-09-19T06:46:38.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.286+0000 7f9091d9b640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90800a33b0 0x7f90800a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f907c009230 tx=0x7f907c0058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:38.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.289+0000 7f908b7fe640 1 -- 172.21.15.37:0/4099349731 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f908c09c8a0 con 0x7f9094100e20 2024-09-19T06:46:38.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:38 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:46:38.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:38 smithi151 ceph-mon[35236]: pgmap v149: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:46:38.424 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:38 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:46:38.424 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:38 smithi096 ceph-mon[35004]: pgmap v149: 97 pgs: 97 active+clean; 33 MiB data, 936 MiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 1.7 MiB/s wr, 77 op/s 2024-09-19T06:46:38.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.522+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9094101200 con 0x7f9094100e20 2024-09-19T06:46:38.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.523+0000 7f908b7fe640 1 -- 172.21.15.37:0/4099349731 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f908c0a0ba0 con 0x7f9094100e20 2024-09-19T06:46:38.525 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:46:38.525 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-19T06:45:42:465959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:41.617479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:46:38.526 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90800a33b0 msgr2=0x7f90800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90800a33b0 0x7f90800a5870 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f907c009230 tx=0x7f907c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094100e20 msgr2=0x7f90941973c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094100e20 0x7f90941973c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f908c00ebc0 tx=0x7f908c00c6a0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 shutdown_connections 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9094102ef0 0x7f909419bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9094102570 0x7f9094197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f90800a33b0 0x7f90800a5870 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 --2- 172.21.15.37:0/4099349731 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9094100e20 0x7f90941973c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 >> 172.21.15.37:0/4099349731 conn(0x7f90940fc7d0 msgr2=0x7f90940febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:38.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 shutdown_connections 2024-09-19T06:46:38.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:38.527+0000 7f9098fe9640 1 -- 172.21.15.37:0/4099349731 wait complete. 2024-09-19T06:46:39.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 320315 7.77 MB/sec execute 66 sec latency 15.957 ms 2024-09-19T06:46:39.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:39 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4099349731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:39.498 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:39 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4099349731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:39.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:39 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4099349731' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:46:39.528 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0' 2024-09-19T06:46:39.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 149.8 --await-for 45.0 2024-09-19T06:46:39.809 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:46:40.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323379 7.91 MB/sec execute 67 sec latency 10.377 ms 2024-09-19T06:46:40.235 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:40 smithi037 ceph-mon[31211]: pgmap v150: 97 pgs: 97 active+clean; 33 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T06:46:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 -- 172.21.15.37:0/292988164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 msgr2=0x7f46941029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 --2- 172.21.15.37:0/292988164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 0x7f46941029b0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f4684009900 tx=0x7f468402a480 comp rx=0 tx=0).stop 2024-09-19T06:46:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 -- 172.21.15.37:0/292988164 shutdown_connections 2024-09-19T06:46:40.344 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 --2- 172.21.15.37:0/292988164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4694102ef0 0x7f469410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 --2- 172.21.15.37:0/292988164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 0x7f46941029b0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 --2- 172.21.15.37:0/292988164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4694100e20 0x7f4694101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 -- 172.21.15.37:0/292988164 >> 172.21.15.37:0/292988164 conn(0x7f46940fc7d0 msgr2=0x7f46940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 -- 172.21.15.37:0/292988164 shutdown_connections 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 -- 172.21.15.37:0/292988164 wait complete. 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 Processor -- start 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.340+0000 7f469b489640 1 -- start start 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f469b489640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4694100e20 0x7f4694198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f469b489640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 0x7f4694198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f469b489640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4694102ef0 0x7f469419cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f469b489640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4694110fd0 con 0x7f4694102570 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f469b489640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4694110e50 con 0x7f4694100e20 2024-09-19T06:46:40.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f469b489640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4694111150 con 0x7f4694102ef0 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4698a24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 0x7f4694198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699225640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4694100e20 0x7f4694198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4698a24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 0x7f4694198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49914/0 (socket says 172.21.15.37:49914) 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4698a24640 1 -- 172.21.15.37:0/3226589915 learned_addr learned my addr 172.21.15.37:0/3226589915 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699a26640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4694102ef0 0x7f469419cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699225640 1 -- 172.21.15.37:0/3226589915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4694102ef0 msgr2=0x7f469419cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699225640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4694102ef0 0x7f469419cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699225640 1 -- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 msgr2=0x7f4694198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699225640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 0x7f4694198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699225640 1 -- 172.21.15.37:0/3226589915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f469419d4b0 con 0x7f4694100e20 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.341+0000 7f4699225640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4694100e20 0x7f4694198500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f469000e9f0 tx=0x7f469000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.342+0000 7f468a7fc640 1 -- 172.21.15.37:0/3226589915 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469000ccc0 con 0x7f4694100e20 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.342+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f469419d7a0 con 0x7f4694100e20 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.342+0000 7f468a7fc640 1 -- 172.21.15.37:0/3226589915 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4690010910 con 0x7f4694100e20 2024-09-19T06:46:40.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.342+0000 7f468a7fc640 1 -- 172.21.15.37:0/3226589915 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690018960 con 0x7f4694100e20 2024-09-19T06:46:40.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.342+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46941a5080 con 0x7f4694100e20 2024-09-19T06:46:40.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.344+0000 7f469d862740 1 Processor -- start 2024-09-19T06:46:40.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f469d862740 1 -- start start 2024-09-19T06:46:40.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f468a7fc640 1 -- 172.21.15.37:0/3226589915 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4690010ac0 con 0x7f4694100e20 2024-09-19T06:46:40.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f469d862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800e45e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f469d862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800e4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f469d862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 0x5647800fa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f469d862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56478004a2a0 con 0x5647800e5cd0 2024-09-19T06:46:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f469d862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56477fff1b10 con 0x5647800e4b20 2024-09-19T06:46:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f469d862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564780046260 con 0x5647800f7fe0 2024-09-19T06:46:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699a26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800e45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699225640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800e4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699225640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800e4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:42646/0 (socket says 172.21.15.37:42646) 2024-09-19T06:46:40.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699225640 1 -- 172.21.15.37:0/1028736976 learned_addr learned my addr 172.21.15.37:0/1028736976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:40.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699a26640 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800e45e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49928/0 (socket says 172.21.15.37:49928) 2024-09-19T06:46:40.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4698a24640 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 0x5647800fa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699a26640 1 -- 172.21.15.37:0/1028736976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 msgr2=0x5647800fa3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699a26640 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 0x5647800fa3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699a26640 1 -- 172.21.15.37:0/1028736976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 msgr2=0x5647800e4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699a26640 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800e4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.345+0000 7f4699a26640 1 -- 172.21.15.37:0/1028736976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56477fdbbb90 con 0x5647800e5cd0 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f4699225640 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f468a7fc640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f46800a33b0 0x7f46800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f4699a26640 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800e45e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f468c007dc0 tx=0x7f468c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61f9a92713f0190 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f4699225640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f46800a33b0 0x7f46800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f467f7fe640 1 -- 172.21.15.37:0/1028736976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c00fea0 con 0x5647800e5cd0 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f468a7fc640 1 -- 172.21.15.37:0/3226589915 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f46900d00b0 con 0x7f4694100e20 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f467f7fe640 1 -- 172.21.15.37:0/1028736976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c003b70 con 0x5647800e5cd0 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.346+0000 7f467f7fe640 1 -- 172.21.15.37:0/1028736976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c0109c0 con 0x5647800e5cd0 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 -- 172.21.15.37:0/1028736976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 msgr2=0x5647800e45e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f4699225640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f46800a33b0 0x7f46800a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46900cd860 tx=0x7f4690098040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800e45e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f468c007dc0 tx=0x7f468c00f790 comp rx=0 tx=0).stop 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 -- 172.21.15.37:0/1028736976 shutdown_connections 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 0x5647800fa3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800e4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 --2- 172.21.15.37:0/1028736976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800e45e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 -- 172.21.15.37:0/1028736976 >> 172.21.15.37:0/1028736976 conn(0x5647800e0cf0 msgr2=0x5647800e10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 -- 172.21.15.37:0/1028736976 shutdown_connections 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 -- 172.21.15.37:0/1028736976 wait complete. 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 -- start start 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.347+0000 7f469d862740 1 Processor -- start 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f469d862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x5647800e8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f469d862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647800e88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f469d862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x5647800e8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f469d862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5647800fad40 con 0x5647800e4b20 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f469d862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56477ffd26e0 con 0x5647800f7fe0 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f469d862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5647800faec0 con 0x5647800e5cd0 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f4699a26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x5647800e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f469d862740 10 client.0 _ll_register_callbacks cb 0x56477fff6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f4699a26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x5647800e8360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49936/0 (socket says 172.21.15.37:49936) 2024-09-19T06:46:40.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f4699a26640 1 -- 172.21.15.37:0/3111314774 learned_addr learned my addr 172.21.15.37:0/3111314774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f4698a24640 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647800e88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.348+0000 7f4699a26640 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x5647800e8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f4698a24640 1 -- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 msgr2=0x5647800e8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f4698a24640 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x5647800e8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f4698a24640 1 -- 172.21.15.37:0/3111314774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 msgr2=0x5647800e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f4698a24640 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x5647800e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f4698a24640 1 -- 172.21.15.37:0/3111314774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564780104270 con 0x5647800e5cd0 2024-09-19T06:46:40.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f4698a24640 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647800e88a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4684000c00 tx=0x7f46840043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f044486de4d10263 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f467cff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:46:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f467cff9640 1 -- 172.21.15.37:0/3111314774 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684035040 con 0x5647800e5cd0 2024-09-19T06:46:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c003b70 con 0x5647800e5cd0 2024-09-19T06:46:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f467cff9640 1 -- 172.21.15.37:0/3111314774 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4684005220 con 0x5647800e5cd0 2024-09-19T06:46:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f467cff9640 1 -- 172.21.15.37:0/3111314774 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4684005420 con 0x5647800e5cd0 2024-09-19T06:46:40.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f469d862740 10 client.26185 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.349+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564780106c00 con 0x5647800e5cd0 2024-09-19T06:46:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f467cff9640 1 -- 172.21.15.37:0/3111314774 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c003b70 con 0x5647800e5cd0 2024-09-19T06:46:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f467cff9640 1 -- 172.21.15.37:0/3111314774 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f4684003920 con 0x5647800e5cd0 2024-09-19T06:46:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f469d862740 10 client.26185 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:40.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f469d862740 10 client.26185 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:46:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f469d862740 20 client.26185 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f469d862740 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5647801080c0 0x56478010a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f469d862740 4 client.26185 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647800579d0 con 0x5647801080c0 2024-09-19T06:46:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f4699a26640 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5647801080c0 0x56478010a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.350+0000 7f467cff9640 10 client.26185 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:40.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.351+0000 7f4699a26640 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5647801080c0 0x56478010a4d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ad2ed8b308769fe server_cookie=f208c389729baca6 in_seq=0 out_seq=0 2024-09-19T06:46:40.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:40 smithi151 ceph-mon[35236]: pgmap v150: 97 pgs: 97 active+clean; 33 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T06:46:40.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f467cff9640 1 -- 172.21.15.37:0/3111314774 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647800579d0 con 0x5647801080c0 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f467cff9640 10 client.26185 handle_command_reply: tid=0 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 client.26185 shutdown 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 msgr2=0x5647800e88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647800e88a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4684000c00 tx=0x7f46840043b0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 shutdown_connections 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5647801080c0 0x56478010a4d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x5647800e8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647800e88a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f467cff9640 0 client.26185 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 --2- 172.21.15.37:0/3111314774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x5647800e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 >> 172.21.15.37:0/3111314774 conn(0x5647800e0cf0 msgr2=0x5647800e6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 shutdown_connections 2024-09-19T06:46:40.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 1 -- 172.21.15.37:0/3111314774 wait complete. 2024-09-19T06:46:40.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.352+0000 7f469d862740 20 client.26185 trim_cache size 0 max 16384 2024-09-19T06:46:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 Processor -- start 2024-09-19T06:46:40.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 -- start start 2024-09-19T06:46:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x56478010fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647801102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x564780100140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46941ae0f0 con 0x5647800e4b20 2024-09-19T06:46:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46941ab9c0 con 0x5647800f7fe0 2024-09-19T06:46:40.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f469d862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46941a9810 con 0x5647800e5cd0 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f4699a26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x56478010fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.373+0000 7f4699a26640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x56478010fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49952/0 (socket says 172.21.15.37:49952) 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 -- 172.21.15.37:0/4276973581 learned_addr learned my addr 172.21.15.37:0/4276973581 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x564780100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4698a24640 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647801102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 -- 172.21.15.37:0/4276973581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 msgr2=0x5647801102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647801102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 -- 172.21.15.37:0/4276973581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 msgr2=0x564780100140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x564780100140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 -- 172.21.15.37:0/4276973581 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647800dc260 con 0x5647800e4b20 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f4699a26640 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x56478010fda0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f468c004940 tx=0x7f468c0106d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94ed8d901c1dffee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:40.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f467effd640 1 -- 172.21.15.37:0/4276973581 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c01d040 con 0x5647800e4b20 2024-09-19T06:46:40.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f467effd640 1 -- 172.21.15.37:0/4276973581 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c010b40 con 0x5647800e4b20 2024-09-19T06:46:40.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.374+0000 7f467effd640 1 -- 172.21.15.37:0/4276973581 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f468c018400 con 0x5647800e4b20 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.375+0000 7f469d862740 1 -- 172.21.15.37:0/4276973581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 msgr2=0x56478010fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.375+0000 7f469d862740 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x56478010fda0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f468c004940 tx=0x7f468c0106d0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.375+0000 7f469d862740 1 -- 172.21.15.37:0/4276973581 shutdown_connections 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.375+0000 7f469d862740 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800f7fe0 0x564780100140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.375+0000 7f469d862740 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800e5cd0 0x5647801102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.375+0000 7f469d862740 1 --2- 172.21.15.37:0/4276973581 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e4b20 0x56478010fda0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.375+0000 7f469d862740 1 -- 172.21.15.37:0/4276973581 >> 172.21.15.37:0/4276973581 conn(0x5647800e0cf0 msgr2=0x5647800dfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.376+0000 7f469d862740 1 -- 172.21.15.37:0/4276973581 shutdown_connections 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.376+0000 7f469d862740 1 -- 172.21.15.37:0/4276973581 wait complete. 2024-09-19T06:46:40.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.376+0000 7f469d862740 1 -- start start 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.376+0000 7f469d862740 1 Processor -- start 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.376+0000 7f469d862740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800fd020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f469d862740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800fd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f469d862740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 0x5647800fdaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f469d862740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f46941a9e80 con 0x5647800e5cd0 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f469d862740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46941ab2f0 con 0x5647800e4b20 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f469d862740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f46941aa550 con 0x5647800f7fe0 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f469d862740 10 client.0 _ll_register_callbacks cb 0x7f4684004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800fd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800fd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49966/0 (socket says 172.21.15.37:49966) 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 -- 172.21.15.37:0/3583567715 learned_addr learned my addr 172.21.15.37:0/3583567715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4699a26640 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800fd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 -- 172.21.15.37:0/3583567715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 msgr2=0x5647800fdaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 0x5647800fdaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 -- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 msgr2=0x5647800fd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800fd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 -- 172.21.15.37:0/3583567715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564780172ef0 con 0x5647800e5cd0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f4698a24640 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800fd560 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4684009850 tx=0x7f4684003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=210505c09d65d050 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f467cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f467cff9640 1 -- 172.21.15.37:0/3583567715 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4684035070 con 0x5647800e5cd0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f467cff9640 1 -- 172.21.15.37:0/3583567715 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4684031860 con 0x5647800e5cd0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.377+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f468c010b40 con 0x5647800e5cd0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f467cff9640 1 -- 172.21.15.37:0/3583567715 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4684051040 con 0x5647800e5cd0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 10 client.16878 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5647801741c0 con 0x5647800e5cd0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f467cff9640 1 -- 172.21.15.37:0/3583567715 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f468c010b40 con 0x5647800e5cd0 2024-09-19T06:46:40.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f467cff9640 1 -- 172.21.15.37:0/3583567715 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f4684054070 con 0x5647800e5cd0 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 10 client.16878 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 10 client.16878 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 20 client.16878 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564780174a30 0x564780176e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 4 client.16878 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}] 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 149.8, "await_for": 45.0}) -- 0x564780051d20 con 0x564780174a30 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.378+0000 7f4699a26640 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564780174a30 0x564780176e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.379+0000 7f467cff9640 10 client.16878 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:40.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.379+0000 7f4699a26640 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564780174a30 0x564780176e50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383726e2f8d82394 server_cookie=aeecba3c33bd49f6 in_seq=0 out_seq=0 2024-09-19T06:46:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f467cff9640 1 -- 172.21.15.37:0/3583567715 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x564780051d20 con 0x564780174a30 2024-09-19T06:46:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f467cff9640 10 client.16878 handle_command_reply: tid=0 2024-09-19T06:46:40.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 client.16878 shutdown 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 msgr2=0x5647800fd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800fd560 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f4684009850 tx=0x7f4684003f60 comp rx=0 tx=0).stop 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 shutdown_connections 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564780174a30 0x564780176e50 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5647800f7fe0 0x5647800fdaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5647800e5cd0 0x5647800fd560 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f467cff9640 0 client.16878 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 --2- 172.21.15.37:0/3583567715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5647800e4b20 0x5647800fd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 >> 172.21.15.37:0/3583567715 conn(0x5647800e0cf0 msgr2=0x5647800e8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 shutdown_connections 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 1 -- 172.21.15.37:0/3583567715 wait complete. 2024-09-19T06:46:40.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.456+0000 7f469d862740 20 client.16878 trim_cache size 0 max 16384 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 31, 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 22, 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "d57a1d24": { 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 22, 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:46:40.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.8, 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:46:40.461 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:40.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:40.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:40.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:40.462 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:46:40.462 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:46:40.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f46800a33b0 msgr2=0x7f46800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f46800a33b0 0x7f46800a5870 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f46900cd860 tx=0x7f4690098040 comp rx=0 tx=0).stop 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4694100e20 msgr2=0x7f4694198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4694100e20 0x7f4694198500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f469000e9f0 tx=0x7f469000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 shutdown_connections 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4694102ef0 0x7f469419cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f46800a33b0 0x7f46800a5870 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4694102570 0x7f4694198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 --2- 172.21.15.37:0/3226589915 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4694100e20 0x7f4694198500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 >> 172.21.15.37:0/3226589915 conn(0x7f46940fc7d0 msgr2=0x7f469406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 shutdown_connections 2024-09-19T06:46:40.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:46:40.457+0000 7f469b489640 1 -- 172.21.15.37:0/3226589915 wait complete. 2024-09-19T06:46:40.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:40 smithi096 ceph-mon[35004]: pgmap v150: 97 pgs: 97 active+clean; 33 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 144 op/s 2024-09-19T06:46:41.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.81 MB/sec execute 68 sec latency 798.190 ms 2024-09-19T06:46:41.253 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd57a1d24', quiesce took 3.7 seconds. Will release after: 26.1 2024-09-19T06:46:42.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.69 MB/sec execute 69 sec latency 1798.291 ms 2024-09-19T06:46:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:42 smithi151 ceph-mon[35236]: pgmap v151: 97 pgs: 97 active+clean; 33 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-09-19T06:46:42.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:42 smithi096 ceph-mon[35004]: pgmap v151: 97 pgs: 97 active+clean; 33 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-09-19T06:46:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:42 smithi037 ceph-mon[31211]: pgmap v151: 97 pgs: 97 active+clean; 33 MiB data, 930 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.8 MiB/s wr, 107 op/s 2024-09-19T06:46:43.203 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.58 MB/sec execute 70 sec latency 2798.365 ms 2024-09-19T06:46:44.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.48 MB/sec execute 71 sec latency 3798.475 ms 2024-09-19T06:46:44.327 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:44 smithi037 ceph-mon[31211]: pgmap v152: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-19T06:46:44.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:44 smithi151 ceph-mon[35236]: pgmap v152: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-19T06:46:44.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:44 smithi096 ceph-mon[35004]: pgmap v152: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.6 MiB/s wr, 180 op/s 2024-09-19T06:46:45.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.37 MB/sec execute 72 sec latency 4798.584 ms 2024-09-19T06:46:46.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.27 MB/sec execute 73 sec latency 5798.708 ms 2024-09-19T06:46:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:46 smithi151 ceph-mon[35236]: pgmap v153: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 146 op/s 2024-09-19T06:46:46.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:46 smithi096 ceph-mon[35004]: pgmap v153: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 146 op/s 2024-09-19T06:46:46.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:46 smithi037 ceph-mon[31211]: pgmap v153: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 2.0 MiB/s wr, 146 op/s 2024-09-19T06:46:47.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.17 MB/sec execute 74 sec latency 6798.814 ms 2024-09-19T06:46:48.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 7.08 MB/sec execute 75 sec latency 7798.927 ms 2024-09-19T06:46:48.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:48 smithi151 ceph-mon[35236]: pgmap v154: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-19T06:46:48.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:48 smithi096 ceph-mon[35004]: pgmap v154: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-19T06:46:48.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:48 smithi037 ceph-mon[31211]: pgmap v154: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 139 op/s 2024-09-19T06:46:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 -- 172.21.15.37:0/1365592112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 msgr2=0x7fb3cc074980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 --2- 172.21.15.37:0/1365592112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc074980 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc009900 tx=0x7fb3bc02a480 comp rx=0 tx=0).stop 2024-09-19T06:46:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 -- 172.21.15.37:0/1365592112 shutdown_connections 2024-09-19T06:46:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 --2- 172.21.15.37:0/1365592112 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3cc10e3a0 0x7fb3cc110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 --2- 172.21.15.37:0/1365592112 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3cc074ec0 0x7fb3cc10de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 --2- 172.21.15.37:0/1365592112 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc074980 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 -- 172.21.15.37:0/1365592112 >> 172.21.15.37:0/1365592112 conn(0x7fb3cc0fd7e0 msgr2=0x7fb3cc0ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 -- 172.21.15.37:0/1365592112 shutdown_connections 2024-09-19T06:46:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 -- 172.21.15.37:0/1365592112 wait complete. 2024-09-19T06:46:48.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.719+0000 7fb3d20af640 1 Processor -- start 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3d20af640 1 -- start start 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3d20af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc19a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3d20af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3cc074ec0 0x7fb3cc19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3d20af640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3cc10e3a0 0x7fb3cc19ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3d20af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1140b0 con 0x7fb3cc0745a0 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3d20af640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc113f30 con 0x7fb3cc074ec0 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3d20af640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc114230 con 0x7fb3cc10e3a0 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc19a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc19a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49980/0 (socket says 172.21.15.37:49980) 2024-09-19T06:46:48.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3cb7fe640 1 -- 172.21.15.37:0/469660428 learned_addr learned my addr 172.21.15.37:0/469660428 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3cc10e3a0 0x7fb3cc19ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.720+0000 7fb3caffd640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3cc074ec0 0x7fb3cc19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3cb7fe640 1 -- 172.21.15.37:0/469660428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3cc10e3a0 msgr2=0x7fb3cc19ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3cc10e3a0 0x7fb3cc19ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3cb7fe640 1 -- 172.21.15.37:0/469660428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3cc074ec0 msgr2=0x7fb3cc19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3cc074ec0 0x7fb3cc19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3cb7fe640 1 -- 172.21.15.37:0/469660428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3cc19f530 con 0x7fb3cc0745a0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc19a540 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc006010 tx=0x7fb3bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3c8ff9640 1 -- 172.21.15.37:0/469660428 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc03f030 con 0x7fb3cc0745a0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3c8ff9640 1 -- 172.21.15.37:0/469660428 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3bc004440 con 0x7fb3cc0745a0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3cc19f7c0 con 0x7fb3cc0745a0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3c8ff9640 1 -- 172.21.15.37:0/469660428 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3bc035070 con 0x7fb3cc0745a0 2024-09-19T06:46:48.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.721+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3cc1a70b0 con 0x7fb3cc0745a0 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3c8ff9640 1 -- 172.21.15.37:0/469660428 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb3bc042030 con 0x7fb3cc0745a0 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 Processor -- start 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 -- start start 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b913ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b91e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b91ecb0 0x55e18b9210a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e18b8782f0 con 0x55e18b914410 2024-09-19T06:46:48.731 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e18b820270 con 0x55e18b913af0 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e18b8742b0 con 0x55e18b91ecb0 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b91e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3c8ff9640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3b40a3460 0x7fb3b40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b91e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49982/0 (socket says 172.21.15.37:49982) 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3cb7fe640 1 -- 172.21.15.37:0/263439533 learned_addr learned my addr 172.21.15.37:0/263439533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b913ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.724+0000 7fb3caffd640 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b91ecb0 0x55e18b9210a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3b40a3460 0x7fb3b40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3c8ff9640 1 -- 172.21.15.37:0/469660428 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb3bc0f0060 con 0x7fb3cc0745a0 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 -- 172.21.15.37:0/263439533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b91ecb0 msgr2=0x55e18b9210a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.732 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b91ecb0 0x55e18b9210a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 -- 172.21.15.37:0/263439533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 msgr2=0x55e18b91e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b91e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 -- 172.21.15.37:0/263439533 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18b81f340 con 0x55e18b913af0 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3caffd640 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b91ecb0 0x55e18b9210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b91e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b913ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4011cc0 tx=0x7fb3c400f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1e704c8a43dc5b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3b40a3460 0x7fb3b40a5920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400fc90 tx=0x7fb3c4018000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3b1ffb640 1 -- 172.21.15.37:0/263439533 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4004b20 con 0x55e18b913af0 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3b1ffb640 1 -- 172.21.15.37:0/263439533 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c4009ce0 con 0x55e18b913af0 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.725+0000 7fb3b1ffb640 1 -- 172.21.15.37:0/263439533 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4005350 con 0x55e18b913af0 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 -- 172.21.15.37:0/263439533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 msgr2=0x55e18b913ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b913ed0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4011cc0 tx=0x7fb3c400f790 comp rx=0 tx=0).stop 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 -- 172.21.15.37:0/263439533 shutdown_connections 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b91ecb0 0x55e18b9210a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b91e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/263439533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b913ed0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 -- 172.21.15.37:0/263439533 >> 172.21.15.37:0/263439533 conn(0x55e18b910060 msgr2=0x55e18b910440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 -- 172.21.15.37:0/263439533 shutdown_connections 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.726+0000 7fb3d4e40740 1 -- 172.21.15.37:0/263439533 wait complete. 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 -- start start 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 Processor -- start 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b91bb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b914410 0x55e18b91c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b91c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e18b921880 con 0x55e18b91ecb0 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e18b801290 con 0x55e18b913af0 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e18b921a00 con 0x55e18b914410 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3d4e40740 10 client.0 _ll_register_callbacks cb 0x55e18b832140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b91c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b91c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49986/0 (socket says 172.21.15.37:49986) 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3cbfff640 1 -- 172.21.15.37:0/1056972831 learned_addr learned my addr 172.21.15.37:0/1056972831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b91bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.727+0000 7fb3caffd640 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b914410 0x55e18b91c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3cbfff640 1 -- 172.21.15.37:0/1056972831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b914410 msgr2=0x55e18b91c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b914410 0x55e18b91c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3cbfff640 1 -- 172.21.15.37:0/1056972831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 msgr2=0x55e18b91bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b91bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3cbfff640 1 -- 172.21.15.37:0/1056972831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18b58b210 con 0x55e18b91ecb0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3caffd640 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b914410 0x55e18b91c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b91c5d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4011fb0 tx=0x7fb3c40117d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905eaa44ac80a4ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3aaffd640 1 -- 172.21.15.37:0/1056972831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4033020 con 0x55e18b91ecb0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3aaffd640 1 -- 172.21.15.37:0/1056972831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb3c4010c10 con 0x55e18b91ecb0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c4009ce0 con 0x55e18b91ecb0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3aaffd640 1 -- 172.21.15.37:0/1056972831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3c4010e10 con 0x55e18b91ecb0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3d4e40740 10 client.16890 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e18b657620 con 0x55e18b91ecb0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.728+0000 7fb3aaffd640 1 -- 172.21.15.37:0/1056972831 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c4009ce0 con 0x55e18b91ecb0 2024-09-19T06:46:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3aaffd640 1 -- 172.21.15.37:0/1056972831 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb3c40265e0 con 0x55e18b91ecb0 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3d4e40740 10 client.16890 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3d4e40740 10 client.16890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3d4e40740 20 client.16890 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b92eba0 0x55e18b930f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3d4e40740 4 client.16890 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e18b885a20 con 0x55e18b92eba0 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b92eba0 0x55e18b930f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b92eba0 0x55e18b930f90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=181e3a42f08d7ac2 server_cookie=7bb96e962732c229 in_seq=0 out_seq=0 2024-09-19T06:46:48.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.729+0000 7fb3aaffd640 10 client.16890 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3aaffd640 1 -- 172.21.15.37:0/1056972831 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e18b885a20 con 0x55e18b92eba0 2024-09-19T06:46:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3aaffd640 10 client.16890 handle_command_reply: tid=0 2024-09-19T06:46:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 client.16890 shutdown 2024-09-19T06:46:48.738 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 msgr2=0x55e18b91c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b91c5d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4011fb0 tx=0x7fb3c40117d0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 shutdown_connections 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b92eba0 0x55e18b930f90 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b91c5d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b914410 0x55e18b91c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/1056972831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b913af0 0x55e18b91bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3aaffd640 0 client.16890 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.730+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 >> 172.21.15.37:0/1056972831 conn(0x55e18b910060 msgr2=0x55e18b915dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.731+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 shutdown_connections 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.731+0000 7fb3d4e40740 1 -- 172.21.15.37:0/1056972831 wait complete. 2024-09-19T06:46:48.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.731+0000 7fb3d4e40740 20 client.16890 trim_cache size 0 max 16384 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.745+0000 7fb3d4e40740 1 Processor -- start 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.745+0000 7fb3d4e40740 1 -- start start 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b91c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b914410 0x55e18b936770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b936cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1b2370 con 0x55e18b91ecb0 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1afc40 con 0x55e18b914410 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1ada90 con 0x55e18b913af0 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b936cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b936cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49996/0 (socket says 172.21.15.37:49996) 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 -- 172.21.15.37:0/2641220525 learned_addr learned my addr 172.21.15.37:0/2641220525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b91c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 -- 172.21.15.37:0/2641220525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 msgr2=0x55e18b91c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3caffd640 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b914410 0x55e18b936770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b91c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 -- 172.21.15.37:0/2641220525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b914410 msgr2=0x55e18b936770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b914410 0x55e18b936770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 -- 172.21.15.37:0/2641220525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18b90fe90 con 0x55e18b91ecb0 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b91c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b936cb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4000f80 tx=0x7fb3c4004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2442c8b1a801804d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3b17fa640 1 -- 172.21.15.37:0/2641220525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4033040 con 0x55e18b91ecb0 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3b17fa640 1 -- 172.21.15.37:0/2641220525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c4044070 con 0x55e18b91ecb0 2024-09-19T06:46:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.746+0000 7fb3b17fa640 1 -- 172.21.15.37:0/2641220525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c4005000 con 0x55e18b91ecb0 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 -- 172.21.15.37:0/2641220525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 msgr2=0x55e18b936cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b936cb0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fb3c4000f80 tx=0x7fb3c4004dc0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 -- 172.21.15.37:0/2641220525 shutdown_connections 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b91ecb0 0x55e18b936cb0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b914410 0x55e18b936770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/2641220525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b91c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 -- 172.21.15.37:0/2641220525 >> 172.21.15.37:0/2641220525 conn(0x55e18b910060 msgr2=0x55e18b91dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 -- 172.21.15.37:0/2641220525 shutdown_connections 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 -- 172.21.15.37:0/2641220525 wait complete. 2024-09-19T06:46:48.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.747+0000 7fb3d4e40740 1 -- start start 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 1 Processor -- start 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b926fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b927510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b91ecb0 0x55e18b918580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1ae100 con 0x55e18b914410 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1af570 con 0x55e18b91ecb0 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3cc1ae7d0 con 0x55e18b913af0 2024-09-19T06:46:48.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3d4e40740 10 client.0 _ll_register_callbacks cb 0x7fb3c00097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b927510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b927510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50002/0 (socket says 172.21.15.37:50002) 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 -- 172.21.15.37:0/3251169668 learned_addr learned my addr 172.21.15.37:0/3251169668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b926fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b91ecb0 0x55e18b918580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 -- 172.21.15.37:0/3251169668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 msgr2=0x55e18b926fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b926fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 -- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b91ecb0 msgr2=0x55e18b918580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b91ecb0 0x55e18b918580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3caffd640 1 -- 172.21.15.37:0/3251169668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18b918db0 con 0x55e18b914410 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.748+0000 7fb3cbfff640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b91ecb0 0x55e18b918580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3caffd640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b927510 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55e18b9113e0 tx=0x7fb3c0010b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e634167f32a7711 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:46:48.755 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3aaffd640 1 -- 172.21.15.37:0/3251169668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0004250 con 0x55e18b914410 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3aaffd640 1 -- 172.21.15.37:0/3251169668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb3c00043f0 con 0x55e18b914410 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3c4044070 con 0x55e18b914410 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3aaffd640 1 -- 172.21.15.37:0/3251169668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb3c000f8f0 con 0x55e18b914410 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 10 client.16896 fetch_fsmap learned FSMap version 31 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55e18b999350 con 0x55e18b914410 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3aaffd640 1 -- 172.21.15.37:0/3251169668 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3c4044070 con 0x55e18b914410 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3aaffd640 1 -- 172.21.15.37:0/3251169668 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb3c001d3f0 con 0x55e18b914410 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 10 client.16896 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 10 client.16896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 20 client.16896 populate_metadata read hostname 'smithi037' 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b999b50 0x55e18b99bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:46:48.756 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 4 client.16896 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:46:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.749+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e18b87fd70 con 0x55e18b999b50 2024-09-19T06:46:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.750+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b999b50 0x55e18b99bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:46:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.750+0000 7fb3aaffd640 10 client.16896 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:48.757 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.750+0000 7fb3cb7fe640 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b999b50 0x55e18b99bf70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59cb2591e8f3f4e8 server_cookie=435928d117aa0b18 in_seq=0 out_seq=0 2024-09-19T06:46:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.809+0000 7fb3aaffd640 1 -- 172.21.15.37:0/3251169668 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e18b87fd70 con 0x55e18b999b50 2024-09-19T06:46:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.809+0000 7fb3aaffd640 10 client.16896 handle_command_reply: tid=0 2024-09-19T06:46:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.809+0000 7fb3d4e40740 1 client.16896 shutdown 2024-09-19T06:46:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 msgr2=0x55e18b927510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b927510 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x55e18b9113e0 tx=0x7fb3c0010b40 comp rx=0 tx=0).stop 2024-09-19T06:46:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 shutdown_connections 2024-09-19T06:46:48.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e18b999b50 0x55e18b99bf70 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e18b91ecb0 0x55e18b918580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e18b914410 0x55e18b927510 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 --2- 172.21.15.37:0/3251169668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e18b913af0 0x55e18b926fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3aaffd640 0 client.16896 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 >> 172.21.15.37:0/3251169668 conn(0x55e18b910060 msgr2=0x55e18b915d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 shutdown_connections 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 1 -- 172.21.15.37:0/3251169668 wait complete. 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.810+0000 7fb3d4e40740 20 client.16896 trim_cache size 0 max 16384 2024-09-19T06:46:48.813 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:46:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:46:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:46:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:46:48.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3b40a3460 msgr2=0x7fb3b40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3b40a3460 0x7fb3b40a5920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb3c400fc90 tx=0x7fb3c4018000 comp rx=0 tx=0).stop 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 msgr2=0x7fb3cc19a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc19a540 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fb3bc006010 tx=0x7fb3bc004040 comp rx=0 tx=0).stop 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 shutdown_connections 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3cc10e3a0 0x7fb3cc19ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3cc074ec0 0x7fb3cc19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3b40a3460 0x7fb3b40a5920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 --2- 172.21.15.37:0/469660428 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3cc0745a0 0x7fb3cc19a540 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 >> 172.21.15.37:0/469660428 conn(0x7fb3cc0fd7e0 msgr2=0x7fb3cc0fe3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 shutdown_connections 2024-09-19T06:46:48.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:46:48.811+0000 7fb3d20af640 1 -- 172.21.15.37:0/469660428 wait complete. 2024-09-19T06:46:49.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.98 MB/sec execute 76 sec latency 8799.041 ms 2024-09-19T06:46:50.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.89 MB/sec execute 77 sec latency 9799.169 ms 2024-09-19T06:46:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:50 smithi151 ceph-mon[35236]: pgmap v155: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-09-19T06:46:50.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:50 smithi096 ceph-mon[35004]: pgmap v155: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-09-19T06:46:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:50 smithi037 ceph-mon[31211]: pgmap v155: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.8 MiB/s wr, 142 op/s 2024-09-19T06:46:51.204 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.81 MB/sec execute 78 sec latency 10799.252 ms 2024-09-19T06:46:52.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.72 MB/sec execute 79 sec latency 11799.384 ms 2024-09-19T06:46:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:52 smithi151 ceph-mon[35236]: pgmap v156: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 817 KiB/s wr, 76 op/s 2024-09-19T06:46:52.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:52 smithi096 ceph-mon[35004]: pgmap v156: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 817 KiB/s wr, 76 op/s 2024-09-19T06:46:52.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:52 smithi037 ceph-mon[31211]: pgmap v156: 97 pgs: 97 active+clean; 38 MiB data, 970 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 817 KiB/s wr, 76 op/s 2024-09-19T06:46:53.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.64 MB/sec execute 80 sec latency 12799.478 ms 2024-09-19T06:46:54.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.55 MB/sec execute 81 sec latency 13799.595 ms 2024-09-19T06:46:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:54 smithi151 ceph-mon[35236]: pgmap v157: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 840 KiB/s wr, 121 op/s 2024-09-19T06:46:54.377 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:54 smithi037 ceph-mon[31211]: pgmap v157: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 840 KiB/s wr, 121 op/s 2024-09-19T06:46:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:54 smithi096 ceph-mon[35004]: pgmap v157: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 840 KiB/s wr, 121 op/s 2024-09-19T06:46:55.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.47 MB/sec execute 82 sec latency 14799.706 ms 2024-09-19T06:46:56.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.40 MB/sec execute 83 sec latency 15799.815 ms 2024-09-19T06:46:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:56 smithi151 ceph-mon[35236]: pgmap v158: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 48 op/s 2024-09-19T06:46:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:56 smithi096 ceph-mon[35004]: pgmap v158: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 48 op/s 2024-09-19T06:46:56.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:56 smithi037 ceph-mon[31211]: pgmap v158: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 48 op/s 2024-09-19T06:46:57.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.32 MB/sec execute 84 sec latency 16799.941 ms 2024-09-19T06:46:58.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.24 MB/sec execute 85 sec latency 17800.053 ms 2024-09-19T06:46:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:46:58 smithi151 ceph-mon[35236]: pgmap v159: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 48 op/s 2024-09-19T06:46:58.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:46:58 smithi096 ceph-mon[35004]: pgmap v159: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 48 op/s 2024-09-19T06:46:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:46:58 smithi037 ceph-mon[31211]: pgmap v159: 97 pgs: 97 active+clean; 30 MiB data, 956 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 27 KiB/s wr, 48 op/s 2024-09-19T06:46:59.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.17 MB/sec execute 86 sec latency 18800.126 ms 2024-09-19T06:47:00.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.10 MB/sec execute 87 sec latency 19800.182 ms 2024-09-19T06:47:00.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:00 smithi151 ceph-mon[35236]: pgmap v160: 97 pgs: 97 active+clean; 30 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 KiB/s wr, 52 op/s 2024-09-19T06:47:00.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:00 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:00 smithi096 ceph-mon[35004]: pgmap v160: 97 pgs: 97 active+clean; 30 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 KiB/s wr, 52 op/s 2024-09-19T06:47:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:00 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:00 smithi037 ceph-mon[31211]: pgmap v160: 97 pgs: 97 active+clean; 30 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 29 KiB/s wr, 52 op/s 2024-09-19T06:47:00.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:00 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:47:01.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 6.03 MB/sec execute 88 sec latency 20800.221 ms 2024-09-19T06:47:01.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:01.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:01.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:47:01.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:01.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:01.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:47:01.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:47:01.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:47:01.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:47:01.790 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/3532748757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481009b0 msgr2=0x7f0f48100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/3532748757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481009b0 0x7f0f48100e10 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009930 tx=0x7f0f3c02a420 comp rx=0 tx=0).stop 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/3532748757 shutdown_connections 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/3532748757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f48101350 0x7f0f4810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/3532748757 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481009b0 0x7f0f48100e10 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/3532748757 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f481069c0 0x7f0f48106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/3532748757 >> 172.21.15.37:0/3532748757 conn(0x7f0f48077990 msgr2=0x7f0f480fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.906+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/3532748757 shutdown_connections 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.907+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/3532748757 wait complete. 2024-09-19T06:47:01.909 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.907+0000 7f0f4e9bd640 1 Processor -- start 2024-09-19T06:47:01.910 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.907+0000 7f0f4e9bd640 1 -- start start 2024-09-19T06:47:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.907+0000 7f0f4e9bd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f481009b0 0x7f0f48197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.907+0000 7f0f4e9bd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48101350 0x7f0f481977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.907+0000 7f0f4e9bd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481069c0 0x7f0f4819bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.907+0000 7f0f4e9bd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110e00 con 0x7f0f481069c0 2024-09-19T06:47:01.911 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4e9bd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110c80 con 0x7f0f481009b0 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4e9bd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f48110f80 con 0x7f0f48101350 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481069c0 0x7f0f4819bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481069c0 0x7f0f4819bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45716/0 (socket says 172.21.15.37:45716) 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 -- 172.21.15.37:0/1026176976 learned_addr learned my addr 172.21.15.37:0/1026176976 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f481009b0 0x7f0f48197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f477fe640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48101350 0x7f0f481977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 -- 172.21.15.37:0/1026176976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48101350 msgr2=0x7f0f481977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48101350 0x7f0f481977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 -- 172.21.15.37:0/1026176976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f481009b0 msgr2=0x7f0f48197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f481009b0 0x7f0f48197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 -- 172.21.15.37:0/1026176976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f4819c280 con 0x7f0f481069c0 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4cf5a640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481069c0 0x7f0f4819bb50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000eca0 tx=0x7f0f4000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.912 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f457fa640 1 -- 172.21.15.37:0/1026176976 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40004b20 con 0x7f0f481069c0 2024-09-19T06:47:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f457fa640 1 -- 172.21.15.37:0/1026176976 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f40004cc0 con 0x7f0f481069c0 2024-09-19T06:47:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f457fa640 1 -- 172.21.15.37:0/1026176976 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f40017650 con 0x7f0f481069c0 2024-09-19T06:47:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f4819c570 con 0x7f0f481069c0 2024-09-19T06:47:01.913 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.908+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f480698a0 con 0x7f0f481069c0 2024-09-19T06:47:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.911+0000 7f0f457fa640 1 -- 172.21.15.37:0/1026176976 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0f40005020 con 0x7f0f481069c0 2024-09-19T06:47:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f457fa640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f300a3460 0x7f0f300a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f300a3460 0x7f0f300a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f457fa640 1 -- 172.21.15.37:0/1026176976 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0f400d1910 con 0x7f0f481069c0 2024-09-19T06:47:01.915 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 Processor -- start 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 -- start start 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f300a3460 0x7f0f300a5920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009ae0 tx=0x7f0f38005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0baf0 0x5556dae0bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae16770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae16cb0 0x5556dae190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556dad702f0 con 0x5556dae16cb0 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5556dad18270 con 0x5556dae0c410 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f51832740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5556dad6c2b0 con 0x5556dae0baf0 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f4cf5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae16cb0 0x5556dae190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.912+0000 7f0f4cf5a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae16cb0 0x5556dae190a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45722/0 (socket says 172.21.15.37:45722) 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f4cf5a640 1 -- 172.21.15.37:0/2997802649 learned_addr learned my addr 172.21.15.37:0/2997802649 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f47fff640 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0baf0 0x5556dae0bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.916 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f477fe640 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae16770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f47fff640 1 -- 172.21.15.37:0/2997802649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 msgr2=0x5556dae16770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f47fff640 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae16770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f47fff640 1 -- 172.21.15.37:0/2997802649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae16cb0 msgr2=0x5556dae190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f47fff640 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae16cb0 0x5556dae190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f47fff640 1 -- 172.21.15.37:0/2997802649 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556dad17340 con 0x5556dae0baf0 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f4cf5a640 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae16cb0 0x5556dae190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f47fff640 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0baf0 0x5556dae0bed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f380117b0 tx=0x7f0f38014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997ecfb2c2f30bbc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f2e7fc640 1 -- 172.21.15.37:0/2997802649 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38023070 con 0x5556dae0baf0 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f2e7fc640 1 -- 172.21.15.37:0/2997802649 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f38011e60 con 0x5556dae0baf0 2024-09-19T06:47:01.917 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.913+0000 7f0f2e7fc640 1 -- 172.21.15.37:0/2997802649 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3801f680 con 0x5556dae0baf0 2024-09-19T06:47:01.921 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 -- 172.21.15.37:0/2997802649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0baf0 msgr2=0x5556dae0bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0baf0 0x5556dae0bed0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0f380117b0 tx=0x7f0f38014000 comp rx=0 tx=0).stop 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 -- 172.21.15.37:0/2997802649 shutdown_connections 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae16cb0 0x5556dae190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae16770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 --2- 172.21.15.37:0/2997802649 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0baf0 0x5556dae0bed0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 -- 172.21.15.37:0/2997802649 >> 172.21.15.37:0/2997802649 conn(0x5556dae08060 msgr2=0x5556dae08440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 -- 172.21.15.37:0/2997802649 shutdown_connections 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.914+0000 7f0f51832740 1 -- 172.21.15.37:0/2997802649 wait complete. 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.915+0000 7f0f51832740 1 -- start start 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.915+0000 7f0f51832740 1 Processor -- start 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.915+0000 7f0f51832740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0baf0 0x5556dae13bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f51832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0c410 0x5556dae14130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.922 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f51832740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae14670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f51832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5556dae19880 con 0x5556dae0c410 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f51832740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5556dacf9290 con 0x5556dae0baf0 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f51832740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5556dae19a00 con 0x5556dae16cb0 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f51832740 10 client.0 _ll_register_callbacks cb 0x5556dad2a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0c410 0x5556dae14130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0c410 0x5556dae14130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45724/0 (socket says 172.21.15.37:45724) 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 -- 172.21.15.37:0/4168560344 learned_addr learned my addr 172.21.15.37:0/4168560344 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f47fff640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0baf0 0x5556dae13bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f4cf5a640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae14670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 -- 172.21.15.37:0/4168560344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 msgr2=0x5556dae14670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae14670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 -- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0baf0 msgr2=0x5556dae13bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0baf0 0x5556dae13bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.923 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 -- 172.21.15.37:0/4168560344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556daa83210 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f47fff640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0baf0 0x5556dae13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f477fe640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0c410 0x5556dae14130 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c02a940 tx=0x7f0f3c0317c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40f3431b41e26e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f277fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.916+0000 7f0f277fe640 1 -- 172.21.15.37:0/4168560344 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c035040 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f38011e60 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f277fe640 1 -- 172.21.15.37:0/4168560344 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0f3c004040 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f277fe640 1 -- 172.21.15.37:0/4168560344 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0f3c004240 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 10 client.16908 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5556dab4f620 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f277fe640 1 -- 172.21.15.37:0/4168560344 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f38011e60 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f277fe640 1 -- 172.21.15.37:0/4168560344 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0f3c0049c0 con 0x5556dae0c410 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 10 client.16908 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 10 client.16908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 20 client.16908 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:01.924 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae26c50 0x5556dae29040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 4 client.16908 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556dad7da20 con 0x5556dae26c50 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.917+0000 7f0f47fff640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae26c50 0x5556dae29040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.918+0000 7f0f277fe640 10 client.16908 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.918+0000 7f0f47fff640 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae26c50 0x5556dae29040 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd6a54af6a49cb8 server_cookie=2fba16848551040e in_seq=0 out_seq=0 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.918+0000 7f0f277fe640 1 -- 172.21.15.37:0/4168560344 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5556dad7da20 con 0x5556dae26c50 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.918+0000 7f0f277fe640 10 client.16908 handle_command_reply: tid=0 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.918+0000 7f0f51832740 1 client.16908 shutdown 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0c410 msgr2=0x5556dae14130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0c410 0x5556dae14130 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c02a940 tx=0x7f0f3c0317c0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 shutdown_connections 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae26c50 0x5556dae29040 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae14670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0c410 0x5556dae14130 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.925 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f277fe640 0 client.16908 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 --2- 172.21.15.37:0/4168560344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0baf0 0x5556dae13bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 >> 172.21.15.37:0/4168560344 conn(0x5556dae08060 msgr2=0x5556dae0dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 shutdown_connections 2024-09-19T06:47:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 1 -- 172.21.15.37:0/4168560344 wait complete. 2024-09-19T06:47:01.926 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.919+0000 7f0f51832740 20 client.16908 trim_cache size 0 max 16384 2024-09-19T06:47:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.933+0000 7f0f51832740 1 Processor -- start 2024-09-19T06:47:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.933+0000 7f0f51832740 1 -- start start 2024-09-19T06:47:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f51832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae0e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.936 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f51832740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae144c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f51832740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae2e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f51832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481adf60 con 0x5556dae0baf0 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f51832740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ab830 con 0x5556dae0c410 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f51832740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481a9680 con 0x5556dae16cb0 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae0e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae0e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45734/0 (socket says 172.21.15.37:45734) 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 -- 172.21.15.37:0/1038262632 learned_addr learned my addr 172.21.15.37:0/1038262632 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f4cf5a640 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae2e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f477fe640 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae144c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 -- 172.21.15.37:0/1038262632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 msgr2=0x5556dae2e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae2e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 -- 172.21.15.37:0/1038262632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 msgr2=0x5556dae144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae144c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.937 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 -- 172.21.15.37:0/1038262632 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556dae293b0 con 0x5556dae0baf0 2024-09-19T06:47:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae0e560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0f38003fa0 tx=0x7f0f38003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ea1bde9ff2aadd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.934+0000 7f0f2dffb640 1 -- 172.21.15.37:0/1038262632 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38023030 con 0x5556dae0baf0 2024-09-19T06:47:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f2dffb640 1 -- 172.21.15.37:0/1038262632 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f38009810 con 0x5556dae0baf0 2024-09-19T06:47:01.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f2dffb640 1 -- 172.21.15.37:0/1038262632 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38004870 con 0x5556dae0baf0 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 -- 172.21.15.37:0/1038262632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 msgr2=0x5556dae0e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae0e560 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f0f38003fa0 tx=0x7f0f38003f30 comp rx=0 tx=0).stop 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 -- 172.21.15.37:0/1038262632 shutdown_connections 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae16cb0 0x5556dae2e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae0c410 0x5556dae144c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 --2- 172.21.15.37:0/1038262632 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae0e560 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 -- 172.21.15.37:0/1038262632 >> 172.21.15.37:0/1038262632 conn(0x5556dae08060 msgr2=0x5556dae15c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 -- 172.21.15.37:0/1038262632 shutdown_connections 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.935+0000 7f0f51832740 1 -- 172.21.15.37:0/1038262632 wait complete. 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 -- start start 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 Processor -- start 2024-09-19T06:47:01.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae1f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0c410 0x5556dae10670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae16cb0 0x5556dae10bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481a9cf0 con 0x5556dae0baf0 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481ab160 con 0x5556dae16cb0 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f481aa3c0 con 0x5556dae0c410 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f51832740 10 client.0 _ll_register_callbacks cb 0x7f0f38005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae1f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f47fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae1f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45740/0 (socket says 172.21.15.37:45740) 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f47fff640 1 -- 172.21.15.37:0/1712396171 learned_addr learned my addr 172.21.15.37:0/1712396171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f477fe640 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0c410 0x5556dae10670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f4cf5a640 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae16cb0 0x5556dae10bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.936+0000 7f0f47fff640 1 -- 172.21.15.37:0/1712396171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0c410 msgr2=0x5556dae10670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0c410 0x5556dae10670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f47fff640 1 -- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae16cb0 msgr2=0x5556dae10bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:01.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae16cb0 0x5556dae10bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f47fff640 1 -- 172.21.15.37:0/1712396171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556dae110f0 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f47fff640 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae1f120 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005b60 tx=0x7f0f38011910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9e0d45b27d22ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f277fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f277fe640 1 -- 172.21.15.37:0/1712396171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38023030 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f277fe640 1 -- 172.21.15.37:0/1712396171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0f38003b90 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5556dad77d70 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f277fe640 1 -- 172.21.15.37:0/1712396171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0f3804d050 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f51832740 10 client.16914 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x5556dae91640 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.937+0000 7f0f277fe640 1 -- 172.21.15.37:0/1712396171 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5556dad77d70 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f277fe640 1 -- 172.21.15.37:0/1712396171 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0f38004b30 con 0x5556dae0baf0 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f51832740 10 client.16914 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f51832740 10 client.16914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:01.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f51832740 20 client.16914 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f51832740 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae91b30 0x5556dae93f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f51832740 4 client.16914 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:47:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5556dacf4900 con 0x5556dae91b30 2024-09-19T06:47:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f477fe640 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae91b30 0x5556dae93f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f277fe640 10 client.16914 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:01.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.938+0000 7f0f477fe640 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae91b30 0x5556dae93f50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d912ae8b4ebb046 server_cookie=95d568c7845355f in_seq=0 out_seq=0 2024-09-19T06:47:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f277fe640 1 -- 172.21.15.37:0/1712396171 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5556dacf4900 con 0x5556dae91b30 2024-09-19T06:47:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f277fe640 10 client.16914 handle_command_reply: tid=0 2024-09-19T06:47:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 client.16914 shutdown 2024-09-19T06:47:01.963 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 msgr2=0x5556dae1f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae1f120 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f0f38005b60 tx=0x7f0f38011910 comp rx=0 tx=0).stop 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 shutdown_connections 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5556dae91b30 0x5556dae93f50 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5556dae16cb0 0x5556dae10bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5556dae0c410 0x5556dae10670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 --2- 172.21.15.37:0/1712396171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5556dae0baf0 0x5556dae1f120 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f277fe640 0 client.16914 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.961+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 >> 172.21.15.37:0/1712396171 conn(0x5556dae08060 msgr2=0x5556dae15c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 shutdown_connections 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f51832740 1 -- 172.21.15.37:0/1712396171 wait complete. 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f51832740 20 client.16914 trim_cache size 0 max 16384 2024-09-19T06:47:01.964 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:47:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:47:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:47:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:47:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f300a3460 msgr2=0x7f0f300a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f300a3460 0x7f0f300a5920 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0f38009ae0 tx=0x7f0f38005f90 comp rx=0 tx=0).stop 2024-09-19T06:47:01.965 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481069c0 msgr2=0x7f0f4819bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481069c0 0x7f0f4819bb50 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f0f4000eca0 tx=0x7f0f4000b690 comp rx=0 tx=0).stop 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 shutdown_connections 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f300a3460 0x7f0f300a5920 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f481069c0 0x7f0f4819bb50 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f48101350 0x7f0f481977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 --2- 172.21.15.37:0/1026176976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f481009b0 0x7f0f48197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 >> 172.21.15.37:0/1026176976 conn(0x7f0f48077990 msgr2=0x7f0f48103710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 shutdown_connections 2024-09-19T06:47:01.966 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:01.962+0000 7f0f4e9bd640 1 -- 172.21.15.37:0/1026176976 wait complete. 2024-09-19T06:47:02.087 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:02.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.96 MB/sec execute 89 sec latency 21800.283 ms 2024-09-19T06:47:02.337 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:02 smithi037 ceph-mon[31211]: pgmap v161: 97 pgs: 97 active+clean; 30 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 25 KiB/s wr, 48 op/s 2024-09-19T06:47:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:02 smithi151 ceph-mon[35236]: pgmap v161: 97 pgs: 97 active+clean; 30 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 25 KiB/s wr, 48 op/s 2024-09-19T06:47:02.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:02 smithi096 ceph-mon[35004]: pgmap v161: 97 pgs: 97 active+clean; 30 MiB data, 960 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 25 KiB/s wr, 48 op/s 2024-09-19T06:47:02.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.554+0000 7f79ac8df640 1 -- 172.21.15.37:0/1038947617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 msgr2=0x7f79a810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.554+0000 7f79ac8df640 1 --2- 172.21.15.37:0/1038947617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a810e610 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009ca0 tx=0x7f79a002a950 comp rx=0 tx=0).stop 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 -- 172.21.15.37:0/1038947617 shutdown_connections 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 --2- 172.21.15.37:0/1038947617 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a810e610 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 --2- 172.21.15.37:0/1038947617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79a8103660 0x7f79a810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 --2- 172.21.15.37:0/1038947617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79a8102d40 0x7f79a8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 -- 172.21.15.37:0/1038947617 >> 172.21.15.37:0/1038947617 conn(0x7f79a80fc7a0 msgr2=0x7f79a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 -- 172.21.15.37:0/1038947617 shutdown_connections 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 -- 172.21.15.37:0/1038947617 wait complete. 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 Processor -- start 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.555+0000 7f79ac8df640 1 -- start start 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79ac8df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79a8102d40 0x7f79a81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79ac8df640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79a8103660 0x7f79a8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79ac8df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79ac8df640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8111f30 con 0x7f79a810c170 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79ac8df640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79a8111db0 con 0x7f79a8102d40 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79ac8df640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81120b0 con 0x7f79a8103660 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a8101420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45762/0 (socket says 172.21.15.37:45762) 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 -- 172.21.15.37:0/2429752462 learned_addr learned my addr 172.21.15.37:0/2429752462 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79a8102d40 0x7f79a81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79a8103660 0x7f79a8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 -- 172.21.15.37:0/2429752462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79a8103660 msgr2=0x7f79a8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79a8103660 0x7f79a8100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 -- 172.21.15.37:0/2429752462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79a8102d40 msgr2=0x7f79a81009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79a8102d40 0x7f79a81009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 -- 172.21.15.37:0/2429752462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79a81a6900 con 0x7f79a810c170 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.556+0000 7f79a6d9d640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a8101420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f79a002ae70 tx=0x7f79a002f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.557+0000 7f79977fe640 1 -- 172.21.15.37:0/2429752462 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0034040 con 0x7f79a810c170 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.557+0000 7f79977fe640 1 -- 172.21.15.37:0/2429752462 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79a0030920 con 0x7f79a810c170 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.557+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79a81a6b90 con 0x7f79a810c170 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.557+0000 7f79977fe640 1 -- 172.21.15.37:0/2429752462 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79a0045480 con 0x7f79a810c170 2024-09-19T06:47:02.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.557+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79a81a7070 con 0x7f79a810c170 2024-09-19T06:47:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.559+0000 7f79977fe640 1 -- 172.21.15.37:0/2429752462 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f79a0037020 con 0x7f79a810c170 2024-09-19T06:47:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.559+0000 7f79aecb8740 1 Processor -- start 2024-09-19T06:47:02.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.559+0000 7f79aecb8740 1 -- start start 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ec4a0 0x560f023ec880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023f7680 0x560f023f9a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79aecb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560f023522e0 con 0x560f023f7680 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79aecb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560f022f9ce0 con 0x560f023ec4a0 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79aecb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f0234e2a0 con 0x560f023ecdc0 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79977fe640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f79900a33b0 0x7f79900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023f7680 0x560f023f9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023f7680 0x560f023f9a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45768/0 (socket says 172.21.15.37:45768) 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3291167432 learned_addr learned my addr 172.21.15.37:0/3291167432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ec4a0 0x560f023ec880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a659c640 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a6d9d640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f79900a33b0 0x7f79900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3291167432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 msgr2=0x560f023f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3291167432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ec4a0 msgr2=0x560f023ec880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ec4a0 0x560f023ec880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3291167432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f022de440 con 0x560f023f7680 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79a659c640 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.560+0000 7f79977fe640 1 -- 172.21.15.37:0/2429752462 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f79a00f0b10 con 0x7f79a810c170 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.561+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023f7680 0x560f023f9a70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f799c00dfd0 tx=0x7f799c00d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a60d7d23e6f20e47 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.561+0000 7f79a6d9d640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f79900a33b0 0x7f79900a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b98d0 tx=0x7f79a008fd30 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.561+0000 7f7994ff9640 1 -- 172.21.15.37:0/3291167432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c005220 con 0x560f023f7680 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.561+0000 7f7994ff9640 1 -- 172.21.15.37:0/3291167432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c00ae30 con 0x560f023f7680 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.561+0000 7f7994ff9640 1 -- 172.21.15.37:0/3291167432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c011800 con 0x560f023f7680 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.561+0000 7f79aecb8740 1 -- 172.21.15.37:0/3291167432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023f7680 msgr2=0x560f023f9a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.561+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023f7680 0x560f023f9a70 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f799c00dfd0 tx=0x7f799c00d360 comp rx=0 tx=0).stop 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- 172.21.15.37:0/3291167432 shutdown_connections 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023f7680 0x560f023f9a70 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3291167432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ec4a0 0x560f023ec880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- 172.21.15.37:0/3291167432 >> 172.21.15.37:0/3291167432 conn(0x560f023e8a30 msgr2=0x560f023e8e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- 172.21.15.37:0/3291167432 shutdown_connections 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- 172.21.15.37:0/3291167432 wait complete. 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- start start 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 Processor -- start 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ec4a0 0x560f023f7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ecdc0 0x560f023f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f023f4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560f023fa250 con 0x560f023ecdc0 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560f0230d050 con 0x560f023f7680 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560f023fa3d0 con 0x560f023ec4a0 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79aecb8740 10 client.0 _ll_register_callbacks cb 0x560f0234d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ecdc0 0x560f023f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.562+0000 7f79a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ecdc0 0x560f023f42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45770/0 (socket says 172.21.15.37:45770) 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a659c640 1 -- 172.21.15.37:0/1885861381 learned_addr learned my addr 172.21.15.37:0/1885861381 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ec4a0 0x560f023f7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f023f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a659c640 1 -- 172.21.15.37:0/1885861381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ec4a0 msgr2=0x560f023f7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a659c640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ec4a0 0x560f023f7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a659c640 1 -- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 msgr2=0x560f023f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a659c640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f023f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a659c640 1 -- 172.21.15.37:0/1885861381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f020c33c0 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f023f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79a659c640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ecdc0 0x560f023f42b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f799800bf90 tx=0x7f79980128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c31612cab01e5a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f798dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f798dffb640 1 -- 172.21.15.37:0/1885861381 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7998021060 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f798dffb640 1 -- 172.21.15.37:0/1885861381 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f79980107b0 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c00ae30 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f798dffb640 1 -- 172.21.15.37:0/1885861381 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f79980109b0 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79aecb8740 10 client.16926 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.563+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560f02405cd0 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f798dffb640 1 -- 172.21.15.37:0/1885861381 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c00ae30 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f798dffb640 1 -- 172.21.15.37:0/1885861381 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f799800e350 con 0x560f023ecdc0 2024-09-19T06:47:02.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79aecb8740 10 client.16926 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79aecb8740 10 client.16926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79aecb8740 20 client.16926 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79aecb8740 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024071e0 0x560f02409600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79aecb8740 4 client.16926 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560f0235f850 con 0x560f024071e0 2024-09-19T06:47:02.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024071e0 0x560f02409600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024071e0 0x560f02409600 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=563a85dbdf6ba4a4 server_cookie=ac238a8072288868 in_seq=0 out_seq=0 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.564+0000 7f798dffb640 10 client.16926 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f798dffb640 1 -- 172.21.15.37:0/1885861381 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560f0235f850 con 0x560f024071e0 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f798dffb640 10 client.16926 handle_command_reply: tid=0 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 client.16926 shutdown 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ecdc0 msgr2=0x560f023f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ecdc0 0x560f023f42b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f799800bf90 tx=0x7f79980128e0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 shutdown_connections 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024071e0 0x560f02409600 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f023f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ecdc0 0x560f023f42b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 --2- 172.21.15.37:0/1885861381 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ec4a0 0x560f023f7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f798dffb640 0 client.16926 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 >> 172.21.15.37:0/1885861381 conn(0x560f023e8a30 msgr2=0x560f023ee710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 shutdown_connections 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 1 -- 172.21.15.37:0/1885861381 wait complete. 2024-09-19T06:47:02.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.565+0000 7f79aecb8740 20 client.16926 trim_cache size 0 max 16384 2024-09-19T06:47:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.582+0000 7f79aecb8740 1 Processor -- start 2024-09-19T06:47:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.582+0000 7f79aecb8740 1 -- start start 2024-09-19T06:47:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f024034e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023ef7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f0240eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79aecb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81b2330 con 0x560f023ec4a0 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79aecb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81afc00 con 0x560f023f7680 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79aecb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81ada50 con 0x560f023ecdc0 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f024034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f024034e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45786/0 (socket says 172.21.15.37:45786) 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 -- 172.21.15.37:0/298198185 learned_addr learned my addr 172.21.15.37:0/298198185 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a659c640 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023ef7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 -- 172.21.15.37:0/298198185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 msgr2=0x560f023ef7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023ef7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 -- 172.21.15.37:0/298198185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 msgr2=0x560f0240eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f0240eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 -- 172.21.15.37:0/298198185 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f023f4ba0 con 0x560f023ec4a0 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f024034e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f799c00dfd0 tx=0x7f799c00d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cd0cd3d2d8f4da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f798ffff640 1 -- 172.21.15.37:0/298198185 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c02c070 con 0x560f023ec4a0 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f798ffff640 1 -- 172.21.15.37:0/298198185 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c005380 con 0x560f023ec4a0 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.583+0000 7f798ffff640 1 -- 172.21.15.37:0/298198185 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c027640 con 0x560f023ec4a0 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 -- 172.21.15.37:0/298198185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 msgr2=0x560f024034e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f024034e0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f799c00dfd0 tx=0x7f799c00d970 comp rx=0 tx=0).stop 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 -- 172.21.15.37:0/298198185 shutdown_connections 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023f7680 0x560f0240eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023ecdc0 0x560f023ef7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 --2- 172.21.15.37:0/298198185 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f024034e0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 -- 172.21.15.37:0/298198185 >> 172.21.15.37:0/298198185 conn(0x560f023e8a30 msgr2=0x560f023ef0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 -- 172.21.15.37:0/298198185 shutdown_connections 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 -- 172.21.15.37:0/298198185 wait complete. 2024-09-19T06:47:02.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.584+0000 7f79aecb8740 1 -- start start 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 1 Processor -- start 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f023ffc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ecdc0 0x560f023faf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023f7680 0x560f023fb480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81ae0c0 con 0x560f023ec4a0 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81af530 con 0x560f023ecdc0 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f79a81ae790 con 0x560f023f7680 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79aecb8740 10 client.0 _ll_register_callbacks cb 0x7f799800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f023ffc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f023ffc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45792/0 (socket says 172.21.15.37:45792) 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3982403683 learned_addr learned my addr 172.21.15.37:0/3982403683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a659c640 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ecdc0 0x560f023faf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023f7680 0x560f023fb480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3982403683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023f7680 msgr2=0x560f023fb480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023f7680 0x560f023fb480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ecdc0 msgr2=0x560f023faf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ecdc0 0x560f023faf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 -- 172.21.15.37:0/3982403683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f023fbaf0 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f79a5d9b640 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f023ffc10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f799c00abf0 tx=0x7f799c005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5920591050ee924a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.585+0000 7f798dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f798dffb640 1 -- 172.21.15.37:0/3982403683 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f799c02c040 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f798dffb640 1 -- 172.21.15.37:0/3982403683 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f799c004860 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f799c005380 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f798dffb640 1 -- 172.21.15.37:0/3982403683 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f799c004a60 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 10 client.16932 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560f02471d00 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f798dffb640 1 -- 172.21.15.37:0/3982403683 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f799c004c00 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f798dffb640 1 -- 172.21.15.37:0/3982403683 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f799c032030 con 0x560f023ec4a0 2024-09-19T06:47:02.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 10 client.16932 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 10 client.16932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 20 client.16932 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024720d0 0x560f024744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 4 client.16932 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:47:02.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.586+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f799c005380 con 0x560f024720d0 2024-09-19T06:47:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.587+0000 7f79a659c640 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024720d0 0x560f024744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.587+0000 7f79a659c640 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024720d0 0x560f024744c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67a4d3f24e1bcc82 server_cookie=1fa9b784fff9c5e in_seq=0 out_seq=0 2024-09-19T06:47:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.587+0000 7f798dffb640 10 client.16932 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.587+0000 7f798dffb640 1 -- 172.21.15.37:0/3982403683 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f799c005380 con 0x560f024720d0 2024-09-19T06:47:02.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.587+0000 7f798dffb640 10 client.16932 handle_command_reply: tid=0 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.587+0000 7f79aecb8740 1 client.16932 shutdown 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 msgr2=0x560f023ffc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f023ffc10 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f799c00abf0 tx=0x7f799c005be0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 shutdown_connections 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560f024720d0 0x560f024744c0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560f023f7680 0x560f023fb480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560f023ecdc0 0x560f023faf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 --2- 172.21.15.37:0/3982403683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560f023ec4a0 0x560f023ffc10 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 >> 172.21.15.37:0/3982403683 conn(0x560f023e8a30 msgr2=0x560f023f6330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f798dffb640 0 client.16932 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 shutdown_connections 2024-09-19T06:47:02.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 1 -- 172.21.15.37:0/3982403683 wait complete. 2024-09-19T06:47:02.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.588+0000 7f79aecb8740 20 client.16932 trim_cache size 0 max 16384 2024-09-19T06:47:02.596 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:47:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:47:02.596 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:47:02.596 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f79900a33b0 msgr2=0x7f79900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f79900a33b0 0x7f79900a5870 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f79a00b98d0 tx=0x7f79a008fd30 comp rx=0 tx=0).stop 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 msgr2=0x7f79a8101420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a8101420 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f79a002ae70 tx=0x7f79a002f5b0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 shutdown_connections 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f79900a33b0 0x7f79900a5870 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f79a810c170 0x7f79a8101420 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f79a8103660 0x7f79a8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 --2- 172.21.15.37:0/2429752462 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f79a8102d40 0x7f79a81009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:02.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 >> 172.21.15.37:0/2429752462 conn(0x7f79a80fc7a0 msgr2=0x7f79a80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:02.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 shutdown_connections 2024-09-19T06:47:02.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:02.589+0000 7f79ac8df640 1 -- 172.21.15.37:0/2429752462 wait complete. 2024-09-19T06:47:03.205 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.90 MB/sec execute 90 sec latency 22800.368 ms 2024-09-19T06:47:03.484 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:47:03.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:03.743 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:04.131 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:04 smithi037 ceph-mon[31211]: pgmap v162: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 886 KiB/s wr, 103 op/s 2024-09-19T06:47:04.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.83 MB/sec execute 91 sec latency 23800.431 ms 2024-09-19T06:47:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 -- 172.21.15.37:0/890710578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc106a80 msgr2=0x7fb3bc106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 --2- 172.21.15.37:0/890710578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc106a80 0x7fb3bc106e60 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009930 tx=0x7fb3b402a420 comp rx=0 tx=0).stop 2024-09-19T06:47:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 -- 172.21.15.37:0/890710578 shutdown_connections 2024-09-19T06:47:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 --2- 172.21.15.37:0/890710578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3bc101420 0x7fb3bc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 --2- 172.21.15.37:0/890710578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3bc100a80 0x7fb3bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 --2- 172.21.15.37:0/890710578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc106a80 0x7fb3bc106e60 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 -- 172.21.15.37:0/890710578 >> 172.21.15.37:0/890710578 conn(0x7fb3bc0fc7a0 msgr2=0x7fb3bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 -- 172.21.15.37:0/890710578 shutdown_connections 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.212+0000 7fb3c3875640 1 -- 172.21.15.37:0/890710578 wait complete. 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 Processor -- start 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 -- start start 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc100a80 0x7fb3bc19a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3bc101420 0x7fb3bc19aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3bc106a80 0x7fb3bc19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc110e80 con 0x7fb3bc100a80 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc110d00 con 0x7fb3bc106a80 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c3875640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb3bc111000 con 0x7fb3bc101420 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c1611640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc100a80 0x7fb3bc19a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:04.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c1611640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc100a80 0x7fb3bc19a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45816/0 (socket says 172.21.15.37:45816) 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c1611640 1 -- 172.21.15.37:0/1109409279 learned_addr learned my addr 172.21.15.37:0/1109409279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c0e10640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3bc101420 0x7fb3bc19aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.213+0000 7fb3c1e12640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3bc106a80 0x7fb3bc19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c1611640 1 -- 172.21.15.37:0/1109409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3bc101420 msgr2=0x7fb3bc19aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c1611640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3bc101420 0x7fb3bc19aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c1611640 1 -- 172.21.15.37:0/1109409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3bc106a80 msgr2=0x7fb3bc19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c1611640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3bc106a80 0x7fb3bc19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c1611640 1 -- 172.21.15.37:0/1109409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3bc19f610 con 0x7fb3bc100a80 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c0e10640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3bc101420 0x7fb3bc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c1611640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc100a80 0x7fb3bc19a660 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009230 tx=0x7fb3b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3ba7fc640 1 -- 172.21.15.37:0/1109409279 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b403f030 con 0x7fb3bc100a80 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3ba7fc640 1 -- 172.21.15.37:0/1109409279 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3b4003ad0 con 0x7fb3bc100a80 2024-09-19T06:47:04.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3bc19f8a0 con 0x7fb3bc100a80 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3ba7fc640 1 -- 172.21.15.37:0/1109409279 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3b4035070 con 0x7fb3bc100a80 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.214+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3bc1a71e0 con 0x7fb3bc100a80 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.215+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3bc100ee0 con 0x7fb3bc100a80 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.216+0000 7fb3ba7fc640 1 -- 172.21.15.37:0/1109409279 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb3b4042030 con 0x7fb3bc100a80 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.217+0000 7fb3ba7fc640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a40a3430 0x7fb3a40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.217+0000 7fb3c0e10640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a40a3430 0x7fb3a40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.217+0000 7fb3ba7fc640 1 -- 172.21.15.37:0/1109409279 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fb3b40f0640 con 0x7fb3bc100a80 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.218+0000 7fb3c0e10640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a40a3430 0x7fb3a40a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005fd0 tx=0x7fb3b0005ea0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:04.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.221+0000 7fb3ba7fc640 1 -- 172.21.15.37:0/1109409279 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb3b40bd020 con 0x7fb3bc100a80 2024-09-19T06:47:04.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:04 smithi151 ceph-mon[35236]: pgmap v162: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 886 KiB/s wr, 103 op/s 2024-09-19T06:47:04.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.435+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb3bc19b9c0 con 0x7fb3bc100a80 2024-09-19T06:47:04.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.436+0000 7fb3ba7fc640 1 -- 172.21.15.37:0/1109409279 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7fb3bc19b9c0 con 0x7fb3bc100a80 2024-09-19T06:47:04.437 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:47:04.437 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-19T06:45:42:465959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:41.617479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:47:04.437 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-19T06:47:04.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a40a3430 msgr2=0x7fb3a40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:04.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a40a3430 0x7fb3a40a58f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0005fd0 tx=0x7fb3b0005ea0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc100a80 msgr2=0x7fb3bc19a660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:04.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc100a80 0x7fb3bc19a660 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fb3b4009230 tx=0x7fb3b4004270 comp rx=0 tx=0).stop 2024-09-19T06:47:04.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 shutdown_connections 2024-09-19T06:47:04.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3bc106a80 0x7fb3bc19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3bc101420 0x7fb3bc19aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3a40a3430 0x7fb3a40a58f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 --2- 172.21.15.37:0/1109409279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3bc100a80 0x7fb3bc19a660 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:04.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.438+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 >> 172.21.15.37:0/1109409279 conn(0x7fb3bc0fc7a0 msgr2=0x7fb3bc06a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:04.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.439+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 shutdown_connections 2024-09-19T06:47:04.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:04.439+0000 7fb3c3875640 1 -- 172.21.15.37:0/1109409279 wait complete. 2024-09-19T06:47:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:04 smithi096 ceph-mon[35004]: pgmap v162: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 886 KiB/s wr, 103 op/s 2024-09-19T06:47:05.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.77 MB/sec execute 92 sec latency 24800.466 ms 2024-09-19T06:47:05.216 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:47:05.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:05 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1109409279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:05.479 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:05.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:05 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1109409279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:05.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:05 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1109409279' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2808307536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 msgr2=0x7f6d641067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2808307536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d641067e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f6d54009900 tx=0x7f6d5402a410 comp rx=0 tx=0).stop 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2808307536 shutdown_connections 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2808307536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d64106f10 0x7f6d6410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2808307536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d641067e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2808307536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d64101de0 0x7f6d641021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2808307536 >> 172.21.15.37:0/2808307536 conn(0x7f6d640fd8c0 msgr2=0x7f6d640ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:05.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2808307536 shutdown_connections 2024-09-19T06:47:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2808307536 wait complete. 2024-09-19T06:47:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.967+0000 7f6d6ba81640 1 Processor -- start 2024-09-19T06:47:05.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6ba81640 1 -- start start 2024-09-19T06:47:05.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6ba81640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d64101de0 0x7f6d64198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6ba81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d64198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6ba81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d64106f10 0x7f6d6419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6ba81640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64111f70 con 0x7f6d64102790 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6ba81640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d64111df0 con 0x7f6d64106f10 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6ba81640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641120f0 con 0x7f6d64101de0 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d64198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d64198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45838/0 (socket says 172.21.15.37:45838) 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 -- 172.21.15.37:0/2256595639 learned_addr learned my addr 172.21.15.37:0/2256595639 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d64106f10 0x7f6d6419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 -- 172.21.15.37:0/2256595639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d64101de0 msgr2=0x7f6d64198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6981d640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d64101de0 0x7f6d64198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d64101de0 0x7f6d64198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 -- 172.21.15.37:0/2256595639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d64106f10 msgr2=0x7f6d6419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d64106f10 0x7f6d6419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 -- 172.21.15.37:0/2256595639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d6419d400 con 0x7f6d64102790 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d64106f10 0x7f6d6419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6981d640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d64101de0 0x7f6d64198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.968+0000 7f6d6901c640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d64198990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d540042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.969+0000 7f6d5affd640 1 -- 172.21.15.37:0/2256595639 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5403f030 con 0x7f6d64102790 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.969+0000 7f6d5affd640 1 -- 172.21.15.37:0/2256595639 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d54003ce0 con 0x7f6d64102790 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.969+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d6419d690 con 0x7f6d64102790 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.969+0000 7f6d5affd640 1 -- 172.21.15.37:0/2256595639 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54035070 con 0x7f6d64102790 2024-09-19T06:47:05.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.969+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d641a4fd0 con 0x7f6d64102790 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.971+0000 7f6d5affd640 1 -- 172.21.15.37:0/2256595639 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6d54042030 con 0x7f6d64102790 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.971+0000 7f6d6de5a740 1 Processor -- start 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.971+0000 7f6d6de5a740 1 -- start start 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cd3090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cd3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 0x55f765ce4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f765c392f0 con 0x55f765cd4b40 2024-09-19T06:47:05.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f765be0cf0 con 0x55f765cd35d0 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f765c352b0 con 0x55f765ce2780 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cd3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cd3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45854/0 (socket says 172.21.15.37:45854) 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3448992427 learned_addr learned my addr 172.21.15.37:0/3448992427 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6981d640 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cd3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d5affd640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d500a3430 0x7f6d500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6901c640 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 0x55f765ce4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3448992427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 msgr2=0x55f765ce4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 0x55f765ce4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3448992427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 msgr2=0x55f765cd3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cd3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3448992427 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f765bc53a0 con 0x55f765cd4b40 2024-09-19T06:47:05.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6981d640 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cd3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.972+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d500a3430 0x7f6d500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.973+0000 7f6d5affd640 1 -- 172.21.15.37:0/2256595639 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6d540f0390 con 0x7f6d64102790 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.973+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cd3090 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009fa0 tx=0x7f6d5c00f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26eb4099220fa272 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.973+0000 7f6d4ffff640 1 -- 172.21.15.37:0/3448992427 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c01a610 con 0x55f765cd4b40 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.973+0000 7f6d4ffff640 1 -- 172.21.15.37:0/3448992427 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c01ac90 con 0x55f765cd4b40 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.973+0000 7f6d4ffff640 1 -- 172.21.15.37:0/3448992427 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c010920 con 0x55f765cd4b40 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3448992427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 msgr2=0x55f765cd3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cd3090 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009fa0 tx=0x7f6d5c00f7f0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d500a3430 0x7f6d500a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c020000 tx=0x7f6d5c01d000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3448992427 shutdown_connections 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 0x55f765ce4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cd3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3448992427 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cd3090 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3448992427 >> 172.21.15.37:0/3448992427 conn(0x55f765c9c310 msgr2=0x55f765ccfc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3448992427 shutdown_connections 2024-09-19T06:47:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3448992427 wait complete. 2024-09-19T06:47:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 -- start start 2024-09-19T06:47:05.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.974+0000 7f6d6de5a740 1 Processor -- start 2024-09-19T06:47:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ce2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765cdf400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765cdf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f765ce5480 con 0x55f765ce2780 2024-09-19T06:47:05.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f765bf4060 con 0x55f765cd4b40 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f765ce5600 con 0x55f765cd35d0 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6de5a740 10 client.0 _ll_register_callbacks cb 0x55f7659478c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765cdf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765cdf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45856/0 (socket says 172.21.15.37:45856) 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 -- 172.21.15.37:0/4213104071 learned_addr learned my addr 172.21.15.37:0/4213104071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6981d640 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ce2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6901c640 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765cdf400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 -- 172.21.15.37:0/4213104071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 msgr2=0x55f765ce2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ce2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 -- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 msgr2=0x55f765cdf400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765cdf400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 -- 172.21.15.37:0/4213104071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7659aa3c0 con 0x55f765ce2780 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.975+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765cdf9f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005b70 tx=0x7f6d5c010f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ef27fd6698ab36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:05.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/4213104071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c032060 con 0x55f765ce2780 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/4213104071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6d5c012240 con 0x55f765ce2780 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d5c01ac90 con 0x55f765ce2780 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/4213104071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6d5c009af0 con 0x55f765ce2780 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 10 client.16950 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f765cf0ef0 con 0x55f765ce2780 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/4213104071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d5c01ac90 con 0x55f765ce2780 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/4213104071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6d5c025540 con 0x55f765ce2780 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 10 client.16950 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:05.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 10 client.16950 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:47:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 20 client.16950 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765cf23b0 0x55f765cf47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 4 client.16950 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.976+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f765c46860 con 0x55f765cf23b0 2024-09-19T06:47:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.977+0000 7f6d6981d640 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765cf23b0 0x55f765cf47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.977+0000 7f6d6981d640 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765cf23b0 0x55f765cf47a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5056624e6d47ef32 server_cookie=e0cfdb43d94ca4bf in_seq=0 out_seq=0 2024-09-19T06:47:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.977+0000 7f6d4d7fa640 10 client.16950 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/4213104071 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f765c46860 con 0x55f765cf23b0 2024-09-19T06:47:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d4d7fa640 10 client.16950 handle_command_reply: tid=0 2024-09-19T06:47:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 client.16950 shutdown 2024-09-19T06:47:05.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 msgr2=0x55f765cdf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765cdf9f0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c005b70 tx=0x7f6d5c010f20 comp rx=0 tx=0).stop 2024-09-19T06:47:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 shutdown_connections 2024-09-19T06:47:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765cf23b0 0x55f765cf47a0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765cdf9f0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765cdf400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/4213104071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ce2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 >> 172.21.15.37:0/4213104071 conn(0x55f765c9c310 msgr2=0x55f765cd5960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d4d7fa640 0 client.16950 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 shutdown_connections 2024-09-19T06:47:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 1 -- 172.21.15.37:0/4213104071 wait complete. 2024-09-19T06:47:05.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.978+0000 7f6d6de5a740 20 client.16950 trim_cache size 0 max 16384 2024-09-19T06:47:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.991+0000 7f6d6de5a740 1 Processor -- start 2024-09-19T06:47:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.991+0000 7f6d6de5a740 1 -- start start 2024-09-19T06:47:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cee720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cde3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 0x55f765cf9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641adfc0 con 0x55f765cd4b40 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641ab890 con 0x55f765cd35d0 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641a96e0 con 0x55f765ce2780 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cde3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cde3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45862/0 (socket says 172.21.15.37:45862) 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 -- 172.21.15.37:0/2070858366 learned_addr learned my addr 172.21.15.37:0/2070858366 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6981d640 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cee720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:05.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 -- 172.21.15.37:0/2070858366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 msgr2=0x55f765cf9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 0x55f765cf9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 -- 172.21.15.37:0/2070858366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 msgr2=0x55f765cee720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cee720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 -- 172.21.15.37:0/2070858366 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f765ccfed0 con 0x55f765cd4b40 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d6901c640 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cde3e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d540bc3d0 tx=0x7f6d5403b5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dbfad4b2f1827ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d4f7fe640 1 -- 172.21.15.37:0/2070858366 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d54049070 con 0x55f765cd4b40 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d4f7fe640 1 -- 172.21.15.37:0/2070858366 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d540bcdd0 con 0x55f765cd4b40 2024-09-19T06:47:06.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.992+0000 7f6d4f7fe640 1 -- 172.21.15.37:0/2070858366 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d540b8d40 con 0x55f765cd4b40 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 -- 172.21.15.37:0/2070858366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 msgr2=0x55f765cde3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cde3e0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f6d540bc3d0 tx=0x7f6d5403b5d0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 -- 172.21.15.37:0/2070858366 shutdown_connections 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765ce2780 0x55f765cf9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765cd4b40 0x55f765cde3e0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/2070858366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd35d0 0x55f765cee720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 -- 172.21.15.37:0/2070858366 >> 172.21.15.37:0/2070858366 conn(0x55f765c9c310 msgr2=0x55f765cddcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:06.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 -- 172.21.15.37:0/2070858366 shutdown_connections 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 -- 172.21.15.37:0/2070858366 wait complete. 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.993+0000 7f6d6de5a740 1 -- start start 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 1 Processor -- start 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ceb180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765ce64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765ce6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641a9d50 con 0x55f765ce2780 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641ab1c0 con 0x55f765cd4b40 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6d641aa420 con 0x55f765cd35d0 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6de5a740 10 client.0 _ll_register_callbacks cb 0x7f6d5c009e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6a01e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765ce6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6901c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765ce64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:06.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6a01e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765ce6bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45874/0 (socket says 172.21.15.37:45874) 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6901c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765ce64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56406/0 (socket says 172.21.15.37:56406) 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3328710251 learned_addr learned my addr 172.21.15.37:0/3328710251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.994+0000 7f6d6981d640 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ceb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3328710251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 msgr2=0x55f765ceb180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ceb180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 msgr2=0x55f765ce64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765ce64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6a01e640 1 -- 172.21.15.37:0/3328710251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f765d5bd70 con 0x55f765ce2780 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6901c640 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765ce64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6a01e640 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765ce6bf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0259e0 tx=0x7f6d5c012d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20f9eab56b811da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/3328710251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c012240 con 0x55f765ce2780 2024-09-19T06:47:06.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/3328710251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f6d5c025440 con 0x55f765ce2780 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6d540bcdd0 con 0x55f765ce2780 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/3328710251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f6d5c009af0 con 0x55f765ce2780 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6de5a740 10 client.16956 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55f765d5cf10 con 0x55f765ce2780 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.995+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/3328710251 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6d540bcdd0 con 0x55f765ce2780 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/3328710251 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f6d5c01aab0 con 0x55f765ce2780 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6de5a740 10 client.16956 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6de5a740 10 client.16956 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6de5a740 20 client.16956 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765d5d5a0 0x55f765d5f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6de5a740 4 client.16956 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f765c40d70 con 0x55f765d5d5a0 2024-09-19T06:47:06.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6981d640 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765d5d5a0 0x55f765d5f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d6981d640 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765d5d5a0 0x55f765d5f9c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=32b6ab3da26eaf98 server_cookie=940d486c97e8d845 in_seq=0 out_seq=0 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.996+0000 7f6d4d7fa640 10 client.16956 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d4d7fa640 1 -- 172.21.15.37:0/3328710251 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f765c40d70 con 0x55f765d5d5a0 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d4d7fa640 10 client.16956 handle_command_reply: tid=0 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 client.16956 shutdown 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 msgr2=0x55f765ce6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765ce6bf0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c0259e0 tx=0x7f6d5c012d40 comp rx=0 tx=0).stop 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 shutdown_connections 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55f765d5d5a0 0x55f765d5f9c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f765ce2780 0x55f765ce6bf0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f765cd4b40 0x55f765ce64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d4d7fa640 0 client.16956 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 --2- 172.21.15.37:0/3328710251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f765cd35d0 0x55f765ceb180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 >> 172.21.15.37:0/3328710251 conn(0x55f765c9c310 msgr2=0x55f765cde4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 shutdown_connections 2024-09-19T06:47:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 1 -- 172.21.15.37:0/3328710251 wait complete. 2024-09-19T06:47:06.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.997+0000 7f6d6de5a740 20 client.16956 trim_cache size 0 max 16384 2024-09-19T06:47:06.006 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:47:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d500a3430 msgr2=0x7f6d500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d500a3430 0x7f6d500a58f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c020000 tx=0x7f6d5c01d000 comp rx=0 tx=0).stop 2024-09-19T06:47:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 msgr2=0x7f6d64198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d64198990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006010 tx=0x7f6d540042f0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 shutdown_connections 2024-09-19T06:47:06.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6d64106f10 0x7f6d6419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6d500a3430 0x7f6d500a58f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6d64102790 0x7f6d64198990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 --2- 172.21.15.37:0/2256595639 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6d64101de0 0x7f6d64198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:06.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 >> 172.21.15.37:0/2256595639 conn(0x7f6d640fd8c0 msgr2=0x7f6d640fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:06.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 shutdown_connections 2024-09-19T06:47:06.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:05.998+0000 7f6d6ba81640 1 -- 172.21.15.37:0/2256595639 wait complete. 2024-09-19T06:47:06.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.71 MB/sec execute 93 sec latency 25800.549 ms 2024-09-19T06:47:06.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:06 smithi037 ceph-mon[31211]: pgmap v163: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 863 KiB/s wr, 58 op/s 2024-09-19T06:47:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:06 smithi151 ceph-mon[35236]: pgmap v163: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 863 KiB/s wr, 58 op/s 2024-09-19T06:47:06.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:06 smithi096 ceph-mon[35004]: pgmap v163: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 863 KiB/s wr, 58 op/s 2024-09-19T06:47:06.818 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:47:07.077 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:07.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.65 MB/sec execute 94 sec latency 26800.623 ms 2024-09-19T06:47:07.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 -- 172.21.15.37:0/3040453711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4101de0 msgr2=0x7efff41021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 --2- 172.21.15.37:0/3040453711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4101de0 0x7efff41021c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7effe8009900 tx=0x7effe802a410 comp rx=0 tx=0).stop 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 -- 172.21.15.37:0/3040453711 shutdown_connections 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 --2- 172.21.15.37:0/3040453711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff4106f10 0x7efff410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 --2- 172.21.15.37:0/3040453711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efff4102790 0x7efff41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 --2- 172.21.15.37:0/3040453711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4101de0 0x7efff41021c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 -- 172.21.15.37:0/3040453711 >> 172.21.15.37:0/3040453711 conn(0x7efff40fd8c0 msgr2=0x7efff40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 -- 172.21.15.37:0/3040453711 shutdown_connections 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.587+0000 7efffb33d640 1 -- 172.21.15.37:0/3040453711 wait complete. 2024-09-19T06:47:07.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 Processor -- start 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 -- start start 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efff4101de0 0x7efff40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff4102790 0x7efff4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4106f10 0x7efff4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efff4111f70 con 0x7efff4106f10 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efff4111df0 con 0x7efff4101de0 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efffb33d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efff41120f0 con 0x7efff4102790 2024-09-19T06:47:07.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4106f10 0x7efff4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4106f10 0x7efff4074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45892/0 (socket says 172.21.15.37:45892) 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 -- 172.21.15.37:0/1304770052 learned_addr learned my addr 172.21.15.37:0/1304770052 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff90d9640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efff4101de0 0x7efff40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff88d8640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff4102790 0x7efff4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 -- 172.21.15.37:0/1304770052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff4102790 msgr2=0x7efff4074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff4102790 0x7efff4074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 -- 172.21.15.37:0/1304770052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efff4101de0 msgr2=0x7efff40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efff4101de0 0x7efff40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff98da640 1 -- 172.21.15.37:0/1304770052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efff41a68f0 con 0x7efff4106f10 2024-09-19T06:47:07.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff88d8640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff4102790 0x7efff4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.588+0000 7efff90d9640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efff4101de0 0x7efff40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.589+0000 7efff98da640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4106f10 0x7efff4074f70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7effec00cad0 tx=0x7effec00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.589+0000 7effe67fc640 1 -- 172.21.15.37:0/1304770052 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00ae20 con 0x7efff4106f10 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.589+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efff41a6be0 con 0x7efff4106f10 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.589+0000 7effe67fc640 1 -- 172.21.15.37:0/1304770052 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effec005020 con 0x7efff4106f10 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.589+0000 7effe67fc640 1 -- 172.21.15.37:0/1304770052 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00f6b0 con 0x7efff4106f10 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.589+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efff41a7170 con 0x7efff4106f10 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.591+0000 7effe67fc640 1 -- 172.21.15.37:0/1304770052 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7effec0051d0 con 0x7efff4106f10 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 Processor -- start 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 -- start start 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f30f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f31c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555b7f2752f0 con 0x555b7f30f5d0 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555b7f21ccf0 con 0x555b7f31a370 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efffd716740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b7f2712b0 con 0x555b7f310b40 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45902/0 (socket says 172.21.15.37:45902) 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 -- 172.21.15.37:0/2124982054 learned_addr learned my addr 172.21.15.37:0/2124982054 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff88d8640 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f30f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff88d8640 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f31c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7effe67fc640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effe00a33b0 0x7effe00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 -- 172.21.15.37:0/2124982054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 msgr2=0x555b7f30f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f30f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 -- 172.21.15.37:0/2124982054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 msgr2=0x555b7f31c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f31c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff90d9640 1 -- 172.21.15.37:0/2124982054 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7f2013a0 con 0x555b7f30f5d0 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.592+0000 7efff88d8640 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f31c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.593+0000 7effe67fc640 1 -- 172.21.15.37:0/1304770052 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7effec0d00b0 con 0x7efff4106f10 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.593+0000 7efff90d9640 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30fa10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7effe8009900 tx=0x7effe80316e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=107cda43a7c7f493 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.593+0000 7efff98da640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effe00a33b0 0x7effe00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.593+0000 7effdf7fe640 1 -- 172.21.15.37:0/2124982054 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe803f070 con 0x555b7f30f5d0 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.593+0000 7effdf7fe640 1 -- 172.21.15.37:0/2124982054 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe80035f0 con 0x555b7f30f5d0 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.593+0000 7effdf7fe640 1 -- 172.21.15.37:0/2124982054 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8035070 con 0x555b7f30f5d0 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- 172.21.15.37:0/2124982054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 msgr2=0x555b7f30fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30fa10 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7effe8009900 tx=0x7effe80316e0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efff98da640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effe00a33b0 0x7effe00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7effec0cd8b0 tx=0x7effec005960 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- 172.21.15.37:0/2124982054 shutdown_connections 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f31c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30fa10 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 --2- 172.21.15.37:0/2124982054 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f30f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- 172.21.15.37:0/2124982054 >> 172.21.15.37:0/2124982054 conn(0x555b7f2d8310 msgr2=0x555b7f30bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- 172.21.15.37:0/2124982054 shutdown_connections 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- 172.21.15.37:0/2124982054 wait complete. 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- start start 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 Processor -- start 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f30f5d0 0x555b7f319d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f310b40 0x555b7f317020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f317610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555b7f31d010 con 0x555b7f310b40 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555b7f230060 con 0x555b7f31a370 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.594+0000 7efffd716740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555b7f31d190 con 0x555b7f30f5d0 2024-09-19T06:47:07.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efffd716740 10 client.0 _ll_register_callbacks cb 0x555b7ef838c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f310b40 0x555b7f317020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f310b40 0x555b7f317020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45916/0 (socket says 172.21.15.37:45916) 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 -- 172.21.15.37:0/3153597851 learned_addr learned my addr 172.21.15.37:0/3153597851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff88d8640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f30f5d0 0x555b7f319d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff88d8640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f317610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 -- 172.21.15.37:0/3153597851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f30f5d0 msgr2=0x555b7f319d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f30f5d0 0x555b7f319d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 -- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 msgr2=0x555b7f317610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f317610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 -- 172.21.15.37:0/3153597851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7efe63c0 con 0x555b7f310b40 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff88d8640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f30f5d0 0x555b7f319d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efff90d9640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f310b40 0x555b7f317020 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7effe8000c00 tx=0x7effe8003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219256c496b44809 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:07.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7effdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7effdcff9640 1 -- 172.21.15.37:0/3153597851 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8035070 con 0x555b7f310b40 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7effdcff9640 1 -- 172.21.15.37:0/3153597851 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7effe803b430 con 0x555b7f310b40 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.595+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effe80035f0 con 0x555b7f310b40 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7effdcff9640 1 -- 172.21.15.37:0/3153597851 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7effe8055050 con 0x555b7f310b40 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 10 client.16968 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b7f328a50 con 0x555b7f310b40 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7effdcff9640 1 -- 172.21.15.37:0/3153597851 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effe80035f0 con 0x555b7f310b40 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7effdcff9640 1 -- 172.21.15.37:0/3153597851 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7effe8043730 con 0x555b7f310b40 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 10 client.16968 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 10 client.16968 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 20 client.16968 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f329da0 0x555b7f32c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 4 client.16968 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b7f282860 con 0x555b7f329da0 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.596+0000 7efff88d8640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f329da0 0x555b7f32c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.597+0000 7efff88d8640 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f329da0 0x555b7f32c190 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a1dcbbadbddabbdf server_cookie=c4fce88adc1d7215 in_seq=0 out_seq=0 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.597+0000 7effdcff9640 10 client.16968 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.597+0000 7effdcff9640 1 -- 172.21.15.37:0/3153597851 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555b7f282860 con 0x555b7f329da0 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.597+0000 7effdcff9640 10 client.16968 handle_command_reply: tid=0 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.597+0000 7efffd716740 1 client.16968 shutdown 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f310b40 msgr2=0x555b7f317020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f310b40 0x555b7f317020 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7effe8000c00 tx=0x7effe8003d80 comp rx=0 tx=0).stop 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 shutdown_connections 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f329da0 0x555b7f32c190 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f317610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f310b40 0x555b7f317020 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 --2- 172.21.15.37:0/3153597851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f30f5d0 0x555b7f319d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7effdcff9640 0 client.16968 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:07.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 >> 172.21.15.37:0/3153597851 conn(0x555b7f2d8310 msgr2=0x555b7f311930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:07.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 shutdown_connections 2024-09-19T06:47:07.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 1 -- 172.21.15.37:0/3153597851 wait complete. 2024-09-19T06:47:07.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.598+0000 7efffd716740 20 client.16968 trim_cache size 0 max 16384 2024-09-19T06:47:07.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 Processor -- start 2024-09-19T06:47:07.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 -- start start 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f3262e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f331730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efff41b2400 con 0x555b7f30f5d0 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efff41afcd0 con 0x555b7f31a370 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.612+0000 7efffd716740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efff41adb20 con 0x555b7f310b40 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45924/0 (socket says 172.21.15.37:45924) 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 -- 172.21.15.37:0/549751747 learned_addr learned my addr 172.21.15.37:0/549751747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 -- 172.21.15.37:0/549751747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 msgr2=0x555b7f3262e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 --2- 172.21.15.37:0/549751747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f3262e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 -- 172.21.15.37:0/549751747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 msgr2=0x555b7f331730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 --2- 172.21.15.37:0/549751747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f331730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 -- 172.21.15.37:0/549751747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b7f30c960 con 0x555b7f30f5d0 2024-09-19T06:47:07.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7efff88d8640 1 --2- 172.21.15.37:0/549751747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30be10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efff000df90 tx=0x7efff000d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9044e73ef64274bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:07.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7effdeffd640 1 -- 172.21.15.37:0/549751747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0020070 con 0x555b7f30f5d0 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7effdeffd640 1 -- 172.21.15.37:0/549751747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff001cab0 con 0x555b7f30f5d0 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.613+0000 7effdeffd640 1 -- 172.21.15.37:0/549751747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff001b040 con 0x555b7f30f5d0 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 -- 172.21.15.37:0/549751747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 msgr2=0x555b7f30be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 --2- 172.21.15.37:0/549751747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30be10 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7efff000df90 tx=0x7efff000d360 comp rx=0 tx=0).stop 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 -- 172.21.15.37:0/549751747 shutdown_connections 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 --2- 172.21.15.37:0/549751747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f31a370 0x555b7f331730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 --2- 172.21.15.37:0/549751747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f3262e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 --2- 172.21.15.37:0/549751747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f30f5d0 0x555b7f30be10 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 -- 172.21.15.37:0/549751747 >> 172.21.15.37:0/549751747 conn(0x555b7f2d8310 msgr2=0x555b7f30b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 -- 172.21.15.37:0/549751747 shutdown_connections 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 -- 172.21.15.37:0/549751747 wait complete. 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.614+0000 7efffd716740 1 -- start start 2024-09-19T06:47:07.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 1 Processor -- start 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f30f5d0 0x555b7f313f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f314440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f31a370 0x555b7f314980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efff41ae190 con 0x555b7f31a370 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efff41af600 con 0x555b7f30f5d0 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efff41ae860 con 0x555b7f310b40 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efffd716740 10 client.0 _ll_register_callbacks cb 0x7effe80315b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efff88d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f31a370 0x555b7f314980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efff88d8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f31a370 0x555b7f314980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45940/0 (socket says 172.21.15.37:45940) 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efff88d8640 1 -- 172.21.15.37:0/3264594401 learned_addr learned my addr 172.21.15.37:0/3264594401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efff88d8640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f30f5d0 0x555b7f313f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.615+0000 7efff90d9640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f314440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efff88d8640 1 -- 172.21.15.37:0/3264594401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 msgr2=0x555b7f314440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efff88d8640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f314440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efff88d8640 1 -- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f30f5d0 msgr2=0x555b7f313f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efff88d8640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f30f5d0 0x555b7f313f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efff88d8640 1 -- 172.21.15.37:0/3264594401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b7f393700 con 0x555b7f31a370 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efff90d9640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f314440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efff88d8640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f31a370 0x555b7f314980 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff00107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8139ed0eff0f836f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7effdcff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7effdcff9640 1 -- 172.21.15.37:0/3264594401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efff0020040 con 0x555b7f31a370 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7effdcff9640 1 -- 172.21.15.37:0/3264594401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7efff001c5d0 con 0x555b7f31a370 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efff001cab0 con 0x555b7f31a370 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7effdcff9640 1 -- 172.21.15.37:0/3264594401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7efff001c7d0 con 0x555b7f31a370 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efffd716740 10 client.16974 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:07.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x555b7f3948e0 con 0x555b7f31a370 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.616+0000 7effdcff9640 1 -- 172.21.15.37:0/3264594401 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efff001cab0 con 0x555b7f31a370 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7effdcff9640 1 -- 172.21.15.37:0/3264594401 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7efff0032060 con 0x555b7f31a370 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efffd716740 10 client.16974 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efffd716740 10 client.16974 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efffd716740 20 client.16974 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efffd716740 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f394ef0 0x555b7f3972e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efffd716740 4 client.16974 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b7f27cd70 con 0x555b7f394ef0 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efff90d9640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f394ef0 0x555b7f3972e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efff90d9640 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f394ef0 0x555b7f3972e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f7e4ad03aca6df92 server_cookie=6f75baad7690793a in_seq=0 out_seq=0 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7effdcff9640 10 client.16974 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7effdcff9640 1 -- 172.21.15.37:0/3264594401 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b7f27cd70 con 0x555b7f394ef0 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7effdcff9640 10 client.16974 handle_command_reply: tid=0 2024-09-19T06:47:07.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.617+0000 7efffd716740 1 client.16974 shutdown 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f31a370 msgr2=0x555b7f314980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f31a370 0x555b7f314980 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7efff0006010 tx=0x7efff00107b0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 shutdown_connections 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x555b7f394ef0 0x555b7f3972e0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555b7f31a370 0x555b7f314980 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555b7f310b40 0x555b7f314440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 --2- 172.21.15.37:0/3264594401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555b7f30f5d0 0x555b7f313f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 >> 172.21.15.37:0/3264594401 conn(0x555b7f2d8310 msgr2=0x555b7f30a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7effdcff9640 0 client.16974 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 shutdown_connections 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 1 -- 172.21.15.37:0/3264594401 wait complete. 2024-09-19T06:47:07.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffd716740 20 client.16974 trim_cache size 0 max 16384 2024-09-19T06:47:07.624 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:47:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effe00a33b0 msgr2=0x7effe00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effe00a33b0 0x7effe00a5870 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7effec0cd8b0 tx=0x7effec005960 comp rx=0 tx=0).stop 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4106f10 msgr2=0x7efff4074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4106f10 0x7efff4074f70 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7effec00cad0 tx=0x7effec00cf90 comp rx=0 tx=0).stop 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 shutdown_connections 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effe00a33b0 0x7effe00a5870 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efff4106f10 0x7efff4074f70 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efff4102790 0x7efff4074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 --2- 172.21.15.37:0/1304770052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efff4101de0 0x7efff40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.618+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 >> 172.21.15.37:0/1304770052 conn(0x7efff40fd8c0 msgr2=0x7efff40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.619+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 shutdown_connections 2024-09-19T06:47:07.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:07.619+0000 7efffb33d640 1 -- 172.21.15.37:0/1304770052 wait complete. 2024-09-19T06:47:08.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.59 MB/sec execute 95 sec latency 27800.669 ms 2024-09-19T06:47:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:08 smithi037 ceph-mon[31211]: pgmap v164: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 863 KiB/s wr, 58 op/s 2024-09-19T06:47:08.354 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:47:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:08 smithi151 ceph-mon[35236]: pgmap v164: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 863 KiB/s wr, 58 op/s 2024-09-19T06:47:08.432 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:08.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:08 smithi096 ceph-mon[35004]: pgmap v164: 97 pgs: 97 active+clean; 29 MiB data, 965 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 863 KiB/s wr, 58 op/s 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/942470803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc4103760 msgr2=0x7f8fc4107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/942470803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc4103760 0x7f8fc4107ba0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f8fb8009930 tx=0x7f8fb802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/942470803 shutdown_connections 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/942470803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc41082e0 0x7f8fc410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/942470803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc4103760 0x7f8fc4107ba0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/942470803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4102db0 0x7f8fc4103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/942470803 >> 172.21.15.37:0/942470803 conn(0x7f8fc40fc7d0 msgr2=0x7f8fc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/942470803 shutdown_connections 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/942470803 wait complete. 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.867+0000 7f8fc9ae2640 1 Processor -- start 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc9ae2640 1 -- start start 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc9ae2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4102db0 0x7f8fc4100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:08.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc9ae2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4103760 0x7f8fc4100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc9ae2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc41082e0 0x7f8fc4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc9ae2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4112120 con 0x7f8fc41082e0 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc9ae2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc4111fa0 con 0x7f8fc4103760 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc9ae2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fc41122a0 con 0x7f8fc4102db0 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc41082e0 0x7f8fc4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc41082e0 0x7f8fc4101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45964/0 (socket says 172.21.15.37:45964) 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 -- 172.21.15.37:0/390309835 learned_addr learned my addr 172.21.15.37:0/390309835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc37fe640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4102db0 0x7f8fc4100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc2ffd640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4103760 0x7f8fc4100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 -- 172.21.15.37:0/390309835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4102db0 msgr2=0x7f8fc4100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4102db0 0x7f8fc4100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 -- 172.21.15.37:0/390309835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4103760 msgr2=0x7f8fc4100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:08.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4103760 0x7f8fc4100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.868+0000 7f8fc3fff640 1 -- 172.21.15.37:0/390309835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fc41a6ae0 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.869+0000 7f8fc3fff640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc41082e0 0x7f8fc4101490 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b970 tx=0x7f8fbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.869+0000 7f8fc0ff9640 1 -- 172.21.15.37:0/390309835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc019070 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.869+0000 7f8fc0ff9640 1 -- 172.21.15.37:0/390309835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fbc00cb40 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.869+0000 7f8fc0ff9640 1 -- 172.21.15.37:0/390309835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc007650 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.869+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fc41a6dd0 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.869+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fc41a72e0 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.870+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fc4103190 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.871+0000 7f8fc0ff9640 1 -- 172.21.15.37:0/390309835 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8fbc01c070 con 0x7f8fc41082e0 2024-09-19T06:47:08.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.872+0000 7f8fc0ff9640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fac0a3430 0x7f8fac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:08.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.872+0000 7f8fc37fe640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fac0a3430 0x7f8fac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:08.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.872+0000 7f8fc0ff9640 1 -- 172.21.15.37:0/390309835 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f8fbc0cf640 con 0x7f8fc41082e0 2024-09-19T06:47:08.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.872+0000 7f8fc37fe640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fac0a3430 0x7f8fac0a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40058c0 tx=0x7f8fb400a4c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:08.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:08.875+0000 7f8fc0ff9640 1 -- 172.21.15.37:0/390309835 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8fbc09c0d0 con 0x7f8fc41082e0 2024-09-19T06:47:08.968 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:09.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.092+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8fc4101f40 con 0x7f8fc41082e0 2024-09-19T06:47:09.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.093+0000 7f8fc0ff9640 1 -- 172.21.15.37:0/390309835 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f8fc4101f40 con 0x7f8fc41082e0 2024-09-19T06:47:09.094 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:47:09.094 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-19T06:45:42:465959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:41.617479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:47:09.094 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-19T06:47:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fac0a3430 msgr2=0x7f8fac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fac0a3430 0x7f8fac0a58f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f8fb40058c0 tx=0x7f8fb400a4c0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc41082e0 msgr2=0x7f8fc4101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc41082e0 0x7f8fc4101490 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc00b970 tx=0x7f8fbc00be30 comp rx=0 tx=0).stop 2024-09-19T06:47:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 shutdown_connections 2024-09-19T06:47:09.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fac0a3430 0x7f8fac0a58f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fc41082e0 0x7f8fc4101490 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fc4103760 0x7f8fc4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 --2- 172.21.15.37:0/390309835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fc4102db0 0x7f8fc4100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 >> 172.21.15.37:0/390309835 conn(0x7f8fc40fc7d0 msgr2=0x7f8fc4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 shutdown_connections 2024-09-19T06:47:09.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:09.095+0000 7f8fc9ae2640 1 -- 172.21.15.37:0/390309835 wait complete. 2024-09-19T06:47:09.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.53 MB/sec execute 96 sec latency 28800.757 ms 2024-09-19T06:47:09.916 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id d57a1d24 --release --await' 2024-09-19T06:47:09.917 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id d57a1d24 --release --await 2024-09-19T06:47:10.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.078+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/424279691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074620 msgr2=0x7f7d64074a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.078+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/424279691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074620 0x7f7d64074a00 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d6002a480 comp rx=0 tx=0).stop 2024-09-19T06:47:10.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.078+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/424279691 shutdown_connections 2024-09-19T06:47:10.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.078+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/424279691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6410e3d0 0x7f7d64110820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.078+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/424279691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d64074f40 0x7f7d6410de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.078+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/424279691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074620 0x7f7d64074a00 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.078+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/424279691 >> 172.21.15.37:0/424279691 conn(0x7f7d640fd8f0 msgr2=0x7f7d640ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:10.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.079+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/424279691 shutdown_connections 2024-09-19T06:47:10.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.079+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/424279691 wait complete. 2024-09-19T06:47:10.081 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.079+0000 7f7d6bdf9640 1 Processor -- start 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.079+0000 7f7d6bdf9640 1 -- start start 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d6bdf9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d64074620 0x7f7d6419a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d6bdf9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074f40 0x7f7d6419ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d6bdf9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6410e3d0 0x7f7d6419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d6bdf9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64114180 con 0x7f7d64074f40 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d6bdf9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64114000 con 0x7f7d6410e3d0 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d6bdf9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d64114300 con 0x7f7d64074620 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074f40 0x7f7d6419ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074f40 0x7f7d6419ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52856/0 (socket says 172.21.15.37:52856) 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 -- 172.21.15.37:0/3953168492 learned_addr learned my addr 172.21.15.37:0/3953168492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d6a396640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6410e3d0 0x7f7d6419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.082 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69b95640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d64074620 0x7f7d6419a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 -- 172.21.15.37:0/3953168492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d64074620 msgr2=0x7f7d6419a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d64074620 0x7f7d6419a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 -- 172.21.15.37:0/3953168492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6410e3d0 msgr2=0x7f7d6419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6410e3d0 0x7f7d6419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.080+0000 7f7d69394640 1 -- 172.21.15.37:0/3953168492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d6419f670 con 0x7f7d64074f40 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.081+0000 7f7d69394640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074f40 0x7f7d6419ac00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c970 tx=0x7f7d5400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.081+0000 7f7d5affd640 1 -- 172.21.15.37:0/3953168492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d54007bb0 con 0x7f7d64074f40 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.081+0000 7f7d5affd640 1 -- 172.21.15.37:0/3953168492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d54007d50 con 0x7f7d64074f40 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.081+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d6419f960 con 0x7f7d64074f40 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.081+0000 7f7d5affd640 1 -- 172.21.15.37:0/3953168492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400f450 con 0x7f7d64074f40 2024-09-19T06:47:10.083 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.081+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d641a7240 con 0x7f7d64074f40 2024-09-19T06:47:10.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.083+0000 7f7d5affd640 1 -- 172.21.15.37:0/3953168492 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7d54005020 con 0x7f7d64074f40 2024-09-19T06:47:10.084 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.083+0000 7f7d6e1d2740 1 Processor -- start 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6e1d2740 1 -- start start 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9e2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e25d0 0x56521b9e2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9ef760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56521b9482f0 con 0x56521b9ed370 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56521b8efcf0 con 0x56521b9e3b40 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56521b9442b0 con 0x56521b9e25d0 2024-09-19T06:47:10.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d69394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9ef760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d69394640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9ef760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52870/0 (socket says 172.21.15.37:52870) 2024-09-19T06:47:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d69394640 1 -- 172.21.15.37:0/462878233 learned_addr learned my addr 172.21.15.37:0/462878233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d69b95640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e25d0 0x56521b9e2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d6a396640 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9e2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.084+0000 7f7d5affd640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d500a3430 0x7f7d500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 -- 172.21.15.37:0/462878233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e25d0 msgr2=0x56521b9e2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e25d0 0x56521b9e2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 -- 172.21.15.37:0/462878233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 msgr2=0x56521b9ef760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9ef760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 -- 172.21.15.37:0/462878233 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521b8d43a0 con 0x56521b9e3b40 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d69b95640 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e25d0 0x56521b9e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d500a3430 0x7f7d500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d5affd640 1 -- 172.21.15.37:0/3953168492 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7d540cfae0 con 0x7f7d64074f40 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9e2090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0103f0 tx=0x7f7d5c0108b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33a1ad292858c11c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d4ffff640 1 -- 172.21.15.37:0/462878233 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c024070 con 0x56521b9e3b40 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d6a396640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d500a3430 0x7f7d500a58f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c021000 tx=0x7f7d5c01e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d4ffff640 1 -- 172.21.15.37:0/462878233 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c01e960 con 0x56521b9e3b40 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.085+0000 7f7d4ffff640 1 -- 172.21.15.37:0/462878233 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c01c790 con 0x56521b9e3b40 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/462878233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 msgr2=0x56521b9e2090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9e2090 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c0103f0 tx=0x7f7d5c0108b0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/462878233 shutdown_connections 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9ef760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e25d0 0x56521b9e2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/462878233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9e2090 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/462878233 >> 172.21.15.37:0/462878233 conn(0x56521b9ab310 msgr2=0x56521b9dec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/462878233 shutdown_connections 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/462878233 wait complete. 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.086+0000 7f7d6e1d2740 1 -- start start 2024-09-19T06:47:10.089 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 1 Processor -- start 2024-09-19T06:47:10.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9ecd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9ea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9ed370 0x56521b9ea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56521b9f0070 con 0x56521b9e25d0 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56521b903060 con 0x56521b9ed370 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56521b9f01f0 con 0x56521b9e3b40 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6e1d2740 10 client.0 _ll_register_callbacks cb 0x56521b6568c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d69b95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9ecd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d69b95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9ecd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52884/0 (socket says 172.21.15.37:52884) 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d69b95640 1 -- 172.21.15.37:0/4284004525 learned_addr learned my addr 172.21.15.37:0/4284004525 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d69394640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.087+0000 7f7d6a396640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9ed370 0x56521b9ea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d69b95640 1 -- 172.21.15.37:0/4284004525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 msgr2=0x56521b9ea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d69b95640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9ea000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d69b95640 1 -- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9ed370 msgr2=0x56521b9ea5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d69b95640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9ed370 0x56521b9ea5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d69b95640 1 -- 172.21.15.37:0/4284004525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521b6b93c0 con 0x56521b9e25d0 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d6a396640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9ed370 0x56521b9ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d69b95640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9ecd50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d60031ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa26699df72a3466 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/4284004525 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60035040 con 0x56521b9e25d0 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5c01e960 con 0x56521b9e25d0 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/4284004525 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7d600042c0 con 0x56521b9e25d0 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/4284004525 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d600044c0 con 0x56521b9e25d0 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d6e1d2740 10 client.16992 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:10.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.088+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56521b9fbb40 con 0x56521b9e25d0 2024-09-19T06:47:10.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/4284004525 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c01e960 con 0x56521b9e25d0 2024-09-19T06:47:10.093 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/4284004525 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7d600035f0 con 0x56521b9e25d0 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d6e1d2740 10 client.16992 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d6e1d2740 10 client.16992 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d6e1d2740 20 client.16992 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521b9fcf50 0x56521b9ff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d6e1d2740 4 client.16992 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.092+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56521b955860 con 0x56521b9fcf50 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.093+0000 7f7d69394640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521b9fcf50 0x56521b9ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.093+0000 7f7d69394640 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521b9fcf50 0x56521b9ff340 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc8468673fb6f993 server_cookie=a2850ab00eefdac5 in_seq=0 out_seq=0 2024-09-19T06:47:10.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.093+0000 7f7d4d7fa640 10 client.16992 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:10.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/4284004525 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56521b955860 con 0x56521b9fcf50 2024-09-19T06:47:10.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d4d7fa640 10 client.16992 handle_command_reply: tid=0 2024-09-19T06:47:10.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 client.16992 shutdown 2024-09-19T06:47:10.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 msgr2=0x56521b9ecd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9ecd50 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7d60009900 tx=0x7f7d60031ad0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 shutdown_connections 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521b9fcf50 0x56521b9ff340 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9ed370 0x56521b9ea5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d4d7fa640 0 client.16992 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9ea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/4284004525 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9ecd50 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 >> 172.21.15.37:0/4284004525 conn(0x56521b9ab310 msgr2=0x56521b9e4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 shutdown_connections 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/4284004525 wait complete. 2024-09-19T06:47:10.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.094+0000 7f7d6e1d2740 20 client.16992 trim_cache size 0 max 16384 2024-09-19T06:47:10.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 Processor -- start 2024-09-19T06:47:10.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 -- start start 2024-09-19T06:47:10.109 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e25d0 0x56521b9e98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9e5120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9e5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641b2480 con 0x56521b9ed370 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641afd50 con 0x56521b9e25d0 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641adba0 con 0x56521b9e3b40 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9e5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9e5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52898/0 (socket says 172.21.15.37:52898) 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 -- 172.21.15.37:0/184569214 learned_addr learned my addr 172.21.15.37:0/184569214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d69394640 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9e5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d69b95640 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e25d0 0x56521b9e98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 -- 172.21.15.37:0/184569214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 msgr2=0x56521b9e5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9e5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.110 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 -- 172.21.15.37:0/184569214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e25d0 msgr2=0x56521b9e98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e25d0 0x56521b9e98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d6a396640 1 -- 172.21.15.37:0/184569214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56521b9e8fb0 con 0x56521b9ed370 2024-09-19T06:47:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.108+0000 7f7d69b95640 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e25d0 0x56521b9e98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:10.111 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.109+0000 7f7d6a396640 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9e5660 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c003fe0 tx=0x7f7d5c009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1f4d47639a4b85 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.109+0000 7f7d4f7fe640 1 -- 172.21.15.37:0/184569214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c024040 con 0x56521b9ed370 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.109+0000 7f7d4f7fe640 1 -- 172.21.15.37:0/184569214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c0139a0 con 0x56521b9ed370 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.109+0000 7f7d4f7fe640 1 -- 172.21.15.37:0/184569214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c01e880 con 0x56521b9ed370 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/184569214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 msgr2=0x56521b9e5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9e5660 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c003fe0 tx=0x7f7d5c009a00 comp rx=0 tx=0).stop 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/184569214 shutdown_connections 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9ed370 0x56521b9e5660 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9e3b40 0x56521b9e5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/184569214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e25d0 0x56521b9e98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/184569214 >> 172.21.15.37:0/184569214 conn(0x56521b9ab310 msgr2=0x56521b9ddd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/184569214 shutdown_connections 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/184569214 wait complete. 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 -- start start 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.110+0000 7f7d6e1d2740 1 Processor -- start 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9f1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9f1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6e1d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9ed370 0x56521b9f1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ae210 con 0x56521b9e25d0 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641af680 con 0x56521b9e3b40 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6e1d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d641ae8e0 con 0x56521b9ed370 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9f1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52906/0 (socket says 172.21.15.37:52906) 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 -- 172.21.15.37:0/1198621779 learned_addr learned my addr 172.21.15.37:0/1198621779 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:10.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6e1d2740 10 client.0 _ll_register_callbacks cb 0x7f7d60009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69394640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9f1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6a396640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9ed370 0x56521b9f1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 -- 172.21.15.37:0/1198621779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9ed370 msgr2=0x56521b9f1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9ed370 0x56521b9f1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 -- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 msgr2=0x56521b9f1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9f1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d69b95640 1 -- 172.21.15.37:0/1198621779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56521ba66da0 con 0x56521b9e25d0 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.111+0000 7f7d6a396640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9ed370 0x56521b9f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d69b95640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9f1430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7d60006010 tx=0x7f7d60003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6928f2dc5a5bc5b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:10.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d4d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:10.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/1198621779 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d60035070 con 0x56521b9e25d0 2024-09-19T06:47:10.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d5c0139a0 con 0x56521b9e25d0 2024-09-19T06:47:10.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/1198621779 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7d600046c0 con 0x56521b9e25d0 2024-09-19T06:47:10.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/1198621779 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d6005e050 con 0x56521b9e25d0 2024-09-19T06:47:10.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d6e1d2740 10 client.16998 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:10.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56521ba680c0 con 0x56521b9e25d0 2024-09-19T06:47:10.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/1198621779 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d5c0139a0 con 0x56521b9e25d0 2024-09-19T06:47:10.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.112+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/1198621779 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7d60045590 con 0x56521b9e25d0 2024-09-19T06:47:10.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d6e1d2740 10 client.16998 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:10.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d6e1d2740 10 client.16998 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d6e1d2740 20 client.16998 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521ba68810 0x56521ba6ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d6e1d2740 4 client.16998 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56521b94fd70 con 0x56521ba68810 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d69394640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521ba68810 0x56521ba6ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.113+0000 7f7d4d7fa640 10 client.16998 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d69394640 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521ba68810 0x56521ba6ac70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5f93fe485c11cfd server_cookie=c7700040f684b2da in_seq=0 out_seq=0 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d4d7fa640 1 -- 172.21.15.37:0/1198621779 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56521b94fd70 con 0x56521ba68810 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d4d7fa640 10 client.16998 handle_command_reply: tid=0 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 client.16998 shutdown 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 msgr2=0x56521b9f1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9f1430 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7d60006010 tx=0x7f7d60003f10 comp rx=0 tx=0).stop 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 shutdown_connections 2024-09-19T06:47:10.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56521ba68810 0x56521ba6ac70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d4d7fa640 0 client.16998 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56521b9ed370 0x56521b9f1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56521b9e3b40 0x56521b9f1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 --2- 172.21.15.37:0/1198621779 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56521b9e25d0 0x56521b9f1430 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.114+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 >> 172.21.15.37:0/1198621779 conn(0x56521b9ab310 msgr2=0x56521b9eb860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.115+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 shutdown_connections 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.115+0000 7f7d6e1d2740 1 -- 172.21.15.37:0/1198621779 wait complete. 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.115+0000 7f7d6e1d2740 20 client.16998 trim_cache size 0 max 16384 2024-09-19T06:47:10.119 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:47:10.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d500a3430 msgr2=0x7f7d500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d500a3430 0x7f7d500a58f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c021000 tx=0x7f7d5c01e000 comp rx=0 tx=0).stop 2024-09-19T06:47:10.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074f40 msgr2=0x7f7d6419ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:10.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074f40 0x7f7d6419ac00 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400c970 tx=0x7f7d5400ce30 comp rx=0 tx=0).stop 2024-09-19T06:47:10.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 shutdown_connections 2024-09-19T06:47:10.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d6410e3d0 0x7f7d6419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d500a3430 0x7f7d500a58f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d64074f40 0x7f7d6419ac00 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 --2- 172.21.15.37:0/3953168492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d64074620 0x7f7d6419a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:10.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 >> 172.21.15.37:0/3953168492 conn(0x7f7d640fd8f0 msgr2=0x7f7d640fe5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:10.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 shutdown_connections 2024-09-19T06:47:10.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:10.116+0000 7f7d6bdf9640 1 -- 172.21.15.37:0/3953168492 wait complete. 2024-09-19T06:47:10.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.47 MB/sec execute 97 sec latency 29801.231 ms 2024-09-19T06:47:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:10 smithi037 ceph-mon[31211]: pgmap v165: 97 pgs: 97 active+clean; 29 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 871 KiB/s wr, 61 op/s 2024-09-19T06:47:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:10 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/390309835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:10 smithi151 ceph-mon[35236]: pgmap v165: 97 pgs: 97 active+clean; 29 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 871 KiB/s wr, 61 op/s 2024-09-19T06:47:10.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:10 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/390309835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:10 smithi096 ceph-mon[35004]: pgmap v165: 97 pgs: 97 active+clean; 29 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 871 KiB/s wr, 61 op/s 2024-09-19T06:47:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:10 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/390309835' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:10.947 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:11.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 323773 5.42 MB/sec execute 98 sec latency 30801.278 ms 2024-09-19T06:47:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 -- 172.21.15.37:0/2922393276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c075110 msgr2=0x7f7d3c10dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/2922393276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c075110 0x7f7d3c10dce0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7d30009900 tx=0x7f7d3002a480 comp rx=0 tx=0).stop 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 -- 172.21.15.37:0/2922393276 shutdown_connections 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/2922393276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3c10e300 0x7f7d3c110790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/2922393276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c075110 0x7f7d3c10dce0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/2922393276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d3c0747f0 0x7f7d3c074bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 -- 172.21.15.37:0/2922393276 >> 172.21.15.37:0/2922393276 conn(0x7f7d3c0fd700 msgr2=0x7f7d3c0ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 -- 172.21.15.37:0/2922393276 shutdown_connections 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.392+0000 7f7d3bfff640 1 -- 172.21.15.37:0/2922393276 wait complete. 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 Processor -- start 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 -- start start 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d3c0747f0 0x7f7d3c19a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3c075110 0x7f7d3c19aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c10e300 0x7f7d3c19ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1140d0 con 0x7f7d3c10e300 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c113f50 con 0x7f7d3c0747f0 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.393+0000 7f7d3bfff640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c114250 con 0x7f7d3c075110 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c10e300 0x7f7d3c19ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c10e300 0x7f7d3c19ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52920/0 (socket says 172.21.15.37:52920) 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/1199367117 learned_addr learned my addr 172.21.15.37:0/1199367117 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3affd640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d3c0747f0 0x7f7d3c19a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3c075110 0x7f7d3c19aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/1199367117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3c075110 msgr2=0x7f7d3c19aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3c075110 0x7f7d3c19aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/1199367117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d3c0747f0 msgr2=0x7f7d3c19a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d3c0747f0 0x7f7d3c19a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 -- 172.21.15.37:0/1199367117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d3c19f560 con 0x7f7d3c10e300 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3c075110 0x7f7d3c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3b7fe640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c10e300 0x7f7d3c19ee80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400cad0 tx=0x7f7d3400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d23fff640 1 -- 172.21.15.37:0/1199367117 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400ad60 con 0x7f7d3c10e300 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d23fff640 1 -- 172.21.15.37:0/1199367117 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d34005020 con 0x7f7d3c10e300 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d23fff640 1 -- 172.21.15.37:0/1199367117 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3400f6b0 con 0x7f7d3c10e300 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d3c19f850 con 0x7f7d3c10e300 2024-09-19T06:47:11.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.394+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d3c1a7130 con 0x7f7d3c10e300 2024-09-19T06:47:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d23fff640 1 -- 172.21.15.37:0/1199367117 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7d34005560 con 0x7f7d3c10e300 2024-09-19T06:47:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 Processor -- start 2024-09-19T06:47:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 -- start start 2024-09-19T06:47:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ec27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ceb7e1a620 con 0x55ceb7eb5b30 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ceb7dc1e50 con 0x55ceb7eb5210 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d439d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ceb7e165e0 con 0x55ceb7ec03f0 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebfeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52924/0 (socket says 172.21.15.37:52924) 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d3affd640 1 -- 172.21.15.37:0/2294138565 learned_addr learned my addr 172.21.15.37:0/2294138565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.397+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ec27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d23fff640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d240a3430 0x7f7d240a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3affd640 1 -- 172.21.15.37:0/2294138565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 msgr2=0x55ceb7ec27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3affd640 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ec27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3affd640 1 -- 172.21.15.37:0/2294138565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 msgr2=0x55ceb7eb55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3affd640 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3affd640 1 -- 172.21.15.37:0/2294138565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceb7db2d40 con 0x55ceb7eb5b30 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3b7fe640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d240a3430 0x7f7d240a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d3affd640 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebfeb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00ad00 tx=0x7f7d2c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2435ce7402a5ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d23fff640 1 -- 172.21.15.37:0/1199367117 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7d340cfc80 con 0x7f7d3c10e300 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d217fa640 1 -- 172.21.15.37:0/2294138565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c01d070 con 0x55ceb7eb5b30 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d217fa640 1 -- 172.21.15.37:0/2294138565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c018b20 con 0x55ceb7eb5b30 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.398+0000 7f7d217fa640 1 -- 172.21.15.37:0/2294138565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c010c20 con 0x55ceb7eb5b30 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 -- 172.21.15.37:0/2294138565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 msgr2=0x55ceb7ebfeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d3b7fe640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d240a3430 0x7f7d240a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7d3409bc60 tx=0x7f7d3406ac50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebfeb0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00ad00 tx=0x7f7d2c00f790 comp rx=0 tx=0).stop 2024-09-19T06:47:11.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 -- 172.21.15.37:0/2294138565 shutdown_connections 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ec27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebfeb0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2294138565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 -- 172.21.15.37:0/2294138565 >> 172.21.15.37:0/2294138565 conn(0x55ceb7eb17a0 msgr2=0x55ceb7eb1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 -- 172.21.15.37:0/2294138565 shutdown_connections 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 -- 172.21.15.37:0/2294138565 wait complete. 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 -- start start 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.399+0000 7f7d439d8740 1 Processor -- start 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5210 0x55ceb7ebd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ebdd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d439d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ceb7ec2fc0 con 0x55ceb7eb5210 2024-09-19T06:47:11.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d439d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ceb7da2f10 con 0x55ceb7ec03f0 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d439d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ceb7ec3140 con 0x55ceb7eb5b30 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d439d8740 10 client.0 _ll_register_callbacks cb 0x55ceb7dc6db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5210 0x55ceb7ebd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5210 0x55ceb7ebd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52938/0 (socket says 172.21.15.37:52938) 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 -- 172.21.15.37:0/2267024480 learned_addr learned my addr 172.21.15.37:0/2267024480 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3affd640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ebdd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 -- 172.21.15.37:0/2267024480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7eb5b30 msgr2=0x55ceb7ebd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 -- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7ec03f0 msgr2=0x55ceb7ebdd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ebdd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 -- 172.21.15.37:0/2267024480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceb7b29580 con 0x55ceb7eb5210 2024-09-19T06:47:11.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3affd640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5210 0x55ceb7ebd2d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7d30033000 tx=0x7f7d30009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5b0e01be7e27e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.400+0000 7f7d1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d1eff5640 1 -- 172.21.15.37:0/2267024480 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d30035040 con 0x55ceb7eb5210 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d1eff5640 1 -- 172.21.15.37:0/2267024480 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7d30005250 con 0x55ceb7eb5210 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2c018b20 con 0x55ceb7eb5210 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d1eff5640 1 -- 172.21.15.37:0/2267024480 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d30050050 con 0x55ceb7eb5210 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 10 client.17010 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ceb7bf3960 con 0x55ceb7eb5210 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d1eff5640 1 -- 172.21.15.37:0/2267024480 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c018b20 con 0x55ceb7eb5210 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d1eff5640 1 -- 172.21.15.37:0/2267024480 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7d3003a070 con 0x55ceb7eb5210 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 10 client.17010 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 10 client.17010 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 20 client.17010 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:11.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7ecff60 0x55ceb7ed23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 4 client.17010 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.401+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ceb7e27b90 con 0x55ceb7ecff60 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.402+0000 7f7d3affd640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7ecff60 0x55ceb7ed23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.402+0000 7f7d3affd640 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7ecff60 0x55ceb7ed23b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77b55d54e9313c9b server_cookie=56db72f9e74689f5 in_seq=0 out_seq=0 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.402+0000 7f7d1eff5640 10 client.17010 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d1eff5640 1 -- 172.21.15.37:0/2267024480 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ceb7e27b90 con 0x55ceb7ecff60 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d1eff5640 10 client.17010 handle_command_reply: tid=0 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 client.17010 shutdown 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5210 msgr2=0x55ceb7ebd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5210 0x55ceb7ebd2d0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7d30033000 tx=0x7f7d30009260 comp rx=0 tx=0).stop 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 shutdown_connections 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7ecff60 0x55ceb7ed23b0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ebdd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ebd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 --2- 172.21.15.37:0/2267024480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5210 0x55ceb7ebd2d0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 >> 172.21.15.37:0/2267024480 conn(0x55ceb7eb17a0 msgr2=0x55ceb7eb0900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d1eff5640 0 client.17010 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 shutdown_connections 2024-09-19T06:47:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 1 -- 172.21.15.37:0/2267024480 wait complete. 2024-09-19T06:47:11.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.403+0000 7f7d439d8740 20 client.17010 trim_cache size 0 max 16384 2024-09-19T06:47:11.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 Processor -- start 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 -- start start 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ed7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ed81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1b22f0 con 0x55ceb7eb5b30 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1afbc0 con 0x55ceb7eb5210 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d439d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1ada10 con 0x55ceb7ec03f0 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ed7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ed7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52954/0 (socket says 172.21.15.37:52954) 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.420+0000 7f7d3affd640 1 -- 172.21.15.37:0/168307827 learned_addr learned my addr 172.21.15.37:0/168307827 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ed81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3affd640 1 -- 172.21.15.37:0/168307827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 msgr2=0x55ceb7ed81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3affd640 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ed81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3affd640 1 -- 172.21.15.37:0/168307827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 msgr2=0x55ceb7eb7370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3affd640 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3affd640 1 -- 172.21.15.37:0/168307827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ceb7eb5960 con 0x55ceb7eb5b30 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d3affd640 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ed7c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c01b040 tx=0x7f7d2c00a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137084e2b286e836 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d20ff9640 1 -- 172.21.15.37:0/168307827 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c032070 con 0x55ceb7eb5b30 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d20ff9640 1 -- 172.21.15.37:0/168307827 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c00a730 con 0x55ceb7eb5b30 2024-09-19T06:47:11.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.421+0000 7f7d20ff9640 1 -- 172.21.15.37:0/168307827 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c01d040 con 0x55ceb7eb5b30 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 -- 172.21.15.37:0/168307827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 msgr2=0x55ceb7ed7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ed7c80 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c01b040 tx=0x7f7d2c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 -- 172.21.15.37:0/168307827 shutdown_connections 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7ed81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ed7c80 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 --2- 172.21.15.37:0/168307827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7eb7370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 -- 172.21.15.37:0/168307827 >> 172.21.15.37:0/168307827 conn(0x55ceb7eb17a0 msgr2=0x55ceb7eb6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 -- 172.21.15.37:0/168307827 shutdown_connections 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 -- 172.21.15.37:0/168307827 wait complete. 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 -- start start 2024-09-19T06:47:11.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.422+0000 7f7d439d8740 1 Processor -- start 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7ec4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ec52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d439d8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7f32c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d439d8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1ae080 con 0x55ceb7eb5b30 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d439d8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1af4f0 con 0x55ceb7eb5210 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d439d8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d3c1ae750 con 0x55ceb7ec03f0 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d439d8740 10 client.0 _ll_register_callbacks cb 0x7f7d30031990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ec52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ec52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52956/0 (socket says 172.21.15.37:52956) 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 -- 172.21.15.37:0/723852898 learned_addr learned my addr 172.21.15.37:0/723852898 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7f32c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7ec4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 -- 172.21.15.37:0/723852898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 msgr2=0x55ceb7f32c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7f32c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 -- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 msgr2=0x55ceb7ec4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7ec4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 -- 172.21.15.37:0/723852898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ceb7f33440 con 0x55ceb7eb5b30 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7f32c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d3affd640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ec52f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00f7d0 tx=0x7f7d2c005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f550ac5f31acdcd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d1eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.423+0000 7f7d1eff5640 1 -- 172.21.15.37:0/723852898 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c03b060 con 0x55ceb7eb5b30 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d1eff5640 1 -- 172.21.15.37:0/723852898 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7d2c010b60 con 0x55ceb7eb5b30 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d2c00a730 con 0x55ceb7eb5b30 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d1eff5640 1 -- 172.21.15.37:0/723852898 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7d2c032040 con 0x55ceb7eb5b30 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 10 client.17016 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:11.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55ceb7f3b7d0 con 0x55ceb7eb5b30 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d1eff5640 1 -- 172.21.15.37:0/723852898 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d2c00a730 con 0x55ceb7eb5b30 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d1eff5640 1 -- 172.21.15.37:0/723852898 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7d2c004660 con 0x55ceb7eb5b30 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 10 client.17016 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 10 client.17016 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 20 client.17016 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7f3bc70 0x55ceb7f3e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 4 client.17016 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d57a1d24", "release": true, "await": true}] 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d57a1d24", "release": true, "await": true}) -- 0x55ceb7e220a0 con 0x55ceb7f3bc70 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.424+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7f3bc70 0x55ceb7f3e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.425+0000 7f7d3a7fc640 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7f3bc70 0x55ceb7f3e090 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a84e7572ccca446d server_cookie=c50b00db96a1ec8e in_seq=0 out_seq=0 2024-09-19T06:47:11.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.425+0000 7f7d1eff5640 10 client.17016 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:11.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d1eff5640 1 -- 172.21.15.37:0/723852898 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55ceb7e220a0 con 0x55ceb7f3bc70 2024-09-19T06:47:11.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d1eff5640 10 client.17016 handle_command_reply: tid=0 2024-09-19T06:47:11.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 client.17016 shutdown 2024-09-19T06:47:11.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 msgr2=0x55ceb7ec52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ec52f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c00f7d0 tx=0x7f7d2c005ed0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 shutdown_connections 2024-09-19T06:47:11.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ceb7f3bc70 0x55ceb7f3e090 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d1eff5640 0 client.17016 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ceb7ec03f0 0x55ceb7f32c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ceb7eb5b30 0x55ceb7ec52f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 --2- 172.21.15.37:0/723852898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ceb7eb5210 0x55ceb7ec4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 >> 172.21.15.37:0/723852898 conn(0x55ceb7eb17a0 msgr2=0x55ceb7eb7480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 shutdown_connections 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 1 -- 172.21.15.37:0/723852898 wait complete. 2024-09-19T06:47:11.436 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.434+0000 7f7d439d8740 20 client.17016 trim_cache size 0 max 16384 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 31, 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 24, 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "d57a1d24": { 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 24, 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:47:11.437 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 149.8, 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:11.438 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:11.439 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:47:11.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d240a3430 msgr2=0x7f7d240a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d240a3430 0x7f7d240a58f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7d3409bc60 tx=0x7f7d3406ac50 comp rx=0 tx=0).stop 2024-09-19T06:47:11.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c10e300 msgr2=0x7f7d3c19ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:11.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c10e300 0x7f7d3c19ee80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7d3400cad0 tx=0x7f7d3400ced0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 shutdown_connections 2024-09-19T06:47:11.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d240a3430 0x7f7d240a58f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d3c10e300 0x7f7d3c19ee80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d3c075110 0x7f7d3c19aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 --2- 172.21.15.37:0/1199367117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d3c0747f0 0x7f7d3c19a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 >> 172.21.15.37:0/1199367117 conn(0x7f7d3c0fd700 msgr2=0x7f7d3c0fe3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 shutdown_connections 2024-09-19T06:47:11.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:11.435+0000 7f7d3bfff640 1 -- 172.21.15.37:0/1199367117 wait complete. 2024-09-19T06:47:11.915 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:47:11.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:47:12.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 326478 5.53 MB/sec execute 99 sec latency 31033.284 ms 2024-09-19T06:47:12.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:12 smithi037 ceph-mon[31211]: pgmap v166: 97 pgs: 97 active+clean; 29 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 869 KiB/s wr, 57 op/s 2024-09-19T06:47:12.311 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd57a1d24', total seconds elapsed: 34.7 2024-09-19T06:47:12.343 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:12 smithi151 ceph-mon[35236]: pgmap v166: 97 pgs: 97 active+clean; 29 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 869 KiB/s wr, 57 op/s 2024-09-19T06:47:12.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:12 smithi096 ceph-mon[35004]: pgmap v166: 97 pgs: 97 active+clean; 29 MiB data, 964 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 869 KiB/s wr, 57 op/s 2024-09-19T06:47:12.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 -- 172.21.15.37:0/2480579945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8103780 msgr2=0x7fc4a8107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 --2- 172.21.15.37:0/2480579945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8103780 0x7fc4a8107bc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009930 tx=0x7fc49c02a420 comp rx=0 tx=0).stop 2024-09-19T06:47:12.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 -- 172.21.15.37:0/2480579945 shutdown_connections 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 --2- 172.21.15.37:0/2480579945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4a8108300 0x7fc4a810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 --2- 172.21.15.37:0/2480579945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8103780 0x7fc4a8107bc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 --2- 172.21.15.37:0/2480579945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4a8102dd0 0x7fc4a81031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 -- 172.21.15.37:0/2480579945 >> 172.21.15.37:0/2480579945 conn(0x7fc4a80fc7d0 msgr2=0x7fc4a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 -- 172.21.15.37:0/2480579945 shutdown_connections 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.894+0000 7fc4aca05640 1 -- 172.21.15.37:0/2480579945 wait complete. 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 Processor -- start 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 -- start start 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4a8102dd0 0x7fc4a8100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4a8103780 0x7fc4a8100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8108300 0x7fc4a81014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.917 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8112070 con 0x7fc4a8108300 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a8111ef0 con 0x7fc4a8102dd0 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4aca05640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81121f0 con 0x7fc4a8103780 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8108300 0x7fc4a81014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4a6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8108300 0x7fc4a81014b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52978/0 (socket says 172.21.15.37:52978) 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4a6d9d640 1 -- 172.21.15.37:0/655942098 learned_addr learned my addr 172.21.15.37:0/655942098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4a8103780 0x7fc4a8100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.895+0000 7fc4a659c640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4a8102dd0 0x7fc4a8100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4a6d9d640 1 -- 172.21.15.37:0/655942098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4a8103780 msgr2=0x7fc4a8100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4a6d9d640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4a8103780 0x7fc4a8100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4a6d9d640 1 -- 172.21.15.37:0/655942098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4a8102dd0 msgr2=0x7fc4a8100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4a6d9d640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4a8102dd0 0x7fc4a8100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4a6d9d640 1 -- 172.21.15.37:0/655942098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4a81a6ab0 con 0x7fc4a8108300 2024-09-19T06:47:12.918 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4a659c640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4a8102dd0 0x7fc4a8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4a6d9d640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8108300 0x7fc4a81014b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c970 tx=0x7fc4a000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4977fe640 1 -- 172.21.15.37:0/655942098 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0007bb0 con 0x7fc4a8108300 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4977fe640 1 -- 172.21.15.37:0/655942098 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4a0007d50 con 0x7fc4a8108300 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4a81a6da0 con 0x7fc4a8108300 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4977fe640 1 -- 172.21.15.37:0/655942098 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000f450 con 0x7fc4a8108300 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.896+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4a81a72b0 con 0x7fc4a8108300 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.898+0000 7fc4977fe640 1 -- 172.21.15.37:0/655942098 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc4a0004f20 con 0x7fc4a8108300 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 Processor -- start 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 -- start start 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8def40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8df480 0x55587f8e9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f8ec130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55587f844280 con 0x55587f8deb60 2024-09-19T06:47:12.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55587f7ebb10 con 0x55587f8df480 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4aedde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55587f840240 con 0x55587f8e9d40 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8def40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4977fe640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc4900a3430 0x7fc4900a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8def40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52988/0 (socket says 172.21.15.37:52988) 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4a5d9b640 1 -- 172.21.15.37:0/2224850395 learned_addr learned my addr 172.21.15.37:0/2224850395 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.899+0000 7fc4a659c640 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8df480 0x55587f8e9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a6d9d640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc4900a3430 0x7fc4900a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f8ec130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4977fe640 1 -- 172.21.15.37:0/655942098 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc4a00cf830 con 0x7fc4a8108300 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a659c640 1 -- 172.21.15.37:0/2224850395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 msgr2=0x55587f8ec130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a659c640 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f8ec130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a659c640 1 -- 172.21.15.37:0/2224850395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 msgr2=0x55587f8def40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a659c640 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8def40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a659c640 1 -- 172.21.15.37:0/2224850395 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587f7f81f0 con 0x55587f8df480 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a659c640 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8df480 0x55587f8e9800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc498005fd0 tx=0x7fc498011c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fd0bfcb647debcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc4a6d9d640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc4900a3430 0x7fc4900a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0098530 tx=0x7fc4a0090ff0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc494ff9640 1 -- 172.21.15.37:0/2224850395 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498003c30 con 0x55587f8df480 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc494ff9640 1 -- 172.21.15.37:0/2224850395 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49800fe80 con 0x55587f8df480 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.900+0000 7fc494ff9640 1 -- 172.21.15.37:0/2224850395 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980193e0 con 0x55587f8df480 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 -- 172.21.15.37:0/2224850395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8df480 msgr2=0x55587f8e9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8df480 0x55587f8e9800 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc498005fd0 tx=0x7fc498011c70 comp rx=0 tx=0).stop 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 -- 172.21.15.37:0/2224850395 shutdown_connections 2024-09-19T06:47:12.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f8ec130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8df480 0x55587f8e9800 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2224850395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8def40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 -- 172.21.15.37:0/2224850395 >> 172.21.15.37:0/2224850395 conn(0x55587f8db0f0 msgr2=0x55587f8db4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 -- 172.21.15.37:0/2224850395 shutdown_connections 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 -- 172.21.15.37:0/2224850395 wait complete. 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.901+0000 7fc4aedde740 1 -- start start 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 1 Processor -- start 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8e9d40 0x55587f8e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55587f8ec910 con 0x55587f8e9d40 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55587f7fef10 con 0x55587f8deb60 2024-09-19T06:47:12.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55587f8eca90 con 0x55587f8df480 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4aedde740 10 client.0 _ll_register_callbacks cb 0x55587f7c7a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8e9d40 0x55587f8e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8e70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:58474/0 (socket says 172.21.15.37:58474) 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a659c640 1 -- 172.21.15.37:0/2085737987 learned_addr learned my addr 172.21.15.37:0/2085737987 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8e9d40 0x55587f8e75f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:52992/0 (socket says 172.21.15.37:52992) 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 -- 172.21.15.37:0/2085737987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 msgr2=0x55587f8e70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8e70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 -- 172.21.15.37:0/2085737987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 msgr2=0x55587f8e6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a5d9b640 1 -- 172.21.15.37:0/2085737987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587f5b56a0 con 0x55587f8e9d40 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.902+0000 7fc4a659c640 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:12.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8e9d40 0x55587f8e75f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000c00 tx=0x7fc49c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f357dc084d19576 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc48dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc48dffb640 1 -- 172.21.15.37:0/2085737987 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49c035040 con 0x55587f8e9d40 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc48dffb640 1 -- 172.21.15.37:0/2085737987 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc49c004480 con 0x55587f8e9d40 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55587f8517f0 con 0x55587f8e9d40 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc48dffb640 1 -- 172.21.15.37:0/2085737987 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc49c005040 con 0x55587f8e9d40 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc4aedde740 10 client.17028 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55587f8f84c0 con 0x55587f8e9d40 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc48dffb640 1 -- 172.21.15.37:0/2085737987 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55587f8517f0 con 0x55587f8e9d40 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.903+0000 7fc48dffb640 1 -- 172.21.15.37:0/2085737987 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc49c04a070 con 0x55587f8e9d40 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4aedde740 10 client.17028 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4aedde740 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4aedde740 20 client.17028 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:12.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f8f98d0 0x55587f8fbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4aedde740 4 client.17028 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55587f84bd00 con 0x55587f8f98d0 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4a659c640 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f8f98d0 0x55587f8fbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc4a659c640 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f8f98d0 0x55587f8fbcc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35893f43b50ccfdf server_cookie=904914c19ac163fa in_seq=0 out_seq=0 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.904+0000 7fc48dffb640 10 client.17028 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc48dffb640 1 -- 172.21.15.37:0/2085737987 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55587f84bd00 con 0x55587f8f98d0 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc48dffb640 10 client.17028 handle_command_reply: tid=0 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 client.17028 shutdown 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8e9d40 msgr2=0x55587f8e75f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8e9d40 0x55587f8e75f0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fc49c000c00 tx=0x7fc49c005fd0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 shutdown_connections 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f8f98d0 0x55587f8fbcc0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8e9d40 0x55587f8e75f0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc48dffb640 0 client.17028 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8e70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 --2- 172.21.15.37:0/2085737987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 >> 172.21.15.37:0/2085737987 conn(0x55587f8db0f0 msgr2=0x55587f8da270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 shutdown_connections 2024-09-19T06:47:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 1 -- 172.21.15.37:0/2085737987 wait complete. 2024-09-19T06:47:12.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.906+0000 7fc4aedde740 20 client.17028 trim_cache size 0 max 16384 2024-09-19T06:47:12.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 Processor -- start 2024-09-19T06:47:12.929 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 -- start start 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8df480 0x55587f9014f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f901a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81b24c0 con 0x55587f8df480 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81afd90 con 0x55587f8deb60 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4aedde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81adbe0 con 0x55587f8e9d40 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8df480 0x55587f9014f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4a659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8df480 0x55587f9014f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53000/0 (socket says 172.21.15.37:53000) 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4a659c640 1 -- 172.21.15.37:0/3547522072 learned_addr learned my addr 172.21.15.37:0/3547522072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.928+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f901a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc4a659c640 1 -- 172.21.15.37:0/3547522072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 msgr2=0x55587f901a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.930 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc4a659c640 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f901a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc4a659c640 1 -- 172.21.15.37:0/3547522072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 msgr2=0x55587f8e15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc4a659c640 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc4a659c640 1 -- 172.21.15.37:0/3547522072 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55587f8e5b40 con 0x55587f8df480 2024-09-19T06:47:12.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc4a659c640 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8df480 0x55587f9014f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc49800f790 tx=0x7fc4980092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7735aabaef7710 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:12.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc48ffff640 1 -- 172.21.15.37:0/3547522072 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc49801f040 con 0x55587f8df480 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc48ffff640 1 -- 172.21.15.37:0/3547522072 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49802a030 con 0x55587f8df480 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.929+0000 7fc48ffff640 1 -- 172.21.15.37:0/3547522072 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4980049a0 con 0x55587f8df480 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 -- 172.21.15.37:0/3547522072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8df480 msgr2=0x55587f9014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8df480 0x55587f9014f0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fc49800f790 tx=0x7fc4980092a0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 -- 172.21.15.37:0/3547522072 shutdown_connections 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8e9d40 0x55587f901a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8df480 0x55587f9014f0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 --2- 172.21.15.37:0/3547522072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8deb60 0x55587f8e15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 -- 172.21.15.37:0/3547522072 >> 172.21.15.37:0/3547522072 conn(0x55587f8db0f0 msgr2=0x55587f8e8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 -- 172.21.15.37:0/3547522072 shutdown_connections 2024-09-19T06:47:12.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.930+0000 7fc4aedde740 1 -- 172.21.15.37:0/3547522072 wait complete. 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 -- start start 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 Processor -- start 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8f2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8e9d40 0x55587f8eed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ae250 con 0x55587f8deb60 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81af6c0 con 0x55587f8e9d40 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc4a81ae920 con 0x55587f8df480 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4aedde740 10 client.0 _ll_register_callbacks cb 0x7fc49c031910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4a5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53012/0 (socket says 172.21.15.37:53012) 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4a659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8f2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4a5d9b640 1 -- 172.21.15.37:0/4071399905 learned_addr learned my addr 172.21.15.37:0/4071399905 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:12.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.931+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8e9d40 0x55587f8eed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4a659c640 1 -- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8e9d40 msgr2=0x55587f8eed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4a659c640 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8e9d40 0x55587f8eed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4a659c640 1 -- 172.21.15.37:0/4071399905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 msgr2=0x55587f8f1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4a659c640 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8f1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4a659c640 1 -- 172.21.15.37:0/4071399905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55587f8fc030 con 0x55587f8df480 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8e9d40 0x55587f8eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4a659c640 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8f2280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55587f8dab40 tx=0x7fc498005f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f33fb3a169595cd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc48dffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc48dffb640 1 -- 172.21.15.37:0/4071399905 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc498004600 con 0x55587f8df480 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc48dffb640 1 -- 172.21.15.37:0/4071399905 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fc498003c50 con 0x55587f8df480 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc49802a030 con 0x55587f8df480 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc48dffb640 1 -- 172.21.15.37:0/4071399905 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fc498003e70 con 0x55587f8df480 2024-09-19T06:47:12.936 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4aedde740 10 client.26293 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.932+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55587f96d480 con 0x55587f8df480 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc48dffb640 1 -- 172.21.15.37:0/4071399905 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc49802a030 con 0x55587f8df480 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc48dffb640 1 -- 172.21.15.37:0/4071399905 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fc498018070 con 0x55587f8df480 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4aedde740 10 client.26293 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4aedde740 10 client.26293 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4aedde740 20 client.26293 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4aedde740 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f96d940 0x55587f96fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4aedde740 4 client.26293 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55587f7c8110 con 0x55587f96d940 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f96d940 0x55587f96fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc4a5d9b640 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f96d940 0x55587f96fd30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6073fad1bae1603d server_cookie=8dbfd1dfa1da660c in_seq=0 out_seq=0 2024-09-19T06:47:12.937 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.933+0000 7fc48dffb640 10 client.26293 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:12.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc48dffb640 1 -- 172.21.15.37:0/4071399905 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55587f7c8110 con 0x55587f96d940 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc48dffb640 10 client.26293 handle_command_reply: tid=0 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 client.26293 shutdown 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 msgr2=0x55587f8f2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8f2280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55587f8dab40 tx=0x7fc498005f80 comp rx=0 tx=0).stop 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 shutdown_connections 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55587f96d940 0x55587f96fd30 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55587f8e9d40 0x55587f8eed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55587f8df480 0x55587f8f2280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 --2- 172.21.15.37:0/4071399905 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55587f8deb60 0x55587f8f1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc48dffb640 0 client.26293 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:12.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 >> 172.21.15.37:0/4071399905 conn(0x55587f8db0f0 msgr2=0x55587f8d9a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 shutdown_connections 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 1 -- 172.21.15.37:0/4071399905 wait complete. 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.935+0000 7fc4aedde740 20 client.26293 trim_cache size 0 max 16384 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "1fb22cca-ef24-4fac-8966-e269b47f9507", 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:47:12.940 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc4900a3430 msgr2=0x7fc4900a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc4900a3430 0x7fc4900a58f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4a0098530 tx=0x7fc4a0090ff0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8108300 msgr2=0x7fc4a81014b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8108300 0x7fc4a81014b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c970 tx=0x7fc4a000ce30 comp rx=0 tx=0).stop 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 shutdown_connections 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc4900a3430 0x7fc4900a58f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc4a8108300 0x7fc4a81014b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc4a8103780 0x7fc4a8100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 --2- 172.21.15.37:0/655942098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4a8102dd0 0x7fc4a8100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:12.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 >> 172.21.15.37:0/655942098 conn(0x7fc4a80fc7d0 msgr2=0x7fc4a8105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:12.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 shutdown_connections 2024-09-19T06:47:12.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:12.936+0000 7fc4aca05640 1 -- 172.21.15.37:0/655942098 wait complete. 2024-09-19T06:47:13.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 328934 5.57 MB/sec execute 100 sec latency 15.624 ms 2024-09-19T06:47:13.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:13 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:47:13.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:13 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:13.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:13 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:47:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:13 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:13.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:13 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:47:13.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:13 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:13.690 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:47:13.993 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:14.206 INFO:tasks.workunit.client.0.smithi037.stdout: 1 331504 5.68 MB/sec execute 101 sec latency 11.949 ms 2024-09-19T06:47:14.288 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:14 smithi037 ceph-mon[31211]: pgmap v167: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-09-19T06:47:14.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:14 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:47:14.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:14 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:14.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:14 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:47:14.289 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:14 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:14 smithi151 ceph-mon[35236]: pgmap v167: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-09-19T06:47:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:14 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:47:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:14 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:14 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:47:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:14 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 -- 172.21.15.37:0/500478748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54109140 msgr2=0x7f9c54110cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 --2- 172.21.15.37:0/500478748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54109140 0x7f9c54110cc0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009ca0 tx=0x7f9c4c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 -- 172.21.15.37:0/500478748 shutdown_connections 2024-09-19T06:47:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 --2- 172.21.15.37:0/500478748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54109140 0x7f9c54110cc0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 --2- 172.21.15.37:0/500478748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c541049c0 0x7f9c54108a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 --2- 172.21.15.37:0/500478748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c54104010 0x7f9c541043f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 -- 172.21.15.37:0/500478748 >> 172.21.15.37:0/500478748 conn(0x7f9c540ffa50 msgr2=0x7f9c54101e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 -- 172.21.15.37:0/500478748 shutdown_connections 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 -- 172.21.15.37:0/500478748 wait complete. 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.504+0000 7f9c5c494640 1 Processor -- start 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5c494640 1 -- start start 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5c494640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54104010 0x7f9c5419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5c494640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c541049c0 0x7f9c5419ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5c494640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54109140 0x7f9c5419ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5c494640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54114170 con 0x7f9c54104010 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5c494640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c54113ff0 con 0x7f9c541049c0 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5c494640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541142f0 con 0x7f9c54109140 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54104010 0x7f9c5419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c59a2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c541049c0 0x7f9c5419ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54104010 0x7f9c5419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53020/0 (socket says 172.21.15.37:53020) 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 -- 172.21.15.37:0/4034480797 learned_addr learned my addr 172.21.15.37:0/4034480797 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54109140 0x7f9c5419ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 -- 172.21.15.37:0/4034480797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54109140 msgr2=0x7f9c5419ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54109140 0x7f9c5419ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 -- 172.21.15.37:0/4034480797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c541049c0 msgr2=0x7f9c5419ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c541049c0 0x7f9c5419ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c5a230640 1 -- 172.21.15.37:0/4034480797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c5419f5e0 con 0x7f9c54104010 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.505+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c541049c0 0x7f9c5419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.506+0000 7f9c5a230640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54104010 0x7f9c5419a630 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000e9f0 tx=0x7f9c5000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.506+0000 7f9c4b7fe640 1 -- 172.21.15.37:0/4034480797 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c5000cd80 con 0x7f9c54104010 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.506+0000 7f9c4b7fe640 1 -- 172.21.15.37:0/4034480797 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c50010910 con 0x7f9c54104010 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.506+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c5419f8d0 con 0x7f9c54104010 2024-09-19T06:47:14.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.506+0000 7f9c4b7fe640 1 -- 172.21.15.37:0/4034480797 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c50018990 con 0x7f9c54104010 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.506+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c541a71b0 con 0x7f9c54104010 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.508+0000 7f9c4b7fe640 1 -- 172.21.15.37:0/4034480797 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9c50010ac0 con 0x7f9c54104010 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.508+0000 7f9c5e86d740 1 Processor -- start 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5e86d740 1 -- start start 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e09880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e14140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e14680 0x564011e16a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564011d6f2e0 con 0x564011e09dc0 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564011d16ce0 con 0x564011e094a0 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564011d6b2a0 con 0x564011e14680 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c4b7fe640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c400a3430 0x7f9c400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5a230640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e14140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5a230640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e14140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53028/0 (socket says 172.21.15.37:53028) 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5a230640 1 -- 172.21.15.37:0/1334238506 learned_addr learned my addr 172.21.15.37:0/1334238506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e09880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e14680 0x564011e16a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c400a3430 0x7f9c400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c4b7fe640 1 -- 172.21.15.37:0/4034480797 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9c500cff90 con 0x7f9c54104010 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c59a2f640 1 -- 172.21.15.37:0/1334238506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 msgr2=0x564011e09880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e09880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c59a2f640 1 -- 172.21.15.37:0/1334238506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 msgr2=0x564011e14140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e14140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.509+0000 7f9c59a2f640 1 -- 172.21.15.37:0/1334238506 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564011cfb440 con 0x564011e14680 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.510+0000 7f9c5a230640 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e14140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.510+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c400a3430 0x7f9c400a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004af0 tx=0x7f9c4c004a80 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.510+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e14680 0x564011e16a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009f80 tx=0x7f9c440118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba225c2063160f03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.510+0000 7f9c48ff9640 1 -- 172.21.15.37:0/1334238506 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44004310 con 0x564011e14680 2024-09-19T06:47:14.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.510+0000 7f9c48ff9640 1 -- 172.21.15.37:0/1334238506 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c440044b0 con 0x564011e14680 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.510+0000 7f9c48ff9640 1 -- 172.21.15.37:0/1334238506 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c440035f0 con 0x564011e14680 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1334238506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e14680 msgr2=0x564011e16a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e14680 0x564011e16a70 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009f80 tx=0x7f9c440118d0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1334238506 shutdown_connections 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e14680 0x564011e16a70 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e14140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1334238506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e09880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1334238506 >> 172.21.15.37:0/1334238506 conn(0x564011e05a30 msgr2=0x564011e05e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1334238506 shutdown_connections 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1334238506 wait complete. 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 -- start start 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.511+0000 7f9c5e86d740 1 Processor -- start 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e13ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.523 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e111a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e14680 0x564011e116e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564011e17250 con 0x564011e09dc0 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564011d2a050 con 0x564011e14680 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564011e173d0 con 0x564011e094a0 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5e86d740 10 client.0 _ll_register_callbacks cb 0x564011d6a160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e111a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e111a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53040/0 (socket says 172.21.15.37:53040) 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 -- 172.21.15.37:0/3125757911 learned_addr learned my addr 172.21.15.37:0/3125757911 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5a230640 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e13ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e14680 0x564011e116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 -- 172.21.15.37:0/3125757911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 msgr2=0x564011e13ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e13ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 -- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e14680 msgr2=0x564011e116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e14680 0x564011e116e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 -- 172.21.15.37:0/3125757911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564011ae03c0 con 0x564011e09dc0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e111a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9c44004bc0 tx=0x7f9c44004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ceb8d6a257628ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c3dffb640 1 -- 172.21.15.37:0/3125757911 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c44029070 con 0x564011e09dc0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.512+0000 7f9c3dffb640 1 -- 172.21.15.37:0/3125757911 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9c44003a00 con 0x564011e09dc0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c440044b0 con 0x564011e09dc0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c3dffb640 1 -- 172.21.15.37:0/3125757911 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9c44035050 con 0x564011e09dc0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 10 client.17046 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564011e22b40 con 0x564011e09dc0 2024-09-19T06:47:14.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c3dffb640 1 -- 172.21.15.37:0/3125757911 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c440044b0 con 0x564011e09dc0 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c3dffb640 1 -- 172.21.15.37:0/3125757911 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f9c4400f480 con 0x564011e09dc0 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 10 client.17046 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 10 client.17046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 20 client.17046 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e23e90 0x564011e26280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 4 client.17046 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564011d7c850 con 0x564011e23e90 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.513+0000 7f9c5a230640 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e23e90 0x564011e26280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.514+0000 7f9c5a230640 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e23e90 0x564011e26280 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27fb873744e6e735 server_cookie=9cfdb27933ac3033 in_seq=0 out_seq=0 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.514+0000 7f9c3dffb640 10 client.17046 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c3dffb640 1 -- 172.21.15.37:0/3125757911 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564011d7c850 con 0x564011e23e90 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c3dffb640 10 client.17046 handle_command_reply: tid=0 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 client.17046 shutdown 2024-09-19T06:47:14.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 msgr2=0x564011e111a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e111a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f9c44004bc0 tx=0x7f9c44004340 comp rx=0 tx=0).stop 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 shutdown_connections 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e23e90 0x564011e26280 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c3dffb640 0 client.17046 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e14680 0x564011e116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e111a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3125757911 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e13ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 >> 172.21.15.37:0/3125757911 conn(0x564011e05a30 msgr2=0x564011e0b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:14.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 shutdown_connections 2024-09-19T06:47:14.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3125757911 wait complete. 2024-09-19T06:47:14.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.515+0000 7f9c5e86d740 20 client.17046 trim_cache size 0 max 16384 2024-09-19T06:47:14.528 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:14 smithi096 ceph-mon[35004]: pgmap v167: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 730 KiB/s rd, 1.1 MiB/s wr, 84 op/s 2024-09-19T06:47:14.528 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:14 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:47:14.528 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:14 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:14.528 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:14 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:47:14.528 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:14 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 Processor -- start 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 -- start start 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e05a30 0x564011e04a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e2bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e2bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541b2390 con 0x564011e09dc0 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541afc60 con 0x564011e05a30 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541adab0 con 0x564011e094a0 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5aa31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e2bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5aa31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e2bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53054/0 (socket says 172.21.15.37:53054) 2024-09-19T06:47:14.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5aa31640 1 -- 172.21.15.37:0/3054597299 learned_addr learned my addr 172.21.15.37:0/3054597299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e2bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.533+0000 7f9c5a230640 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e05a30 0x564011e04a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c5aa31640 1 -- 172.21.15.37:0/3054597299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 msgr2=0x564011e2bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e2bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c5aa31640 1 -- 172.21.15.37:0/3054597299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e05a30 msgr2=0x564011e04a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e05a30 0x564011e04a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c5aa31640 1 -- 172.21.15.37:0/3054597299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564011e09bf0 con 0x564011e09dc0 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c5a230640 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e05a30 0x564011e04a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e2bff0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c035000 tx=0x7f9c4c036000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30f13c3e97901b11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c3ffff640 1 -- 172.21.15.37:0/3054597299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c02fb10 con 0x564011e09dc0 2024-09-19T06:47:14.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c3ffff640 1 -- 172.21.15.37:0/3054597299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4c036070 con 0x564011e09dc0 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.534+0000 7f9c3ffff640 1 -- 172.21.15.37:0/3054597299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c04b6b0 con 0x564011e09dc0 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3054597299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 msgr2=0x564011e2bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e2bff0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c035000 tx=0x7f9c4c036000 comp rx=0 tx=0).stop 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3054597299 shutdown_connections 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e2bff0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e094a0 0x564011e2bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/3054597299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e05a30 0x564011e04a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3054597299 >> 172.21.15.37:0/3054597299 conn(0x564011e11840 msgr2=0x564011e04350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3054597299 shutdown_connections 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 -- 172.21.15.37:0/3054597299 wait complete. 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 -- start start 2024-09-19T06:47:14.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.535+0000 7f9c5e86d740 1 Processor -- start 2024-09-19T06:47:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e05a30 0x564011e15710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e15c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5e86d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e16190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541ae120 con 0x564011e09dc0 2024-09-19T06:47:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541af590 con 0x564011e094a0 2024-09-19T06:47:14.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5e86d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9c541ae7f0 con 0x564011e05a30 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5e86d740 10 client.0 _ll_register_callbacks cb 0x7f9c5009c0f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e16190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5a230640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e05a30 0x564011e15710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c59a2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e15c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e16190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53062/0 (socket says 172.21.15.37:53062) 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 -- 172.21.15.37:0/1147753228 learned_addr learned my addr 172.21.15.37:0/1147753228 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 -- 172.21.15.37:0/1147753228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e05a30 msgr2=0x564011e15710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e05a30 0x564011e15710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 -- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 msgr2=0x564011e15c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e15c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 -- 172.21.15.37:0/1147753228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564011e14260 con 0x564011e09dc0 2024-09-19T06:47:14.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c59a2f640 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e15c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c5aa31640 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e16190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c0099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff988bf4f1bd7e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c3dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.536+0000 7f9c3dffb640 1 -- 172.21.15.37:0/1147753228 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c05f060 con 0x564011e09dc0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c4c036070 con 0x564011e09dc0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c3dffb640 1 -- 172.21.15.37:0/1147753228 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9c4c0307d0 con 0x564011e09dc0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c3dffb640 1 -- 172.21.15.37:0/1147753228 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9c4c030eb0 con 0x564011e09dc0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 10 client.17052 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x564011e8eae0 con 0x564011e09dc0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c3dffb640 1 -- 172.21.15.37:0/1147753228 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c4c036070 con 0x564011e09dc0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c3dffb640 1 -- 172.21.15.37:0/1147753228 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f9c4c048040 con 0x564011e09dc0 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 10 client.17052 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 20 client.17052 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:14.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e14680 0x564011e911a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 4 client.17052 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564011d76d60 con 0x564011e14680 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.537+0000 7f9c5a230640 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e14680 0x564011e911a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.538+0000 7f9c5a230640 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e14680 0x564011e911a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9f8a0c25c24bb43 server_cookie=b4f2d48290f2fc05 in_seq=0 out_seq=0 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.538+0000 7f9c3dffb640 10 client.17052 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.538+0000 7f9c3dffb640 1 -- 172.21.15.37:0/1147753228 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564011d76d60 con 0x564011e14680 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.538+0000 7f9c3dffb640 10 client.17052 handle_command_reply: tid=0 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.538+0000 7f9c5e86d740 1 client.17052 shutdown 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 msgr2=0x564011e16190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e16190 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c006010 tx=0x7f9c4c0099c0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 shutdown_connections 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564011e14680 0x564011e911a0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564011e09dc0 0x564011e16190 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564011e094a0 0x564011e15c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 --2- 172.21.15.37:0/1147753228 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564011e05a30 0x564011e15710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 >> 172.21.15.37:0/1147753228 conn(0x564011e11840 msgr2=0x564011e13220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c3dffb640 0 client.17052 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 shutdown_connections 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 1 -- 172.21.15.37:0/1147753228 wait complete. 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5e86d740 20 client.17052 trim_cache size 0 max 16384 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:47:14.544 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "1fb22cca-ef24-4fac-8966-e269b47f9507": { 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "1fb22cca-ef24-4fac-8966-e269b47f9507", 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:14.545 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c400a3430 msgr2=0x7f9c400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5c494640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c400a3430 0x7f9c400a58f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c004af0 tx=0x7f9c4c004a80 comp rx=0 tx=0).stop 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54104010 msgr2=0x7f9c5419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.539+0000 7f9c5c494640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54104010 0x7f9c5419a630 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f9c5000e9f0 tx=0x7f9c5000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 shutdown_connections 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9c54109140 0x7f9c5419ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c541049c0 0x7f9c5419ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9c400a3430 0x7f9c400a58f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.546 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 --2- 172.21.15.37:0/4034480797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9c54104010 0x7f9c5419a630 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 >> 172.21.15.37:0/4034480797 conn(0x7f9c540ffa50 msgr2=0x7f9c54100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 shutdown_connections 2024-09-19T06:47:14.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:14.540+0000 7f9c5c494640 1 -- 172.21.15.37:0/4034480797 wait complete. 2024-09-19T06:47:15.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.060+0000 7fef86ffd640 1 -- 172.21.15.37:0/2196281020 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef780319b0 con 0x7fef881069c0 2024-09-19T06:47:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 -- 172.21.15.37:0/2196281020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef881069c0 msgr2=0x7fef88106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2196281020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef881069c0 0x7fef88106da0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fef78009930 tx=0x7fef7802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 -- 172.21.15.37:0/2196281020 shutdown_connections 2024-09-19T06:47:15.064 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2196281020 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef88101350 0x7fef8810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2196281020 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881009b0 0x7fef88100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2196281020 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef881069c0 0x7fef88106da0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 -- 172.21.15.37:0/2196281020 >> 172.21.15.37:0/2196281020 conn(0x7fef88077990 msgr2=0x7fef880fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 -- 172.21.15.37:0/2196281020 shutdown_connections 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 -- 172.21.15.37:0/2196281020 wait complete. 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 Processor -- start 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.061+0000 7fef8ea53640 1 -- start start 2024-09-19T06:47:15.065 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef8ea53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881009b0 0x7fef88198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef8ea53640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88101350 0x7fef88198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef8ea53640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef881069c0 0x7fef8819ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef8ea53640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef88110e50 con 0x7fef88101350 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef8ea53640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef88110cd0 con 0x7fef881069c0 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef8ea53640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef88110fd0 con 0x7fef881009b0 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88101350 0x7fef88198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88101350 0x7fef88198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53078/0 (socket says 172.21.15.37:53078) 2024-09-19T06:47:15.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 -- 172.21.15.37:0/2607698599 learned_addr learned my addr 172.21.15.37:0/2607698599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef8cff0640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef881069c0 0x7fef8819ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 -- 172.21.15.37:0/2607698599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881009b0 msgr2=0x7fef88198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881009b0 0x7fef88198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 -- 172.21.15.37:0/2607698599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef881069c0 msgr2=0x7fef8819ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef881069c0 0x7fef8819ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 -- 172.21.15.37:0/2607698599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef8819d400 con 0x7fef88101350 2024-09-19T06:47:15.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.062+0000 7fef877fe640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88101350 0x7fef88198950 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.063+0000 7fef857fa640 1 -- 172.21.15.37:0/2607698599 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c00cd80 con 0x7fef88101350 2024-09-19T06:47:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.063+0000 7fef857fa640 1 -- 172.21.15.37:0/2607698599 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef7c005500 con 0x7fef88101350 2024-09-19T06:47:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.063+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef8819d6f0 con 0x7fef88101350 2024-09-19T06:47:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.063+0000 7fef857fa640 1 -- 172.21.15.37:0/2607698599 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c010690 con 0x7fef88101350 2024-09-19T06:47:15.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.063+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef881a4f80 con 0x7fef88101350 2024-09-19T06:47:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.065+0000 7fef857fa640 1 -- 172.21.15.37:0/2607698599 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fef7c0046c0 con 0x7fef88101350 2024-09-19T06:47:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.065+0000 7fef917e9740 1 Processor -- start 2024-09-19T06:47:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.065+0000 7fef917e9740 1 -- start start 2024-09-19T06:47:15.077 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef917e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d015887ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef917e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d015892770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef917e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d0158950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef917e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d0157ec2f0 con 0x55d015892cb0 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef917e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d015794270 con 0x55d015887af0 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef917e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d0157e82b0 con 0x55d015888410 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d015887ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d0158950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d015887ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:55316/0 (socket says 172.21.15.37:55316) 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 -- 172.21.15.37:0/2624378515 learned_addr learned my addr 172.21.15.37:0/2624378515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef87fff640 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d015892770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef857fa640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fef700a3460 0x7fef700a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.078 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 -- 172.21.15.37:0/2624378515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 msgr2=0x55d015892770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d015892770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 -- 172.21.15.37:0/2624378515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 msgr2=0x55d0158950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d0158950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 -- 172.21.15.37:0/2624378515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d015793340 con 0x55d015887af0 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef877fe640 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d0158950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef857fa640 1 -- 172.21.15.37:0/2607698599 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fef7c0d01f0 con 0x7fef88101350 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.066+0000 7fef8cff0640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fef700a3460 0x7fef700a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.067+0000 7fef8cff0640 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d015887ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef80011590 tx=0x7fef80011a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71c5fdccb41cbf8f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.067+0000 7fef6e7fc640 1 -- 172.21.15.37:0/2624378515 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef80017050 con 0x55d015887af0 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.067+0000 7fef6e7fc640 1 -- 172.21.15.37:0/2624378515 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef8001bb90 con 0x55d015887af0 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.067+0000 7fef6e7fc640 1 -- 172.21.15.37:0/2624378515 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8000e3b0 con 0x55d015887af0 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.067+0000 7fef8cff0640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fef700a3460 0x7fef700a5920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef80022000 tx=0x7fef80023000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.067+0000 7fef917e9740 1 -- 172.21.15.37:0/2624378515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 msgr2=0x55d015887ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.067+0000 7fef917e9740 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d015887ed0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fef80011590 tx=0x7fef80011a50 comp rx=0 tx=0).stop 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 -- 172.21.15.37:0/2624378515 shutdown_connections 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d0158950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d015892770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 --2- 172.21.15.37:0/2624378515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d015887ed0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 -- 172.21.15.37:0/2624378515 >> 172.21.15.37:0/2624378515 conn(0x55d015884060 msgr2=0x55d015884440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 -- 172.21.15.37:0/2624378515 shutdown_connections 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 -- 172.21.15.37:0/2624378515 wait complete. 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 -- start start 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.068+0000 7fef917e9740 1 Processor -- start 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d01588fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d0158900e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d015890620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d015895880 con 0x55d015892cb0 2024-09-19T06:47:15.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d015775290 con 0x55d015887af0 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d015895a00 con 0x55d015888410 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 10 client.0 _ll_register_callbacks cb 0x55d0157a6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d015890620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d015890620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53088/0 (socket says 172.21.15.37:53088) 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 -- 172.21.15.37:0/1739082361 learned_addr learned my addr 172.21.15.37:0/1739082361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef87fff640 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d01588fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef877fe640 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d0158900e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 -- 172.21.15.37:0/1739082361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 msgr2=0x55d0158900e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d0158900e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 -- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 msgr2=0x55d01588fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d01588fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 -- 172.21.15.37:0/1739082361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0154ff210 con 0x55d015892cb0 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef8cff0640 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d015890620 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fef80009cd0 tx=0x7fef80009d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac93f6f71a28869c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:15.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef677fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef677fe640 1 -- 172.21.15.37:0/1739082361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef800049c0 con 0x55d015892cb0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef677fe640 1 -- 172.21.15.37:0/1739082361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fef80004b60 con 0x55d015892cb0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.069+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef8001bb90 con 0x55d015892cb0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef677fe640 1 -- 172.21.15.37:0/1739082361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fef80017020 con 0x55d015892cb0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 10 client.17064 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d0155cb620 con 0x55d015892cb0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef677fe640 1 -- 172.21.15.37:0/1739082361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef8001bb90 con 0x55d015892cb0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef677fe640 1 -- 172.21.15.37:0/1739082361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fef8000ea30 con 0x55d015892cb0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 10 client.17064 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 10 client.17064 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 20 client.17064 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d0158a2ba0 0x55d0158a4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 4 client.17064 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0157f9a20 con 0x55d0158a2ba0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.070+0000 7fef87fff640 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d0158a2ba0 0x55d0158a4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.071+0000 7fef87fff640 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d0158a2ba0 0x55d0158a4f90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45fdfddd3454c746 server_cookie=2d7fcaec0bc81e30 in_seq=0 out_seq=0 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.071+0000 7fef677fe640 10 client.17064 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef677fe640 1 -- 172.21.15.37:0/1739082361 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d0157f9a20 con 0x55d0158a2ba0 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef677fe640 10 client.17064 handle_command_reply: tid=0 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 client.17064 shutdown 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 msgr2=0x55d015890620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d015890620 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fef80009cd0 tx=0x7fef80009d00 comp rx=0 tx=0).stop 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 shutdown_connections 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d0158a2ba0 0x55d0158a4f90 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015892cb0 0x55d015890620 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015888410 0x55d0158900e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 --2- 172.21.15.37:0/1739082361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015887af0 0x55d01588fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 >> 172.21.15.37:0/1739082361 conn(0x55d015884060 msgr2=0x55d0158831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef677fe640 0 client.17064 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:15.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 shutdown_connections 2024-09-19T06:47:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.072+0000 7fef917e9740 1 -- 172.21.15.37:0/1739082361 wait complete. 2024-09-19T06:47:15.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.073+0000 7fef917e9740 20 client.17064 trim_cache size 0 max 16384 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.087+0000 7fef917e9740 1 Processor -- start 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.087+0000 7fef917e9740 1 -- start start 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef917e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015887af0 0x55d015891ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef917e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015888410 0x55d0158aa560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef917e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015892cb0 0x55d0158aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef917e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef881b01e0 con 0x55d015887af0 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef917e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef881adab0 con 0x55d015888410 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef917e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef881ab900 con 0x55d015892cb0 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015887af0 0x55d015891ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015887af0 0x55d015891ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53090/0 (socket says 172.21.15.37:53090) 2024-09-19T06:47:15.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 -- 172.21.15.37:0/2568161627 learned_addr learned my addr 172.21.15.37:0/2568161627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef877fe640 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015888410 0x55d0158aa560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef8cff0640 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015892cb0 0x55d0158aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 -- 172.21.15.37:0/2568161627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015892cb0 msgr2=0x55d0158aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015892cb0 0x55d0158aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 -- 172.21.15.37:0/2568161627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015888410 msgr2=0x55d0158aa560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015888410 0x55d0158aa560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 -- 172.21.15.37:0/2568161627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d015885110 con 0x55d015887af0 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef87fff640 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015887af0 0x55d015891ed0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fef7802a940 tx=0x7fef78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2b8b47cadd8308 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef6dffb640 1 -- 172.21.15.37:0/2568161627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78035040 con 0x55d015887af0 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef6dffb640 1 -- 172.21.15.37:0/2568161627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef78049070 con 0x55d015887af0 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.088+0000 7fef6dffb640 1 -- 172.21.15.37:0/2568161627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef78004920 con 0x55d015887af0 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 -- 172.21.15.37:0/2568161627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015887af0 msgr2=0x55d015891ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015887af0 0x55d015891ed0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fef7802a940 tx=0x7fef78000bc0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 -- 172.21.15.37:0/2568161627 shutdown_connections 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015892cb0 0x55d0158aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015888410 0x55d0158aa560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 --2- 172.21.15.37:0/2568161627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015887af0 0x55d015891ed0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 -- 172.21.15.37:0/2568161627 >> 172.21.15.37:0/2568161627 conn(0x55d015884060 msgr2=0x55d0158831c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 -- 172.21.15.37:0/2568161627 shutdown_connections 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 -- 172.21.15.37:0/2568161627 wait complete. 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.089+0000 7fef917e9740 1 -- start start 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 1 Processor -- start 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015887af0 0x55d015898320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015888410 0x55d015898860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015892cb0 0x55d015905570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.095 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fef881abf70 con 0x55d015888410 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fef881ad3e0 con 0x55d015892cb0 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fef881ac640 con 0x55d015887af0 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef917e9740 10 client.0 _ll_register_callbacks cb 0x7fef80011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015888410 0x55d015898860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015888410 0x55d015898860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:53100/0 (socket says 172.21.15.37:53100) 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 -- 172.21.15.37:0/610263135 learned_addr learned my addr 172.21.15.37:0/610263135 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef87fff640 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015887af0 0x55d015898320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef8cff0640 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015892cb0 0x55d015905570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 -- 172.21.15.37:0/610263135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015887af0 msgr2=0x55d015898320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015887af0 0x55d015898320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 -- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015892cb0 msgr2=0x55d015905570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015892cb0 0x55d015905570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.096 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 -- 172.21.15.37:0/610263135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d015905be0 con 0x55d015888410 2024-09-19T06:47:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef877fe640 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015888410 0x55d015898860 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fef7c098930 tx=0x7fef7c0d3cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9825d57d576d582 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.090+0000 7fef677fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef677fe640 1 -- 172.21.15.37:0/610263135 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef7c098770 con 0x55d015888410 2024-09-19T06:47:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef677fe640 1 -- 172.21.15.37:0/610263135 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fef7c0a4eb0 con 0x55d015888410 2024-09-19T06:47:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef78049070 con 0x55d015888410 2024-09-19T06:47:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef677fe640 1 -- 172.21.15.37:0/610263135 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fef7c0cddd0 con 0x55d015888410 2024-09-19T06:47:15.097 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 10 client.17070 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d01590cfd0 con 0x55d015888410 2024-09-19T06:47:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef677fe640 1 -- 172.21.15.37:0/610263135 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fef78049070 con 0x55d015888410 2024-09-19T06:47:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef677fe640 1 -- 172.21.15.37:0/610263135 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fef7c09b390 con 0x55d015888410 2024-09-19T06:47:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 10 client.17070 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 10 client.17070 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:15.098 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 20 client.17070 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d01590d700 0x55d01590fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 4 client.17070 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:47:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d0157f3d70 con 0x55d01590d700 2024-09-19T06:47:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.091+0000 7fef87fff640 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d01590d700 0x55d01590fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.092+0000 7fef677fe640 10 client.17070 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:15.099 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.092+0000 7fef87fff640 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d01590d700 0x55d01590fb20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eae75a504a1ace36 server_cookie=3f76613a6bf9cece in_seq=0 out_seq=0 2024-09-19T06:47:15.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 334095 5.76 MB/sec execute 102 sec latency 27.028 ms 2024-09-19T06:47:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.233+0000 7fef677fe640 1 -- 172.21.15.37:0/610263135 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d0157f3d70 con 0x55d01590d700 2024-09-19T06:47:15.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.233+0000 7fef677fe640 10 client.17070 handle_command_reply: tid=0 2024-09-19T06:47:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 client.17070 shutdown 2024-09-19T06:47:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015888410 msgr2=0x55d015898860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015888410 0x55d015898860 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fef7c098930 tx=0x7fef7c0d3cc0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 shutdown_connections 2024-09-19T06:47:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d01590d700 0x55d01590fb20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d015892cb0 0x55d015905570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d015888410 0x55d015898860 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 --2- 172.21.15.37:0/610263135 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d015887af0 0x55d015898320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 >> 172.21.15.37:0/610263135 conn(0x55d015884060 msgr2=0x55d01589db10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef677fe640 0 client.17070 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 shutdown_connections 2024-09-19T06:47:15.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 1 -- 172.21.15.37:0/610263135 wait complete. 2024-09-19T06:47:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.234+0000 7fef917e9740 20 client.17070 trim_cache size 0 max 16384 2024-09-19T06:47:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:47:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:47:15.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:47:15.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:47:15.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fef700a3460 msgr2=0x7fef700a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fef700a3460 0x7fef700a5920 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fef80022000 tx=0x7fef80023000 comp rx=0 tx=0).stop 2024-09-19T06:47:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88101350 msgr2=0x7fef88198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88101350 0x7fef88198950 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fef7c00e9f0 tx=0x7fef7c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 shutdown_connections 2024-09-19T06:47:15.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fef881069c0 0x7fef8819ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fef700a3460 0x7fef700a5920 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fef88101350 0x7fef88198950 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 --2- 172.21.15.37:0/2607698599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fef881009b0 0x7fef88198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 >> 172.21.15.37:0/2607698599 conn(0x7fef88077990 msgr2=0x7fef880699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 shutdown_connections 2024-09-19T06:47:15.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:15.235+0000 7fef8ea53640 1 -- 172.21.15.37:0/2607698599 wait complete. 2024-09-19T06:47:15.434 INFO:tasks.cephfs.filesystem:scrub status for tag:1fb22cca-ef24-4fac-8966-e269b47f9507 - {'path': '/', 'tag': '1fb22cca-ef24-4fac-8966-e269b47f9507', 'options': 'recursive,force'} 2024-09-19T06:47:16.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 337208 5.88 MB/sec execute 103 sec latency 23.501 ms 2024-09-19T06:47:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:16 smithi151 ceph-mon[35236]: pgmap v168: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 267 KiB/s wr, 29 op/s 2024-09-19T06:47:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:16 smithi096 ceph-mon[35004]: pgmap v168: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 267 KiB/s wr, 29 op/s 2024-09-19T06:47:16.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:16 smithi037 ceph-mon[31211]: pgmap v168: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 267 KiB/s wr, 29 op/s 2024-09-19T06:47:17.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.88 MB/sec execute 104 sec latency 447.739 ms 2024-09-19T06:47:18.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.82 MB/sec execute 105 sec latency 1447.869 ms 2024-09-19T06:47:18.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:18 smithi151 ceph-mon[35236]: pgmap v169: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 267 KiB/s wr, 29 op/s 2024-09-19T06:47:18.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:18 smithi096 ceph-mon[35004]: pgmap v169: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 267 KiB/s wr, 29 op/s 2024-09-19T06:47:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:18 smithi037 ceph-mon[31211]: pgmap v169: 97 pgs: 97 active+clean; 32 MiB data, 954 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 267 KiB/s wr, 29 op/s 2024-09-19T06:47:19.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.77 MB/sec execute 106 sec latency 2447.996 ms 2024-09-19T06:47:20.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.71 MB/sec execute 107 sec latency 3448.106 ms 2024-09-19T06:47:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:20 smithi096 ceph-mon[35004]: pgmap v170: 97 pgs: 97 active+clean; 36 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-19T06:47:20.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:20 smithi037 ceph-mon[31211]: pgmap v170: 97 pgs: 97 active+clean; 36 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-19T06:47:20.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:20 smithi151 ceph-mon[35236]: pgmap v170: 97 pgs: 97 active+clean; 36 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 164 op/s 2024-09-19T06:47:21.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.66 MB/sec execute 108 sec latency 4448.237 ms 2024-09-19T06:47:22.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.61 MB/sec execute 109 sec latency 5448.310 ms 2024-09-19T06:47:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:22 smithi096 ceph-mon[35004]: pgmap v171: 97 pgs: 97 active+clean; 36 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-09-19T06:47:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:22 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:47:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:22 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:47:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:22 smithi037 ceph-mon[31211]: pgmap v171: 97 pgs: 97 active+clean; 36 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-09-19T06:47:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:22 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:47:22.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:22 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:47:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:22 smithi151 ceph-mon[35236]: pgmap v171: 97 pgs: 97 active+clean; 36 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.1 MiB/s wr, 161 op/s 2024-09-19T06:47:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:22 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:47:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:22 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:47:23.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.56 MB/sec execute 110 sec latency 6448.438 ms 2024-09-19T06:47:24.207 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.51 MB/sec execute 111 sec latency 7448.545 ms 2024-09-19T06:47:24.378 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:24 smithi037 ceph-mon[31211]: pgmap v172: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-09-19T06:47:24.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:24 smithi096 ceph-mon[35004]: pgmap v172: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-09-19T06:47:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:24 smithi151 ceph-mon[35236]: pgmap v172: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 1.1 MiB/s wr, 167 op/s 2024-09-19T06:47:25.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.46 MB/sec execute 112 sec latency 8448.666 ms 2024-09-19T06:47:26.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 338378 5.41 MB/sec execute 113 sec latency 9448.802 ms 2024-09-19T06:47:26.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:26 smithi096 ceph-mon[35004]: pgmap v173: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 KiB/s wr, 140 op/s 2024-09-19T06:47:26.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:26 smithi037 ceph-mon[31211]: pgmap v173: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 KiB/s wr, 140 op/s 2024-09-19T06:47:26.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:26 smithi151 ceph-mon[35236]: pgmap v173: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 KiB/s wr, 140 op/s 2024-09-19T06:47:27.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 340151 5.43 MB/sec execute 114 sec latency 9890.931 ms 2024-09-19T06:47:28.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 342294 5.49 MB/sec execute 115 sec latency 15.429 ms 2024-09-19T06:47:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3179363109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 msgr2=0x7fe5dc103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3179363109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc103060 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8009900 tx=0x7fe5d802a480 comp rx=0 tx=0).stop 2024-09-19T06:47:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3179363109 shutdown_connections 2024-09-19T06:47:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3179363109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5dc10c0d0 0x7fe5dc10e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3179363109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc1035a0 0x7fe5dc10ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.330 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3179363109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc103060 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3179363109 >> 172.21.15.37:0/3179363109 conn(0x7fe5dc077990 msgr2=0x7fe5dc0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3179363109 shutdown_connections 2024-09-19T06:47:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.329+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3179363109 wait complete. 2024-09-19T06:47:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 Processor -- start 2024-09-19T06:47:28.331 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 -- start start 2024-09-19T06:47:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc1008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc1035a0 0x7fe5dc100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5dc10c0d0 0x7fe5dc101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc111f00 con 0x7fe5dc102c80 2024-09-19T06:47:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc111d80 con 0x7fe5dc1035a0 2024-09-19T06:47:28.333 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e3ad7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc112080 con 0x7fe5dc10c0d0 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc1008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc1008d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49588/0 (socket says 172.21.15.37:49588) 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 -- 172.21.15.37:0/3950804922 learned_addr learned my addr 172.21.15.37:0/3950804922 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc1035a0 0x7fe5dc100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5dc10c0d0 0x7fe5dc101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 -- 172.21.15.37:0/3950804922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5dc10c0d0 msgr2=0x7fe5dc101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5dc10c0d0 0x7fe5dc101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 -- 172.21.15.37:0/3950804922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc1035a0 msgr2=0x7fe5dc100e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc1035a0 0x7fe5dc100e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.330+0000 7fe5e1873640 1 -- 172.21.15.37:0/3950804922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5dc1a6840 con 0x7fe5dc102c80 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.331+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc1008d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8004770 tx=0x7fe5d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.334 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.331+0000 7fe5d2ffd640 1 -- 172.21.15.37:0/3950804922 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d803f070 con 0x7fe5dc102c80 2024-09-19T06:47:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.331+0000 7fe5d2ffd640 1 -- 172.21.15.37:0/3950804922 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d8031ca0 con 0x7fe5dc102c80 2024-09-19T06:47:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.331+0000 7fe5d2ffd640 1 -- 172.21.15.37:0/3950804922 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8035030 con 0x7fe5dc102c80 2024-09-19T06:47:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.331+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5dc1a6ad0 con 0x7fe5dc102c80 2024-09-19T06:47:28.335 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.331+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5dc1a6f70 con 0x7fe5dc102c80 2024-09-19T06:47:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.333+0000 7fe5d2ffd640 1 -- 172.21.15.37:0/3950804922 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe5d8042070 con 0x7fe5dc102c80 2024-09-19T06:47:28.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 Processor -- start 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 -- start start 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b61ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b62410 0x55d1f3b6c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d1f3ac62f0 con 0x55d1f3b62410 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1f3a6e270 con 0x55d1f3b61af0 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e6873740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1f3ac22b0 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b62410 0x55d1f3b6c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e2074640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b61ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b62410 0x55d1f3b6c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49594/0 (socket says 172.21.15.37:49594) 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e1873640 1 -- 172.21.15.37:0/3714440278 learned_addr learned my addr 172.21.15.37:0/3714440278 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.334+0000 7fe5d2ffd640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5c80a34e0 0x7fe5c80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5c80a34e0 0x7fe5c80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e1873640 1 -- 172.21.15.37:0/3714440278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b6ccb0 msgr2=0x55d1f3b6f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e1873640 1 -- 172.21.15.37:0/3714440278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 msgr2=0x55d1f3b61ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b61ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e1873640 1 -- 172.21.15.37:0/3714440278 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f3a6d340 con 0x55d1f3b62410 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5d2ffd640 1 -- 172.21.15.37:0/3950804922 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fe5d80f0310 con 0x7fe5dc102c80 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b62410 0x55d1f3b6c770 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80bc640 tx=0x7fe5d8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a830d02a98a4a57f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5c80a34e0 0x7fe5c80a59a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40094d0 tx=0x7fe5d4004ab0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5c7fff640 1 -- 172.21.15.37:0/3714440278 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8049030 con 0x55d1f3b62410 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5c7fff640 1 -- 172.21.15.37:0/3714440278 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d80bcdb0 con 0x55d1f3b62410 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.335+0000 7fe5c7fff640 1 -- 172.21.15.37:0/3714440278 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d80c0a60 con 0x55d1f3b62410 2024-09-19T06:47:28.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 -- 172.21.15.37:0/3714440278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b62410 msgr2=0x55d1f3b6c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b62410 0x55d1f3b6c770 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80bc640 tx=0x7fe5d8009260 comp rx=0 tx=0).stop 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 -- 172.21.15.37:0/3714440278 shutdown_connections 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b62410 0x55d1f3b6c770 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3714440278 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b61ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 -- 172.21.15.37:0/3714440278 >> 172.21.15.37:0/3714440278 conn(0x55d1f3b5e060 msgr2=0x55d1f3b5e440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 -- 172.21.15.37:0/3714440278 shutdown_connections 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 -- 172.21.15.37:0/3714440278 wait complete. 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.336+0000 7fe5e6873740 1 -- start start 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 1 Processor -- start 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b69bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b6a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d1f3b6f640 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d1f3a4f290 con 0x55d1f3b61af0 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d1f3b6f7c0 con 0x55d1f3b62410 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e6873740 10 client.0 _ll_register_callbacks cb 0x55d1f3a80140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b69bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e1072640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b6a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b69bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:40072/0 (socket says 172.21.15.37:40072) 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e1873640 1 -- 172.21.15.37:0/3647112192 learned_addr learned my addr 172.21.15.37:0/3647112192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.337+0000 7fe5e1873640 1 -- 172.21.15.37:0/3647112192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 msgr2=0x55d1f3b6a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b6a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e1873640 1 -- 172.21.15.37:0/3647112192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 msgr2=0x55d1f3b6a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e1873640 1 -- 172.21.15.37:0/3647112192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f37d9210 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b69bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80b9080 tx=0x7fe5d80b8320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9354ec9d8ba360b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3647112192 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8049070 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3647112192 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fe5d80b8d20 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d80bcdb0 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3647112192 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe5d80c5320 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e6873740 10 client.26483 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1f38a5620 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.338+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3647112192 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d80a9030 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3647112192 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe5d80bc910 con 0x55d1f3b61af0 2024-09-19T06:47:28.346 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e6873740 10 client.26483 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e6873740 10 client.26483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e6873740 20 client.26483 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3b7c7f0 0x55d1f3b7ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e6873740 4 client.26483 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1f3ad3a20 con 0x55d1f3b7c7f0 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3b7c7f0 0x55d1f3b7ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5c57fa640 10 client.26483 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.339+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3b7c7f0 0x55d1f3b7ebe0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1916ec64fb985f95 server_cookie=5628bf4b41ac4d4d in_seq=0 out_seq=0 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.340+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3647112192 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d1f3ad3a20 con 0x55d1f3b7c7f0 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.340+0000 7fe5c57fa640 10 client.26483 handle_command_reply: tid=0 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.340+0000 7fe5e6873740 1 client.26483 shutdown 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 msgr2=0x55d1f3b69bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.347 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b69bb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80b9080 tx=0x7fe5d80b8320 comp rx=0 tx=0).stop 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 shutdown_connections 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3b7c7f0 0x55d1f3b7ebe0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b6a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b6a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3647112192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b69bb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5c57fa640 0 client.26483 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 >> 172.21.15.37:0/3647112192 conn(0x55d1f3b5e060 msgr2=0x55d1f3b63dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 shutdown_connections 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 1 -- 172.21.15.37:0/3647112192 wait complete. 2024-09-19T06:47:28.348 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.341+0000 7fe5e6873740 20 client.26483 trim_cache size 0 max 16384 2024-09-19T06:47:28.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.355+0000 7fe5e6873740 1 Processor -- start 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e6873740 1 -- start start 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b6a460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b84430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b84970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e6873740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1b2290 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e6873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1afb60 con 0x55d1f3b61af0 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e6873740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1ad9b0 con 0x55d1f3b62410 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b84970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b84970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49614/0 (socket says 172.21.15.37:49614) 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 -- 172.21.15.37:0/3545018600 learned_addr learned my addr 172.21.15.37:0/3545018600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b6a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b84430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 -- 172.21.15.37:0/3545018600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 msgr2=0x55d1f3b84430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b84430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 -- 172.21.15.37:0/3545018600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 msgr2=0x55d1f3b6a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b6a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 -- 172.21.15.37:0/3545018600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1f3b5de90 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b84430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.356+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b84970 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4012050 tx=0x7fe5d4009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e3f8e94a462572 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.357+0000 7fe5c77fe640 1 -- 172.21.15.37:0/3545018600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d400bd30 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.357+0000 7fe5c77fe640 1 -- 172.21.15.37:0/3545018600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d4028070 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.357+0000 7fe5c77fe640 1 -- 172.21.15.37:0/3545018600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40236d0 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.357+0000 7fe5e6873740 1 -- 172.21.15.37:0/3545018600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 msgr2=0x55d1f3b84970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.362 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.357+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b84970 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4012050 tx=0x7fe5d4009a70 comp rx=0 tx=0).stop 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.357+0000 7fe5e6873740 1 -- 172.21.15.37:0/3545018600 shutdown_connections 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.357+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b84970 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b84430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3545018600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b61af0 0x55d1f3b6a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 -- 172.21.15.37:0/3545018600 >> 172.21.15.37:0/3545018600 conn(0x55d1f3b5e060 msgr2=0x55d1f3b6bc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 -- 172.21.15.37:0/3545018600 shutdown_connections 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 -- 172.21.15.37:0/3545018600 wait complete. 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 -- start start 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 Processor -- start 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b61af0 0x55d1f3b66d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b67290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b677d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1ae020 con 0x55d1f3b61af0 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1af490 con 0x55d1f3b6ccb0 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.358+0000 7fe5e6873740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe5dc1ae6f0 con 0x55d1f3b62410 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e6873740 10 client.0 _ll_register_callbacks cb 0x7fe5d40099c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b61af0 0x55d1f3b66d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b61af0 0x55d1f3b66d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49628/0 (socket says 172.21.15.37:49628) 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e2074640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b677d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 -- 172.21.15.37:0/3376875004 learned_addr learned my addr 172.21.15.37:0/3376875004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b67290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 -- 172.21.15.37:0/3376875004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 msgr2=0x55d1f3b67290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b67290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 -- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b6ccb0 msgr2=0x55d1f3b677d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b677d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 -- 172.21.15.37:0/3376875004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1f3be6560 con 0x55d1f3b61af0 2024-09-19T06:47:28.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e2074640 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e1873640 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b61af0 0x55d1f3b66d50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80bcc80 tx=0x7fe5d802a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9375bbc1e10566 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5c57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3376875004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5d8049040 con 0x55d1f3b61af0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3376875004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fe5d80bbb00 con 0x55d1f3b61af0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5d4028070 con 0x55d1f3b61af0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3376875004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fe5d8005d50 con 0x55d1f3b61af0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e6873740 10 client.17088 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.359+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55d1f3be6560 con 0x55d1f3b61af0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3376875004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5d4028070 con 0x55d1f3b61af0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3376875004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fe5d80b8400 con 0x55d1f3b61af0 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e6873740 10 client.17088 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e6873740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e6873740 20 client.17088 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:28.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3be7b20 0x55d1f3be9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e6873740 4 client.17088 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:47:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d1f3acdd70 con 0x55d1f3be7b20 2024-09-19T06:47:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3be7b20 0x55d1f3be9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5c57fa640 10 client.17088 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:28.366 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.360+0000 7fe5e1072640 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3be7b20 0x55d1f3be9f10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891679f320e8f167 server_cookie=ba92ecc09da1ff5c in_seq=0 out_seq=0 2024-09-19T06:47:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5c57fa640 1 -- 172.21.15.37:0/3376875004 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d1f3acdd70 con 0x55d1f3be7b20 2024-09-19T06:47:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5c57fa640 10 client.17088 handle_command_reply: tid=0 2024-09-19T06:47:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 client.17088 shutdown 2024-09-19T06:47:28.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b61af0 msgr2=0x55d1f3b66d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b61af0 0x55d1f3b66d50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80bcc80 tx=0x7fe5d802a480 comp rx=0 tx=0).stop 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 shutdown_connections 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d1f3be7b20 0x55d1f3be9f10 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d1f3b6ccb0 0x55d1f3b677d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d1f3b62410 0x55d1f3b67290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 --2- 172.21.15.37:0/3376875004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d1f3b61af0 0x55d1f3b66d50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5c57fa640 0 client.17088 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 >> 172.21.15.37:0/3376875004 conn(0x55d1f3b5e060 msgr2=0x55d1f3b634c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 shutdown_connections 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.417+0000 7fe5e6873740 1 -- 172.21.15.37:0/3376875004 wait complete. 2024-09-19T06:47:28.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e6873740 20 client.17088 trim_cache size 0 max 16384 2024-09-19T06:47:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:47:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:47:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:47:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:47:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5c80a34e0 msgr2=0x7fe5c80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5c80a34e0 0x7fe5c80a59a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40094d0 tx=0x7fe5d4004ab0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 msgr2=0x7fe5dc1008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc1008d0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe5d8004770 tx=0x7fe5d8003c10 comp rx=0 tx=0).stop 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 shutdown_connections 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe5dc10c0d0 0x7fe5dc101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe5dc1035a0 0x7fe5dc100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5c80a34e0 0x7fe5c80a59a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 --2- 172.21.15.37:0/3950804922 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5dc102c80 0x7fe5dc1008d0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 >> 172.21.15.37:0/3950804922 conn(0x7fe5dc077990 msgr2=0x7fe5dc0fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 shutdown_connections 2024-09-19T06:47:28.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:28.418+0000 7fe5e3ad7640 1 -- 172.21.15.37:0/3950804922 wait complete. 2024-09-19T06:47:28.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:28 smithi096 ceph-mon[35004]: pgmap v174: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 KiB/s wr, 140 op/s 2024-09-19T06:47:28.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:28 smithi037 ceph-mon[31211]: pgmap v174: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 KiB/s wr, 140 op/s 2024-09-19T06:47:28.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:28 smithi151 ceph-mon[35236]: pgmap v174: 97 pgs: 97 active+clean; 36 MiB data, 980 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 870 KiB/s wr, 140 op/s 2024-09-19T06:47:29.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 344980 5.56 MB/sec execute 116 sec latency 25.465 ms 2024-09-19T06:47:30.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 347995 5.67 MB/sec execute 117 sec latency 12.535 ms 2024-09-19T06:47:30.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:30 smithi096 ceph-mon[35004]: pgmap v175: 97 pgs: 97 active+clean; 39 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2024-09-19T06:47:30.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:30 smithi037 ceph-mon[31211]: pgmap v175: 97 pgs: 97 active+clean; 39 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2024-09-19T06:47:30.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:30 smithi151 ceph-mon[35236]: pgmap v175: 97 pgs: 97 active+clean; 39 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 1.1 MiB/s wr, 162 op/s 2024-09-19T06:47:31.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 350959 5.75 MB/sec execute 118 sec latency 10.917 ms 2024-09-19T06:47:32.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 353004 5.78 MB/sec execute 119 sec latency 15.736 ms 2024-09-19T06:47:32.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:32 smithi096 ceph-mon[35004]: pgmap v176: 97 pgs: 97 active+clean; 39 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 236 KiB/s wr, 27 op/s 2024-09-19T06:47:32.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:32 smithi037 ceph-mon[31211]: pgmap v176: 97 pgs: 97 active+clean; 39 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 236 KiB/s wr, 27 op/s 2024-09-19T06:47:32.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:32 smithi151 ceph-mon[35236]: pgmap v176: 97 pgs: 97 active+clean; 39 MiB data, 981 MiB used, 1.0 TiB / 1.0 TiB avail; 469 KiB/s rd, 236 KiB/s wr, 27 op/s 2024-09-19T06:47:33.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 355627 5.86 MB/sec execute 120 sec latency 22.004 ms 2024-09-19T06:47:34.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 358573 5.96 MB/sec execute 121 sec latency 10.976 ms 2024-09-19T06:47:34.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:34 smithi096 ceph-mon[35004]: pgmap v177: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-09-19T06:47:34.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:34 smithi037 ceph-mon[31211]: pgmap v177: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-09-19T06:47:34.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:34 smithi151 ceph-mon[35236]: pgmap v177: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 162 op/s 2024-09-19T06:47:35.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 361228 6.01 MB/sec execute 122 sec latency 18.586 ms 2024-09-19T06:47:36.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 363483 6.06 MB/sec execute 123 sec latency 15.708 ms 2024-09-19T06:47:36.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:36 smithi096 ceph-mon[35004]: pgmap v178: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-19T06:47:36.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:36 smithi037 ceph-mon[31211]: pgmap v178: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-19T06:47:36.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:36 smithi151 ceph-mon[35236]: pgmap v178: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-19T06:47:37.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 366038 6.13 MB/sec execute 124 sec latency 11.338 ms 2024-09-19T06:47:38.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 368271 6.18 MB/sec execute 125 sec latency 16.363 ms 2024-09-19T06:47:38.414 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:38 smithi151 ceph-mon[35236]: pgmap v179: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-19T06:47:38.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:38 smithi096 ceph-mon[35004]: pgmap v179: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-19T06:47:38.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:38 smithi037 ceph-mon[31211]: pgmap v179: 97 pgs: 97 active+clean; 44 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.2 MiB/s wr, 156 op/s 2024-09-19T06:47:39.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 370470 6.22 MB/sec execute 126 sec latency 15.907 ms 2024-09-19T06:47:40.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 373093 6.28 MB/sec execute 127 sec latency 14.778 ms 2024-09-19T06:47:40.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:40 smithi096 ceph-mon[35004]: pgmap v180: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 248 op/s 2024-09-19T06:47:40.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:40 smithi037 ceph-mon[31211]: pgmap v180: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 248 op/s 2024-09-19T06:47:40.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:40 smithi151 ceph-mon[35236]: pgmap v180: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.1 MiB/s wr, 248 op/s 2024-09-19T06:47:41.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 376203 6.38 MB/sec execute 128 sec latency 12.157 ms 2024-09-19T06:47:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 -- 172.21.15.37:0/1452604155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 msgr2=0x7f0cac100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/1452604155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac100e10 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0c98009930 tx=0x7f0c9802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 -- 172.21.15.37:0/1452604155 shutdown_connections 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/1452604155 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac101350 0x7f0cac10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/1452604155 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac100e10 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/1452604155 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cac1069c0 0x7f0cac106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 -- 172.21.15.37:0/1452604155 >> 172.21.15.37:0/1452604155 conn(0x7f0cac077990 msgr2=0x7f0cac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 -- 172.21.15.37:0/1452604155 shutdown_connections 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.510+0000 7f0cb2f66640 1 -- 172.21.15.37:0/1452604155 wait complete. 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 Processor -- start 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 -- start start 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cac101350 0x7f0cac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac1069c0 0x7f0cac19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac110e50 con 0x7f0cac1009b0 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac110cd0 con 0x7f0cac1069c0 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb2f66640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac110fd0 con 0x7f0cac101350 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44800/0 (socket says 172.21.15.37:44800) 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2778420830 learned_addr learned my addr 172.21.15.37:0/2778420830 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cb1503640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac1069c0 0x7f0cac19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.511+0000 7f0cabfff640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cac101350 0x7f0cac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb1503640 1 -- 172.21.15.37:0/2778420830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cac101350 msgr2=0x7f0cac198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb1503640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cac101350 0x7f0cac198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb1503640 1 -- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 msgr2=0x7f0cac198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb1503640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb1503640 1 -- 172.21.15.37:0/2778420830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cac19d400 con 0x7f0cac1069c0 2024-09-19T06:47:41.516 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb1503640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac1069c0 0x7f0cac19ccd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009fd0 tx=0x7f0ca400ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0ca9ffb640 1 -- 172.21.15.37:0/2778420830 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca400ce20 con 0x7f0cac1069c0 2024-09-19T06:47:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0ca9ffb640 1 -- 172.21.15.37:0/2778420830 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca4005020 con 0x7f0cac1069c0 2024-09-19T06:47:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cac19d6f0 con 0x7f0cac1069c0 2024-09-19T06:47:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0ca9ffb640 1 -- 172.21.15.37:0/2778420830 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4010720 con 0x7f0cac1069c0 2024-09-19T06:47:41.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.512+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cac1a4f80 con 0x7f0cac1069c0 2024-09-19T06:47:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0ca9ffb640 1 -- 172.21.15.37:0/2778420830 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0ca4005540 con 0x7f0cac1069c0 2024-09-19T06:47:41.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 Processor -- start 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 -- start start 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d339ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d344770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3470a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f6d29e2f0 con 0x561f6d344cb0 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561f6d246270 con 0x561f6d33a410 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.515+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561f6d29a2b0 con 0x561f6d339af0 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d344770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0ca9ffb640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c9c0a3460 0x7f0c9c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d344770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44036/0 (socket says 172.21.15.37:44036) 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cabfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d339ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2881013553 learned_addr learned my addr 172.21.15.37:0/2881013553 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cabfff640 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb1503640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c9c0a3460 0x7f0c9c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2881013553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 msgr2=0x561f6d339ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d339ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2881013553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 msgr2=0x561f6d3470a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3470a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2881013553 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6d245340 con 0x561f6d33a410 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0ca9ffb640 1 -- 172.21.15.37:0/2778420830 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0ca40d0490 con 0x7f0cac1069c0 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cabfff640 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d339ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d344770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00115f0 tx=0x7f0ca0011ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fa314efb8db4fe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0cb1503640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c9c0a3460 0x7f0c9c0a5920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ca409c410 tx=0x7f0ca40cb040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0c96ffd640 1 -- 172.21.15.37:0/2881013553 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca001e070 con 0x561f6d33a410 2024-09-19T06:47:41.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0c96ffd640 1 -- 172.21.15.37:0/2881013553 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca000fdd0 con 0x561f6d33a410 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.516+0000 7f0c96ffd640 1 -- 172.21.15.37:0/2881013553 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca000e3b0 con 0x561f6d33a410 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2881013553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 msgr2=0x561f6d344770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d344770 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0ca00115f0 tx=0x7f0ca0011ab0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2881013553 shutdown_connections 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3470a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d344770 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2881013553 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d339ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2881013553 >> 172.21.15.37:0/2881013553 conn(0x561f6d336060 msgr2=0x561f6d336440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2881013553 shutdown_connections 2024-09-19T06:47:41.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2881013553 wait complete. 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.517+0000 7f0cb5de0740 1 -- start start 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 1 Processor -- start 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d341c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d342190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3426d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561f6d347880 con 0x561f6d344cb0 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561f6d227290 con 0x561f6d33a410 2024-09-19T06:47:41.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561f6d347a00 con 0x561f6d339af0 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb5de0740 10 client.0 _ll_register_callbacks cb 0x561f6d258140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d342190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cabfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d341c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d342190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44052/0 (socket says 172.21.15.37:44052) 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 -- 172.21.15.37:0/1494662515 learned_addr learned my addr 172.21.15.37:0/1494662515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cabfff640 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 -- 172.21.15.37:0/1494662515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 msgr2=0x561f6d341c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d341c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 -- 172.21.15.37:0/1494662515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 msgr2=0x561f6d3426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.518+0000 7f0cb0d02640 1 -- 172.21.15.37:0/1494662515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6cfb1210 con 0x561f6d33a410 2024-09-19T06:47:41.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d342190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0011fd0 tx=0x7f0ca0019640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d4e6288b51506ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0c8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0c8bfff640 1 -- 172.21.15.37:0/1494662515 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca001e050 con 0x561f6d33a410 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0c8bfff640 1 -- 172.21.15.37:0/1494662515 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0ca000fad0 con 0x561f6d33a410 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ca000fdd0 con 0x561f6d33a410 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0c8bfff640 1 -- 172.21.15.37:0/1494662515 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ca0033050 con 0x561f6d33a410 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 10 client.26501 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f6d07d620 con 0x561f6d33a410 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0c8bfff640 1 -- 172.21.15.37:0/1494662515 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0ca000fdd0 con 0x561f6d33a410 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0c8bfff640 1 -- 172.21.15.37:0/1494662515 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0ca0004880 con 0x561f6d33a410 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 10 client.26501 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 10 client.26501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:41.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 20 client.26501 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d354c00 0x561f6d356ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 4 client.26501 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.519+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f6d2aba20 con 0x561f6d354c00 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.520+0000 7f0cabfff640 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d354c00 0x561f6d356ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.520+0000 7f0c8bfff640 10 client.26501 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.520+0000 7f0cabfff640 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d354c00 0x561f6d356ff0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b23a4b6f5989592 server_cookie=b6bfa918e6f2fed in_seq=0 out_seq=0 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0c8bfff640 1 -- 172.21.15.37:0/1494662515 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f6d2aba20 con 0x561f6d354c00 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0c8bfff640 10 client.26501 handle_command_reply: tid=0 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 client.26501 shutdown 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 msgr2=0x561f6d342190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d342190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0011fd0 tx=0x7f0ca0019640 comp rx=0 tx=0).stop 2024-09-19T06:47:41.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 shutdown_connections 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d354c00 0x561f6d356ff0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d3426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0c8bfff640 0 client.26501 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d342190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/1494662515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d341c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 >> 172.21.15.37:0/1494662515 conn(0x561f6d336060 msgr2=0x561f6d33bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 shutdown_connections 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 1 -- 172.21.15.37:0/1494662515 wait complete. 2024-09-19T06:47:41.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.521+0000 7f0cb5de0740 20 client.26501 trim_cache size 0 max 16384 2024-09-19T06:47:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 Processor -- start 2024-09-19T06:47:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 -- start start 2024-09-19T06:47:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d339af0 0x561f6d342500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d33a410 0x561f6d33bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d344cb0 0x561f6d35c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1b0090 con 0x561f6d339af0 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ad960 con 0x561f6d344cb0 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.537+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ab7b0 con 0x561f6d33a410 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d339af0 0x561f6d342500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d339af0 0x561f6d342500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44816/0 (socket says 172.21.15.37:44816) 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 -- 172.21.15.37:0/631927950 learned_addr learned my addr 172.21.15.37:0/631927950 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d344cb0 0x561f6d35c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d33a410 0x561f6d33bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 -- 172.21.15.37:0/631927950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d33a410 msgr2=0x561f6d33bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d33a410 0x561f6d33bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 -- 172.21.15.37:0/631927950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d344cb0 msgr2=0x561f6d35c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d344cb0 0x561f6d35c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 -- 172.21.15.37:0/631927950 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6d339920 con 0x561f6d339af0 2024-09-19T06:47:41.541 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0cabfff640 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d339af0 0x561f6d342500 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0c9802a9e0 tx=0x7f0c9802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b83e584d2c6bd0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0c967fc640 1 -- 172.21.15.37:0/631927950 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98035040 con 0x561f6d339af0 2024-09-19T06:47:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0c967fc640 1 -- 172.21.15.37:0/631927950 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c98005410 con 0x561f6d339af0 2024-09-19T06:47:41.542 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.538+0000 7f0c967fc640 1 -- 172.21.15.37:0/631927950 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c98031770 con 0x561f6d339af0 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 -- 172.21.15.37:0/631927950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d339af0 msgr2=0x561f6d342500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d339af0 0x561f6d342500 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f0c9802a9e0 tx=0x7f0c9802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 -- 172.21.15.37:0/631927950 shutdown_connections 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d344cb0 0x561f6d35c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d33a410 0x561f6d33bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/631927950 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d339af0 0x561f6d342500 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 -- 172.21.15.37:0/631927950 >> 172.21.15.37:0/631927950 conn(0x561f6d336060 msgr2=0x561f6d33b550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:41.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 -- 172.21.15.37:0/631927950 shutdown_connections 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 -- 172.21.15.37:0/631927950 wait complete. 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.539+0000 7f0cb5de0740 1 -- start start 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 1 Processor -- start 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d34d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d33e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d33ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1abe20 con 0x561f6d344cb0 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ad290 con 0x561f6d33a410 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0cac1ac4f0 con 0x561f6d339af0 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb5de0740 10 client.0 _ll_register_callbacks cb 0x7f0ca001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d33e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cabfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d34d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d33e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44072/0 (socket says 172.21.15.37:44072) 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2079691846 learned_addr learned my addr 172.21.15.37:0/2079691846 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cabfff640 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d34d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:55800/0 (socket says 172.21.15.37:55800) 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cabfff640 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d33ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2079691846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 msgr2=0x561f6d34d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d34d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2079691846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 msgr2=0x561f6d33ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d33ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.540+0000 7f0cb0d02640 1 -- 172.21.15.37:0/2079691846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f6d33f180 con 0x561f6d33a410 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb0d02640 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d33e6d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009810 tx=0x7f0ca0019960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7e64633be8054fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0c8bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0c8bfff640 1 -- 172.21.15.37:0/2079691846 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ca003b060 con 0x561f6d33a410 2024-09-19T06:47:41.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0c98005410 con 0x561f6d33a410 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0c8bfff640 1 -- 172.21.15.37:0/2079691846 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f0ca0004680 con 0x561f6d33a410 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0c8bfff640 1 -- 172.21.15.37:0/2079691846 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f0ca001e040 con 0x561f6d33a410 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 10 client.26507 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561f6d3bf440 con 0x561f6d33a410 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0c8bfff640 1 -- 172.21.15.37:0/2079691846 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0c98005410 con 0x561f6d33a410 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0c8bfff640 1 -- 172.21.15.37:0/2079691846 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f0ca0004f20 con 0x561f6d33a410 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 10 client.26507 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 10 client.26507 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 20 client.26507 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d3bfa40 0x561f6d3c1e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 4 client.26507 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.541+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561f6d2a5d70 con 0x561f6d3bfa40 2024-09-19T06:47:41.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.542+0000 7f0cabfff640 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d3bfa40 0x561f6d3c1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.542+0000 7f0c8bfff640 10 client.26507 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:41.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.542+0000 7f0cabfff640 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d3bfa40 0x561f6d3c1e60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9b82b6e8d7978d8 server_cookie=61e0dd795c569c9e in_seq=0 out_seq=0 2024-09-19T06:47:41.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0c8bfff640 1 -- 172.21.15.37:0/2079691846 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561f6d2a5d70 con 0x561f6d3bfa40 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0c8bfff640 10 client.26507 handle_command_reply: tid=0 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 client.26507 shutdown 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 msgr2=0x561f6d33e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d33e6d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009810 tx=0x7f0ca0019960 comp rx=0 tx=0).stop 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 shutdown_connections 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561f6d3bfa40 0x561f6d3c1e60 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561f6d344cb0 0x561f6d33ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0c8bfff640 0 client.26507 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561f6d33a410 0x561f6d33e6d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 --2- 172.21.15.37:0/2079691846 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561f6d339af0 0x561f6d34d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.687+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 >> 172.21.15.37:0/2079691846 conn(0x561f6d336060 msgr2=0x561f6d3351c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:41.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.688+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 shutdown_connections 2024-09-19T06:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.688+0000 7f0cb5de0740 1 -- 172.21.15.37:0/2079691846 wait complete. 2024-09-19T06:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.688+0000 7f0cb5de0740 20 client.26507 trim_cache size 0 max 16384 2024-09-19T06:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:47:41.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c9c0a3460 msgr2=0x7f0c9c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c9c0a3460 0x7f0c9c0a5920 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0ca409c410 tx=0x7f0ca40cb040 comp rx=0 tx=0).stop 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac1069c0 msgr2=0x7f0cac19ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac1069c0 0x7f0cac19ccd0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009fd0 tx=0x7f0ca400ef10 comp rx=0 tx=0).stop 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 shutdown_connections 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0cac1069c0 0x7f0cac19ccd0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0cac101350 0x7f0cac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0c9c0a3460 0x7f0c9c0a5920 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 --2- 172.21.15.37:0/2778420830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0cac1009b0 0x7f0cac198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 >> 172.21.15.37:0/2778420830 conn(0x7f0cac077990 msgr2=0x7f0cac069b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 shutdown_connections 2024-09-19T06:47:41.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:41.689+0000 7f0cb2f66640 1 -- 172.21.15.37:0/2778420830 wait complete. 2024-09-19T06:47:42.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 378274 6.38 MB/sec execute 129 sec latency 47.274 ms 2024-09-19T06:47:42.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:42 smithi096 ceph-mon[35004]: pgmap v181: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-09-19T06:47:42.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:42 smithi037 ceph-mon[31211]: pgmap v181: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-09-19T06:47:42.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:42 smithi151 ceph-mon[35236]: pgmap v181: 97 pgs: 97 active+clean; 54 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 1.9 MiB/s wr, 227 op/s 2024-09-19T06:47:43.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 380603 6.44 MB/sec execute 130 sec latency 12.064 ms 2024-09-19T06:47:44.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 383492 6.53 MB/sec execute 131 sec latency 19.297 ms 2024-09-19T06:47:44.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:44 smithi096 ceph-mon[35004]: pgmap v182: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 441 op/s 2024-09-19T06:47:44.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:44 smithi037 ceph-mon[31211]: pgmap v182: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 441 op/s 2024-09-19T06:47:44.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:44 smithi151 ceph-mon[35236]: pgmap v182: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 3.1 MiB/s wr, 441 op/s 2024-09-19T06:47:45.208 INFO:tasks.workunit.client.0.smithi037.stdout: 1 386150 6.57 MB/sec execute 132 sec latency 17.435 ms 2024-09-19T06:47:45.435 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:47:45.697 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 -- 172.21.15.37:0/2950353152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7570102d30 msgr2=0x7f7570103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 --2- 172.21.15.37:0/2950353152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7570102d30 0x7f7570103110 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7560009cd0 tx=0x7f756002a910 comp rx=0 tx=0).stop 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 -- 172.21.15.37:0/2950353152 shutdown_connections 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 --2- 172.21.15.37:0/2950353152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75701082a0 0x7f757010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 --2- 172.21.15.37:0/2950353152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75701036e0 0x7f7570107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 --2- 172.21.15.37:0/2950353152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7570102d30 0x7f7570103110 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 -- 172.21.15.37:0/2950353152 >> 172.21.15.37:0/2950353152 conn(0x7f75700fc7d0 msgr2=0x7f75700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 -- 172.21.15.37:0/2950353152 shutdown_connections 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.175+0000 7f75758f0640 1 -- 172.21.15.37:0/2950353152 wait complete. 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 Processor -- start 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 -- start start 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570102d30 0x7f7570198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75701036e0 0x7f7570198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75701082a0 0x7f757019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75701120e0 con 0x7f75701036e0 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7570111f60 con 0x7f7570102d30 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f75758f0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7570112260 con 0x7f75701082a0 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f756effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570102d30 0x7f7570198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f756e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75701036e0 0x7f7570198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75701082a0 0x7f757019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f756effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570102d30 0x7f7570198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44104/0 (socket says 172.21.15.37:44104) 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.176+0000 7f756effd640 1 -- 172.21.15.37:0/3065784872 learned_addr learned my addr 172.21.15.37:0/3065784872 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f756effd640 1 -- 172.21.15.37:0/3065784872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75701082a0 msgr2=0x7f757019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f756effd640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75701082a0 0x7f757019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f756effd640 1 -- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75701036e0 msgr2=0x7f7570198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f756effd640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75701036e0 0x7f7570198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f756effd640 1 -- 172.21.15.37:0/3065784872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757019d5d0 con 0x7f7570102d30 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f756effd640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570102d30 0x7f7570198620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7560005e30 tx=0x7f7560003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f75748ee640 1 -- 172.21.15.37:0/3065784872 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7560031660 con 0x7f7570102d30 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757019d860 con 0x7f7570102d30 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f75748ee640 1 -- 172.21.15.37:0/3065784872 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7560031800 con 0x7f7570102d30 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f75748ee640 1 -- 172.21.15.37:0/3065784872 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756002fb30 con 0x7f7570102d30 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.177+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75701a51a0 con 0x7f7570102d30 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 Processor -- start 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f75748ee640 1 -- 172.21.15.37:0/3065784872 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f75600045b0 con 0x7f7570102d30 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 -- start start 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8d4a0 0x560e33b8d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8ddc0 0x560e33b98140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e33af32e0 con 0x560e33b8d4a0 2024-09-19T06:47:46.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e33a9ace0 con 0x560e33b98680 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f7577cc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e33aef2a0 con 0x560e33b8ddc0 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8d4a0 0x560e33b8d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8d4a0 0x560e33b8d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44846/0 (socket says 172.21.15.37:44846) 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 -- 172.21.15.37:0/2291572726 learned_addr learned my addr 172.21.15.37:0/2291572726 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756e7fc640 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756effd640 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8ddc0 0x560e33b98140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 -- 172.21.15.37:0/2291572726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8ddc0 msgr2=0x560e33b98140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8ddc0 0x560e33b98140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 -- 172.21.15.37:0/2291572726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 msgr2=0x560e33b9aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b9aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.180+0000 7f756f7fe640 1 -- 172.21.15.37:0/2291572726 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e33a7f440 con 0x560e33b8d4a0 2024-09-19T06:47:46.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.181+0000 7f756f7fe640 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8d4a0 0x560e33b8d880 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7568009ca0 tx=0x7f7568011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a0101c1085c7b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.181+0000 7f75748ee640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f755c0a33b0 0x7f755c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.181+0000 7f7565ffb640 1 -- 172.21.15.37:0/2291572726 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756801e070 con 0x560e33b8d4a0 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.181+0000 7f756effd640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f755c0a33b0 0x7f755c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.181+0000 7f7565ffb640 1 -- 172.21.15.37:0/2291572726 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7568009e50 con 0x560e33b8d4a0 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.181+0000 7f7565ffb640 1 -- 172.21.15.37:0/2291572726 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756800ea80 con 0x560e33b8d4a0 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.181+0000 7f75748ee640 1 -- 172.21.15.37:0/3065784872 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f75600f0e80 con 0x7f7570102d30 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 -- 172.21.15.37:0/2291572726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8d4a0 msgr2=0x560e33b8d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f756effd640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f755c0a33b0 0x7f755c0a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75600ee790 tx=0x7f75600dd040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8d4a0 0x560e33b8d880 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7568009ca0 tx=0x7f7568011810 comp rx=0 tx=0).stop 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 -- 172.21.15.37:0/2291572726 shutdown_connections 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b9aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8ddc0 0x560e33b98140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2291572726 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8d4a0 0x560e33b8d880 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 -- 172.21.15.37:0/2291572726 >> 172.21.15.37:0/2291572726 conn(0x560e33b89a30 msgr2=0x560e33b89e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 -- 172.21.15.37:0/2291572726 shutdown_connections 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 -- 172.21.15.37:0/2291572726 wait complete. 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 -- start start 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.182+0000 7f7577cc9740 1 Processor -- start 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8d4a0 0x560e33b97f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b95220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b95760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f7577cc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e33b9b250 con 0x560e33b8ddc0 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f7577cc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e33aae050 con 0x560e33b98680 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f7577cc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e33b9b3d0 con 0x560e33b8d4a0 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f7577cc9740 10 client.0 _ll_register_callbacks cb 0x560e33aee160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b95220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b95220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44850/0 (socket says 172.21.15.37:44850) 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 -- 172.21.15.37:0/1942963741 learned_addr learned my addr 172.21.15.37:0/1942963741 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756f7fe640 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8d4a0 0x560e33b97f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756f7fe640 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b95760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 -- 172.21.15.37:0/1942963741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8d4a0 msgr2=0x560e33b97f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8d4a0 0x560e33b97f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 -- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 msgr2=0x560e33b95760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b95760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 -- 172.21.15.37:0/1942963741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e338643c0 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f756e7fc640 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b95220 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7558011570 tx=0x7f7558011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5333b82184580a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.183+0000 7f754effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f754effd640 1 -- 172.21.15.37:0/1942963741 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558013670 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7568009e50 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f754effd640 1 -- 172.21.15.37:0/1942963741 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f7558013810 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f754effd640 1 -- 172.21.15.37:0/1942963741 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f7558013a10 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 10 client.17118 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e33ba6c70 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f754effd640 1 -- 172.21.15.37:0/1942963741 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7568009e50 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f754effd640 1 -- 172.21.15.37:0/1942963741 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f75580253b0 con 0x560e33b8ddc0 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 10 client.17118 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 10 client.17118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:46.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 20 client.17118 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33ba8080 0x560e33baa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 4 client.17118 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.184+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e33b00850 con 0x560e33ba8080 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.185+0000 7f756f7fe640 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33ba8080 0x560e33baa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.185+0000 7f756f7fe640 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33ba8080 0x560e33baa4a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a8a980957de9530 server_cookie=39a85173cc56a5f9 in_seq=0 out_seq=0 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.185+0000 7f754effd640 10 client.17118 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f754effd640 1 -- 172.21.15.37:0/1942963741 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e33b00850 con 0x560e33ba8080 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f754effd640 10 client.17118 handle_command_reply: tid=0 2024-09-19T06:47:46.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 client.17118 shutdown 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 msgr2=0x560e33b95220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b95220 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7558011570 tx=0x7f7558011a30 comp rx=0 tx=0).stop 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 shutdown_connections 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33ba8080 0x560e33baa4a0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b98680 0x560e33b95760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b95220 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 --2- 172.21.15.37:0/1942963741 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b8d4a0 0x560e33b97f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 >> 172.21.15.37:0/1942963741 conn(0x560e33b89a30 msgr2=0x560e33b8f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f754effd640 0 client.17118 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:46.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 shutdown_connections 2024-09-19T06:47:46.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.186+0000 7f7577cc9740 1 -- 172.21.15.37:0/1942963741 wait complete. 2024-09-19T06:47:46.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.187+0000 7f7577cc9740 20 client.17118 trim_cache size 0 max 16384 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 Processor -- start 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 -- start start 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b97190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33bafd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75701b0260 con 0x560e33b8ddc0 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75701adb30 con 0x560e33b8d4a0 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f7577cc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75701ab980 con 0x560e33b98680 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f756e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b97190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f756e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b97190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44864/0 (socket says 172.21.15.37:44864) 2024-09-19T06:47:46.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f756e7fc640 1 -- 172.21.15.37:0/520410102 learned_addr learned my addr 172.21.15.37:0/520410102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:46.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.206+0000 7f756f7fe640 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33bafd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f756e7fc640 1 -- 172.21.15.37:0/520410102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 msgr2=0x560e33bafd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f756e7fc640 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33bafd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f756e7fc640 1 -- 172.21.15.37:0/520410102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 msgr2=0x560e33ba44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f756e7fc640 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba44b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f756e7fc640 1 -- 172.21.15.37:0/520410102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e33b894d0 con 0x560e33b8ddc0 2024-09-19T06:47:46.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f756e7fc640 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b97190 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7558025850 tx=0x7f7558009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258785d1b2a78cda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 388316 6.61 MB/sec execute 133 sec latency 39.247 ms 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f75657fa640 1 -- 172.21.15.37:0/520410102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f755802a070 con 0x560e33b8ddc0 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f75657fa640 1 -- 172.21.15.37:0/520410102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7558025e90 con 0x560e33b8ddc0 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.207+0000 7f75657fa640 1 -- 172.21.15.37:0/520410102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7558013bb0 con 0x560e33b8ddc0 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 -- 172.21.15.37:0/520410102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 msgr2=0x560e33b97190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b97190 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7558025850 tx=0x7f7558009e30 comp rx=0 tx=0).stop 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 -- 172.21.15.37:0/520410102 shutdown_connections 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33bafd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b97190 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 --2- 172.21.15.37:0/520410102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 -- 172.21.15.37:0/520410102 >> 172.21.15.37:0/520410102 conn(0x560e33b89a30 msgr2=0x560e33b96a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 -- 172.21.15.37:0/520410102 shutdown_connections 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 -- 172.21.15.37:0/520410102 wait complete. 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.208+0000 7f7577cc9740 1 -- start start 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 1 Processor -- start 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b9bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33b9c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75701abff0 con 0x560e33b8ddc0 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f75701ad460 con 0x560e33b8d4a0 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f75701ac6c0 con 0x560e33b98680 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b9bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f7577cc9740 10 client.0 _ll_register_callbacks cb 0x7f75580094f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44144/0 (socket says 172.21.15.37:44144) 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 -- 172.21.15.37:0/2709325876 learned_addr learned my addr 172.21.15.37:0/2709325876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b9bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44866/0 (socket says 172.21.15.37:44866) 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33b9c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 -- 172.21.15.37:0/2709325876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 msgr2=0x560e33b9c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33b9c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 -- 172.21.15.37:0/2709325876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 msgr2=0x560e33b9bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b9bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.209+0000 7f756f7fe640 1 -- 172.21.15.37:0/2709325876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e33b9cab0 con 0x560e33b8d4a0 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f756e7fc640 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b9bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:46.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f756f7fe640 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba0bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f756800e590 tx=0x7f7568005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75735fed7d7203df server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f754effd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f754effd640 1 -- 172.21.15.37:0/2709325876 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756801e040 con 0x560e33b8d4a0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f754effd640 1 -- 172.21.15.37:0/2709325876 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f756800e990 con 0x560e33b8d4a0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e33afad60 con 0x560e33b8d4a0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f754effd640 1 -- 172.21.15.37:0/2709325876 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f756800eb90 con 0x560e33b8d4a0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f7577cc9740 10 client.26525 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.210+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x560e33c1b3b0 con 0x560e33b8d4a0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f754effd640 1 -- 172.21.15.37:0/2709325876 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560e33afad60 con 0x560e33b8d4a0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f754effd640 1 -- 172.21.15.37:0/2709325876 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f7568024040 con 0x560e33b8d4a0 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f7577cc9740 10 client.26525 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f7577cc9740 10 client.26525 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:46.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f7577cc9740 20 client.26525 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:46.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33c1b940 0x560e33c1dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:46.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f7577cc9740 4 client.26525 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:47:46.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e33a77210 con 0x560e33c1b940 2024-09-19T06:47:46.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.211+0000 7f756e7fc640 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33c1b940 0x560e33c1dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:46.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.212+0000 7f756e7fc640 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33c1b940 0x560e33c1dd30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2afc75389cf67b39 server_cookie=7c9ba1c1a8c4d024 in_seq=0 out_seq=0 2024-09-19T06:47:46.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.212+0000 7f754effd640 10 client.26525 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f754effd640 1 -- 172.21.15.37:0/2709325876 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e33a77210 con 0x560e33c1b940 2024-09-19T06:47:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f754effd640 10 client.26525 handle_command_reply: tid=0 2024-09-19T06:47:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 client.26525 shutdown 2024-09-19T06:47:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 msgr2=0x560e33ba0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba0bd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f756800e590 tx=0x7f7568005f90 comp rx=0 tx=0).stop 2024-09-19T06:47:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 shutdown_connections 2024-09-19T06:47:46.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e33c1b940 0x560e33c1dd30 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e33b98680 0x560e33b9c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e33b8ddc0 0x560e33b9bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f754effd640 0 client.26525 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 --2- 172.21.15.37:0/2709325876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e33b8d4a0 0x560e33ba0bd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 >> 172.21.15.37:0/2709325876 conn(0x560e33b89a30 msgr2=0x560e33b972a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 shutdown_connections 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 1 -- 172.21.15.37:0/2709325876 wait complete. 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.213+0000 7f7577cc9740 20 client.26525 trim_cache size 0 max 16384 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:47:46.219 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:47:46.220 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:47:46.220 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:47:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f755c0a33b0 msgr2=0x7f755c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f755c0a33b0 0x7f755c0a5870 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75600ee790 tx=0x7f75600dd040 comp rx=0 tx=0).stop 2024-09-19T06:47:46.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570102d30 msgr2=0x7f7570198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570102d30 0x7f7570198620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7560005e30 tx=0x7f7560003f10 comp rx=0 tx=0).stop 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 shutdown_connections 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75701082a0 0x7f757019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f755c0a33b0 0x7f755c0a5870 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f75701036e0 0x7f7570198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 --2- 172.21.15.37:0/3065784872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7570102d30 0x7f7570198620 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 >> 172.21.15.37:0/3065784872 conn(0x7f75700fc7d0 msgr2=0x7f7570105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 shutdown_connections 2024-09-19T06:47:46.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:46.214+0000 7f75758f0640 1 -- 172.21.15.37:0/3065784872 wait complete. 2024-09-19T06:47:46.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:46 smithi096 ceph-mon[35004]: pgmap v183: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-09-19T06:47:46.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:46 smithi037 ceph-mon[31211]: pgmap v183: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-09-19T06:47:46.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:46 smithi151 ceph-mon[35236]: pgmap v183: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-09-19T06:47:47.025 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:47:47.025 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:47:47.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 390974 6.67 MB/sec execute 134 sec latency 12.544 ms 2024-09-19T06:47:47.337 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:47.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 -- 172.21.15.37:0/2504316025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc101380 msgr2=0x7f2acc10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 --2- 172.21.15.37:0/2504316025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc101380 0x7f2acc10dfb0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009900 tx=0x7f2ac002a410 comp rx=0 tx=0).stop 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 -- 172.21.15.37:0/2504316025 shutdown_connections 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 --2- 172.21.15.37:0/2504316025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc10e4f0 0x7f2acc110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 --2- 172.21.15.37:0/2504316025 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc101380 0x7f2acc10dfb0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 --2- 172.21.15.37:0/2504316025 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2acc1009d0 0x7f2acc100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 -- 172.21.15.37:0/2504316025 >> 172.21.15.37:0/2504316025 conn(0x7f2acc0fc7d0 msgr2=0x7f2acc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 -- 172.21.15.37:0/2504316025 shutdown_connections 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.859+0000 7f2ad1959640 1 -- 172.21.15.37:0/2504316025 wait complete. 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 Processor -- start 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 -- start start 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2acc1009d0 0x7f2acc1a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:47.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc101380 0x7f2acc1a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc10e4f0 0x7f2acc19a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc114260 con 0x7f2acc10e4f0 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc1140e0 con 0x7f2acc101380 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2ad1959640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2acc1143e0 con 0x7f2acc1009d0 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2acb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc10e4f0 0x7f2acc19a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2aca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc101380 0x7f2acc1a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2acb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc10e4f0 0x7f2acc19a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44882/0 (socket says 172.21.15.37:44882) 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2aca7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc101380 0x7f2acc1a0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44152/0 (socket says 172.21.15.37:44152) 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2acb7fe640 1 -- 172.21.15.37:0/1892031935 learned_addr learned my addr 172.21.15.37:0/1892031935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.860+0000 7f2acaffd640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2acc1009d0 0x7f2acc1a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2acb7fe640 1 -- 172.21.15.37:0/1892031935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2acc1009d0 msgr2=0x7f2acc1a03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2acb7fe640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2acc1009d0 0x7f2acc1a03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:47.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2acb7fe640 1 -- 172.21.15.37:0/1892031935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc101380 msgr2=0x7f2acc1a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2acb7fe640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc101380 0x7f2acc1a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2acb7fe640 1 -- 172.21.15.37:0/1892031935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2acc19acc0 con 0x7f2acc10e4f0 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2aca7fc640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc101380 0x7f2acc1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2acb7fe640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc10e4f0 0x7f2acc19a590 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400eb40 tx=0x7f2ac400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2ad0957640 1 -- 172.21.15.37:0/1892031935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4004b20 con 0x7f2acc10e4f0 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2acc19afb0 con 0x7f2acc10e4f0 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2ad0957640 1 -- 172.21.15.37:0/1892031935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2ac4004cc0 con 0x7f2acc10e4f0 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2acc1a72a0 con 0x7f2acc10e4f0 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.861+0000 7f2ad0957640 1 -- 172.21.15.37:0/1892031935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ac4010850 con 0x7f2acc10e4f0 2024-09-19T06:47:47.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.862+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2acc06a7d0 con 0x7f2acc10e4f0 2024-09-19T06:47:47.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.864+0000 7f2ad0957640 1 -- 172.21.15.37:0/1892031935 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2ac4005000 con 0x7f2acc10e4f0 2024-09-19T06:47:47.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.865+0000 7f2ad0957640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2ab40a33b0 0x7f2ab40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:47.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.865+0000 7f2acaffd640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2ab40a33b0 0x7f2ab40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:47.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.865+0000 7f2ad0957640 1 -- 172.21.15.37:0/1892031935 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f2ac40d0420 con 0x7f2acc10e4f0 2024-09-19T06:47:47.869 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.866+0000 7f2acaffd640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2ab40a33b0 0x7f2ab40a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005fd0 tx=0x7f2abc00b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:47.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:47.868+0000 7f2ad0957640 1 -- 172.21.15.37:0/1892031935 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2ac409ce00 con 0x7f2acc10e4f0 2024-09-19T06:47:48.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.097+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2acc109680 con 0x7f2acc10e4f0 2024-09-19T06:47:48.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.098+0000 7f2ad0957640 1 -- 172.21.15.37:0/1892031935 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) ==== 94+0+11903 (secure 0 0 0) 0x7f2ac409c700 con 0x7f2acc10e4f0 2024-09-19T06:47:48.100 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:47:48.100 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":31,"btime":"2024-09-19T06:45:42:465959+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":31,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:45:41.617479+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:47:48.100 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 31 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2ab40a33b0 msgr2=0x7f2ab40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2ab40a33b0 0x7f2ab40a5870 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2abc005fd0 tx=0x7f2abc00b390 comp rx=0 tx=0).stop 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc10e4f0 msgr2=0x7f2acc19a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc10e4f0 0x7f2acc19a590 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f2ac400eb40 tx=0x7f2ac400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 shutdown_connections 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2ab40a33b0 0x7f2ab40a5870 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2acc10e4f0 0x7f2acc19a590 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2acc101380 0x7f2acc1a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 --2- 172.21.15.37:0/1892031935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2acc1009d0 0x7f2acc1a03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:48.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 >> 172.21.15.37:0/1892031935 conn(0x7f2acc0fc7d0 msgr2=0x7f2acc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:48.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 shutdown_connections 2024-09-19T06:47:48.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:48.100+0000 7f2ad1959640 1 -- 172.21.15.37:0/1892031935 wait complete. 2024-09-19T06:47:48.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 393555 6.72 MB/sec execute 135 sec latency 68.059 ms 2024-09-19T06:47:48.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:48 smithi037 ceph-mon[31211]: pgmap v184: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-09-19T06:47:48.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:48 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1892031935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:48.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:48 smithi096 ceph-mon[35004]: pgmap v184: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-09-19T06:47:48.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:48 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1892031935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:48.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:48 smithi151 ceph-mon[35236]: pgmap v184: 97 pgs: 97 active+clean; 45 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.1 MiB/s wr, 306 op/s 2024-09-19T06:47:48.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:48 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1892031935' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:47:48.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:47:49.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 395830 6.75 MB/sec execute 136 sec latency 15.547 ms 2024-09-19T06:47:49.250 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:49.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 -- 172.21.15.37:0/2794704985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1036e0 msgr2=0x7fee8c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 --2- 172.21.15.37:0/2794704985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1036e0 0x7fee8c107b90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009930 tx=0x7fee7c02a420 comp rx=0 tx=0).stop 2024-09-19T06:47:49.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 -- 172.21.15.37:0/2794704985 shutdown_connections 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 --2- 172.21.15.37:0/2794704985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c1082a0 0x7fee8c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 --2- 172.21.15.37:0/2794704985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1036e0 0x7fee8c107b90 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 --2- 172.21.15.37:0/2794704985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 0x7fee8c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 -- 172.21.15.37:0/2794704985 >> 172.21.15.37:0/2794704985 conn(0x7fee8c0fc7d0 msgr2=0x7fee8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 -- 172.21.15.37:0/2794704985 shutdown_connections 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.720+0000 7fee93d93640 1 -- 172.21.15.37:0/2794704985 wait complete. 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 Processor -- start 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 -- start start 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 0x7fee8c1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c1036e0 0x7fee8c198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1082a0 0x7fee8c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1120a0 con 0x7fee8c1082a0 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c111f20 con 0x7fee8c102d30 2024-09-19T06:47:49.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee93d93640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c112220 con 0x7fee8c1036e0 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee92330640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1082a0 0x7fee8c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 0x7fee8c1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 0x7fee8c1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58554/0 (socket says 172.21.15.37:58554) 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee91b2f640 1 -- 172.21.15.37:0/2308410233 learned_addr learned my addr 172.21.15.37:0/2308410233 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee9132e640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c1036e0 0x7fee8c198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee92330640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1082a0 0x7fee8c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36436/0 (socket says 172.21.15.37:36436) 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee91b2f640 1 -- 172.21.15.37:0/2308410233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c1036e0 msgr2=0x7fee8c198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.721+0000 7fee91b2f640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c1036e0 0x7fee8c198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee91b2f640 1 -- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1082a0 msgr2=0x7fee8c19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee91b2f640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1082a0 0x7fee8c19ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee91b2f640 1 -- 172.21.15.37:0/2308410233 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8c19d560 con 0x7fee8c102d30 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee9132e640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c1036e0 0x7fee8c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:49.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee92330640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1082a0 0x7fee8c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:49.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee91b2f640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 0x7fee8c1985b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee8800e9f0 tx=0x7fee8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee82ffd640 1 -- 172.21.15.37:0/2308410233 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8800cd80 con 0x7fee8c102d30 2024-09-19T06:47:49.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee82ffd640 1 -- 172.21.15.37:0/2308410233 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee88010910 con 0x7fee8c102d30 2024-09-19T06:47:49.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8c19d850 con 0x7fee8c102d30 2024-09-19T06:47:49.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee82ffd640 1 -- 172.21.15.37:0/2308410233 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee88018990 con 0x7fee8c102d30 2024-09-19T06:47:49.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.722+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee8c1a5130 con 0x7fee8c102d30 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.724+0000 7fee82ffd640 1 -- 172.21.15.37:0/2308410233 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fee88010ac0 con 0x7fee8c102d30 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.724+0000 7fee9616c740 1 Processor -- start 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9616c740 1 -- start start 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9616c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d310090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9616c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d310a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9616c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d31f780 0x55c53d321b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9616c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c53d2762f0 con 0x55c53d3105d0 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9616c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c53d21dcf0 con 0x55c53d31f780 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9616c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c53d2722b0 con 0x55c53d311b40 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d310a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d310a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36446/0 (socket says 172.21.15.37:36446) 2024-09-19T06:47:49.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 -- 172.21.15.37:0/1340319214 learned_addr learned my addr 172.21.15.37:0/1340319214 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9132e640 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d31f780 0x55c53d321b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee92330640 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d310090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 -- 172.21.15.37:0/1340319214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 msgr2=0x55c53d310090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d310090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee82ffd640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fee780a33b0 0x7fee780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 -- 172.21.15.37:0/1340319214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d31f780 msgr2=0x55c53d321b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d31f780 0x55c53d321b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee91b2f640 1 -- 172.21.15.37:0/1340319214 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53d2023a0 con 0x55c53d3105d0 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.725+0000 7fee9132e640 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d31f780 0x55c53d321b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.726+0000 7fee92330640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fee780a33b0 0x7fee780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.726+0000 7fee82ffd640 1 -- 172.21.15.37:0/2308410233 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fee880d0030 con 0x7fee8c102d30 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.726+0000 7fee91b2f640 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d310a10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fee880cd6f0 tx=0x7fee88098490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8ca30a1815811a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.726+0000 7fee77fff640 1 -- 172.21.15.37:0/1340319214 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8809c190 con 0x55c53d3105d0 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.726+0000 7fee77fff640 1 -- 172.21.15.37:0/1340319214 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee8809c3c0 con 0x55c53d3105d0 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.726+0000 7fee77fff640 1 -- 172.21.15.37:0/1340319214 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8809bd20 con 0x55c53d3105d0 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 -- 172.21.15.37:0/1340319214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 msgr2=0x55c53d310a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee92330640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fee780a33b0 0x7fee780a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee84005c40 tx=0x7fee84009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d310a10 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fee880cd6f0 tx=0x7fee88098490 comp rx=0 tx=0).stop 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 -- 172.21.15.37:0/1340319214 shutdown_connections 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d31f780 0x55c53d321b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d310a10 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 --2- 172.21.15.37:0/1340319214 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d310090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 -- 172.21.15.37:0/1340319214 >> 172.21.15.37:0/1340319214 conn(0x55c53d2d9310 msgr2=0x55c53d30cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 -- 172.21.15.37:0/1340319214 shutdown_connections 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 -- 172.21.15.37:0/1340319214 wait complete. 2024-09-19T06:47:49.740 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 -- start start 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.727+0000 7fee9616c740 1 Processor -- start 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9616c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d31f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9616c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d311b40 0x55c53d31c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9616c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d31ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9616c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c53d322480 con 0x55c53d311b40 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9616c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c53d231060 con 0x55c53d3105d0 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9616c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c53d322600 con 0x55c53d31f780 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9616c740 10 client.0 _ll_register_callbacks cb 0x55c53cf848c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9132e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d311b40 0x55c53d31c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee92330640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d31ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d31f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9132e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d311b40 0x55c53d31c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36452/0 (socket says 172.21.15.37:36452) 2024-09-19T06:47:49.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee9132e640 1 -- 172.21.15.37:0/1633455702 learned_addr learned my addr 172.21.15.37:0/1633455702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee92330640 1 -- 172.21.15.37:0/1633455702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 msgr2=0x55c53d31f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee92330640 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d31f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee92330640 1 -- 172.21.15.37:0/1633455702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d311b40 msgr2=0x55c53d31c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee92330640 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d311b40 0x55c53d31c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee92330640 1 -- 172.21.15.37:0/1633455702 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53cfe73c0 con 0x55c53d31f780 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee92330640 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d31ca10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee84005fd0 tx=0x7fee84004ab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f2f8d82d951d70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.728+0000 7fee757fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee757fa640 1 -- 172.21.15.37:0/1633455702 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84011bd0 con 0x55c53d31f780 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee757fa640 1 -- 172.21.15.37:0/1633455702 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fee84011d70 con 0x55c53d31f780 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee8809c3c0 con 0x55c53d31f780 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee757fa640 1 -- 172.21.15.37:0/1633455702 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fee8401e470 con 0x55c53d31f780 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 10 client.26392 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:49.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c53d32df80 con 0x55c53d31f780 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee757fa640 1 -- 172.21.15.37:0/1633455702 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee8809c3c0 con 0x55c53d31f780 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee757fa640 1 -- 172.21.15.37:0/1633455702 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fee84027070 con 0x55c53d31f780 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 10 client.26392 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 10 client.26392 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 20 client.26392 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d32f440 0x55c53d331880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 4 client.26392 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c53d283860 con 0x55c53d32f440 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.729+0000 7fee91b2f640 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d32f440 0x55c53d331880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.730+0000 7fee757fa640 10 client.26392 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.730+0000 7fee91b2f640 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d32f440 0x55c53d331880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=596a7f1b57d532c4 server_cookie=23dc102df9e4df61 in_seq=0 out_seq=0 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.730+0000 7fee757fa640 1 -- 172.21.15.37:0/1633455702 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c53d283860 con 0x55c53d32f440 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.730+0000 7fee757fa640 10 client.26392 handle_command_reply: tid=0 2024-09-19T06:47:49.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.730+0000 7fee9616c740 1 client.26392 shutdown 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 msgr2=0x55c53d31ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d31ca10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fee84005fd0 tx=0x7fee84004ab0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 shutdown_connections 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d32f440 0x55c53d331880 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d31ca10 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d311b40 0x55c53d31c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee757fa640 0 client.26392 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 --2- 172.21.15.37:0/1633455702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d31f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 >> 172.21.15.37:0/1633455702 conn(0x55c53d2d9310 msgr2=0x55c53d312960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 shutdown_connections 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 1 -- 172.21.15.37:0/1633455702 wait complete. 2024-09-19T06:47:49.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.731+0000 7fee9616c740 20 client.26392 trim_cache size 0 max 16384 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.745+0000 7fee9616c740 1 Processor -- start 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.745+0000 7fee9616c740 1 -- start start 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.745+0000 7fee9616c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d32b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.745+0000 7fee9616c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d311b40 0x55c53d31e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.745+0000 7fee9616c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d3371a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9616c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1ae0c0 con 0x55c53d3105d0 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9616c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1ab990 con 0x55c53d311b40 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9616c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1a97e0 con 0x55c53d31f780 2024-09-19T06:47:49.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d311b40 0x55c53d31e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d32b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d311b40 0x55c53d31e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58586/0 (socket says 172.21.15.37:58586) 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 -- 172.21.15.37:0/3054902156 learned_addr learned my addr 172.21.15.37:0/3054902156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee92330640 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d3371a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 -- 172.21.15.37:0/3054902156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 msgr2=0x55c53d3371a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d3371a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 -- 172.21.15.37:0/3054902156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 msgr2=0x55c53d32b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d32b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 -- 172.21.15.37:0/3054902156 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c53d32bc60 con 0x55c53d311b40 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee92330640 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d3371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee91b2f640 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d32b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee9132e640 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d311b40 0x55c53d31e390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009230 tx=0x7fee7c0316f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16d4c983fca1253a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee777fe640 1 -- 172.21.15.37:0/3054902156 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c035040 con 0x55c53d311b40 2024-09-19T06:47:49.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee777fe640 1 -- 172.21.15.37:0/3054902156 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee7c005040 con 0x55c53d311b40 2024-09-19T06:47:49.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.746+0000 7fee777fe640 1 -- 172.21.15.37:0/3054902156 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c004af0 con 0x55c53d311b40 2024-09-19T06:47:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 -- 172.21.15.37:0/3054902156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d311b40 msgr2=0x55c53d31e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d311b40 0x55c53d31e390 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fee7c009230 tx=0x7fee7c0316f0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 -- 172.21.15.37:0/3054902156 shutdown_connections 2024-09-19T06:47:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d31f780 0x55c53d3371a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d311b40 0x55c53d31e390 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 --2- 172.21.15.37:0/3054902156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d3105d0 0x55c53d32b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 -- 172.21.15.37:0/3054902156 >> 172.21.15.37:0/3054902156 conn(0x55c53d2d9310 msgr2=0x55c53d31dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 -- 172.21.15.37:0/3054902156 shutdown_connections 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.747+0000 7fee9616c740 1 -- 172.21.15.37:0/3054902156 wait complete. 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.748+0000 7fee9616c740 1 -- start start 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.748+0000 7fee9616c740 1 Processor -- start 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.748+0000 7fee9616c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d323930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.748+0000 7fee9616c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d323e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.748+0000 7fee9616c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d31f780 0x55c53d399170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9616c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1a9e50 con 0x55c53d31f780 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d323930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9132e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d323e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9616c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1ab2c0 con 0x55c53d3105d0 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9132e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d323e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:55288/0 (socket says 172.21.15.37:55288) 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d323930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58596/0 (socket says 172.21.15.37:58596) 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9616c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fee8c1aa520 con 0x55c53d311b40 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9132e640 1 -- 172.21.15.37:0/746188335 learned_addr learned my addr 172.21.15.37:0/746188335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9616c740 10 client.0 _ll_register_callbacks cb 0x7fee840125d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee92330640 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d31f780 0x55c53d399170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 -- 172.21.15.37:0/746188335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 msgr2=0x55c53d323e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d323e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 -- 172.21.15.37:0/746188335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d31f780 msgr2=0x55c53d399170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d31f780 0x55c53d399170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 -- 172.21.15.37:0/746188335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c53d324750 con 0x55c53d3105d0 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee91b2f640 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d323930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee880d37e0 tx=0x7fee880cf040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12324cbb7fce7504 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:49.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee757fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee757fa640 1 -- 172.21.15.37:0/746188335 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8809bf00 con 0x55c53d3105d0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee757fa640 1 -- 172.21.15.37:0/746188335 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7fee88098a30 con 0x55c53d3105d0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.749+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7c005040 con 0x55c53d3105d0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee757fa640 1 -- 172.21.15.37:0/746188335 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fee8809c190 con 0x55c53d3105d0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 10 client.26549 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c53d39a770 con 0x55c53d3105d0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee757fa640 1 -- 172.21.15.37:0/746188335 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fee7c005040 con 0x55c53d3105d0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee757fa640 1 -- 172.21.15.37:0/746188335 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fee880d3c30 con 0x55c53d3105d0 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 10 client.26549 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 10 client.26549 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 20 client.26549 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d39aa00 0x55c53d39cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:49.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 4 client.26549 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c53d27dd70 con 0x55c53d39aa00 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.750+0000 7fee9132e640 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d39aa00 0x55c53d39cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9132e640 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d39aa00 0x55c53d39cdf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=783bebd16b0022a2 server_cookie=8d1acd0180a96337 in_seq=0 out_seq=0 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee757fa640 10 client.26549 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee757fa640 1 -- 172.21.15.37:0/746188335 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c53d27dd70 con 0x55c53d39aa00 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee757fa640 10 client.26549 handle_command_reply: tid=0 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 client.26549 shutdown 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 msgr2=0x55c53d323930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d323930 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fee880d37e0 tx=0x7fee880cf040 comp rx=0 tx=0).stop 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 shutdown_connections 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55c53d39aa00 0x55c53d39cdf0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c53d31f780 0x55c53d399170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c53d311b40 0x55c53d323e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee757fa640 0 client.26549 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:47:49.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 --2- 172.21.15.37:0/746188335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c53d3105d0 0x55c53d323930 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 >> 172.21.15.37:0/746188335 conn(0x55c53d2d9310 msgr2=0x55c53d31e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 shutdown_connections 2024-09-19T06:47:49.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 1 -- 172.21.15.37:0/746188335 wait complete. 2024-09-19T06:47:49.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.751+0000 7fee9616c740 20 client.26549 trim_cache size 0 max 16384 2024-09-19T06:47:49.758 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:47:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fee780a33b0 msgr2=0x7fee780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fee780a33b0 0x7fee780a5870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fee84005c40 tx=0x7fee84009500 comp rx=0 tx=0).stop 2024-09-19T06:47:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 msgr2=0x7fee8c1985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 0x7fee8c1985b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fee8800e9f0 tx=0x7fee8800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 shutdown_connections 2024-09-19T06:47:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fee780a33b0 0x7fee780a5870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fee8c1082a0 0x7fee8c19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fee8c1036e0 0x7fee8c198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 --2- 172.21.15.37:0/2308410233 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fee8c102d30 0x7fee8c1985b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:49.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 >> 172.21.15.37:0/2308410233 conn(0x7fee8c0fc7d0 msgr2=0x7fee8c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:49.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 shutdown_connections 2024-09-19T06:47:49.760 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:49.752+0000 7fee93d93640 1 -- 172.21.15.37:0/2308410233 wait complete. 2024-09-19T06:47:50.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 398033 6.80 MB/sec execute 137 sec latency 45.064 ms 2024-09-19T06:47:50.400 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:50 smithi037 ceph-mon[31211]: pgmap v185: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-09-19T06:47:50.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:50 smithi096 ceph-mon[35004]: pgmap v185: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-09-19T06:47:50.579 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:47:50.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:50 smithi151 ceph-mon[35236]: pgmap v185: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 MiB/s rd, 3.7 MiB/s wr, 417 op/s 2024-09-19T06:47:50.857 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:51.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 401077 6.89 MB/sec execute 138 sec latency 13.627 ms 2024-09-19T06:47:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 -- 172.21.15.37:0/3866102609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248102790 msgr2=0x7f42481067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 --2- 172.21.15.37:0/3866102609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248102790 0x7f42481067e0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f4230009900 tx=0x7f423002a410 comp rx=0 tx=0).stop 2024-09-19T06:47:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 -- 172.21.15.37:0/3866102609 shutdown_connections 2024-09-19T06:47:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 --2- 172.21.15.37:0/3866102609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248106f10 0x7f424810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 --2- 172.21.15.37:0/3866102609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248102790 0x7f42481067e0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 --2- 172.21.15.37:0/3866102609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4248101de0 0x7f42481021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 -- 172.21.15.37:0/3866102609 >> 172.21.15.37:0/3866102609 conn(0x7f42480fd8c0 msgr2=0x7f42480ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 -- 172.21.15.37:0/3866102609 shutdown_connections 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 -- 172.21.15.37:0/3866102609 wait complete. 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 Processor -- start 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.495+0000 7f424e537640 1 -- start start 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424e537640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4248101de0 0x7f424819a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424e537640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248102790 0x7f424819ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424e537640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248106f10 0x7f424819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424e537640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4248111f30 con 0x7f4248106f10 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424e537640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4248111db0 con 0x7f4248102790 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424e537640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42481120b0 con 0x7f4248101de0 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248106f10 0x7f424819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248106f10 0x7f424819ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36484/0 (socket says 172.21.15.37:36484) 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 -- 172.21.15.37:0/1759946981 learned_addr learned my addr 172.21.15.37:0/1759946981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f423ffff640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248102790 0x7f424819ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f4247fff640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4248101de0 0x7f424819a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 -- 172.21.15.37:0/1759946981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4248101de0 msgr2=0x7f424819a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4248101de0 0x7f424819a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 -- 172.21.15.37:0/1759946981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248102790 msgr2=0x7f424819ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248102790 0x7f424819ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 -- 172.21.15.37:0/1759946981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f424819f610 con 0x7f4248106f10 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f4247fff640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4248101de0 0x7f424819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f423ffff640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248102790 0x7f424819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424cad4640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248106f10 0x7f424819ef30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f424000efd0 tx=0x7f424000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f4245ffb640 1 -- 172.21.15.37:0/1759946981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4240003b70 con 0x7f4248106f10 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f4245ffb640 1 -- 172.21.15.37:0/1759946981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4240003d10 con 0x7f4248106f10 2024-09-19T06:47:51.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f424819f900 con 0x7f4248106f10 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.496+0000 7f4245ffb640 1 -- 172.21.15.37:0/1759946981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42400109f0 con 0x7f4248106f10 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.497+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42481a71e0 con 0x7f4248106f10 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.499+0000 7f4245ffb640 1 -- 172.21.15.37:0/1759946981 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4240004020 con 0x7f4248106f10 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.499+0000 7f4250910740 1 Processor -- start 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.499+0000 7f4250910740 1 -- start start 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4250910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 0x55dfc1f96090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4250910740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f965d0 0x55dfc1f96a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4250910740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4250910740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dfc1efc2f0 con 0x55dfc1f965d0 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4250910740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfc1ea3cf0 con 0x55dfc1fa1370 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4250910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfc1ef82b0 con 0x55dfc1f97b40 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f965d0 0x55dfc1f96a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f965d0 0x55dfc1f96a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36500/0 (socket says 172.21.15.37:36500) 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 -- 172.21.15.37:0/536937608 learned_addr learned my addr 172.21.15.37:0/536937608 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f423ffff640 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4245ffb640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42340a33b0 0x7f42340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f423ffff640 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 0x55dfc1f96090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f424cad4640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42340a33b0 0x7f42340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 -- 172.21.15.37:0/536937608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 msgr2=0x55dfc1f96090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 0x55dfc1f96090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 -- 172.21.15.37:0/536937608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 msgr2=0x55dfc1fa3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4247fff640 1 -- 172.21.15.37:0/536937608 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc1e883a0 con 0x55dfc1f965d0 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f423ffff640 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.500+0000 7f4245ffb640 1 -- 172.21.15.37:0/1759946981 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f42400d01d0 con 0x7f4248106f10 2024-09-19T06:47:51.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.501+0000 7f4247fff640 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f965d0 0x55dfc1f96a10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4238009c50 tx=0x7f423800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6309d812aa6d24a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.501+0000 7f423e7fc640 1 -- 172.21.15.37:0/536937608 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238016070 con 0x55dfc1f965d0 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.501+0000 7f423e7fc640 1 -- 172.21.15.37:0/536937608 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4238005500 con 0x55dfc1f965d0 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.501+0000 7f423e7fc640 1 -- 172.21.15.37:0/536937608 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238011670 con 0x55dfc1f965d0 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.501+0000 7f4250910740 1 -- 172.21.15.37:0/536937608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f965d0 msgr2=0x55dfc1f96a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.501+0000 7f4250910740 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f965d0 0x55dfc1f96a10 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f4238009c50 tx=0x7f423800e6c0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.501+0000 7f424cad4640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42340a33b0 0x7f42340a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f424000afd0 tx=0x7f424006b140 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- 172.21.15.37:0/536937608 shutdown_connections 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f965d0 0x55dfc1f96a10 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 --2- 172.21.15.37:0/536937608 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 0x55dfc1f96090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- 172.21.15.37:0/536937608 >> 172.21.15.37:0/536937608 conn(0x55dfc1f5f310 msgr2=0x55dfc1f92c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- 172.21.15.37:0/536937608 shutdown_connections 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- 172.21.15.37:0/536937608 wait complete. 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- start start 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 Processor -- start 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 0x55dfc1f9dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1f9e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dfc1fa4070 con 0x55dfc1fa1370 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dfc1eb7060 con 0x55dfc1f965d0 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dfc1fa41f0 con 0x55dfc1f97b40 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.502+0000 7f4250910740 10 client.0 _ll_register_callbacks cb 0x55dfc1c0a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1f9e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1f9e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36508/0 (socket says 172.21.15.37:36508) 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 -- 172.21.15.37:0/3298245255 learned_addr learned my addr 172.21.15.37:0/3298245255 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 -- 172.21.15.37:0/3298245255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 msgr2=0x55dfc1f9dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 0x55dfc1f9dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 -- 172.21.15.37:0/3298245255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 msgr2=0x55dfc1f9e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1f9e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 -- 172.21.15.37:0/3298245255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc1c6d3c0 con 0x55dfc1f965d0 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f423ffff640 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa0d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f423002a930 tx=0x7f423002a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8388d45d0a6c474 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f42277fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f42277fe640 1 -- 172.21.15.37:0/3298245255 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230035030 con 0x55dfc1f965d0 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4238005500 con 0x55dfc1f965d0 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f42277fe640 1 -- 172.21.15.37:0/3298245255 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f42300035f0 con 0x55dfc1f965d0 2024-09-19T06:47:51.519 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f42277fe640 1 -- 172.21.15.37:0/3298245255 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4230047040 con 0x55dfc1f965d0 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f4250910740 10 client.26561 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.503+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dfc1fafb10 con 0x55dfc1f965d0 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f42277fe640 1 -- 172.21.15.37:0/3298245255 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4238005500 con 0x55dfc1f965d0 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f42277fe640 1 -- 172.21.15.37:0/3298245255 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f423003d970 con 0x55dfc1f965d0 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4250910740 10 client.26561 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4250910740 10 client.26561 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4250910740 20 client.26561 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4250910740 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc1fb0f20 0x55dfc1fb3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4250910740 4 client.26561 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:51.520 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dfc1f09860 con 0x55dfc1fb0f20 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4247fff640 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc1fb0f20 0x55dfc1fb3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f4247fff640 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc1fb0f20 0x55dfc1fb3390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=95485cbeae62f0e2 server_cookie=b94e736535bd171c in_seq=0 out_seq=0 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.504+0000 7f42277fe640 10 client.26561 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f42277fe640 1 -- 172.21.15.37:0/3298245255 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dfc1f09860 con 0x55dfc1fb0f20 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f42277fe640 10 client.26561 handle_command_reply: tid=0 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 client.26561 shutdown 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f965d0 msgr2=0x55dfc1fa0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa0d20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f423002a930 tx=0x7f423002a410 comp rx=0 tx=0).stop 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 shutdown_connections 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc1fb0f20 0x55dfc1fb3390 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1f9e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f97b40 0x55dfc1f9dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.521 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f42277fe640 0 client.26561 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:51.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 --2- 172.21.15.37:0/3298245255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa0d20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 >> 172.21.15.37:0/3298245255 conn(0x55dfc1f5f310 msgr2=0x55dfc1f98960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 shutdown_connections 2024-09-19T06:47:51.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 1 -- 172.21.15.37:0/3298245255 wait complete. 2024-09-19T06:47:51.522 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.505+0000 7f4250910740 20 client.26561 trim_cache size 0 max 16384 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 Processor -- start 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 -- start start 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1f91c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42481b2440 con 0x55dfc1fa1370 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42481afd10 con 0x55dfc1f97b40 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4250910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42481adb60 con 0x55dfc1f965d0 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f423ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f4247fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f423ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36512/0 (socket says 172.21.15.37:36512) 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f423ffff640 1 -- 172.21.15.37:0/3302125380 learned_addr learned my addr 172.21.15.37:0/3302125380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.519+0000 7f423ffff640 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1f91c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423ffff640 1 -- 172.21.15.37:0/3302125380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 msgr2=0x55dfc1f91c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423ffff640 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1f91c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423ffff640 1 -- 172.21.15.37:0/3302125380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f97b40 msgr2=0x55dfc1fa2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423ffff640 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423ffff640 1 -- 172.21.15.37:0/3302125380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dfc1f97300 con 0x55dfc1fa1370 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f4247fff640 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423ffff640 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa2ca0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f42300096c0 tx=0x7f423003d450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75778c35214f670e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423dffb640 1 -- 172.21.15.37:0/3302125380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230035040 con 0x55dfc1fa1370 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423dffb640 1 -- 172.21.15.37:0/3302125380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42300044c0 con 0x55dfc1fa1370 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.520+0000 7f423dffb640 1 -- 172.21.15.37:0/3302125380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300039e0 con 0x55dfc1fa1370 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 -- 172.21.15.37:0/3302125380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 msgr2=0x55dfc1fa2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.528 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa2ca0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f42300096c0 tx=0x7f423003d450 comp rx=0 tx=0).stop 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 -- 172.21.15.37:0/3302125380 shutdown_connections 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa2ca0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 --2- 172.21.15.37:0/3302125380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1f91c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 -- 172.21.15.37:0/3302125380 >> 172.21.15.37:0/3302125380 conn(0x55dfc1f5f310 msgr2=0x55dfc1f91500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 -- 172.21.15.37:0/3302125380 shutdown_connections 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 -- 172.21.15.37:0/3302125380 wait complete. 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.521+0000 7f4250910740 1 -- start start 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 1 Processor -- start 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42481ae1d0 con 0x55dfc1f97b40 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42481af640 con 0x55dfc1fa1370 2024-09-19T06:47:51.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42481ae8a0 con 0x55dfc1f965d0 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4250910740 10 client.0 _ll_register_callbacks cb 0x7f42300317f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4247fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4247fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36516/0 (socket says 172.21.15.37:36516) 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f4247fff640 1 -- 172.21.15.37:0/3422254380 learned_addr learned my addr 172.21.15.37:0/3422254380 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f423ffff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.522+0000 7f423ffff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4247fff640 1 -- 172.21.15.37:0/3422254380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 msgr2=0x55dfc1fa9490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4247fff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa9490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4247fff640 1 -- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 msgr2=0x55dfc1fa4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4247fff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4247fff640 1 -- 172.21.15.37:0/3422254380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dfc1fa4980 con 0x55dfc1f97b40 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f423ffff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:51.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4247fff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa99d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f423800ebc0 tx=0x7f423800eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cd605d7db17db1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f42277fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f42277fe640 1 -- 172.21.15.37:0/3422254380 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238016040 con 0x55dfc1f97b40 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f42277fe640 1 -- 172.21.15.37:0/3422254380 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f4238004680 con 0x55dfc1f97b40 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dfc1f03d70 con 0x55dfc1f97b40 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f42277fe640 1 -- 172.21.15.37:0/3422254380 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f4238036050 con 0x55dfc1f97b40 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4250910740 10 client.17166 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55dfc201c3d0 con 0x55dfc1f97b40 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.523+0000 7f42277fe640 1 -- 172.21.15.37:0/3422254380 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55dfc1f03d70 con 0x55dfc1f97b40 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f42277fe640 1 -- 172.21.15.37:0/3422254380 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f423800f040 con 0x55dfc1f97b40 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f4250910740 10 client.17166 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f4250910740 10 client.17166 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f4250910740 20 client.17166 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:51.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f4250910740 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc201c9b0 0x55dfc201edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f4250910740 4 client.17166 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dfc1e80170 con 0x55dfc201c9b0 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f423ffff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc201c9b0 0x55dfc201edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f423ffff640 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc201c9b0 0x55dfc201edd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=be81e18a07f7e224 server_cookie=a579f55d3dd23acf in_seq=0 out_seq=0 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f42277fe640 10 client.17166 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f42277fe640 1 -- 172.21.15.37:0/3422254380 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dfc1e80170 con 0x55dfc201c9b0 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f42277fe640 10 client.17166 handle_command_reply: tid=0 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.524+0000 7f4250910740 1 client.17166 shutdown 2024-09-19T06:47:51.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f97b40 msgr2=0x55dfc1fa99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa99d0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f423800ebc0 tx=0x7f423800eef0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 shutdown_connections 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55dfc201c9b0 0x55dfc201edd0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dfc1fa1370 0x55dfc1fa4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dfc1f97b40 0x55dfc1fa99d0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 --2- 172.21.15.37:0/3422254380 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dfc1f965d0 0x55dfc1fa9490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f42277fe640 0 client.17166 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 >> 172.21.15.37:0/3422254380 conn(0x55dfc1f5f310 msgr2=0x55dfc1f91d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 shutdown_connections 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 1 -- 172.21.15.37:0/3422254380 wait complete. 2024-09-19T06:47:51.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f4250910740 20 client.17166 trim_cache size 0 max 16384 2024-09-19T06:47:51.534 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:47:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42340a33b0 msgr2=0x7f42340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f424e537640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42340a33b0 0x7f42340a5870 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f424000afd0 tx=0x7f424006b140 comp rx=0 tx=0).stop 2024-09-19T06:47:51.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248106f10 msgr2=0x7f424819ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.525+0000 7f424e537640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248106f10 0x7f424819ef30 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f424000efd0 tx=0x7f424000c6a0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 shutdown_connections 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42340a33b0 0x7f42340a5870 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4248106f10 0x7f424819ef30 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4248102790 0x7f424819ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 --2- 172.21.15.37:0/1759946981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4248101de0 0x7f424819a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 >> 172.21.15.37:0/1759946981 conn(0x7f42480fd8c0 msgr2=0x7f42480fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 shutdown_connections 2024-09-19T06:47:51.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:51.526+0000 7f424e537640 1 -- 172.21.15.37:0/1759946981 wait complete. 2024-09-19T06:47:52.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 403467 6.90 MB/sec execute 139 sec latency 13.772 ms 2024-09-19T06:47:52.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:52 smithi037 ceph-mon[31211]: pgmap v186: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-09-19T06:47:52.426 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:47:52.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:52 smithi096 ceph-mon[35004]: pgmap v186: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-09-19T06:47:52.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:52 smithi151 ceph-mon[35236]: pgmap v186: 97 pgs: 97 active+clean; 60 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.8 MiB/s wr, 324 op/s 2024-09-19T06:47:52.695 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:53.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 -- 172.21.15.37:0/1297034240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578100e20 msgr2=0x7f5578101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 --2- 172.21.15.37:0/1297034240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578100e20 0x7f5578101200 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5574009900 tx=0x7f557402a480 comp rx=0 tx=0).stop 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 -- 172.21.15.37:0/1297034240 shutdown_connections 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 --2- 172.21.15.37:0/1297034240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578102ef0 0x7f557810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 --2- 172.21.15.37:0/1297034240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5578102570 0x7f55781029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 --2- 172.21.15.37:0/1297034240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578100e20 0x7f5578101200 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 -- 172.21.15.37:0/1297034240 >> 172.21.15.37:0/1297034240 conn(0x7f55780fc7d0 msgr2=0x7f55780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 -- 172.21.15.37:0/1297034240 shutdown_connections 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.179+0000 7f557f645640 1 -- 172.21.15.37:0/1297034240 wait complete. 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 Processor -- start 2024-09-19T06:47:53.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 -- start start 2024-09-19T06:47:53.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578100e20 0x7f55780fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102570 0x7f55780fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5578102ef0 0x7f55780fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5578110f50 con 0x7f5578102570 2024-09-19T06:47:53.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5578110dd0 con 0x7f5578100e20 2024-09-19T06:47:53.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.180+0000 7f557f645640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55781110d0 con 0x7f5578102ef0 2024-09-19T06:47:53.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578100e20 0x7f55780fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557cbe0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102570 0x7f55780fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578100e20 0x7f55780fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58662/0 (socket says 172.21.15.37:58662) 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557d3e1640 1 -- 172.21.15.37:0/600346498 learned_addr learned my addr 172.21.15.37:0/600346498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557dbe2640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5578102ef0 0x7f55780fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557cbe0640 1 -- 172.21.15.37:0/600346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5578102ef0 msgr2=0x7f55780fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557cbe0640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5578102ef0 0x7f55780fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557cbe0640 1 -- 172.21.15.37:0/600346498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578100e20 msgr2=0x7f55780fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557cbe0640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578100e20 0x7f55780fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557cbe0640 1 -- 172.21.15.37:0/600346498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55781a6960 con 0x7f5578102570 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557d3e1640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578100e20 0x7f55780fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557cbe0640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102570 0x7f55780fd1c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f556800c970 tx=0x7f556800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f556e7fc640 1 -- 172.21.15.37:0/600346498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568007bb0 con 0x7f5578102570 2024-09-19T06:47:53.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55781a6c50 con 0x7f5578102570 2024-09-19T06:47:53.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f556e7fc640 1 -- 172.21.15.37:0/600346498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5568007d50 con 0x7f5578102570 2024-09-19T06:47:53.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f556e7fc640 1 -- 172.21.15.37:0/600346498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568018820 con 0x7f5578102570 2024-09-19T06:47:53.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.181+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55781a7190 con 0x7f5578102570 2024-09-19T06:47:53.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 Processor -- start 2024-09-19T06:47:53.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f556e7fc640 1 -- 172.21.15.37:0/600346498 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f55680189c0 con 0x7f5578102570 2024-09-19T06:47:53.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 -- start start 2024-09-19T06:47:53.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362920090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c362920a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c36292d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c3628862f0 con 0x55c362921b40 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c36282dcf0 con 0x55c3629205d0 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f5581a1e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c3628822b0 con 0x55c36292b370 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f557dbe2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362920090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c362920a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f557dbe2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362920090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36540/0 (socket says 172.21.15.37:36540) 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f557dbe2640 1 -- 172.21.15.37:0/2251804248 learned_addr learned my addr 172.21.15.37:0/2251804248 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.185+0000 7f557cbe0640 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c36292d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557dbe2640 1 -- 172.21.15.37:0/2251804248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 msgr2=0x55c36292d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557dbe2640 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c36292d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557dbe2640 1 -- 172.21.15.37:0/2251804248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 msgr2=0x55c362920a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557dbe2640 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c362920a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557dbe2640 1 -- 172.21.15.37:0/2251804248 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3628123a0 con 0x55c362921b40 2024-09-19T06:47:53.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557d3e1640 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c362920a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:53.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557dbe2640 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362920090 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5570009f20 tx=0x7f557000f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10c0912d62321e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f556e7fc640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55640a33b0 0x7f55640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f55637fe640 1 -- 172.21.15.37:0/2251804248 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570015070 con 0x55c362921b40 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f55637fe640 1 -- 172.21.15.37:0/2251804248 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557001ab40 con 0x55c362921b40 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f55637fe640 1 -- 172.21.15.37:0/2251804248 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570011c50 con 0x55c362921b40 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f556e7fc640 1 -- 172.21.15.37:0/600346498 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f55680cfdb0 con 0x7f5578102570 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.186+0000 7f557d3e1640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55640a33b0 0x7f55640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 -- 172.21.15.37:0/2251804248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 msgr2=0x55c362920090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f557d3e1640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55640a33b0 0x7f55640a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c36291dac0 tx=0x7f5574005a20 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:53.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362920090 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f5570009f20 tx=0x7f557000f7f0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 -- 172.21.15.37:0/2251804248 shutdown_connections 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c36292d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c362920a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 --2- 172.21.15.37:0/2251804248 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362920090 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 -- 172.21.15.37:0/2251804248 >> 172.21.15.37:0/2251804248 conn(0x55c3628e9310 msgr2=0x55c36291cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 -- 172.21.15.37:0/2251804248 shutdown_connections 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.187+0000 7f5581a1e740 1 -- 172.21.15.37:0/2251804248 wait complete. 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 -- start start 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 Processor -- start 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c36292ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362927fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362928560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c36292e070 con 0x55c362921b40 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c362841060 con 0x55c3629205d0 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c36292e1f0 con 0x55c36292b370 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f5581a1e740 10 client.0 _ll_register_callbacks cb 0x55c3625948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f557cbe0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362927fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f557cbe0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362927fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36550/0 (socket says 172.21.15.37:36550) 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c36292ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f557cbe0640 1 -- 172.21.15.37:0/3858067273 learned_addr learned my addr 172.21.15.37:0/3858067273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.188+0000 7f557dbe2640 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362928560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f557cbe0640 1 -- 172.21.15.37:0/3858067273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 msgr2=0x55c362928560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f557cbe0640 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362928560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f557cbe0640 1 -- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 msgr2=0x55c36292ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f557cbe0640 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c36292ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f557cbe0640 1 -- 172.21.15.37:0/3858067273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3625f73c0 con 0x55c362921b40 2024-09-19T06:47:53.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f557d3e1640 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c36292ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f557cbe0640 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362927fc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f556809c020 tx=0x7f556809be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d02cde9622e27f62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5560ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5560ff9640 1 -- 172.21.15.37:0/3858067273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568018cb0 con 0x55c362921b40 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5560ff9640 1 -- 172.21.15.37:0/3858067273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f55680a00d0 con 0x55c362921b40 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f557001ab40 con 0x55c362921b40 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5560ff9640 1 -- 172.21.15.37:0/3858067273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f556809c790 con 0x55c362921b40 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5581a1e740 10 client.17178 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c362939ba0 con 0x55c362921b40 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5560ff9640 1 -- 172.21.15.37:0/3858067273 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f557001ab40 con 0x55c362921b40 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.189+0000 7f5560ff9640 1 -- 172.21.15.37:0/3858067273 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5568098700 con 0x55c362921b40 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f5581a1e740 10 client.17178 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f5581a1e740 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:53.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f5581a1e740 20 client.17178 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c36293aef0 0x55c36293d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f5581a1e740 4 client.17178 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c362893860 con 0x55c36293aef0 2024-09-19T06:47:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f557d3e1640 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c36293aef0 0x55c36293d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f557d3e1640 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c36293aef0 0x55c36293d2e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc07c6a45514410 server_cookie=a23b2da52d79544b in_seq=0 out_seq=0 2024-09-19T06:47:53.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.190+0000 7f5560ff9640 10 client.17178 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.192+0000 7f5560ff9640 1 -- 172.21.15.37:0/3858067273 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c362893860 con 0x55c36293aef0 2024-09-19T06:47:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.192+0000 7f5560ff9640 10 client.17178 handle_command_reply: tid=0 2024-09-19T06:47:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.192+0000 7f5581a1e740 1 client.17178 shutdown 2024-09-19T06:47:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.192+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 msgr2=0x55c362927fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.192+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362927fc0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f556809c020 tx=0x7f556809be60 comp rx=0 tx=0).stop 2024-09-19T06:47:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 shutdown_connections 2024-09-19T06:47:53.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c36293aef0 0x55c36293d2e0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362928560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c362921b40 0x55c362927fc0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3858067273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c3629205d0 0x55c36292ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 >> 172.21.15.37:0/3858067273 conn(0x55c3628e9310 msgr2=0x55c362922930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5560ff9640 0 client.17178 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 shutdown_connections 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 1 -- 172.21.15.37:0/3858067273 wait complete. 2024-09-19T06:47:53.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.193+0000 7f5581a1e740 20 client.17178 trim_cache size 0 max 16384 2024-09-19T06:47:53.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.206+0000 7f5581a1e740 1 Processor -- start 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.206+0000 7f5581a1e740 1 -- start start 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.206+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362937340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.206+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362929f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.206+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362942b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f5581a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55781b2290 con 0x55c3629205d0 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f5581a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55781afb60 con 0x55c362921b40 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f5581a1e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ad9b0 con 0x55c36292b370 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362937340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362937340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36552/0 (socket says 172.21.15.37:36552) 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 -- 172.21.15.37:0/3432671546 learned_addr learned my addr 172.21.15.37:0/3432671546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557cbe0640 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362929f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557dbe2640 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362942b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 -- 172.21.15.37:0/3432671546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 msgr2=0x55c362942b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362942b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 -- 172.21.15.37:0/3432671546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 msgr2=0x55c362929f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362929f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 -- 172.21.15.37:0/3432671546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c36292b080 con 0x55c3629205d0 2024-09-19T06:47:53.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f557d3e1640 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362937340 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5574005ec0 tx=0x7f5574040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981eb1ccadddb3d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:53.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 405810 6.96 MB/sec execute 140 sec latency 17.407 ms 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f5562ffd640 1 -- 172.21.15.37:0/3432671546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574035040 con 0x55c3629205d0 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f5562ffd640 1 -- 172.21.15.37:0/3432671546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5574003e00 con 0x55c3629205d0 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.207+0000 7f5562ffd640 1 -- 172.21.15.37:0/3432671546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557403d450 con 0x55c3629205d0 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 -- 172.21.15.37:0/3432671546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 msgr2=0x55c362937340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362937340 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5574005ec0 tx=0x7f5574040000 comp rx=0 tx=0).stop 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 -- 172.21.15.37:0/3432671546 shutdown_connections 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362942b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362929f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 --2- 172.21.15.37:0/3432671546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362937340 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 -- 172.21.15.37:0/3432671546 >> 172.21.15.37:0/3432671546 conn(0x55c3628e9310 msgr2=0x55c362929820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 -- 172.21.15.37:0/3432671546 shutdown_connections 2024-09-19T06:47:53.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 -- 172.21.15.37:0/3432671546 wait complete. 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.208+0000 7f5581a1e740 1 -- start start 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 1 Processor -- start 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362924ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362925410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362925950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ae020 con 0x55c3629205d0 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55781af490 con 0x55c362921b40 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ae6f0 con 0x55c36292b370 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f5581a1e740 10 client.0 _ll_register_callbacks cb 0x7f55680d3410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362924ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362924ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36558/0 (socket says 172.21.15.37:36558) 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 -- 172.21.15.37:0/511609751 learned_addr learned my addr 172.21.15.37:0/511609751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:53.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557cbe0640 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362925410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557dbe2640 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362925950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 -- 172.21.15.37:0/511609751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 msgr2=0x55c362925950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362925950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 -- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 msgr2=0x55c362925410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362925410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.209+0000 7f557d3e1640 1 -- 172.21.15.37:0/511609751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3629a4b40 con 0x55c3629205d0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f557d3e1640 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362924ed0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5574004650 tx=0x7f55740513c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59854353123238d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5560ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5560ff9640 1 -- 172.21.15.37:0/511609751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5574051670 con 0x55c3629205d0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5560ff9640 1 -- 172.21.15.37:0/511609751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f5574051b40 con 0x55c3629205d0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5574003e00 con 0x55c3629205d0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5560ff9640 1 -- 172.21.15.37:0/511609751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f5574051d40 con 0x55c3629205d0 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 10 client.17184 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:53.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c3629a5d20 con 0x55c3629205d0 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5560ff9640 1 -- 172.21.15.37:0/511609751 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5574003e00 con 0x55c3629205d0 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5560ff9640 1 -- 172.21.15.37:0/511609751 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f5574044040 con 0x55c3629205d0 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 10 client.17184 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 10 client.17184 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 20 client.17184 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c3629a6430 0x55c3629a8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 4 client.17184 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.210+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c36288dd70 con 0x55c3629a6430 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.211+0000 7f557cbe0640 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c3629a6430 0x55c3629a8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.211+0000 7f557cbe0640 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c3629a6430 0x55c3629a8850 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5275f2963163372 server_cookie=c53b6e27ec8faa5d in_seq=0 out_seq=0 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.211+0000 7f5560ff9640 10 client.17184 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5560ff9640 1 -- 172.21.15.37:0/511609751 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c36288dd70 con 0x55c3629a6430 2024-09-19T06:47:53.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5560ff9640 10 client.17184 handle_command_reply: tid=0 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 client.17184 shutdown 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 msgr2=0x55c362924ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362924ed0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f5574004650 tx=0x7f55740513c0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 shutdown_connections 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c3629a6430 0x55c3629a8850 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c36292b370 0x55c362925950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c362921b40 0x55c362925410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 --2- 172.21.15.37:0/511609751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c3629205d0 0x55c362924ed0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 >> 172.21.15.37:0/511609751 conn(0x55c3628e9310 msgr2=0x55c362922110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5560ff9640 0 client.17184 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 shutdown_connections 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 1 -- 172.21.15.37:0/511609751 wait complete. 2024-09-19T06:47:53.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.212+0000 7f5581a1e740 20 client.17184 trim_cache size 0 max 16384 2024-09-19T06:47:53.219 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:47:53.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55640a33b0 msgr2=0x7f55640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55640a33b0 0x7f55640a5870 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x55c36291dac0 tx=0x7f5574005a20 comp rx=0 tx=0).stop 2024-09-19T06:47:53.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102570 msgr2=0x7f55780fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:53.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102570 0x7f55780fd1c0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f556800c970 tx=0x7f556800ce30 comp rx=0 tx=0).stop 2024-09-19T06:47:53.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 shutdown_connections 2024-09-19T06:47:53.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5578102ef0 0x7f55780fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55640a33b0 0x7f55640a5870 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5578102570 0x7f55780fd1c0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 --2- 172.21.15.37:0/600346498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5578100e20 0x7f55780fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:53.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 >> 172.21.15.37:0/600346498 conn(0x7f55780fc7d0 msgr2=0x7f557806a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:53.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 shutdown_connections 2024-09-19T06:47:53.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:53.213+0000 7f557f645640 1 -- 172.21.15.37:0/600346498 wait complete. 2024-09-19T06:47:54.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 408395 7.01 MB/sec execute 141 sec latency 13.155 ms 2024-09-19T06:47:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:54 smithi096 ceph-mon[35004]: pgmap v187: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-09-19T06:47:54.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:54 smithi037 ceph-mon[31211]: pgmap v187: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-09-19T06:47:54.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:54 smithi151 ceph-mon[35236]: pgmap v187: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 MiB/s rd, 3.7 MiB/s wr, 427 op/s 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 -- 172.21.15.37:0/73833070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99241009b0 msgr2=0x7f9924100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 --2- 172.21.15.37:0/73833070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99241009b0 0x7f9924100e10 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9918009930 tx=0x7f991802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 -- 172.21.15.37:0/73833070 shutdown_connections 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 --2- 172.21.15.37:0/73833070 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9924101350 0x7f992410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 --2- 172.21.15.37:0/73833070 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f99241009b0 0x7f9924100e10 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 --2- 172.21.15.37:0/73833070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99241069c0 0x7f9924106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 -- 172.21.15.37:0/73833070 >> 172.21.15.37:0/73833070 conn(0x7f9924077990 msgr2=0x7f99240fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.784 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 -- 172.21.15.37:0/73833070 shutdown_connections 2024-09-19T06:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 -- 172.21.15.37:0/73833070 wait complete. 2024-09-19T06:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 Processor -- start 2024-09-19T06:47:54.785 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.782+0000 7f992a850640 1 -- start start 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f992a850640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99241009b0 0x7f9924197240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f992a850640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9924101350 0x7f9924197780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f992a850640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99241069c0 0x7f992419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f992a850640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9924110e00 con 0x7f9924101350 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f992a850640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9924110c80 con 0x7f99241069c0 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f992a850640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9924110f80 con 0x7f99241009b0 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9924101350 0x7f9924197780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9924101350 0x7f9924197780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36580/0 (socket says 172.21.15.37:36580) 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 -- 172.21.15.37:0/3963289620 learned_addr learned my addr 172.21.15.37:0/3963289620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f9928ded640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99241069c0 0x7f992419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.787 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f9923fff640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99241009b0 0x7f9924197240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 -- 172.21.15.37:0/3963289620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99241009b0 msgr2=0x7f9924197240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99241009b0 0x7f9924197240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 -- 172.21.15.37:0/3963289620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99241069c0 msgr2=0x7f992419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99241069c0 0x7f992419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.783+0000 7f99237fe640 1 -- 172.21.15.37:0/3963289620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f992419c230 con 0x7f9924101350 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.784+0000 7f99237fe640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9924101350 0x7f9924197780 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9918009230 tx=0x7f9918004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.784+0000 7f99217fa640 1 -- 172.21.15.37:0/3963289620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f991803f030 con 0x7f9924101350 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.784+0000 7f99217fa640 1 -- 172.21.15.37:0/3963289620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9918004440 con 0x7f9924101350 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.784+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f992419c4c0 con 0x7f9924101350 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.784+0000 7f99217fa640 1 -- 172.21.15.37:0/3963289620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918035070 con 0x7f9924101350 2024-09-19T06:47:54.788 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.784+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99241a3db0 con 0x7f9924101350 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.786+0000 7f99217fa640 1 -- 172.21.15.37:0/3963289620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9918042030 con 0x7f9924101350 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.786+0000 7f992d5e2740 1 Processor -- start 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.786+0000 7f992d5e2740 1 -- start start 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c330b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c32c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c3386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f992d5e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56460c28b2f0 con 0x56460c32c070 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f992d5e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56460c233270 con 0x56460c32ca30 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f992d5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56460c2872b0 con 0x56460c330770 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c32c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.795 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c32c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36596/0 (socket says 172.21.15.37:36596) 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f99237fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c3386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 -- 172.21.15.37:0/2200673388 learned_addr learned my addr 172.21.15.37:0/2200673388 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9928ded640 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c330b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f99217fa640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f990c0a3460 0x7f990c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 -- 172.21.15.37:0/2200673388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 msgr2=0x56460c330b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c330b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 -- 172.21.15.37:0/2200673388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 msgr2=0x56460c3386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c3386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9923fff640 1 -- 172.21.15.37:0/2200673388 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56460c232340 con 0x56460c32c070 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f99237fe640 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c3386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f9928ded640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f990c0a3460 0x7f990c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.796 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.787+0000 7f99217fa640 1 -- 172.21.15.37:0/3963289620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f99180f0060 con 0x7f9924101350 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.788+0000 7f9923fff640 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c32c4f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9914009c70 tx=0x7f9914011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8e7b31403d1073 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.788+0000 7f9928ded640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f990c0a3460 0x7f990c0a5920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f991c009fd0 tx=0x7f991c004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.788+0000 7f990a7fc640 1 -- 172.21.15.37:0/2200673388 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914004230 con 0x56460c32c070 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.788+0000 7f990a7fc640 1 -- 172.21.15.37:0/2200673388 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99140043d0 con 0x56460c32c070 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.788+0000 7f990a7fc640 1 -- 172.21.15.37:0/2200673388 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914019820 con 0x56460c32c070 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 -- 172.21.15.37:0/2200673388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 msgr2=0x56460c32c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c32c4f0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f9914009c70 tx=0x7f9914011810 comp rx=0 tx=0).stop 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 -- 172.21.15.37:0/2200673388 shutdown_connections 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c3386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c32c4f0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2200673388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c330b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.797 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 -- 172.21.15.37:0/2200673388 >> 172.21.15.37:0/2200673388 conn(0x56460c322b20 msgr2=0x56460c322f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 -- 172.21.15.37:0/2200673388 shutdown_connections 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 -- 172.21.15.37:0/2200673388 wait complete. 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 -- start start 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.789+0000 7f992d5e2740 1 Processor -- start 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c335b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c336050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c336590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f992d5e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56460c338cf0 con 0x56460c32c070 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f992d5e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56460c214290 con 0x56460c32ca30 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f992d5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56460c338e70 con 0x56460c330770 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f992d5e2740 10 client.0 _ll_register_callbacks cb 0x56460c245140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c335b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.798 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c335b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36608/0 (socket says 172.21.15.37:36608) 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 -- 172.21.15.37:0/3929298393 learned_addr learned my addr 172.21.15.37:0/3929298393 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f99237fe640 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c336050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 -- 172.21.15.37:0/3929298393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 msgr2=0x56460c336590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c336590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 -- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 msgr2=0x56460c336050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c336050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 -- 172.21.15.37:0/3929298393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56460bf9e210 con 0x56460c32c070 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f99237fe640 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c336050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f9923fff640 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c335b10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9914005d30 tx=0x7f9914005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54124fbc2c199d2d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.799 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f99037fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f99037fe640 1 -- 172.21.15.37:0/3929298393 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914027070 con 0x56460c32c070 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.790+0000 7f99037fe640 1 -- 172.21.15.37:0/3929298393 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9914003800 con 0x56460c32c070 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99140043d0 con 0x56460c32c070 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f99037fe640 1 -- 172.21.15.37:0/3929298393 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f9914033050 con 0x56460c32c070 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 10 client.17196 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56460c06a620 con 0x56460c32c070 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f99037fe640 1 -- 172.21.15.37:0/3929298393 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99140043d0 con 0x56460c32c070 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f99037fe640 1 -- 172.21.15.37:0/3929298393 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f9914018070 con 0x56460c32c070 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 10 client.17196 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 20 client.17196 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c345f50 0x56460c348340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.800 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 4 client.17196 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56460c298a20 con 0x56460c345f50 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.791+0000 7f99237fe640 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c345f50 0x56460c348340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.792+0000 7f99037fe640 10 client.17196 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.792+0000 7f99237fe640 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c345f50 0x56460c348340 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4ea0ce050420548 server_cookie=d499380edbfd4be6 in_seq=0 out_seq=0 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f99037fe640 1 -- 172.21.15.37:0/3929298393 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56460c298a20 con 0x56460c345f50 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f99037fe640 10 client.17196 handle_command_reply: tid=0 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 client.17196 shutdown 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 msgr2=0x56460c335b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c335b10 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f9914005d30 tx=0x7f9914005d60 comp rx=0 tx=0).stop 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 shutdown_connections 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c345f50 0x56460c348340 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c336590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.801 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f99037fe640 0 client.17196 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c336050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 --2- 172.21.15.37:0/3929298393 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c335b10 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 >> 172.21.15.37:0/3929298393 conn(0x56460c322b20 msgr2=0x56460c3221c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 shutdown_connections 2024-09-19T06:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 1 -- 172.21.15.37:0/3929298393 wait complete. 2024-09-19T06:47:54.802 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.793+0000 7f992d5e2740 20 client.17196 trim_cache size 0 max 16384 2024-09-19T06:47:54.810 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 Processor -- start 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 -- start start 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c32c070 0x56460c32b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32ca30 0x56460c32e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c330770 0x56460c32e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99241adf00 con 0x56460c32ca30 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99241ab7d0 con 0x56460c330770 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f992d5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99241a9620 con 0x56460c32c070 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32ca30 0x56460c32e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32ca30 0x56460c32e1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36622/0 (socket says 172.21.15.37:36622) 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 -- 172.21.15.37:0/2621583621 learned_addr learned my addr 172.21.15.37:0/2621583621 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f9923fff640 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c32c070 0x56460c32b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.811 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f9928ded640 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c330770 0x56460c32e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 -- 172.21.15.37:0/2621583621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c32c070 msgr2=0x56460c32b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c32c070 0x56460c32b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 -- 172.21.15.37:0/2621583621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c330770 msgr2=0x56460c32e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c330770 0x56460c32e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f99237fe640 1 -- 172.21.15.37:0/2621583621 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56460c32f670 con 0x56460c32ca30 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.808+0000 7f9923fff640 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c32c070 0x56460c32b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.809+0000 7f99237fe640 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32ca30 0x56460c32e1d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f99180ed7b0 tx=0x7f99180dc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bd51bfed8ef5bfa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.809+0000 7f9909ffb640 1 -- 172.21.15.37:0/2621583621 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9918049040 con 0x56460c32ca30 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.809+0000 7f9909ffb640 1 -- 172.21.15.37:0/2621583621 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99180bc390 con 0x56460c32ca30 2024-09-19T06:47:54.812 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.809+0000 7f9909ffb640 1 -- 172.21.15.37:0/2621583621 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99180b8a00 con 0x56460c32ca30 2024-09-19T06:47:54.815 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.809+0000 7f992d5e2740 1 -- 172.21.15.37:0/2621583621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32ca30 msgr2=0x56460c32e1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.809+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32ca30 0x56460c32e1d0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f99180ed7b0 tx=0x7f99180dc040 comp rx=0 tx=0).stop 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- 172.21.15.37:0/2621583621 shutdown_connections 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c330770 0x56460c32e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32ca30 0x56460c32e1d0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 --2- 172.21.15.37:0/2621583621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c32c070 0x56460c32b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- 172.21.15.37:0/2621583621 >> 172.21.15.37:0/2621583621 conn(0x56460c322b20 msgr2=0x56460c32b060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- 172.21.15.37:0/2621583621 shutdown_connections 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- 172.21.15.37:0/2621583621 wait complete. 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- start start 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 Processor -- start 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c33e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c33ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c332f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f99241a9c90 con 0x56460c32c070 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f99241ab100 con 0x56460c32ca30 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f99241aa360 con 0x56460c330770 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f992d5e2740 10 client.0 _ll_register_callbacks cb 0x7f99180f39e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.810+0000 7f9923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c33e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c33e4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36634/0 (socket says 172.21.15.37:36634) 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 -- 172.21.15.37:0/205549052 learned_addr learned my addr 172.21.15.37:0/205549052 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f99237fe640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c33ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9928ded640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c332f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 -- 172.21.15.37:0/205549052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 msgr2=0x56460c332f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c332f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 -- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 msgr2=0x56460c33ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c33ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 -- 172.21.15.37:0/205549052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56460c333770 con 0x56460c32c070 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f99237fe640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c33ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9928ded640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c332f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f9923fff640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c33e4c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9914019f00 tx=0x7f9914005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbe4c815228848c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f99037fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f99037fe640 1 -- 172.21.15.37:0/205549052 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9914027040 con 0x56460c32c070 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f99037fe640 1 -- 172.21.15.37:0/205549052 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f9914009860 con 0x56460c32c070 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f99180bc390 con 0x56460c32c070 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f99037fe640 1 -- 172.21.15.37:0/205549052 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f991403d050 con 0x56460c32c070 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f992d5e2740 10 client.17202 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x56460c34ddb0 con 0x56460c32c070 2024-09-19T06:47:54.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.811+0000 7f99037fe640 1 -- 172.21.15.37:0/205549052 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99180bc390 con 0x56460c32c070 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f99037fe640 1 -- 172.21.15.37:0/205549052 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f9914018050 con 0x56460c32c070 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f992d5e2740 10 client.17202 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f992d5e2740 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f992d5e2740 20 client.17202 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f992d5e2740 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c3b0ee0 0x56460c3b3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f992d5e2740 4 client.17202 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56460c292d70 con 0x56460c3b0ee0 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f99237fe640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c3b0ee0 0x56460c3b3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f99037fe640 10 client.17202 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:54.819 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.812+0000 7f99237fe640 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c3b0ee0 0x56460c3b3300 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dd61abbef92b112 server_cookie=5e7df525c4dcb682 in_seq=0 out_seq=0 2024-09-19T06:47:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f99037fe640 1 -- 172.21.15.37:0/205549052 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56460c292d70 con 0x56460c3b0ee0 2024-09-19T06:47:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f99037fe640 10 client.17202 handle_command_reply: tid=0 2024-09-19T06:47:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 client.17202 shutdown 2024-09-19T06:47:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 msgr2=0x56460c33e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c33e4c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f9914019f00 tx=0x7f9914005fa0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 shutdown_connections 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56460c3b0ee0 0x56460c3b3300 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56460c330770 0x56460c332f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56460c32ca30 0x56460c33ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 --2- 172.21.15.37:0/205549052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56460c32c070 0x56460c33e4c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 >> 172.21.15.37:0/205549052 conn(0x56460c322b20 msgr2=0x56460c32b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f99037fe640 0 client.17202 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 shutdown_connections 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 1 -- 172.21.15.37:0/205549052 wait complete. 2024-09-19T06:47:54.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.945+0000 7f992d5e2740 20 client.17202 trim_cache size 0 max 16384 2024-09-19T06:47:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:47:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:47:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:47:54.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f990c0a3460 msgr2=0x7f990c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f990c0a3460 0x7f990c0a5920 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f991c009fd0 tx=0x7f991c004a40 comp rx=0 tx=0).stop 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9924101350 msgr2=0x7f9924197780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9924101350 0x7f9924197780 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9918009230 tx=0x7f9918004040 comp rx=0 tx=0).stop 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 shutdown_connections 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99241069c0 0x7f992419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f990c0a3460 0x7f990c0a5920 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9924101350 0x7f9924197780 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 --2- 172.21.15.37:0/3963289620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99241009b0 0x7f9924197240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.946+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 >> 172.21.15.37:0/3963289620 conn(0x7f9924077990 msgr2=0x7f99241037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.947+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 shutdown_connections 2024-09-19T06:47:54.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:47:54.947+0000 7f992a850640 1 -- 172.21.15.37:0/3963289620 wait complete. 2024-09-19T06:47:54.952 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:47:54.952 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:47:55.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 410928 7.05 MB/sec execute 142 sec latency 57.220 ms 2024-09-19T06:47:55.219 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 -- 172.21.15.37:0/3376052253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3950102b50 msgr2=0x7f3950100090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3376052253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3950102b50 0x7f3950100090 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f3948009900 tx=0x7f394802a400 comp rx=0 tx=0).stop 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 -- 172.21.15.37:0/3376052253 shutdown_connections 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3376052253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f395010e810 0x7f3950110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3376052253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3950102fc0 0x7f39501005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3376052253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3950102b50 0x7f3950100090 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 -- 172.21.15.37:0/3376052253 >> 172.21.15.37:0/3376052253 conn(0x7f3950077210 msgr2=0x7f39500fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 -- 172.21.15.37:0/3376052253 shutdown_connections 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 -- 172.21.15.37:0/3376052253 wait complete. 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 Processor -- start 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.688+0000 7f3956f7a640 1 -- start start 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3956f7a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3950102b50 0x7f395019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3956f7a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f395010e810 0x7f395019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3956f7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f395019ef70 0x7f395019f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3956f7a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39501141c0 con 0x7f395019ef70 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3956f7a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3950114040 con 0x7f3950102b50 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3956f7a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3950114340 con 0x7f395010e810 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f395019ef70 0x7f395019f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f395019ef70 0x7f395019f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36654/0 (socket says 172.21.15.37:36654) 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 -- 172.21.15.37:0/3858723864 learned_addr learned my addr 172.21.15.37:0/3858723864 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3954d16640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3950102b50 0x7f395019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f394ffff640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f395010e810 0x7f395019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 -- 172.21.15.37:0/3858723864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f395010e810 msgr2=0x7f395019abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f395010e810 0x7f395019abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 -- 172.21.15.37:0/3858723864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3950102b50 msgr2=0x7f395019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3950102b50 0x7f395019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 -- 172.21.15.37:0/3858723864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f395019fa30 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.689+0000 7f3955517640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f395019ef70 0x7f395019f420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f393800e970 tx=0x7f393800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.690+0000 7f394dffb640 1 -- 172.21.15.37:0/3858723864 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393800ccc0 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.690+0000 7f394dffb640 1 -- 172.21.15.37:0/3858723864 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f393800ce60 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.690+0000 7f394dffb640 1 -- 172.21.15.37:0/3858723864 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3938010490 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.690+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f395019fd20 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.690+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39501a7500 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.692+0000 7f394dffb640 1 -- 172.21.15.37:0/3858723864 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3938005020 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 Processor -- start 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f394dffb640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f393c0a3430 0x7f393c0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 -- start start 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f394ffff640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f393c0a3430 0x7f393c0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f394dffb640 1 -- 172.21.15.37:0/3858723864 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f39380d0120 con 0x7f395019ef70 2024-09-19T06:47:55.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9ca1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9cac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9caf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561dd9c07280 con 0x561dd9ca1b60 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561dd9baeb10 con 0x561dd9ca2480 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3959353740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561dd9c03240 con 0x561dd9cacd40 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f3954d16640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9ca1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.693+0000 7f394ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9cac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f3954d16640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9ca1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36662/0 (socket says 172.21.15.37:36662) 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f3954d16640 1 -- 172.21.15.37:0/3331548940 learned_addr learned my addr 172.21.15.37:0/3331548940 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f3955517640 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9caf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f394ffff640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f393c0a3430 0x7f393c0a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39500fcd20 tx=0x7f3940009760 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:55.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f394ffff640 1 -- 172.21.15.37:0/3331548940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 msgr2=0x561dd9caf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f394ffff640 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9caf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f394ffff640 1 -- 172.21.15.37:0/3331548940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 msgr2=0x561dd9ca1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f394ffff640 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9ca1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f394ffff640 1 -- 172.21.15.37:0/3331548940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd9bbb1f0 con 0x561dd9ca2480 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f3954d16640 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9ca1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f394ffff640 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9cac800 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f395006a1c0 tx=0x7f3940018000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe1d29bae0cc97d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f3946ffd640 1 -- 172.21.15.37:0/3331548940 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394001d630 con 0x561dd9ca2480 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f3946ffd640 1 -- 172.21.15.37:0/3331548940 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3940021030 con 0x561dd9ca2480 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.694+0000 7f3946ffd640 1 -- 172.21.15.37:0/3331548940 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394001cc20 con 0x561dd9ca2480 2024-09-19T06:47:55.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 -- 172.21.15.37:0/3331548940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 msgr2=0x561dd9cac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9cac800 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f395006a1c0 tx=0x7f3940018000 comp rx=0 tx=0).stop 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 -- 172.21.15.37:0/3331548940 shutdown_connections 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9caf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9cac800 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 --2- 172.21.15.37:0/3331548940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9ca1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 -- 172.21.15.37:0/3331548940 >> 172.21.15.37:0/3331548940 conn(0x561dd9c9e0f0 msgr2=0x561dd9c9e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 -- 172.21.15.37:0/3331548940 shutdown_connections 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 -- 172.21.15.37:0/3331548940 wait complete. 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 -- start start 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.695+0000 7f3959353740 1 Processor -- start 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3959353740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca1b60 0x561dd9ca9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3959353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9caa110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3959353740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9cacd40 0x561dd9caa650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3959353740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561dd9caf910 con 0x561dd9cacd40 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3959353740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561dd9bc1f10 con 0x561dd9ca2480 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3959353740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561dd9cafa90 con 0x561dd9ca1b60 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3959353740 10 client.0 _ll_register_callbacks cb 0x561dd9b8aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9cacd40 0x561dd9caa650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f394ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9caa110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9cacd40 0x561dd9caa650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36674/0 (socket says 172.21.15.37:36674) 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 -- 172.21.15.37:0/2591585636 learned_addr learned my addr 172.21.15.37:0/2591585636 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f394ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9caa110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58794/0 (socket says 172.21.15.37:58794) 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3954d16640 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca1b60 0x561dd9ca9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 -- 172.21.15.37:0/2591585636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca1b60 msgr2=0x561dd9ca9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca1b60 0x561dd9ca9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 -- 172.21.15.37:0/2591585636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 msgr2=0x561dd9caa110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9caa110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.696+0000 7f3955517640 1 -- 172.21.15.37:0/2591585636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd99786a0 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3955517640 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9cacd40 0x561dd9caa650 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3938004a00 tx=0x7f393809cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9baa1e450cac389 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f392ffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f392ffff640 1 -- 172.21.15.37:0/2591585636 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39380a5010 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f392ffff640 1 -- 172.21.15.37:0/2591585636 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f39380b5260 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3940021030 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f392ffff640 1 -- 172.21.15.37:0/2591585636 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3938098a10 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 10 client.17214 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561dd9cbb520 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f392ffff640 1 -- 172.21.15.37:0/2591585636 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3940021030 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f392ffff640 1 -- 172.21.15.37:0/2591585636 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f393809b7f0 con 0x561dd9cacd40 2024-09-19T06:47:55.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 10 client.17214 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 10 client.17214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 20 client.17214 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9cbedd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 4 client.17214 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.697+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561dd9c147f0 con 0x561dd9cbc9e0 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.698+0000 7f3954d16640 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9cbedd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.698+0000 7f392ffff640 10 client.17214 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.698+0000 7f3954d16640 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9cbedd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=acd769b4315e689 server_cookie=167d0b9bfd1687d0 in_seq=0 out_seq=0 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f392ffff640 1 -- 172.21.15.37:0/2591585636 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561dd9c147f0 con 0x561dd9cbc9e0 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f392ffff640 10 client.17214 handle_command_reply: tid=0 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 client.17214 shutdown 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9cacd40 msgr2=0x561dd9caa650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9cacd40 0x561dd9caa650 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f3938004a00 tx=0x7f393809cb60 comp rx=0 tx=0).stop 2024-09-19T06:47:55.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 shutdown_connections 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9cbedd0 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f392ffff640 0 client.17214 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9cacd40 0x561dd9caa650 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca2480 0x561dd9caa110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 --2- 172.21.15.37:0/2591585636 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca1b60 0x561dd9ca9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 >> 172.21.15.37:0/2591585636 conn(0x561dd9c9e0f0 msgr2=0x561dd9c9d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 shutdown_connections 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.699+0000 7f3959353740 1 -- 172.21.15.37:0/2591585636 wait complete. 2024-09-19T06:47:55.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.700+0000 7f3959353740 20 client.17214 trim_cache size 0 max 16384 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 Processor -- start 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 -- start start 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9caba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca2480 0x561dd9cc4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9cacd40 0x561dd9cc4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39501b23f0 con 0x561dd9ca1b60 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39501afcc0 con 0x561dd9cacd40 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3959353740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39501adb10 con 0x561dd9ca2480 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9caba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9caba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36684/0 (socket says 172.21.15.37:36684) 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 -- 172.21.15.37:0/3046283383 learned_addr learned my addr 172.21.15.37:0/3046283383 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:55.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3955517640 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9cacd40 0x561dd9cc4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f394ffff640 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca2480 0x561dd9cc4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 -- 172.21.15.37:0/3046283383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca2480 msgr2=0x561dd9cc4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca2480 0x561dd9cc4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 -- 172.21.15.37:0/3046283383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9cacd40 msgr2=0x561dd9cc4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9cacd40 0x561dd9cc4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.717+0000 7f3954d16640 1 -- 172.21.15.37:0/3046283383 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561dd9c98680 con 0x561dd9ca1b60 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3954d16640 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9caba30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f394802a920 tx=0x7f3948031af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3879a80c94571a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f39467fc640 1 -- 172.21.15.37:0/3046283383 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948035040 con 0x561dd9ca1b60 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f39467fc640 1 -- 172.21.15.37:0/3046283383 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3948005520 con 0x561dd9ca1b60 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f39467fc640 1 -- 172.21.15.37:0/3046283383 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948004c20 con 0x561dd9ca1b60 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3959353740 1 -- 172.21.15.37:0/3046283383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 msgr2=0x561dd9caba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3959353740 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9caba30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f394802a920 tx=0x7f3948031af0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3959353740 1 -- 172.21.15.37:0/3046283383 shutdown_connections 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3959353740 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9cacd40 0x561dd9cc4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3959353740 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9ca2480 0x561dd9cc4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3959353740 1 --2- 172.21.15.37:0/3046283383 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca1b60 0x561dd9caba30 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.718+0000 7f3959353740 1 -- 172.21.15.37:0/3046283383 >> 172.21.15.37:0/3046283383 conn(0x561dd9c9e0f0 msgr2=0x561dd9cab320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 -- 172.21.15.37:0/3046283383 shutdown_connections 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 -- 172.21.15.37:0/3046283383 wait complete. 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 -- start start 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 Processor -- start 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca1b60 0x561dd9cb0c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca2480 0x561dd9cb1170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9cb16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f39501ae180 con 0x561dd9ca2480 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39501af5f0 con 0x561dd9ca1b60 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f39501ae850 con 0x561dd9cacd40 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.719+0000 7f3959353740 10 client.0 _ll_register_callbacks cb 0x7f3938005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f394ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca2480 0x561dd9cb1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca1b60 0x561dd9cb0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca1b60 0x561dd9cb0c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58808/0 (socket says 172.21.15.37:58808) 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 -- 172.21.15.37:0/3696992320 learned_addr learned my addr 172.21.15.37:0/3696992320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3955517640 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9cb16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 -- 172.21.15.37:0/3696992320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 msgr2=0x561dd9cb16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9cb16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 -- 172.21.15.37:0/3696992320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca2480 msgr2=0x561dd9cb1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca2480 0x561dd9cb1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 -- 172.21.15.37:0/3696992320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561dd9d2efc0 con 0x561dd9ca1b60 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f394ffff640 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca2480 0x561dd9cb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3954d16640 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca1b60 0x561dd9cb0c30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3948005940 tx=0x7f3948005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8df340e782899691 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f392ffff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f392ffff640 1 -- 172.21.15.37:0/3696992320 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3948055030 con 0x561dd9ca1b60 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f392ffff640 1 -- 172.21.15.37:0/3696992320 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f39480475d0 con 0x561dd9ca1b60 2024-09-19T06:47:55.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3948005520 con 0x561dd9ca1b60 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.720+0000 7f392ffff640 1 -- 172.21.15.37:0/3696992320 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f3948035040 con 0x561dd9ca1b60 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 10 client.26621 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x561dd9d302e0 con 0x561dd9ca1b60 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f392ffff640 1 -- 172.21.15.37:0/3696992320 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3948005520 con 0x561dd9ca1b60 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f392ffff640 1 -- 172.21.15.37:0/3696992320 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f3948063af0 con 0x561dd9ca1b60 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 10 client.26621 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 10 client.26621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 20 client.26621 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9d32980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 4 client.26621 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561dd9c0ed00 con 0x561dd9cbc9e0 2024-09-19T06:47:55.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.721+0000 7f394ffff640 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9d32980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:55.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.722+0000 7f394ffff640 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9d32980 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13b3a8792124f3d6 server_cookie=21a98b817d663dc3 in_seq=0 out_seq=0 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.722+0000 7f392ffff640 10 client.26621 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.723+0000 7f392ffff640 1 -- 172.21.15.37:0/3696992320 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561dd9c0ed00 con 0x561dd9cbc9e0 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.723+0000 7f392ffff640 10 client.26621 handle_command_reply: tid=0 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.723+0000 7f3959353740 1 client.26621 shutdown 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.723+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca1b60 msgr2=0x561dd9cb0c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.723+0000 7f3959353740 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca1b60 0x561dd9cb0c30 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3948005940 tx=0x7f3948005750 comp rx=0 tx=0).stop 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 shutdown_connections 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561dd9cbc9e0 0x561dd9d32980 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561dd9cacd40 0x561dd9cb16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561dd9ca2480 0x561dd9cb1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f392ffff640 0 client.26621 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 --2- 172.21.15.37:0/3696992320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561dd9ca1b60 0x561dd9cb0c30 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 >> 172.21.15.37:0/3696992320 conn(0x561dd9c9e0f0 msgr2=0x561dd9caa3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 shutdown_connections 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 1 -- 172.21.15.37:0/3696992320 wait complete. 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3959353740 20 client.26621 trim_cache size 0 max 16384 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "668cad5f-916e-4586-aca7-ca024c1c45c1", 2024-09-19T06:47:55.729 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:47:55.730 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:47:55.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f393c0a3430 msgr2=0x7f393c0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f393c0a3430 0x7f393c0a58f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f39500fcd20 tx=0x7f3940009760 comp rx=0 tx=0).stop 2024-09-19T06:47:55.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f395019ef70 msgr2=0x7f395019f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:55.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f395019ef70 0x7f395019f420 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f393800e970 tx=0x7f393800ed70 comp rx=0 tx=0).stop 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.724+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 shutdown_connections 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.725+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f393c0a3430 0x7f393c0a58f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.725+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f395019ef70 0x7f395019f420 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.725+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f395010e810 0x7f395019abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.725+0000 7f3956f7a640 1 --2- 172.21.15.37:0/3858723864 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3950102b50 0x7f395019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.725+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 >> 172.21.15.37:0/3858723864 conn(0x7f3950077210 msgr2=0x7f395010fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.725+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 shutdown_connections 2024-09-19T06:47:55.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:55.725+0000 7f3956f7a640 1 -- 172.21.15.37:0/3858723864 wait complete. 2024-09-19T06:47:56.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 413021 7.08 MB/sec execute 143 sec latency 39.357 ms 2024-09-19T06:47:56.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:56 smithi037 ceph-mon[31211]: pgmap v188: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-19T06:47:56.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:56 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:47:56.501 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:56 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:56.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:56 smithi096 ceph-mon[35004]: pgmap v188: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-19T06:47:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:56 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:47:56.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:56 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:56.528 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:47:56.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:56 smithi151 ceph-mon[35236]: pgmap v188: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-19T06:47:56.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:56 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:47:56.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:56 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:56.792 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:47:57.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 415660 7.13 MB/sec execute 144 sec latency 14.086 ms 2024-09-19T06:47:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:57 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:47:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:57 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:57 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:47:57.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:57 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:57.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:57 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:47:57.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:57 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:57.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:57 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:47:57.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:57 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:57.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:57 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:47:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:57 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:57 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:47:57.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:57 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 -- 172.21.15.37:0/444473250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448106a80 msgr2=0x7f3448106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 --2- 172.21.15.37:0/444473250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448106a80 0x7f3448106e60 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f3438009930 tx=0x7f343802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 -- 172.21.15.37:0/444473250 shutdown_connections 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 --2- 172.21.15.37:0/444473250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3448101420 0x7f344810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 --2- 172.21.15.37:0/444473250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3448100a80 0x7f3448100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 --2- 172.21.15.37:0/444473250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448106a80 0x7f3448106e60 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 -- 172.21.15.37:0/444473250 >> 172.21.15.37:0/444473250 conn(0x7f34480fc7a0 msgr2=0x7f34480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 -- 172.21.15.37:0/444473250 shutdown_connections 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.639+0000 7f344dc9d640 1 -- 172.21.15.37:0/444473250 wait complete. 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 Processor -- start 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 -- start start 2024-09-19T06:47:57.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448100a80 0x7f3448197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3448101420 0x7f34481978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3448106a80 0x7f344819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3448110e90 con 0x7f3448100a80 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3448110d10 con 0x7f3448106a80 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f344dc9d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3448111010 con 0x7f3448101420 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448100a80 0x7f3448197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448100a80 0x7f3448197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36710/0 (socket says 172.21.15.37:36710) 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 -- 172.21.15.37:0/1636625013 learned_addr learned my addr 172.21.15.37:0/1636625013 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f3447fff640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3448106a80 0x7f344819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f343ffff640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3448101420 0x7f34481978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 -- 172.21.15.37:0/1636625013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3448101420 msgr2=0x7f34481978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3448101420 0x7f34481978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 -- 172.21.15.37:0/1636625013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3448106a80 msgr2=0x7f344819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3448106a80 0x7f344819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f34477fe640 1 -- 172.21.15.37:0/1636625013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f344819c330 con 0x7f3448100a80 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.640+0000 7f3447fff640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3448106a80 0x7f344819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.641+0000 7f34477fe640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448100a80 0x7f3448197380 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f343802a940 tx=0x7f3438003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.641+0000 7f34457fa640 1 -- 172.21.15.37:0/1636625013 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f343803f070 con 0x7f3448100a80 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.641+0000 7f34457fa640 1 -- 172.21.15.37:0/1636625013 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3438031c90 con 0x7f3448100a80 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.641+0000 7f34457fa640 1 -- 172.21.15.37:0/1636625013 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438035030 con 0x7f3448100a80 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.641+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f344819c5c0 con 0x7f3448100a80 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.641+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344819ca50 con 0x7f3448100a80 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.644+0000 7f34457fa640 1 -- 172.21.15.37:0/1636625013 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3438042070 con 0x7f3448100a80 2024-09-19T06:47:57.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.644+0000 7f3450076740 1 Processor -- start 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.644+0000 7f3450076740 1 -- start start 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.644+0000 7f3450076740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0fd00 0x557c20e0e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.644+0000 7f3450076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0ead0 0x557c20e0ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.644+0000 7f3450076740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e1b740 0x557c20e1db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f3450076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c20d722e0 con 0x557c20e0ead0 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f3450076740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557c20d19ce0 con 0x557c20e0fd00 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f3450076740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557c20d6e2a0 con 0x557c20e1b740 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f3447fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0fd00 0x557c20e0e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f3447fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0fd00 0x557c20e0e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58830/0 (socket says 172.21.15.37:58830) 2024-09-19T06:47:57.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f3447fff640 1 -- 172.21.15.37:0/242130807 learned_addr learned my addr 172.21.15.37:0/242130807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:57.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f34477fe640 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0ead0 0x557c20e0ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f343ffff640 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e1b740 0x557c20e1db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f34477fe640 1 -- 172.21.15.37:0/242130807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e1b740 msgr2=0x557c20e1db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f34477fe640 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e1b740 0x557c20e1db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f34477fe640 1 -- 172.21.15.37:0/242130807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0fd00 msgr2=0x557c20e0e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f34477fe640 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0fd00 0x557c20e0e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.645+0000 7f34477fe640 1 -- 172.21.15.37:0/242130807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c20cfe440 con 0x557c20e0ead0 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.646+0000 7f34477fe640 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0ead0 0x557c20e0ef30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f34380edab0 tx=0x7f34380f8640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bffd655e2a244d2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.646+0000 7f343dffb640 1 -- 172.21.15.37:0/242130807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34380f2520 con 0x557c20e0ead0 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.646+0000 7f34457fa640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34340a3430 0x7f34340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.646+0000 7f343dffb640 1 -- 172.21.15.37:0/242130807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34380f0c90 con 0x557c20e0ead0 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.646+0000 7f343dffb640 1 -- 172.21.15.37:0/242130807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438102b10 con 0x557c20e0ead0 2024-09-19T06:47:57.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.646+0000 7f3447fff640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34340a3430 0x7f34340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.646+0000 7f34457fa640 1 -- 172.21.15.37:0/1636625013 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f34380f03c0 con 0x7f3448100a80 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 -- 172.21.15.37:0/242130807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0ead0 msgr2=0x557c20e0ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0ead0 0x557c20e0ef30 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f34380edab0 tx=0x7f34380f8640 comp rx=0 tx=0).stop 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3447fff640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34340a3430 0x7f34340a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3440009fd0 tx=0x7f3440009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 -- 172.21.15.37:0/242130807 shutdown_connections 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e1b740 0x557c20e1db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0ead0 0x557c20e0ef30 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 --2- 172.21.15.37:0/242130807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0fd00 0x557c20e0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 -- 172.21.15.37:0/242130807 >> 172.21.15.37:0/242130807 conn(0x557c20e0ac40 msgr2=0x557c20e0b020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 -- 172.21.15.37:0/242130807 shutdown_connections 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.647+0000 7f3450076740 1 -- 172.21.15.37:0/242130807 wait complete. 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 -- start start 2024-09-19T06:47:57.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 Processor -- start 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e1b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e183a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e18940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557c20e1e410 con 0x557c20e1b740 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557c20d2d050 con 0x557c20e0ead0 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557c20e1e590 con 0x557c20e0fd00 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f3450076740 10 client.0 _ll_register_callbacks cb 0x557c20d6d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.648+0000 7f34477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e1b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e1b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58840/0 (socket says 172.21.15.37:58840) 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 -- 172.21.15.37:0/3446715322 learned_addr learned my addr 172.21.15.37:0/3446715322 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f343ffff640 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e183a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f3447fff640 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e18940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 -- 172.21.15.37:0/3446715322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 msgr2=0x557c20e183a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e183a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 -- 172.21.15.37:0/3446715322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 msgr2=0x557c20e18940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e18940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 -- 172.21.15.37:0/3446715322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c20ae33c0 con 0x557c20e0ead0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f34477fe640 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e1b0f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f34380b9010 tx=0x7f343802a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b0cb7f593d0cefb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f3426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f3426ffd640 1 -- 172.21.15.37:0/3446715322 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438049070 con 0x557c20e0ead0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f3426ffd640 1 -- 172.21.15.37:0/3446715322 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f34380ba940 con 0x557c20e0ead0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.649+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34380f0c90 con 0x557c20e0ead0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3426ffd640 1 -- 172.21.15.37:0/3446715322 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f34380b88e0 con 0x557c20e0ead0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 10 client.26633 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557c20e1e410 con 0x557c20e0ead0 2024-09-19T06:47:57.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3426ffd640 1 -- 172.21.15.37:0/3446715322 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34380bcd40 con 0x557c20e0ead0 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3426ffd640 1 -- 172.21.15.37:0/3446715322 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f34380b8cb0 con 0x557c20e0ead0 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 10 client.26633 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 10 client.26633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 20 client.26633 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e2b2a0 0x557c20e2d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 4 client.26633 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f34380f0c90 con 0x557c20e2b2a0 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.650+0000 7f343ffff640 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e2b2a0 0x557c20e2d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.651+0000 7f3426ffd640 10 client.26633 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.651+0000 7f343ffff640 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e2b2a0 0x557c20e2d6c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e949574df708107 server_cookie=b62be6159f704ac4 in_seq=0 out_seq=0 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3426ffd640 1 -- 172.21.15.37:0/3446715322 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f34380f0c90 con 0x557c20e2b2a0 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3426ffd640 10 client.26633 handle_command_reply: tid=0 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 client.26633 shutdown 2024-09-19T06:47:57.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 msgr2=0x557c20e1b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e1b0f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f34380b9010 tx=0x7f343802a420 comp rx=0 tx=0).stop 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 shutdown_connections 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e2b2a0 0x557c20e2d6c0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e18940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e183a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 --2- 172.21.15.37:0/3446715322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e1b0f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3426ffd640 0 client.26633 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 >> 172.21.15.37:0/3446715322 conn(0x557c20e0ac40 msgr2=0x557c20e109a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 shutdown_connections 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 1 -- 172.21.15.37:0/3446715322 wait complete. 2024-09-19T06:47:57.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.652+0000 7f3450076740 20 client.26633 trim_cache size 0 max 16384 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 Processor -- start 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 -- start start 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0ead0 0x557c20e27670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0fd00 0x557c20e16c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e1b740 0x557c20e15e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34481ae0c0 con 0x557c20e0fd00 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34481ab990 con 0x557c20e1b740 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.672+0000 7f3450076740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34481a97e0 con 0x557c20e0ead0 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0fd00 0x557c20e16c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0fd00 0x557c20e16c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36740/0 (socket says 172.21.15.37:36740) 2024-09-19T06:47:57.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f34477fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0ead0 0x557c20e27670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 -- 172.21.15.37:0/2698585348 learned_addr learned my addr 172.21.15.37:0/2698585348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:57.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f3447fff640 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e1b740 0x557c20e15e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 -- 172.21.15.37:0/2698585348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0ead0 msgr2=0x557c20e27670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0ead0 0x557c20e27670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 -- 172.21.15.37:0/2698585348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e1b740 msgr2=0x557c20e15e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e1b740 0x557c20e15e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 -- 172.21.15.37:0/2698585348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c20e17350 con 0x557c20e0fd00 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f34477fe640 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0ead0 0x557c20e27670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343ffff640 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0fd00 0x557c20e16c70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x557c20e17c30 tx=0x7f3430009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce961d30889158d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343d7fa640 1 -- 172.21.15.37:0/2698585348 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34300142d0 con 0x557c20e0fd00 2024-09-19T06:47:57.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343d7fa640 1 -- 172.21.15.37:0/2698585348 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f343001d940 con 0x557c20e0fd00 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.673+0000 7f343d7fa640 1 -- 172.21.15.37:0/2698585348 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3430012af0 con 0x557c20e0fd00 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 -- 172.21.15.37:0/2698585348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0fd00 msgr2=0x557c20e16c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0fd00 0x557c20e16c70 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x557c20e17c30 tx=0x7f3430009f90 comp rx=0 tx=0).stop 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 -- 172.21.15.37:0/2698585348 shutdown_connections 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e1b740 0x557c20e15e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e0fd00 0x557c20e16c70 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 --2- 172.21.15.37:0/2698585348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0ead0 0x557c20e27670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 -- 172.21.15.37:0/2698585348 >> 172.21.15.37:0/2698585348 conn(0x557c20e0ac40 msgr2=0x557c20e15700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 -- 172.21.15.37:0/2698585348 shutdown_connections 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.674+0000 7f3450076740 1 -- 172.21.15.37:0/2698585348 wait complete. 2024-09-19T06:47:57.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 -- start start 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 Processor -- start 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e32d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e33290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e337d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34481a9e50 con 0x557c20e1b740 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34481ab2c0 con 0x557c20e0ead0 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34481aa520 con 0x557c20e0fd00 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3450076740 10 client.0 _ll_register_callbacks cb 0x7f3440009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3447fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e337d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f343ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e33290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3447fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e337d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36744/0 (socket says 172.21.15.37:36744) 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f34477fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e32d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.675+0000 7f3447fff640 1 -- 172.21.15.37:0/2173803290 learned_addr learned my addr 172.21.15.37:0/2173803290 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f34477fe640 1 -- 172.21.15.37:0/2173803290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 msgr2=0x557c20e33290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f34477fe640 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e33290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f34477fe640 1 -- 172.21.15.37:0/2173803290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 msgr2=0x557c20e337d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f34477fe640 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e337d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f34477fe640 1 -- 172.21.15.37:0/2173803290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c20e95140 con 0x557c20e0ead0 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3447fff640 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f343ffff640 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e33290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f34477fe640 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e32d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f343802a9e0 tx=0x7f3438009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49a72da756b0f96b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3426ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3426ffd640 1 -- 172.21.15.37:0/2173803290 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3438049070 con 0x557c20e0ead0 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3426ffd640 1 -- 172.21.15.37:0/2173803290 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8527+0+0 (secure 0 0 0) 0x7f34380ba940 con 0x557c20e0ead0 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f343001d940 con 0x557c20e0ead0 2024-09-19T06:47:57.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3426ffd640 1 -- 172.21.15.37:0/2173803290 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7f34380bbfa0 con 0x557c20e0ead0 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3450076740 10 client.26639 fetch_fsmap learned FSMap version 31 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.676+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x557c20e95140 con 0x557c20e0ead0 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3426ffd640 1 -- 172.21.15.37:0/2173803290 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f343001d940 con 0x557c20e0ead0 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3426ffd640 1 -- 172.21.15.37:0/2173803290 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7f34380edd30 con 0x557c20e0ead0 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3450076740 10 client.26639 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3450076740 10 client.26639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3450076740 20 client.26639 populate_metadata read hostname 'smithi037' 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3450076740 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e96780 0x557c20e98b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3450076740 4 client.26639 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557c20d7f850 con 0x557c20e96780 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f343ffff640 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e96780 0x557c20e98b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f3426ffd640 10 client.26639 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.677+0000 7f343ffff640 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e96780 0x557c20e98b70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41a66c4b2388829 server_cookie=2ebdc24f1d494b14 in_seq=0 out_seq=0 2024-09-19T06:47:57.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.678+0000 7f3426ffd640 1 -- 172.21.15.37:0/2173803290 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x557c20d7f850 con 0x557c20e96780 2024-09-19T06:47:57.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.678+0000 7f3426ffd640 10 client.26639 handle_command_reply: tid=0 2024-09-19T06:47:57.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.678+0000 7f3450076740 1 client.26639 shutdown 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.678+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 msgr2=0x557c20e32d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.678+0000 7f3450076740 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e32d50 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f343802a9e0 tx=0x7f3438009260 comp rx=0 tx=0).stop 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 shutdown_connections 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557c20e96780 0x557c20e98b70 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557c20e1b740 0x557c20e337d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3426ffd640 0 client.26639 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557c20e0fd00 0x557c20e33290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 --2- 172.21.15.37:0/2173803290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557c20e0ead0 0x557c20e32d50 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 >> 172.21.15.37:0/2173803290 conn(0x557c20e0ac40 msgr2=0x557c20e15700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 shutdown_connections 2024-09-19T06:47:57.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 1 -- 172.21.15.37:0/2173803290 wait complete. 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f3450076740 20 client.26639 trim_cache size 0 max 16384 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: "668cad5f-916e-4586-aca7-ca024c1c45c1": { 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "668cad5f-916e-4586-aca7-ca024c1c45c1", 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:47:57.685 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34340a3430 msgr2=0x7f34340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f344dc9d640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34340a3430 0x7f34340a58f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3440009fd0 tx=0x7f3440009500 comp rx=0 tx=0).stop 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448100a80 msgr2=0x7f3448197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.679+0000 7f344dc9d640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448100a80 0x7f3448197380 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f343802a940 tx=0x7f3438003b20 comp rx=0 tx=0).stop 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 shutdown_connections 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3448106a80 0x7f344819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3448101420 0x7f34481978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34340a3430 0x7f34340a58f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 --2- 172.21.15.37:0/1636625013 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3448100a80 0x7f3448197380 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:47:57.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 >> 172.21.15.37:0/1636625013 conn(0x7f34480fc7a0 msgr2=0x7f34480fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:47:57.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 shutdown_connections 2024-09-19T06:47:57.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:47:57.680+0000 7f344dc9d640 1 -- 172.21.15.37:0/1636625013 wait complete. 2024-09-19T06:47:58.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 418537 7.20 MB/sec execute 145 sec latency 47.969 ms 2024-09-19T06:47:58.440 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:47:58 smithi037 ceph-mon[31211]: pgmap v189: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-19T06:47:58.522 INFO:tasks.cephfs.filesystem:scrub status for tag:668cad5f-916e-4586-aca7-ca024c1c45c1 - {'path': '/', 'tag': '668cad5f-916e-4586-aca7-ca024c1c45c1', 'options': 'recursive,force'} 2024-09-19T06:47:58.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:47:58 smithi096 ceph-mon[35004]: pgmap v189: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-19T06:47:58.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:47:58 smithi151 ceph-mon[35236]: pgmap v189: 97 pgs: 97 active+clean; 58 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.5 MiB/s wr, 213 op/s 2024-09-19T06:47:59.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 420498 7.20 MB/sec execute 146 sec latency 15.158 ms 2024-09-19T06:48:00.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 422958 7.25 MB/sec execute 147 sec latency 15.645 ms 2024-09-19T06:48:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:00 smithi096 ceph-mon[35004]: pgmap v190: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-09-19T06:48:00.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:00 smithi037 ceph-mon[31211]: pgmap v190: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-09-19T06:48:00.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:00 smithi151 ceph-mon[35236]: pgmap v190: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 3.6 MiB/s wr, 420 op/s 2024-09-19T06:48:01.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 425389 7.29 MB/sec execute 148 sec latency 29.214 ms 2024-09-19T06:48:01.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:48:01.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:48:01.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:48:01.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:48:01.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:48:01.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:48:01.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:48:01.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:48:01.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:48:01.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:48:01.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:48:01.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:48:02.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 427232 7.31 MB/sec execute 149 sec latency 24.459 ms 2024-09-19T06:48:02.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:02 smithi096 ceph-mon[35004]: pgmap v191: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2024-09-19T06:48:02.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:02 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:48:02.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:02 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:48:02.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:02 smithi037 ceph-mon[31211]: pgmap v191: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2024-09-19T06:48:02.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:02 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:48:02.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:02 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:48:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:02 smithi151 ceph-mon[35236]: pgmap v191: 97 pgs: 97 active+clean; 50 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2024-09-19T06:48:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:02 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:48:02.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:02 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:48:03.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 428665 7.31 MB/sec execute 150 sec latency 175.942 ms 2024-09-19T06:48:03.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:03 smithi096 ceph-mon[35004]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:48:03.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:03 smithi037 ceph-mon[31211]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:48:03.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:03 smithi151 ceph-mon[35236]: osdmap e86: 12 total, 12 up, 12 in 2024-09-19T06:48:04.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 430881 7.34 MB/sec execute 151 sec latency 13.275 ms 2024-09-19T06:48:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:04 smithi096 ceph-mon[35004]: pgmap v193: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:04.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:04 smithi037 ceph-mon[31211]: pgmap v193: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:04 smithi151 ceph-mon[35236]: pgmap v193: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:05.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 433447 7.39 MB/sec execute 152 sec latency 31.539 ms 2024-09-19T06:48:06.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 436521 7.46 MB/sec execute 153 sec latency 12.054 ms 2024-09-19T06:48:06.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:06 smithi096 ceph-mon[35004]: pgmap v194: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:06.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:06 smithi037 ceph-mon[31211]: pgmap v194: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:06.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:06 smithi151 ceph-mon[35236]: pgmap v194: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:07.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 438917 7.47 MB/sec execute 154 sec latency 11.938 ms 2024-09-19T06:48:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/3768116435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 msgr2=0x7fb2e810e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/3768116435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e810e550 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0009ca0 tx=0x7fb2e002a950 comp rx=0 tx=0).stop 2024-09-19T06:48:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/3768116435 shutdown_connections 2024-09-19T06:48:08.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/3768116435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e810e550 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/3768116435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e81035a0 0x7fb2e810ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/3768116435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e8102c80 0x7fb2e8103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/3768116435 >> 172.21.15.37:0/3768116435 conn(0x7fb2e8077990 msgr2=0x7fb2e80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/3768116435 shutdown_connections 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.050+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/3768116435 wait complete. 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 Processor -- start 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 -- start start 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e8102c80 0x7fb2e81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e81035a0 0x7fb2e8198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8111e90 con 0x7fb2e810c0d0 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8111d10 con 0x7fb2e81035a0 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ef8f8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e8112010 con 0x7fb2e8102c80 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ede95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e8102c80 0x7fb2e81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ede95640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e819cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51120/0 (socket says 172.21.15.37:51120) 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e8102c80 0x7fb2e81982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:53320/0 (socket says 172.21.15.37:53320) 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ede95640 1 -- 172.21.15.37:0/4062464412 learned_addr learned my addr 172.21.15.37:0/4062464412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.051+0000 7fb2ece93640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e81035a0 0x7fb2e8198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ede95640 1 -- 172.21.15.37:0/4062464412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e8102c80 msgr2=0x7fb2e81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ede95640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e8102c80 0x7fb2e81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ede95640 1 -- 172.21.15.37:0/4062464412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e81035a0 msgr2=0x7fb2e8198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ede95640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e81035a0 0x7fb2e8198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ede95640 1 -- 172.21.15.37:0/4062464412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2e819d2d0 con 0x7fb2e810c0d0 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ed694640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e8102c80 0x7fb2e81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ece93640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e81035a0 0x7fb2e8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ede95640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e819cba0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0004af0 tx=0x7fb2e002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2de7fc640 1 -- 172.21.15.37:0/4062464412 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0034040 con 0x7fb2e810c0d0 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2e819d560 con 0x7fb2e810c0d0 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2de7fc640 1 -- 172.21.15.37:0/4062464412 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e0030920 con 0x7fb2e810c0d0 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2de7fc640 1 -- 172.21.15.37:0/4062464412 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e0004670 con 0x7fb2e810c0d0 2024-09-19T06:48:08.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.052+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2e81a4e50 con 0x7fb2e810c0d0 2024-09-19T06:48:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 Processor -- start 2024-09-19T06:48:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2de7fc640 1 -- 172.21.15.37:0/4062464412 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb2e0037020 con 0x7fb2e810c0d0 2024-09-19T06:48:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 -- start start 2024-09-19T06:48:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a86ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a87410 0x55c440a91770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a940a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c4409eb2f0 con 0x55c440a86af0 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c440993270 con 0x55c440a91cb0 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2f2695740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c4409e72b0 con 0x55c440a87410 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2ece93640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a86ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a87410 0x55c440a91770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2ece93640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a86ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51130/0 (socket says 172.21.15.37:51130) 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a87410 0x55c440a91770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:53326/0 (socket says 172.21.15.37:53326) 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.056+0000 7fb2ece93640 1 -- 172.21.15.37:0/624149868 learned_addr learned my addr 172.21.15.37:0/624149868 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2ed694640 1 -- 172.21.15.37:0/624149868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 msgr2=0x55c440a940a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2ed694640 1 --2- 172.21.15.37:0/624149868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a940a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2ed694640 1 -- 172.21.15.37:0/624149868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 msgr2=0x55c440a86ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2ed694640 1 --2- 172.21.15.37:0/624149868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a86ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2ed694640 1 -- 172.21.15.37:0/624149868 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c440992340 con 0x55c440a87410 2024-09-19T06:48:08.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2de7fc640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb2d40a3460 0x7fb2d40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2ece93640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb2d40a3460 0x7fb2d40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2ed694640 1 --2- 172.21.15.37:0/624149868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a87410 0x55c440a91770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009fd0 tx=0x7fb2e40118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b0b0d86792026f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2de7fc640 1 -- 172.21.15.37:0/4062464412 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8615+0+0 (secure 0 0 0) 0x7fb2e00f0910 con 0x7fb2e810c0d0 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2d37fe640 1 -- 172.21.15.37:0/624149868 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004020 con 0x55c440a87410 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2d37fe640 1 -- 172.21.15.37:0/624149868 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4005500 con 0x55c440a87410 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.057+0000 7fb2d37fe640 1 -- 172.21.15.37:0/624149868 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4004d40 con 0x55c440a87410 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 -- 172.21.15.37:0/624149868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a87410 msgr2=0x55c440a91770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 --2- 172.21.15.37:0/624149868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a87410 0x55c440a91770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009fd0 tx=0x7fb2e40118d0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2ece93640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb2d40a3460 0x7fb2d40a5920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55c440a843e0 tx=0x7fb2d8009db0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 -- 172.21.15.37:0/624149868 shutdown_connections 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 --2- 172.21.15.37:0/624149868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a940a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 --2- 172.21.15.37:0/624149868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a87410 0x55c440a91770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 --2- 172.21.15.37:0/624149868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a86ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 -- 172.21.15.37:0/624149868 >> 172.21.15.37:0/624149868 conn(0x55c440a83060 msgr2=0x55c440a83440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 -- 172.21.15.37:0/624149868 shutdown_connections 2024-09-19T06:48:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.058+0000 7fb2f2695740 1 -- 172.21.15.37:0/624149868 wait complete. 2024-09-19T06:48:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 -- start start 2024-09-19T06:48:08.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 Processor -- start 2024-09-19T06:48:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a86af0 0x55c440a8eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a8f000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a8f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c440a98c90 con 0x55c440a87410 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c440974290 con 0x55c440a91cb0 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c440a98e10 con 0x55c440a86af0 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2f2695740 10 client.0 _ll_register_callbacks cb 0x55c4409a5140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2ece93640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a8f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a86af0 0x55c440a8eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2ece93640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a8f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51134/0 (socket says 172.21.15.37:51134) 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a86af0 0x55c440a8eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:53332/0 (socket says 172.21.15.37:53332) 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2ece93640 1 -- 172.21.15.37:0/2856369830 learned_addr learned my addr 172.21.15.37:0/2856369830 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.059+0000 7fb2ede95640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a8f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ed694640 1 -- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 msgr2=0x55c440a8f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ed694640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a8f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ed694640 1 -- 172.21.15.37:0/2856369830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 msgr2=0x55c440a8f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ed694640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a8f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ed694640 1 -- 172.21.15.37:0/2856369830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4406fe210 con 0x55c440a86af0 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ece93640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a8f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ede95640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a8f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2ed694640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a86af0 0x55c440a8eac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2e401b040 tx=0x7fb2e40038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5380590ad0de80e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/2856369830 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e401d050 con 0x55c440a86af0 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2e4005500 con 0x55c440a86af0 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/2856369830 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8615+0+0 (secure 0 0 0) 0x7fb2e4016040 con 0x55c440a86af0 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/2856369830 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb2e4031050 con 0x55c440a86af0 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2f2695740 10 client.26488 fetch_fsmap learned FSMap version 31 2024-09-19T06:48:08.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c4407ca620 con 0x55c440a86af0 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/2856369830 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2e4005500 con 0x55c440a86af0 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.060+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/2856369830 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb2e4004ff0 con 0x55c440a86af0 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2f2695740 10 client.26488 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2f2695740 10 client.26488 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2f2695740 20 client.26488 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2f2695740 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440aa5f10 0x55c440aa8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2f2695740 4 client.26488 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4409f8a20 con 0x55c440aa5f10 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2ece93640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440aa5f10 0x55c440aa8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2d0ff9640 10 client.26488 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.061+0000 7fb2ece93640 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440aa5f10 0x55c440aa8300 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c0d12bdec605c00 server_cookie=ec7ad37ab5bc959d in_seq=0 out_seq=0 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.062+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/2856369830 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c4409f8a20 con 0x55c440aa5f10 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.062+0000 7fb2d0ff9640 10 client.26488 handle_command_reply: tid=0 2024-09-19T06:48:08.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.062+0000 7fb2f2695740 1 client.26488 shutdown 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a86af0 msgr2=0x55c440a8eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a86af0 0x55c440a8eac0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb2e401b040 tx=0x7fb2e40038a0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 shutdown_connections 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440aa5f10 0x55c440aa8300 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a91cb0 0x55c440a8f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a8f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 --2- 172.21.15.37:0/2856369830 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a86af0 0x55c440a8eac0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 >> 172.21.15.37:0/2856369830 conn(0x55c440a83060 msgr2=0x55c440a88dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2d0ff9640 0 client.26488 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 shutdown_connections 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 1 -- 172.21.15.37:0/2856369830 wait complete. 2024-09-19T06:48:08.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.063+0000 7fb2f2695740 20 client.26488 trim_cache size 0 max 16384 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 Processor -- start 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 -- start start 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a8da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a87410 0x55c440aadad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440aae010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81adf10 con 0x55c440a86af0 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81ab7e0 con 0x55c440a87410 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2f2695740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81a9650 con 0x55c440a91cb0 2024-09-19T06:48:08.083 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.080+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a8da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a8da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51148/0 (socket says 172.21.15.37:51148) 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ed694640 1 -- 172.21.15.37:0/62051679 learned_addr learned my addr 172.21.15.37:0/62051679 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ece93640 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a87410 0x55c440aadad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ede95640 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440aae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ece93640 1 -- 172.21.15.37:0/62051679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 msgr2=0x55c440aae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ece93640 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440aae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ece93640 1 -- 172.21.15.37:0/62051679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 msgr2=0x55c440a8da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ece93640 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a8da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ece93640 1 -- 172.21.15.37:0/62051679 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c440a82e90 con 0x55c440a87410 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ed694640 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2ece93640 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a87410 0x55c440aadad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55c440a90070 tx=0x7fb2d801a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25a38c31923ebaf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2d2ffd640 1 -- 172.21.15.37:0/62051679 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d800fc00 con 0x55c440a87410 2024-09-19T06:48:08.084 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2d2ffd640 1 -- 172.21.15.37:0/62051679 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d800fda0 con 0x55c440a87410 2024-09-19T06:48:08.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.081+0000 7fb2d2ffd640 1 -- 172.21.15.37:0/62051679 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2d80035f0 con 0x55c440a87410 2024-09-19T06:48:08.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 -- 172.21.15.37:0/62051679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a87410 msgr2=0x55c440aadad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a87410 0x55c440aadad0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x55c440a90070 tx=0x7fb2d801a000 comp rx=0 tx=0).stop 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 -- 172.21.15.37:0/62051679 shutdown_connections 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440aae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a87410 0x55c440aadad0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 --2- 172.21.15.37:0/62051679 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a86af0 0x55c440a8da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 -- 172.21.15.37:0/62051679 >> 172.21.15.37:0/62051679 conn(0x55c440a83060 msgr2=0x55c440a8d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 -- 172.21.15.37:0/62051679 shutdown_connections 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.082+0000 7fb2f2695740 1 -- 172.21.15.37:0/62051679 wait complete. 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 -- start start 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 Processor -- start 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a86af0 0x55c440a921b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a926f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440a92c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81a9ca0 con 0x55c440a87410 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81ab110 con 0x55c440a86af0 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb2e81aa370 con 0x55c440a91cb0 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.083+0000 7fb2f2695740 10 client.0 _ll_register_callbacks cb 0x7fb2d8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a86af0 0x55c440a921b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ece93640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a926f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ede95640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440a92c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ed694640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a86af0 0x55c440a921b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54772/0 (socket says 172.21.15.37:54772) 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ed694640 1 -- 172.21.15.37:0/269533187 learned_addr learned my addr 172.21.15.37:0/269533187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ede95640 1 -- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a86af0 msgr2=0x55c440a921b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ede95640 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a86af0 0x55c440a921b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ede95640 1 -- 172.21.15.37:0/269533187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 msgr2=0x55c440a926f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ede95640 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a926f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ede95640 1 -- 172.21.15.37:0/269533187 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c440a9d340 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ed694640 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a86af0 0x55c440a921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2ede95640 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440a92c30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00ee110 tx=0x7fb2e00bc8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa3050660b1f4240 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/269533187 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2e002c040 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2d800fda0 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/269533187 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8615+0+0 (secure 0 0 0) 0x7fb2e00c5680 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/269533187 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) ==== 24+0+0 (secure 0 0 0) 0x7fb2e00bd3a0 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2f2695740 10 client.26494 fetch_fsmap learned FSMap version 31 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.084+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) -- 0x55c440a9e6a0 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/269533187 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2d800fda0 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/269533187 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 31) ==== 8346+0+0 (secure 0 0 0) 0x7fb2e004d030 con 0x55c440a91cb0 2024-09-19T06:48:08.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2f2695740 10 client.26494 fetch_fsmap finished waiting for FSMap version 31 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2f2695740 10 client.26494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2f2695740 20 client.26494 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2f2695740 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440b10aa0 0x55c440b12e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2f2695740 4 client.26494 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4409f2d70 con 0x55c440b10aa0 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2ed694640 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440b10aa0 0x55c440b12e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2d0ff9640 10 client.26494 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:08.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.085+0000 7fb2ed694640 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440b10aa0 0x55c440b12e90 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d38dca98b5b26c5 server_cookie=9d66bd0c9272b9c2 in_seq=0 out_seq=0 2024-09-19T06:48:08.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 440729 7.49 MB/sec execute 155 sec latency 49.452 ms 2024-09-19T06:48:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2d0ff9640 1 -- 172.21.15.37:0/269533187 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4409f2d70 con 0x55c440b10aa0 2024-09-19T06:48:08.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2d0ff9640 10 client.26494 handle_command_reply: tid=0 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 client.26494 shutdown 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 msgr2=0x55c440a92c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440a92c30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb2e00ee110 tx=0x7fb2e00bc8a0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 shutdown_connections 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c440b10aa0 0x55c440b12e90 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c440a91cb0 0x55c440a92c30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c440a87410 0x55c440a926f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 --2- 172.21.15.37:0/269533187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c440a86af0 0x55c440a921b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2d0ff9640 0 client.26494 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 >> 172.21.15.37:0/269533187 conn(0x55c440a83060 msgr2=0x55c440a8db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:08.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 shutdown_connections 2024-09-19T06:48:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.231+0000 7fb2f2695740 1 -- 172.21.15.37:0/269533187 wait complete. 2024-09-19T06:48:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2f2695740 20 client.26494 trim_cache size 0 max 16384 2024-09-19T06:48:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:48:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:48:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:48:08.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb2d40a3460 msgr2=0x7fb2d40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb2d40a3460 0x7fb2d40a5920 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x55c440a843e0 tx=0x7fb2d8009db0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 msgr2=0x7fb2e819cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e819cba0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0004af0 tx=0x7fb2e002f3a0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 shutdown_connections 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb2d40a3460 0x7fb2d40a5920 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb2e810c0d0 0x7fb2e819cba0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.236 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb2e81035a0 0x7fb2e8198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 --2- 172.21.15.37:0/4062464412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb2e8102c80 0x7fb2e81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 >> 172.21.15.37:0/4062464412 conn(0x7fb2e8077990 msgr2=0x7fb2e80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 shutdown_connections 2024-09-19T06:48:08.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:08.232+0000 7fb2ef8f8640 1 -- 172.21.15.37:0/4062464412 wait complete. 2024-09-19T06:48:08.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:08 smithi096 ceph-mon[35004]: pgmap v195: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:08.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:08 smithi037 ceph-mon[31211]: pgmap v195: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:08.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:08 smithi151 ceph-mon[35236]: pgmap v195: 97 pgs: 32 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 48 active+clean; 62 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 3.2 MiB/s wr, 408 op/s 2024-09-19T06:48:09.209 INFO:tasks.workunit.client.0.smithi037.stdout: 1 443612 7.56 MB/sec execute 156 sec latency 68.710 ms 2024-09-19T06:48:10.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 445872 7.57 MB/sec execute 157 sec latency 17.131 ms 2024-09-19T06:48:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:10 smithi096 ceph-mon[35004]: pgmap v196: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-09-19T06:48:10.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:10 smithi096 ceph-mon[35004]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:48:10.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:10 smithi037 ceph-mon[31211]: pgmap v196: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-09-19T06:48:10.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:10 smithi037 ceph-mon[31211]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:48:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:10 smithi151 ceph-mon[35236]: pgmap v196: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 3.4 MiB/s wr, 292 op/s 2024-09-19T06:48:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:10 smithi151 ceph-mon[35236]: osdmap e87: 12 total, 12 up, 12 in 2024-09-19T06:48:11.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 448226 7.60 MB/sec execute 158 sec latency 21.223 ms 2024-09-19T06:48:12.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 450941 7.66 MB/sec execute 159 sec latency 14.013 ms 2024-09-19T06:48:12.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:12 smithi096 ceph-mon[35004]: pgmap v198: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-09-19T06:48:12.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:12 smithi037 ceph-mon[31211]: pgmap v198: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-09-19T06:48:12.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:12 smithi151 ceph-mon[35236]: pgmap v198: 97 pgs: 97 active+clean; 64 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 MiB/s rd, 4.0 MiB/s wr, 343 op/s 2024-09-19T06:48:13.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 453276 7.68 MB/sec execute 160 sec latency 99.587 ms 2024-09-19T06:48:14.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 455622 7.72 MB/sec execute 161 sec latency 12.005 ms 2024-09-19T06:48:14.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:14 smithi096 ceph-mon[35004]: pgmap v199: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:14.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:14 smithi037 ceph-mon[31211]: pgmap v199: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:14.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:14 smithi151 ceph-mon[35236]: pgmap v199: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:15.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 458269 7.76 MB/sec execute 162 sec latency 14.060 ms 2024-09-19T06:48:16.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 460184 7.87 MB/sec execute 163 sec latency 48.722 ms 2024-09-19T06:48:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:16 smithi096 ceph-mon[35004]: pgmap v200: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:16.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:16 smithi037 ceph-mon[31211]: pgmap v200: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:16.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:16 smithi151 ceph-mon[35236]: pgmap v200: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:17.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 462787 7.91 MB/sec execute 164 sec latency 23.499 ms 2024-09-19T06:48:18.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 465648 7.97 MB/sec execute 165 sec latency 55.812 ms 2024-09-19T06:48:18.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:18 smithi096 ceph-mon[35004]: pgmap v201: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:18.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:18 smithi037 ceph-mon[31211]: pgmap v201: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:18.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:18 smithi151 ceph-mon[35236]: pgmap v201: 97 pgs: 97 active+clean; 52 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 3.7 MiB/s wr, 420 op/s 2024-09-19T06:48:19.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.93 MB/sec execute 166 sec latency 651.590 ms 2024-09-19T06:48:19.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:19 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:19.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:19 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:19.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:19 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:20.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.88 MB/sec execute 167 sec latency 1651.626 ms 2024-09-19T06:48:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:20 smithi096 ceph-mon[35004]: pgmap v202: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 431 op/s 2024-09-19T06:48:20.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:20 smithi037 ceph-mon[31211]: pgmap v202: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 431 op/s 2024-09-19T06:48:20.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:20 smithi151 ceph-mon[35236]: pgmap v202: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.6 MiB/s wr, 431 op/s 2024-09-19T06:48:21.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.83 MB/sec execute 168 sec latency 2651.745 ms 2024-09-19T06:48:21.338 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 -- 172.21.15.37:0/3453261353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8241009f0 msgr2=0x7ff824100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 --2- 172.21.15.37:0/3453261353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8241009f0 0x7ff824100e50 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7ff818009930 tx=0x7ff81802a420 comp rx=0 tx=0).stop 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 -- 172.21.15.37:0/3453261353 shutdown_connections 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 --2- 172.21.15.37:0/3453261353 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff824101390 0x7ff82410d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 --2- 172.21.15.37:0/3453261353 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff8241009f0 0x7ff824100e50 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 --2- 172.21.15.37:0/3453261353 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff824106a00 0x7ff824106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 -- 172.21.15.37:0/3453261353 >> 172.21.15.37:0/3453261353 conn(0x7ff8240fc6b0 msgr2=0x7ff8240fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.335+0000 7ff828f76640 1 -- 172.21.15.37:0/3453261353 shutdown_connections 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.336+0000 7ff828f76640 1 -- 172.21.15.37:0/3453261353 wait complete. 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.336+0000 7ff828f76640 1 Processor -- start 2024-09-19T06:48:21.339 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.336+0000 7ff828f76640 1 -- start start 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff828f76640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8241009f0 0x7ff824197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff828f76640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff824101390 0x7ff8241977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff828f76640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff824106a00 0x7ff82419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff828f76640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff824110e40 con 0x7ff824106a00 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff828f76640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff824110cc0 con 0x7ff8241009f0 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff828f76640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff824110fc0 con 0x7ff824101390 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff824106a00 0x7ff82419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff824106a00 0x7ff82419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43112/0 (socket says 172.21.15.37:43112) 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 -- 172.21.15.37:0/4178064361 learned_addr learned my addr 172.21.15.37:0/4178064361 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff821d9b640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff824101390 0x7ff8241977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff82259c640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8241009f0 0x7ff824197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 -- 172.21.15.37:0/4178064361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff824101390 msgr2=0x7ff8241977d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff824101390 0x7ff8241977d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.340 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 -- 172.21.15.37:0/4178064361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8241009f0 msgr2=0x7ff824197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8241009f0 0x7ff824197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 -- 172.21.15.37:0/4178064361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff82419c280 con 0x7ff824106a00 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff822d9d640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff824106a00 0x7ff82419bb50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eaa0 tx=0x7ff81c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.337+0000 7ff8137fe640 1 -- 172.21.15.37:0/4178064361 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c00cdb0 con 0x7ff824106a00 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.338+0000 7ff8137fe640 1 -- 172.21.15.37:0/4178064361 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81c005020 con 0x7ff824106a00 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.338+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff82419c570 con 0x7ff824106a00 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.338+0000 7ff8137fe640 1 -- 172.21.15.37:0/4178064361 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff81c0116b0 con 0x7ff824106a00 2024-09-19T06:48:21.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.338+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8241a3e00 con 0x7ff824106a00 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.340+0000 7ff8137fe640 1 -- 172.21.15.37:0/4178064361 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff81c005560 con 0x7ff824106a00 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.340+0000 7ff82bded740 1 Processor -- start 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82bded740 1 -- start start 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82bded740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c4af0 0x5640dd9c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82bded740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c5410 0x5640dd9cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82bded740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dd9d20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82bded740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640dd9292f0 con 0x5640dd9c5410 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82bded740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640dd8d1270 con 0x5640dd9cfcb0 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82bded740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5640dd9252b0 con 0x5640dd9c4af0 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff8137fe640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff80c0a34e0 0x7ff80c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c5410 0x5640dd9cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.354 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dd9d20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dd9d20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:37828/0 (socket says 172.21.15.37:37828) 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff82259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c5410 0x5640dd9cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43120/0 (socket says 172.21.15.37:43120) 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff821d9b640 1 -- 172.21.15.37:0/262507171 learned_addr learned my addr 172.21.15.37:0/262507171 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff821d9b640 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c4af0 0x5640dd9c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff822d9d640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff80c0a34e0 0x7ff80c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.341+0000 7ff8137fe640 1 -- 172.21.15.37:0/4178064361 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7ff81c0cffb0 con 0x7ff824106a00 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff82259c640 1 -- 172.21.15.37:0/262507171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c4af0 msgr2=0x5640dd9c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff82259c640 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c4af0 0x5640dd9c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff82259c640 1 -- 172.21.15.37:0/262507171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 msgr2=0x5640dd9d20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff82259c640 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dd9d20a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff82259c640 1 -- 172.21.15.37:0/262507171 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640dd8d0340 con 0x5640dd9c5410 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff82259c640 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c5410 0x5640dd9cf770 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff81400bb60 tx=0x7ff81400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6a96726bce7133 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff822d9d640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff80c0a34e0 0x7ff80c0a59a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0cd780 tx=0x7ff81c0916f0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.355 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff810ff9640 1 -- 172.21.15.37:0/262507171 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814015070 con 0x5640dd9c5410 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff810ff9640 1 -- 172.21.15.37:0/262507171 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81400be50 con 0x5640dd9c5410 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.342+0000 7ff810ff9640 1 -- 172.21.15.37:0/262507171 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff814011440 con 0x5640dd9c5410 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 -- 172.21.15.37:0/262507171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c5410 msgr2=0x5640dd9cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c5410 0x5640dd9cf770 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ff81400bb60 tx=0x7ff81400f730 comp rx=0 tx=0).stop 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 -- 172.21.15.37:0/262507171 shutdown_connections 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dd9d20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c5410 0x5640dd9cf770 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 --2- 172.21.15.37:0/262507171 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c4af0 0x5640dd9c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 -- 172.21.15.37:0/262507171 >> 172.21.15.37:0/262507171 conn(0x5640dd9c1060 msgr2=0x5640dd9c1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 -- 172.21.15.37:0/262507171 shutdown_connections 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 -- 172.21.15.37:0/262507171 wait complete. 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 -- start start 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.343+0000 7ff82bded740 1 Processor -- start 2024-09-19T06:48:21.356 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82bded740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c4af0 0x5640dd9ccbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82bded740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82bded740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9cfcb0 0x5640dd9cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82bded740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640dd9d2880 con 0x5640dd9cfcb0 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82bded740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640dd8b2290 con 0x5640dd9c4af0 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82bded740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5640dd9d2a00 con 0x5640dd9c5410 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82bded740 10 client.0 _ll_register_callbacks cb 0x5640dd8e3140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9cfcb0 0x5640dd9cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9cfcb0 0x5640dd9cd650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43128/0 (socket says 172.21.15.37:43128) 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 -- 172.21.15.37:0/2939002787 learned_addr learned my addr 172.21.15.37:0/2939002787 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c4af0 0x5640dd9ccbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82259c640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 -- 172.21.15.37:0/2939002787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 msgr2=0x5640dd9cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.357 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 -- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c4af0 msgr2=0x5640dd9ccbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c4af0 0x5640dd9ccbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 -- 172.21.15.37:0/2939002787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640dd63c210 con 0x5640dd9cfcb0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff82259c640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff821d9b640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9cfcb0 0x5640dd9cd650 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff818009230 tx=0x7ff818009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4bcd0d35d73e19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.344+0000 7ff809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff809ffb640 1 -- 172.21.15.37:0/2939002787 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818035040 con 0x5640dd9cfcb0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff809ffb640 1 -- 172.21.15.37:0/2939002787 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7ff818004460 con 0x5640dd9cfcb0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81400be50 con 0x5640dd9cfcb0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff809ffb640 1 -- 172.21.15.37:0/2939002787 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff818051050 con 0x5640dd9cfcb0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 10 client.17268 fetch_fsmap learned FSMap version 32 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5640dd708620 con 0x5640dd9cfcb0 2024-09-19T06:48:21.358 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff809ffb640 1 -- 172.21.15.37:0/2939002787 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81400be50 con 0x5640dd9cfcb0 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff809ffb640 1 -- 172.21.15.37:0/2939002787 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7ff81803a040 con 0x5640dd9cfcb0 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 10 client.17268 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 10 client.17268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 20 client.17268 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dd9dfc20 0x5640dd9e2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 4 client.17268 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.345+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640dd936a20 con 0x5640dd9dfc20 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.346+0000 7ff82259c640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dd9dfc20 0x5640dd9e2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.346+0000 7ff82259c640 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dd9dfc20 0x5640dd9e2070 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a5b037cb125284 server_cookie=80d7c434a7c7e461 in_seq=0 out_seq=0 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.346+0000 7ff809ffb640 10 client.17268 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff809ffb640 1 -- 172.21.15.37:0/2939002787 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640dd936a20 con 0x5640dd9dfc20 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff809ffb640 10 client.17268 handle_command_reply: tid=0 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 client.17268 shutdown 2024-09-19T06:48:21.359 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9cfcb0 msgr2=0x5640dd9cd650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9cfcb0 0x5640dd9cd650 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ff818009230 tx=0x7ff818009660 comp rx=0 tx=0).stop 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 shutdown_connections 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dd9dfc20 0x5640dd9e2070 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9cfcb0 0x5640dd9cd650 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff809ffb640 0 client.17268 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 --2- 172.21.15.37:0/2939002787 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c4af0 0x5640dd9ccbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 >> 172.21.15.37:0/2939002787 conn(0x5640dd9c1060 msgr2=0x5640dd9c6d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 shutdown_connections 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 1 -- 172.21.15.37:0/2939002787 wait complete. 2024-09-19T06:48:21.360 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.347+0000 7ff82bded740 20 client.17268 trim_cache size 0 max 16384 2024-09-19T06:48:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 Processor -- start 2024-09-19T06:48:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 -- start start 2024-09-19T06:48:21.363 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9c7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c5410 0x5640dd9cd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9cfcb0 0x5640dd9e7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241b01d0 con 0x5640dd9c4af0 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241adaa0 con 0x5640dd9c5410 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82bded740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ab910 con 0x5640dd9cfcb0 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9c7560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43140/0 (socket says 172.21.15.37:43140) 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 -- 172.21.15.37:0/3203137373 learned_addr learned my addr 172.21.15.37:0/3203137373 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82259c640 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c5410 0x5640dd9cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 -- 172.21.15.37:0/3203137373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9cfcb0 msgr2=0x5640dd9e7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9cfcb0 0x5640dd9e7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.364 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 -- 172.21.15.37:0/3203137373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c5410 msgr2=0x5640dd9cd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c5410 0x5640dd9cd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 -- 172.21.15.37:0/3203137373 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640dd9c4920 con 0x5640dd9c4af0 2024-09-19T06:48:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff82259c640 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c5410 0x5640dd9cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.362+0000 7ff821d9b640 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9c7560 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff818005ec0 tx=0x7ff818009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b801816df5c0fb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff80bfff640 1 -- 172.21.15.37:0/3203137373 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818035070 con 0x5640dd9c4af0 2024-09-19T06:48:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff80bfff640 1 -- 172.21.15.37:0/3203137373 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81803da50 con 0x5640dd9c4af0 2024-09-19T06:48:21.365 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff80bfff640 1 -- 172.21.15.37:0/3203137373 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818003d30 con 0x5640dd9c4af0 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 -- 172.21.15.37:0/3203137373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 msgr2=0x5640dd9c7560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9c7560 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7ff818005ec0 tx=0x7ff818009260 comp rx=0 tx=0).stop 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 -- 172.21.15.37:0/3203137373 shutdown_connections 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9cfcb0 0x5640dd9e7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9c5410 0x5640dd9cd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 --2- 172.21.15.37:0/3203137373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9c7560 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 -- 172.21.15.37:0/3203137373 >> 172.21.15.37:0/3203137373 conn(0x5640dd9c1060 msgr2=0x5640dd9cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.369 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 -- 172.21.15.37:0/3203137373 shutdown_connections 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.363+0000 7ff82bded740 1 -- 172.21.15.37:0/3203137373 wait complete. 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 -- start start 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 Processor -- start 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9e7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9e7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dda46070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241abf60 con 0x5640dd9c4af0 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ad3d0 con 0x5640dd9cfcb0 2024-09-19T06:48:21.370 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff8241ac630 con 0x5640dd9c5410 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82bded740 10 client.0 _ll_register_callbacks cb 0x7ff818009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9e7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9e7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43146/0 (socket says 172.21.15.37:43146) 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 -- 172.21.15.37:0/3111615293 learned_addr learned my addr 172.21.15.37:0/3111615293 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff82259c640 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9e7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 -- 172.21.15.37:0/3111615293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 msgr2=0x5640dd9e7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9e7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 -- 172.21.15.37:0/3111615293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 msgr2=0x5640dda46070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dda46070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.364+0000 7ff821d9b640 1 -- 172.21.15.37:0/3111615293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640dd9bc2d0 con 0x5640dd9c4af0 2024-09-19T06:48:21.371 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff821d9b640 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9e7930 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff818005b90 tx=0x7ff81802aa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db62dd770203f93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff809ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff809ffb640 1 -- 172.21.15.37:0/3111615293 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818035040 con 0x5640dd9c4af0 2024-09-19T06:48:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff809ffb640 1 -- 172.21.15.37:0/3111615293 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7ff818032e30 con 0x5640dd9c4af0 2024-09-19T06:48:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff81803da50 con 0x5640dd9c4af0 2024-09-19T06:48:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff809ffb640 1 -- 172.21.15.37:0/3111615293 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7ff818003a30 con 0x5640dd9c4af0 2024-09-19T06:48:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff82bded740 10 client.17274 fetch_fsmap learned FSMap version 32 2024-09-19T06:48:21.372 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5640dda4eaf0 con 0x5640dd9c4af0 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff809ffb640 1 -- 172.21.15.37:0/3111615293 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff81803da50 con 0x5640dd9c4af0 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff809ffb640 1 -- 172.21.15.37:0/3111615293 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7ff818067930 con 0x5640dd9c4af0 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff82bded740 10 client.17274 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff82bded740 10 client.17274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff82bded740 20 client.17274 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.365+0000 7ff82bded740 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dda4f100 0x5640dda514f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.366+0000 7ff82bded740 4 client.17274 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.366+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640dd930d70 con 0x5640dda4f100 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.366+0000 7ff82259c640 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dda4f100 0x5640dda514f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:21.373 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.366+0000 7ff809ffb640 10 client.17274 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:21.374 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.366+0000 7ff82259c640 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dda4f100 0x5640dda514f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea975483c0017b47 server_cookie=141da918fe1b0283 in_seq=0 out_seq=0 2024-09-19T06:48:21.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.526+0000 7ff809ffb640 1 -- 172.21.15.37:0/3111615293 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640dd930d70 con 0x5640dda4f100 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.526+0000 7ff809ffb640 10 client.17274 handle_command_reply: tid=0 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.526+0000 7ff82bded740 1 client.17274 shutdown 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.526+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 msgr2=0x5640dd9e7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.526+0000 7ff82bded740 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9e7930 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7ff818005b90 tx=0x7ff81802aa60 comp rx=0 tx=0).stop 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 shutdown_connections 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640dda4f100 0x5640dda514f0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640dd9cfcb0 0x5640dda46070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640dd9c5410 0x5640dd9e7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 --2- 172.21.15.37:0/3111615293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640dd9c4af0 0x5640dd9e7930 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 >> 172.21.15.37:0/3111615293 conn(0x5640dd9c1060 msgr2=0x5640dd9bf9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff809ffb640 0 client.17274 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:21.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 shutdown_connections 2024-09-19T06:48:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 1 -- 172.21.15.37:0/3111615293 wait complete. 2024-09-19T06:48:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.527+0000 7ff82bded740 20 client.17274 trim_cache size 0 max 16384 2024-09-19T06:48:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:48:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:48:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:48:21.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff80c0a34e0 msgr2=0x7ff80c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff80c0a34e0 0x7ff80c0a59a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0cd780 tx=0x7ff81c0916f0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff824106a00 msgr2=0x7ff82419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff824106a00 0x7ff82419bb50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7ff81c00eaa0 tx=0x7ff81c00ef60 comp rx=0 tx=0).stop 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 shutdown_connections 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff80c0a34e0 0x7ff80c0a59a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff824106a00 0x7ff82419bb50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff824101390 0x7ff8241977d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 --2- 172.21.15.37:0/4178064361 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff8241009f0 0x7ff824197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:21.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 >> 172.21.15.37:0/4178064361 conn(0x7ff8240fc6b0 msgr2=0x7ff8240fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 shutdown_connections 2024-09-19T06:48:21.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:21.528+0000 7ff828f76640 1 -- 172.21.15.37:0/4178064361 wait complete. 2024-09-19T06:48:22.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.79 MB/sec execute 169 sec latency 3651.820 ms 2024-09-19T06:48:22.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:22 smithi096 ceph-mon[35004]: pgmap v203: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 396 op/s 2024-09-19T06:48:22.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:22 smithi037 ceph-mon[31211]: pgmap v203: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 396 op/s 2024-09-19T06:48:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:22 smithi151 ceph-mon[35236]: pgmap v203: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 396 op/s 2024-09-19T06:48:23.210 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.74 MB/sec execute 170 sec latency 4651.884 ms 2024-09-19T06:48:24.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.70 MB/sec execute 171 sec latency 5652.002 ms 2024-09-19T06:48:24.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:24 smithi096 ceph-mon[35004]: pgmap v204: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-09-19T06:48:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:24 smithi037 ceph-mon[31211]: pgmap v204: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-09-19T06:48:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:24 smithi151 ceph-mon[35236]: pgmap v204: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.8 MiB/s wr, 512 op/s 2024-09-19T06:48:25.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.65 MB/sec execute 172 sec latency 6652.103 ms 2024-09-19T06:48:26.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.61 MB/sec execute 173 sec latency 7652.253 ms 2024-09-19T06:48:26.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:26 smithi096 ceph-mon[35004]: pgmap v205: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-19T06:48:26.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:26 smithi037 ceph-mon[31211]: pgmap v205: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-19T06:48:26.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:26 smithi151 ceph-mon[35236]: pgmap v205: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-19T06:48:27.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466058 7.56 MB/sec execute 174 sec latency 8652.325 ms 2024-09-19T06:48:28.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 466372 7.52 MB/sec execute 175 sec latency 9507.156 ms 2024-09-19T06:48:28.523 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:48:28.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:28 smithi096 ceph-mon[35004]: pgmap v206: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-19T06:48:28.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:28 smithi037 ceph-mon[31211]: pgmap v206: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-19T06:48:28.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:28 smithi151 ceph-mon[35236]: pgmap v206: 97 pgs: 97 active+clean; 46 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.1 MiB/s wr, 272 op/s 2024-09-19T06:48:28.793 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:29.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 469245 7.58 MB/sec execute 176 sec latency 50.003 ms 2024-09-19T06:48:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 -- 172.21.15.37:0/1868937640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe810c180 msgr2=0x7f8fe810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/1868937640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe810c180 0x7f8fe810e600 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009ca0 tx=0x7f8fe002a950 comp rx=0 tx=0).stop 2024-09-19T06:48:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 -- 172.21.15.37:0/1868937640 shutdown_connections 2024-09-19T06:48:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/1868937640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe810c180 0x7f8fe810e600 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/1868937640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe81036c0 0x7f8fe810bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/1868937640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe8102da0 0x7f8fe8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 -- 172.21.15.37:0/1868937640 >> 172.21.15.37:0/1868937640 conn(0x7f8fe80fc7a0 msgr2=0x7f8fe80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 -- 172.21.15.37:0/1868937640 shutdown_connections 2024-09-19T06:48:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 -- 172.21.15.37:0/1868937640 wait complete. 2024-09-19T06:48:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.325+0000 7f8fec9a2640 1 Processor -- start 2024-09-19T06:48:29.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fec9a2640 1 -- start start 2024-09-19T06:48:29.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fec9a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe8102da0 0x7f8fe81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fec9a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe81036c0 0x7f8fe8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fec9a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe810c180 0x7f8fe819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fec9a2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8111f20 con 0x7f8fe8102da0 2024-09-19T06:48:29.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fec9a2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe8111da0 con 0x7f8fe810c180 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fec9a2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81120a0 con 0x7f8fe81036c0 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe8102da0 0x7f8fe81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe8102da0 0x7f8fe81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43154/0 (socket says 172.21.15.37:43154) 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe81036c0 0x7f8fe8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 -- 172.21.15.37:0/168731659 learned_addr learned my addr 172.21.15.37:0/168731659 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe6d9d640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe810c180 0x7f8fe819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 -- 172.21.15.37:0/168731659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe81036c0 msgr2=0x7f8fe8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe81036c0 0x7f8fe8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 -- 172.21.15.37:0/168731659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe810c180 msgr2=0x7f8fe819ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe810c180 0x7f8fe819ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 -- 172.21.15.37:0/168731659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fe819d390 con 0x7f8fe8102da0 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.326+0000 7f8fe659c640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe8102da0 0x7f8fe81983e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800d950 tx=0x7f8fd800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:29.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.327+0000 7f8fdf7fe640 1 -- 172.21.15.37:0/168731659 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8017070 con 0x7f8fe8102da0 2024-09-19T06:48:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.327+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fe819d680 con 0x7f8fe8102da0 2024-09-19T06:48:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.327+0000 7f8fdf7fe640 1 -- 172.21.15.37:0/168731659 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd8009c30 con 0x7f8fe8102da0 2024-09-19T06:48:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.327+0000 7f8fdf7fe640 1 -- 172.21.15.37:0/168731659 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd800f6b0 con 0x7f8fe8102da0 2024-09-19T06:48:29.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.327+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fe81a4f60 con 0x7f8fe8102da0 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.329+0000 7f8fdf7fe640 1 -- 172.21.15.37:0/168731659 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8fd80109d0 con 0x7f8fe8102da0 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.329+0000 7f8feed7b740 1 Processor -- start 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8feed7b740 1 -- start start 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bf4a0 0x5603222bf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222ca140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8feed7b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5603222252e0 con 0x5603222bfdc0 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8feed7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603221ccce0 con 0x5603222ca680 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8feed7b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603222212a0 con 0x5603222bf4a0 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222ca140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222ca140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43158/0 (socket says 172.21.15.37:43158) 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 -- 172.21.15.37:0/2012488348 learned_addr learned my addr 172.21.15.37:0/2012488348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe6d9d640 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bf4a0 0x5603222bf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fdffff640 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 -- 172.21.15.37:0/2012488348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bf4a0 msgr2=0x5603222bf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bf4a0 0x5603222bf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 -- 172.21.15.37:0/2012488348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 msgr2=0x5603222cca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fdf7fe640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fd40a33b0 0x7f8fd40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222cca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.330+0000 7f8fe659c640 1 -- 172.21.15.37:0/2012488348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603221b1440 con 0x5603222bfdc0 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.331+0000 7f8fe6d9d640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fd40a33b0 0x7f8fd40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.331+0000 7f8fdf7fe640 1 -- 172.21.15.37:0/168731659 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7f8fd80cfc30 con 0x7f8fe8102da0 2024-09-19T06:48:29.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.331+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222ca140 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80cd2c0 tx=0x7f8fd80982f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8835fbc6ff9dc08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:29.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.331+0000 7f8fdcff9640 1 -- 172.21.15.37:0/2012488348 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8020070 con 0x5603222bfdc0 2024-09-19T06:48:29.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.331+0000 7f8fdcff9640 1 -- 172.21.15.37:0/2012488348 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd80d3e40 con 0x5603222bfdc0 2024-09-19T06:48:29.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.331+0000 7f8fdcff9640 1 -- 172.21.15.37:0/2012488348 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd809bad0 con 0x5603222bfdc0 2024-09-19T06:48:29.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 -- 172.21.15.37:0/2012488348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 msgr2=0x5603222ca140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222ca140 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f8fd80cd2c0 tx=0x7f8fd80982f0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8fe6d9d640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fd40a33b0 0x7f8fd40a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0005e30 tx=0x7f8fe0005c50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 -- 172.21.15.37:0/2012488348 shutdown_connections 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222cca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222ca140 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2012488348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bf4a0 0x5603222bf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 -- 172.21.15.37:0/2012488348 >> 172.21.15.37:0/2012488348 conn(0x5603222bba30 msgr2=0x5603222bbe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 -- 172.21.15.37:0/2012488348 shutdown_connections 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 -- 172.21.15.37:0/2012488348 wait complete. 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.332+0000 7f8feed7b740 1 -- start start 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 1 Processor -- start 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222c9fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222c7250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222c7790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5603222cd250 con 0x5603222bfdc0 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603221e0050 con 0x5603222bf4a0 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603222cd3d0 con 0x5603222ca680 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8feed7b740 10 client.0 _ll_register_callbacks cb 0x560322220160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222c9fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222c7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43166/0 (socket says 172.21.15.37:43166) 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 -- 172.21.15.37:0/2872203959 learned_addr learned my addr 172.21.15.37:0/2872203959 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fe6d9d640 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 -- 172.21.15.37:0/2872203959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 msgr2=0x5603222c7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 -- 172.21.15.37:0/2872203959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 msgr2=0x5603222c9fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222c9fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.333+0000 7f8fdffff640 1 -- 172.21.15.37:0/2872203959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560321f963c0 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8fdffff640 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222c7250 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0012700 tx=0x7f8fd0012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5cbcb3cf5ec5203 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8fcdffb640 1 -- 172.21.15.37:0/2872203959 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd000f8c0 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8fcdffb640 1 -- 172.21.15.37:0/2872203959 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7f8fd000fa60 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560322232850 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8fcdffb640 1 -- 172.21.15.37:0/2872203959 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fd000fc60 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8feed7b740 10 client.17286 fetch_fsmap learned FSMap version 32 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5603222dd090 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.334+0000 7f8fcdffb640 1 -- 172.21.15.37:0/2872203959 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x560322232850 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8fcdffb640 1 -- 172.21.15.37:0/2872203959 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f8fd0010550 con 0x5603222bfdc0 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8feed7b740 10 client.17286 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:48:29.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8feed7b740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8feed7b740 20 client.17286 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603222de5a0 0x5603222e09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8feed7b740 4 client.17286 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56032222cd60 con 0x5603222de5a0 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603222de5a0 0x5603222e09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603222de5a0 0x5603222e09e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37957e8490f51c1d server_cookie=bb2f14fcb35c5554 in_seq=0 out_seq=0 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.335+0000 7f8fcdffb640 10 client.17286 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8fcdffb640 1 -- 172.21.15.37:0/2872203959 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56032222cd60 con 0x5603222de5a0 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8fcdffb640 10 client.17286 handle_command_reply: tid=0 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 client.17286 shutdown 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 msgr2=0x5603222c7250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222c7250 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0012700 tx=0x7f8fd0012bc0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 shutdown_connections 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603222de5a0 0x5603222e09e0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222c7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222c7250 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2872203959 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222c9fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 >> 172.21.15.37:0/2872203959 conn(0x5603222bba30 msgr2=0x5603222c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8fcdffb640 0 client.17286 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 shutdown_connections 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 1 -- 172.21.15.37:0/2872203959 wait complete. 2024-09-19T06:48:29.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.337+0000 7f8feed7b740 20 client.17286 trim_cache size 0 max 16384 2024-09-19T06:48:29.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.354+0000 7f8feed7b740 1 Processor -- start 2024-09-19T06:48:29.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.354+0000 7f8feed7b740 1 -- start start 2024-09-19T06:48:29.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bf4a0 0x5603222e62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bfdc0 0x5603222e67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8feed7b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81ae040 con 0x5603222bf4a0 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8feed7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81ab910 con 0x5603222ca680 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8feed7b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81a9780 con 0x5603222bfdc0 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bf4a0 0x5603222e62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bf4a0 0x5603222e62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43180/0 (socket says 172.21.15.37:43180) 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 -- 172.21.15.37:0/2652600593 learned_addr learned my addr 172.21.15.37:0/2652600593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe6d9d640 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fdffff640 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bfdc0 0x5603222e67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 -- 172.21.15.37:0/2652600593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bfdc0 msgr2=0x5603222e67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bfdc0 0x5603222e67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 -- 172.21.15.37:0/2652600593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 msgr2=0x5603222d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 -- 172.21.15.37:0/2652600593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603222cd130 con 0x5603222bf4a0 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fe659c640 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bf4a0 0x5603222e62b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009610 tx=0x7f8fd8005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d534f7fdbe9ef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.355+0000 7f8fcffff640 1 -- 172.21.15.37:0/2652600593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8020040 con 0x5603222bf4a0 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8fcffff640 1 -- 172.21.15.37:0/2652600593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd80d2070 con 0x5603222bf4a0 2024-09-19T06:48:29.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8fcffff640 1 -- 172.21.15.37:0/2652600593 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd809b8f0 con 0x5603222bf4a0 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 -- 172.21.15.37:0/2652600593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bf4a0 msgr2=0x5603222e62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bf4a0 0x5603222e62b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8009610 tx=0x7f8fd8005bb0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 -- 172.21.15.37:0/2652600593 shutdown_connections 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222ca680 0x5603222d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222bfdc0 0x5603222e67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 --2- 172.21.15.37:0/2652600593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bf4a0 0x5603222e62b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 -- 172.21.15.37:0/2652600593 >> 172.21.15.37:0/2652600593 conn(0x5603222bba30 msgr2=0x5603222c6340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 -- 172.21.15.37:0/2652600593 shutdown_connections 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.356+0000 7f8feed7b740 1 -- 172.21.15.37:0/2652600593 wait complete. 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 -- start start 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 Processor -- start 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222d2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222d2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81a9dd0 con 0x5603222bfdc0 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81ab240 con 0x5603222bf4a0 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8fe81aa4a0 con 0x5603222ca680 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8feed7b740 10 client.0 _ll_register_callbacks cb 0x7f8fd0009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222d2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8fdffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222d2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43182/0 (socket says 172.21.15.37:43182) 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8fdffff640 1 -- 172.21.15.37:0/927177226 learned_addr learned my addr 172.21.15.37:0/927177226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8fe659c640 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222d2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8fe6d9d640 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8fdffff640 1 -- 172.21.15.37:0/927177226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 msgr2=0x5603222cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.357+0000 7f8fdffff640 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fdffff640 1 -- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 msgr2=0x5603222d2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fdffff640 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222d2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fdffff640 1 -- 172.21.15.37:0/927177226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603222d07e0 con 0x5603222bfdc0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fdffff640 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222d2f50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00059f0 tx=0x7f8fd000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa2159203e5455b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fcdffb640 1 -- 172.21.15.37:0/927177226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fd0028070 con 0x5603222bfdc0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fcdffb640 1 -- 172.21.15.37:0/927177226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7f8fd0010430 con 0x5603222bfdc0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fd80d2070 con 0x5603222bfdc0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fcdffb640 1 -- 172.21.15.37:0/927177226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f8fd0034050 con 0x5603222bfdc0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8feed7b740 10 client.17292 fetch_fsmap learned FSMap version 32 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x560322348f50 con 0x5603222bfdc0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fcdffb640 1 -- 172.21.15.37:0/927177226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8fd80d2070 con 0x5603222bfdc0 2024-09-19T06:48:29.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8fcdffb640 1 -- 172.21.15.37:0/927177226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 32) ==== 8350+0+0 (secure 0 0 0) 0x7f8fd0003a00 con 0x5603222bfdc0 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8feed7b740 10 client.17292 fetch_fsmap finished waiting for FSMap version 32 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8feed7b740 10 client.17292 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.358+0000 7f8feed7b740 20 client.17292 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.359+0000 7f8feed7b740 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560322349670 0x56032234ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.359+0000 7f8feed7b740 4 client.17292 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.359+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5603221a9210 con 0x560322349670 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.359+0000 7f8fe659c640 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560322349670 0x56032234ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.359+0000 7f8fe659c640 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560322349670 0x56032234ba90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed354956d5f4493 server_cookie=7e2c63abafd79d63 in_seq=0 out_seq=0 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.359+0000 7f8fcdffb640 10 client.17292 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8fcdffb640 1 -- 172.21.15.37:0/927177226 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5603221a9210 con 0x560322349670 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8fcdffb640 10 client.17292 handle_command_reply: tid=0 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 client.17292 shutdown 2024-09-19T06:48:29.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 msgr2=0x5603222d2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222d2f50 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f8fd00059f0 tx=0x7f8fd000f810 comp rx=0 tx=0).stop 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 shutdown_connections 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560322349670 0x56032234ba90 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603222ca680 0x5603222cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603222bfdc0 0x5603222d2f50 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 --2- 172.21.15.37:0/927177226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603222bf4a0 0x5603222d2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 >> 172.21.15.37:0/927177226 conn(0x5603222bba30 msgr2=0x5603222ba350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8fcdffb640 0 client.17292 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.360+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 shutdown_connections 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8feed7b740 1 -- 172.21.15.37:0/927177226 wait complete. 2024-09-19T06:48:29.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8feed7b740 20 client.17292 trim_cache size 0 max 16384 2024-09-19T06:48:29.367 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:48:29.367 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:48:29.367 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:48:29.367 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:48:29.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fd40a33b0 msgr2=0x7f8fd40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fd40a33b0 0x7f8fd40a5870 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0005e30 tx=0x7f8fe0005c50 comp rx=0 tx=0).stop 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe8102da0 msgr2=0x7f8fe81983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe8102da0 0x7f8fe81983e0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f8fd800d950 tx=0x7f8fd800de10 comp rx=0 tx=0).stop 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 shutdown_connections 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fe810c180 0x7f8fe819ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8fe81036c0 0x7f8fe8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8fd40a33b0 0x7f8fd40a5870 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 --2- 172.21.15.37:0/168731659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8fe8102da0 0x7f8fe81983e0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 >> 172.21.15.37:0/168731659 conn(0x7f8fe80fc7a0 msgr2=0x7f8fe80fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 shutdown_connections 2024-09-19T06:48:29.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:29.361+0000 7f8fec9a2640 1 -- 172.21.15.37:0/168731659 wait complete. 2024-09-19T06:48:30.095 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:48:30.096 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:48:30.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 471712 7.61 MB/sec execute 177 sec latency 18.425 ms 2024-09-19T06:48:30.368 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:30.489 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:30 smithi037 ceph-mon[31211]: pgmap v207: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-09-19T06:48:30.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:30 smithi096 ceph-mon[35004]: pgmap v207: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-09-19T06:48:30.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:30 smithi151 ceph-mon[35236]: pgmap v207: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.0 MiB/s wr, 306 op/s 2024-09-19T06:48:30.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 -- 172.21.15.37:0/1692330521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc106c80 msgr2=0x7fa0fc107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 --2- 172.21.15.37:0/1692330521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc106c80 0x7fa0fc107060 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009930 tx=0x7fa0ec02a420 comp rx=0 tx=0).stop 2024-09-19T06:48:30.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 -- 172.21.15.37:0/1692330521 shutdown_connections 2024-09-19T06:48:30.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 --2- 172.21.15.37:0/1692330521 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0fc100600 0x7fa0fc10e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 --2- 172.21.15.37:0/1692330521 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0fc0ffc40 0x7fa0fc1000c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 --2- 172.21.15.37:0/1692330521 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc106c80 0x7fa0fc107060 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 -- 172.21.15.37:0/1692330521 >> 172.21.15.37:0/1692330521 conn(0x7fa0fc0fba90 msgr2=0x7fa0fc0fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 -- 172.21.15.37:0/1692330521 shutdown_connections 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.888+0000 7fa102524640 1 -- 172.21.15.37:0/1692330521 wait complete. 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 Processor -- start 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 -- start start 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0fc0ffc40 0x7fa0fc1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc100600 0x7fa0fc198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0fc106c80 0x7fa0fc19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc111f30 con 0x7fa0fc100600 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc111db0 con 0x7fa0fc0ffc40 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa102524640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0fc1120b0 con 0x7fa0fc106c80 2024-09-19T06:48:30.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc100600 0x7fa0fc198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc100600 0x7fa0fc198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49124/0 (socket says 172.21.15.37:49124) 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0fbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0fc0ffc40 0x7fa0fc1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 -- 172.21.15.37:0/2152248783 learned_addr learned my addr 172.21.15.37:0/2152248783 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa100ac1640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0fc106c80 0x7fa0fc19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 -- 172.21.15.37:0/2152248783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0fc106c80 msgr2=0x7fa0fc19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0fc106c80 0x7fa0fc19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 -- 172.21.15.37:0/2152248783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0fc0ffc40 msgr2=0x7fa0fc1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0fc0ffc40 0x7fa0fc1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 -- 172.21.15.37:0/2152248783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0fc19d380 con 0x7fa0fc100600 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0fbfff640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0fc0ffc40 0x7fa0fc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.889+0000 7fa0f3fff640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc100600 0x7fa0fc198910 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400e9f0 tx=0x7fa0e400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:30.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.890+0000 7fa0f9ffb640 1 -- 172.21.15.37:0/2152248783 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400cd80 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.890+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0fc19d670 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.890+0000 7fa0f9ffb640 1 -- 172.21.15.37:0/2152248783 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa0e4010910 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.890+0000 7fa0f9ffb640 1 -- 172.21.15.37:0/2152248783 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e40188f0 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.890+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0fc1a4f50 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.891+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0fc1000c0 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.892+0000 7fa0f9ffb640 1 -- 172.21.15.37:0/2152248783 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa0e4010ac0 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.893+0000 7fa0f9ffb640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0e80a3430 0x7fa0e80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.893+0000 7fa0f9ffb640 1 -- 172.21.15.37:0/2152248783 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7fa0e40d01f0 con 0x7fa0fc100600 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.893+0000 7fa0fbfff640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0e80a3430 0x7fa0e80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:30.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.894+0000 7fa0fbfff640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0e80a3430 0x7fa0e80a58f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009900 tx=0x7fa0ec0058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:30.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:30.898+0000 7fa0f9ffb640 1 -- 172.21.15.37:0/2152248783 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fa0e409cb40 con 0x7fa0fc100600 2024-09-19T06:48:31.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.119+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa0fc10a040 con 0x7fa0fc100600 2024-09-19T06:48:31.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.120+0000 7fa0f9ffb640 1 -- 172.21.15.37:0/2152248783 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) ==== 94+0+11904 (secure 0 0 0) 0x7fa0e409c440 con 0x7fa0fc100600 2024-09-19T06:48:31.121 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:48:31.121 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":32,"btime":"2024-09-19T06:48:18:199849+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":32,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:48:18.039935+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25663":{"gid":25663,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/3248587852","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":3248587852},{"type":"v1","addr":"172.21.15.151:6839","nonce":3248587852}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:48:31.121 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 32 2024-09-19T06:48:31.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0e80a3430 msgr2=0x7fa0e80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:31.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0e80a3430 0x7fa0e80a58f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009900 tx=0x7fa0ec0058d0 comp rx=0 tx=0).stop 2024-09-19T06:48:31.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc100600 msgr2=0x7fa0fc198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:31.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc100600 0x7fa0fc198910 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fa0e400e9f0 tx=0x7fa0e400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 shutdown_connections 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0fc106c80 0x7fa0fc19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0e80a3430 0x7fa0e80a58f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0fc100600 0x7fa0fc198910 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 --2- 172.21.15.37:0/2152248783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0fc0ffc40 0x7fa0fc1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.122+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 >> 172.21.15.37:0/2152248783 conn(0x7fa0fc0fba90 msgr2=0x7fa0fc0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.123+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 shutdown_connections 2024-09-19T06:48:31.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:31.123+0000 7fa102524640 1 -- 172.21.15.37:0/2152248783 wait complete. 2024-09-19T06:48:31.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 474198 7.64 MB/sec execute 178 sec latency 17.135 ms 2024-09-19T06:48:31.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:31 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2152248783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:31.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:31 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2152248783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:31.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:31 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2152248783' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:48:31.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:48:32.137 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:32.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 476955 7.68 MB/sec execute 179 sec latency 11.463 ms 2024-09-19T06:48:32.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:32 smithi096 ceph-mon[35004]: pgmap v208: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:48:32.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:32 smithi037 ceph-mon[31211]: pgmap v208: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:48:32.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:32 smithi151 ceph-mon[35236]: pgmap v208: 97 pgs: 97 active+clean; 45 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 210 KiB/s rd, 2.0 MiB/s wr, 187 op/s 2024-09-19T06:48:32.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 -- 172.21.15.37:0/3680704716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074f40 msgr2=0x7f428810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3680704716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074f40 0x7f428810de80 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f4270009900 tx=0x7f427002a480 comp rx=0 tx=0).stop 2024-09-19T06:48:32.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 -- 172.21.15.37:0/3680704716 shutdown_connections 2024-09-19T06:48:32.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3680704716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f428810e3c0 0x7f4288110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3680704716 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074f40 0x7f428810de80 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3680704716 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288074620 0x7f4288074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 -- 172.21.15.37:0/3680704716 >> 172.21.15.37:0/3680704716 conn(0x7f42880fd8c0 msgr2=0x7f42880ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 -- 172.21.15.37:0/3680704716 shutdown_connections 2024-09-19T06:48:32.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.647+0000 7f428cb5c640 1 -- 172.21.15.37:0/3680704716 wait complete. 2024-09-19T06:48:32.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 Processor -- start 2024-09-19T06:48:32.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 -- start start 2024-09-19T06:48:32.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074620 0x7f428819a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288074f40 0x7f428819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f428810e3c0 0x7f428819ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4288114140 con 0x7f4288074620 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4288113fc0 con 0x7f428810e3c0 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.648+0000 7f428cb5c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42881142c0 con 0x7f4288074f40 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074620 0x7f428819a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f427ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288074f40 0x7f428819abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074620 0x7f428819a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49140/0 (socket says 172.21.15.37:49140) 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 -- 172.21.15.37:0/3229554355 learned_addr learned my addr 172.21.15.37:0/3229554355 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f4286d9d640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f428810e3c0 0x7f428819ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 -- 172.21.15.37:0/3229554355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288074f40 msgr2=0x7f428819abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288074f40 0x7f428819abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 -- 172.21.15.37:0/3229554355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f428810e3c0 msgr2=0x7f428819ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f428810e3c0 0x7f428819ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 -- 172.21.15.37:0/3229554355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f428819f650 con 0x7f4288074620 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428659c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074620 0x7f428819a6a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f427800c920 tx=0x7f427800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f427f7fe640 1 -- 172.21.15.37:0/3229554355 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278007c70 con 0x7f4288074620 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.649+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f428819f940 con 0x7f4288074620 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.650+0000 7f427f7fe640 1 -- 172.21.15.37:0/3229554355 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4278007e10 con 0x7f4288074620 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.650+0000 7f427f7fe640 1 -- 172.21.15.37:0/3229554355 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4278018890 con 0x7f4288074620 2024-09-19T06:48:32.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.650+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42881a7220 con 0x7f4288074620 2024-09-19T06:48:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.652+0000 7f427f7fe640 1 -- 172.21.15.37:0/3229554355 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4278018b30 con 0x7f4288074620 2024-09-19T06:48:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.653+0000 7f428ef35740 1 Processor -- start 2024-09-19T06:48:32.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.653+0000 7f428ef35740 1 -- start start 2024-09-19T06:48:32.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.653+0000 7f427f7fe640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42740a3430 0x7f42740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.653+0000 7f4286d9d640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42740a3430 0x7f42740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.653+0000 7f427f7fe640 1 -- 172.21.15.37:0/3229554355 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7f42780cfde0 con 0x7f4288074620 2024-09-19T06:48:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.653+0000 7f428ef35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce168090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428ef35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce1685d0 0x562dce168a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428ef35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce173370 0x562dce175760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428ef35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dce0ce2f0 con 0x562dce1685d0 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428ef35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562dce075cf0 con 0x562dce173370 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428ef35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562dce0ca2b0 con 0x562dce169b40 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce168090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce168090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:38856/0 (socket says 172.21.15.37:38856) 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 -- 172.21.15.37:0/4124933386 learned_addr learned my addr 172.21.15.37:0/4124933386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f427ffff640 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce1685d0 0x562dce168a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f4286d9d640 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce173370 0x562dce175760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f4286d9d640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42740a3430 0x7f42740a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4280005fd0 tx=0x7f42800107a0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 -- 172.21.15.37:0/4124933386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce173370 msgr2=0x562dce175760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce173370 0x562dce175760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 -- 172.21.15.37:0/4124933386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce1685d0 msgr2=0x562dce168a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce1685d0 0x562dce168a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 -- 172.21.15.37:0/4124933386 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dce05a3a0 con 0x562dce169b40 2024-09-19T06:48:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f427ffff640 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce1685d0 0x562dce168a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f428659c640 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce168090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4278098a60 tx=0x7f4278018f20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd0280c28aa02280 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f427cff9640 1 -- 172.21.15.37:0/4124933386 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780b5070 con 0x562dce169b40 2024-09-19T06:48:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.654+0000 7f427cff9640 1 -- 172.21.15.37:0/4124933386 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42780a0440 con 0x562dce169b40 2024-09-19T06:48:32.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f427cff9640 1 -- 172.21.15.37:0/4124933386 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780cd780 con 0x562dce169b40 2024-09-19T06:48:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f428ef35740 1 -- 172.21.15.37:0/4124933386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 msgr2=0x562dce168090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f428ef35740 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce168090 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f4278098a60 tx=0x7f4278018f20 comp rx=0 tx=0).stop 2024-09-19T06:48:32.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f428ef35740 1 -- 172.21.15.37:0/4124933386 shutdown_connections 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f428ef35740 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce173370 0x562dce175760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f428ef35740 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce1685d0 0x562dce168a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f428ef35740 1 --2- 172.21.15.37:0/4124933386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce168090 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.655+0000 7f428ef35740 1 -- 172.21.15.37:0/4124933386 >> 172.21.15.37:0/4124933386 conn(0x562dce131310 msgr2=0x562dce164c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 -- 172.21.15.37:0/4124933386 shutdown_connections 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 -- 172.21.15.37:0/4124933386 wait complete. 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 -- start start 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 Processor -- start 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce172d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce170060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce1706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562dce176070 con 0x562dce169b40 2024-09-19T06:48:32.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562dce089060 con 0x562dce1685d0 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562dce1761f0 con 0x562dce173370 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f428ef35740 10 client.0 _ll_register_callbacks cb 0x562dcdddc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f427ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce170060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.656+0000 7f427ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce170060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49152/0 (socket says 172.21.15.37:49152) 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f427ffff640 1 -- 172.21.15.37:0/4020824068 learned_addr learned my addr 172.21.15.37:0/4020824068 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f428659c640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce172d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f4286d9d640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce1706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f427ffff640 1 -- 172.21.15.37:0/4020824068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 msgr2=0x562dce1706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f427ffff640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce1706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f427ffff640 1 -- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 msgr2=0x562dce172d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f427ffff640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce172d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f427ffff640 1 -- 172.21.15.37:0/4020824068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dcde3f3c0 con 0x562dce169b40 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f428659c640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce172d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f427ffff640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce170060 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f427002a9a0 tx=0x7f42700318d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818a1fe2a0406f64 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f426dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f426dffb640 1 -- 172.21.15.37:0/4020824068 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4270035040 con 0x562dce169b40 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42780a0440 con 0x562dce169b40 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f426dffb640 1 -- 172.21.15.37:0/4020824068 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7f4270004240 con 0x562dce169b40 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.657+0000 7f426dffb640 1 -- 172.21.15.37:0/4020824068 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4270004440 con 0x562dce169b40 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 10 client.17310 fetch_fsmap learned FSMap version 33 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562dce181b50 con 0x562dce169b40 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f426dffb640 1 -- 172.21.15.37:0/4020824068 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42780a0440 con 0x562dce169b40 2024-09-19T06:48:32.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f426dffb640 1 -- 172.21.15.37:0/4020824068 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f427004c070 con 0x562dce169b40 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 10 client.17310 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 10 client.17310 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 20 client.17310 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce183010 0x562dce185440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 4 client.17310 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dce0db860 con 0x562dce183010 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.658+0000 7f428659c640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce183010 0x562dce185440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.659+0000 7f428659c640 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce183010 0x562dce185440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=41a4ce714f48c998 server_cookie=1a3fa4eea7cc56aa in_seq=0 out_seq=0 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.659+0000 7f426dffb640 10 client.17310 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.659+0000 7f426dffb640 1 -- 172.21.15.37:0/4020824068 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562dce0db860 con 0x562dce183010 2024-09-19T06:48:32.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.659+0000 7f426dffb640 10 client.17310 handle_command_reply: tid=0 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.659+0000 7f428ef35740 1 client.17310 shutdown 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 msgr2=0x562dce170060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce170060 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f427002a9a0 tx=0x7f42700318d0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 shutdown_connections 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce183010 0x562dce185440 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce1706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce170060 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 --2- 172.21.15.37:0/4020824068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce172d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 >> 172.21.15.37:0/4020824068 conn(0x562dce131310 msgr2=0x562dce16a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f426dffb640 0 client.17310 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 shutdown_connections 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.660+0000 7f428ef35740 1 -- 172.21.15.37:0/4020824068 wait complete. 2024-09-19T06:48:32.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.661+0000 7f428ef35740 20 client.17310 trim_cache size 0 max 16384 2024-09-19T06:48:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.674+0000 7f428ef35740 1 Processor -- start 2024-09-19T06:48:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.674+0000 7f428ef35740 1 -- start start 2024-09-19T06:48:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428ef35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce16f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428ef35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce16b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428ef35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce173370 0x562dce16b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428ef35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42881b24b0 con 0x562dce173370 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428ef35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42881afd50 con 0x562dce1685d0 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428ef35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42881adbc0 con 0x562dce169b40 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f4286d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce173370 0x562dce16b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce16f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f4286d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce173370 0x562dce16b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49162/0 (socket says 172.21.15.37:49162) 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f4286d9d640 1 -- 172.21.15.37:0/2842023507 learned_addr learned my addr 172.21.15.37:0/2842023507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f427ffff640 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce16b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428659c640 1 -- 172.21.15.37:0/2842023507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 msgr2=0x562dce16b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428659c640 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce16b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428659c640 1 -- 172.21.15.37:0/2842023507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce173370 msgr2=0x562dce16b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428659c640 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce173370 0x562dce16b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428659c640 1 -- 172.21.15.37:0/2842023507 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dce16f010 con 0x562dce1685d0 2024-09-19T06:48:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f427ffff640 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce16b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:48:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f4286d9d640 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce173370 0x562dce16b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.675+0000 7f428659c640 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce16f920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42780cd5b0 tx=0x7f4278018f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59a0c8eb2f4bf91c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f426ffff640 1 -- 172.21.15.37:0/2842023507 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780b5040 con 0x562dce1685d0 2024-09-19T06:48:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f426ffff640 1 -- 172.21.15.37:0/2842023507 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42780a0440 con 0x562dce1685d0 2024-09-19T06:48:32.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f426ffff640 1 -- 172.21.15.37:0/2842023507 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42780a4ad0 con 0x562dce1685d0 2024-09-19T06:48:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 -- 172.21.15.37:0/2842023507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 msgr2=0x562dce16f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce16f920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f42780cd5b0 tx=0x7f4278018f90 comp rx=0 tx=0).stop 2024-09-19T06:48:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 -- 172.21.15.37:0/2842023507 shutdown_connections 2024-09-19T06:48:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce173370 0x562dce16b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce169b40 0x562dce16b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 --2- 172.21.15.37:0/2842023507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce16f920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 -- 172.21.15.37:0/2842023507 >> 172.21.15.37:0/2842023507 conn(0x562dce131310 msgr2=0x562dce1720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.676+0000 7f428ef35740 1 -- 172.21.15.37:0/2842023507 shutdown_connections 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 -- 172.21.15.37:0/2842023507 wait complete. 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 -- start start 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 Processor -- start 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce177240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce177780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce177cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f42881ae210 con 0x562dce169b40 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f42881af680 con 0x562dce1685d0 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f42881ae8e0 con 0x562dce173370 2024-09-19T06:48:32.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f427ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce177780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce177240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428ef35740 10 client.0 _ll_register_callbacks cb 0x7f4270009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce177240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43684/0 (socket says 172.21.15.37:43684) 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f428659c640 1 -- 172.21.15.37:0/3586667022 learned_addr learned my addr 172.21.15.37:0/3586667022 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.677+0000 7f427ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce177780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49168/0 (socket says 172.21.15.37:49168) 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f4286d9d640 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce177cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428659c640 1 -- 172.21.15.37:0/3586667022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 msgr2=0x562dce177cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428659c640 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce177cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428659c640 1 -- 172.21.15.37:0/3586667022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 msgr2=0x562dce177780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428659c640 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce177780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428659c640 1 -- 172.21.15.37:0/3586667022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dce170a30 con 0x562dce1685d0 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f427ffff640 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce177780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428659c640 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce177240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f42780a47d0 tx=0x7f427800ff30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e83133d24fa15b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:32.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f426dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f426dffb640 1 -- 172.21.15.37:0/3586667022 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f427807e030 con 0x562dce1685d0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42780a0440 con 0x562dce1685d0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f426dffb640 1 -- 172.21.15.37:0/3586667022 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7f42780bc0b0 con 0x562dce1685d0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f426dffb640 1 -- 172.21.15.37:0/3586667022 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f42780b5040 con 0x562dce1685d0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428ef35740 10 client.26711 fetch_fsmap learned FSMap version 33 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.678+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562dce1edda0 con 0x562dce1685d0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f426dffb640 1 -- 172.21.15.37:0/3586667022 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42780a0440 con 0x562dce1685d0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f426dffb640 1 -- 172.21.15.37:0/3586667022 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7f42780a4870 con 0x562dce1685d0 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f428ef35740 10 client.26711 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f428ef35740 10 client.26711 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f428ef35740 20 client.26711 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f428ef35740 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce1ee030 0x562dce1f0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:32.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f428ef35740 4 client.26711 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562dce0d5d70 con 0x562dce1ee030 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f427ffff640 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce1ee030 0x562dce1f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f427ffff640 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce1ee030 0x562dce1f0420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b17797247eec2147 server_cookie=a19a296099c0d5f in_seq=0 out_seq=0 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.679+0000 7f426dffb640 10 client.26711 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f426dffb640 1 -- 172.21.15.37:0/3586667022 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562dce0d5d70 con 0x562dce1ee030 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f426dffb640 10 client.26711 handle_command_reply: tid=0 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 client.26711 shutdown 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 msgr2=0x562dce177240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce177240 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f42780a47d0 tx=0x7f427800ff30 comp rx=0 tx=0).stop 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 shutdown_connections 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562dce1ee030 0x562dce1f0420 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562dce173370 0x562dce177cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562dce169b40 0x562dce177780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 --2- 172.21.15.37:0/3586667022 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562dce1685d0 0x562dce177240 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f426dffb640 0 client.26711 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:48:32.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 >> 172.21.15.37:0/3586667022 conn(0x562dce131310 msgr2=0x562dce1634e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 shutdown_connections 2024-09-19T06:48:32.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 1 -- 172.21.15.37:0/3586667022 wait complete. 2024-09-19T06:48:32.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428ef35740 20 client.26711 trim_cache size 0 max 16384 2024-09-19T06:48:32.688 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:48:32.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42740a3430 msgr2=0x7f42740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42740a3430 0x7f42740a58f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4280005fd0 tx=0x7f42800107a0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074620 msgr2=0x7f428819a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:32.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.680+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074620 0x7f428819a6a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f427800c920 tx=0x7f427800cde0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 shutdown_connections 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f428810e3c0 0x7f428819ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4288074f40 0x7f428819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42740a3430 0x7f42740a58f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 --2- 172.21.15.37:0/3229554355 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4288074620 0x7f428819a6a0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 >> 172.21.15.37:0/3229554355 conn(0x7f42880fd8c0 msgr2=0x7f42880fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 shutdown_connections 2024-09-19T06:48:32.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:32.681+0000 7f428cb5c640 1 -- 172.21.15.37:0/3229554355 wait complete. 2024-09-19T06:48:33.211 INFO:tasks.workunit.client.0.smithi037.stdout: 1 480028 7.74 MB/sec execute 180 sec latency 25.896 ms 2024-09-19T06:48:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:33 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:33.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:33 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:33.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:33 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:33.675 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:48:33.946 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:34.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 482556 7.76 MB/sec execute 181 sec latency 40.632 ms 2024-09-19T06:48:34.338 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:34 smithi037 ceph-mon[31211]: pgmap v209: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.429+0000 7efd922ca640 1 -- 172.21.15.37:0/2767034668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 msgr2=0x7efd8c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.429+0000 7efd922ca640 1 --2- 172.21.15.37:0/2767034668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c101200 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7efd7c009900 tx=0x7efd7c02a480 comp rx=0 tx=0).stop 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 -- 172.21.15.37:0/2767034668 shutdown_connections 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 --2- 172.21.15.37:0/2767034668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c102ef0 0x7efd8c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 --2- 172.21.15.37:0/2767034668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd8c102570 0x7efd8c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 --2- 172.21.15.37:0/2767034668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c101200 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 -- 172.21.15.37:0/2767034668 >> 172.21.15.37:0/2767034668 conn(0x7efd8c0fc7d0 msgr2=0x7efd8c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 -- 172.21.15.37:0/2767034668 shutdown_connections 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 -- 172.21.15.37:0/2767034668 wait complete. 2024-09-19T06:48:34.448 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 Processor -- start 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.430+0000 7efd922ca640 1 -- start start 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd922ca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd922ca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd8c102570 0x7efd8c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd922ca640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c102ef0 0x7efd8c19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd922ca640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c110f80 con 0x7efd8c100e20 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd922ca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c110e00 con 0x7efd8c102570 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd922ca640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c111100 con 0x7efd8c102ef0 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49196/0 (socket says 172.21.15.37:49196) 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 -- 172.21.15.37:0/3231720156 learned_addr learned my addr 172.21.15.37:0/3231720156 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd8c102570 0x7efd8c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 -- 172.21.15.37:0/3231720156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c102ef0 msgr2=0x7efd8c19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.449 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c102ef0 0x7efd8c19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 -- 172.21.15.37:0/3231720156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd8c102570 msgr2=0x7efd8c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd8c102570 0x7efd8c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 -- 172.21.15.37:0/3231720156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd8c19d4b0 con 0x7efd8c100e20 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.431+0000 7efd8bfff640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c198500 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006010 tx=0x7efd7c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.432+0000 7efd897fa640 1 -- 172.21.15.37:0/3231720156 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c03f070 con 0x7efd8c100e20 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.432+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd8c19d740 con 0x7efd8c100e20 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.432+0000 7efd897fa640 1 -- 172.21.15.37:0/3231720156 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7c031c30 con 0x7efd8c100e20 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.432+0000 7efd897fa640 1 -- 172.21.15.37:0/3231720156 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c035030 con 0x7efd8c100e20 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.432+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd8c1a5080 con 0x7efd8c100e20 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.434+0000 7efd897fa640 1 -- 172.21.15.37:0/3231720156 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efd7c042070 con 0x7efd8c100e20 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.434+0000 7efd946a3740 1 Processor -- start 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.434+0000 7efd946a3740 1 -- start start 2024-09-19T06:48:34.450 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd946a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd946a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd946a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd946a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5574a3b8c2f0 con 0x5574a3c27b40 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd946a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574a3b33cf0 con 0x5574a3c265d0 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd946a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574a3b882b0 con 0x5574a3c31370 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c26090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49202/0 (socket says 172.21.15.37:49202) 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 -- 172.21.15.37:0/467560141 learned_addr learned my addr 172.21.15.37:0/467560141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd897fa640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd740a33b0 0x7efd740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 -- 172.21.15.37:0/467560141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 msgr2=0x5574a3c33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.451 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c33760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 -- 172.21.15.37:0/467560141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 msgr2=0x5574a3c26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 -- 172.21.15.37:0/467560141 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a3b183a0 con 0x5574a3c27b40 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd8bfff640 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd897fa640 1 -- 172.21.15.37:0/3231720156 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7efd7c0f0450 con 0x7efd8c100e20 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.435+0000 7efd90867640 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c26090 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7efd84009de0 tx=0x7efd8400d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13aaff8fb034a3ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd90867640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd740a33b0 0x7efd740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd727fc640 1 -- 172.21.15.37:0/467560141 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd84016070 con 0x5574a3c27b40 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd727fc640 1 -- 172.21.15.37:0/467560141 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd84010b40 con 0x5574a3c27b40 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd727fc640 1 -- 172.21.15.37:0/467560141 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8400f3c0 con 0x5574a3c27b40 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd946a3740 1 -- 172.21.15.37:0/467560141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 msgr2=0x5574a3c26090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.452 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd90867640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd740a33b0 0x7efd740a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd84018000 tx=0x7efd8401be90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd946a3740 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c26090 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7efd84009de0 tx=0x7efd8400d6b0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd946a3740 1 -- 172.21.15.37:0/467560141 shutdown_connections 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd946a3740 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd946a3740 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd946a3740 1 --2- 172.21.15.37:0/467560141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c26090 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.436+0000 7efd946a3740 1 -- 172.21.15.37:0/467560141 >> 172.21.15.37:0/467560141 conn(0x5574a3bef310 msgr2=0x5574a3c22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 -- 172.21.15.37:0/467560141 shutdown_connections 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 -- 172.21.15.37:0/467560141 wait complete. 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 -- start start 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 Processor -- start 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c265d0 0x5574a3c30ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c27b40 0x5574a3c2df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c2e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.453 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5574a3c34070 con 0x5574a3c265d0 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5574a3b47060 con 0x5574a3c27b40 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.437+0000 7efd946a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5574a3c341f0 con 0x5574a3c31370 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd946a3740 10 client.0 _ll_register_callbacks cb 0x5574a389a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c265d0 0x5574a3c30ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c27b40 0x5574a3c2df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c265d0 0x5574a3c30ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49210/0 (socket says 172.21.15.37:49210) 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8bfff640 1 -- 172.21.15.37:0/399159224 learned_addr learned my addr 172.21.15.37:0/399159224 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd90867640 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c2e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8b7fe640 1 -- 172.21.15.37:0/399159224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 msgr2=0x5574a3c2e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c2e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8b7fe640 1 -- 172.21.15.37:0/399159224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c265d0 msgr2=0x5574a3c30ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c265d0 0x5574a3c30ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8b7fe640 1 -- 172.21.15.37:0/399159224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a38fd3c0 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8bfff640 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c265d0 0x5574a3c30ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c27b40 0x5574a3c2df90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0fff70 tx=0x7efd80012b60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b8c34f255a01591 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd6b7fe640 1 -- 172.21.15.37:0/399159224 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80004050 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd6b7fe640 1 -- 172.21.15.37:0/399159224 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7efd800041f0 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.438+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd84010b40 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd6b7fe640 1 -- 172.21.15.37:0/399159224 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efd800043f0 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 10 client.26723 fetch_fsmap learned FSMap version 33 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5574a3c3fae0 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd6b7fe640 1 -- 172.21.15.37:0/399159224 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd84010b40 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd6b7fe640 1 -- 172.21.15.37:0/399159224 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7efd80023070 con 0x5574a3c27b40 2024-09-19T06:48:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 10 client.26723 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 10 client.26723 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 20 client.26723 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3c40ff0 0x5574a3c43430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 4 client.26723 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574a3b99860 con 0x5574a3c40ff0 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.439+0000 7efd8bfff640 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3c40ff0 0x5574a3c43430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.440+0000 7efd6b7fe640 10 client.26723 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.440+0000 7efd8bfff640 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3c40ff0 0x5574a3c43430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e6d05702d4c32d1d server_cookie=fa9e170c5781e252 in_seq=0 out_seq=0 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.440+0000 7efd6b7fe640 1 -- 172.21.15.37:0/399159224 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5574a3b99860 con 0x5574a3c40ff0 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.440+0000 7efd6b7fe640 10 client.26723 handle_command_reply: tid=0 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.440+0000 7efd946a3740 1 client.26723 shutdown 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.440+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c27b40 msgr2=0x5574a3c2df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.456 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.440+0000 7efd946a3740 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c27b40 0x5574a3c2df90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7efd8c0fff70 tx=0x7efd80012b60 comp rx=0 tx=0).stop 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 shutdown_connections 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3c40ff0 0x5574a3c43430 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c2e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c27b40 0x5574a3c2df90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 --2- 172.21.15.37:0/399159224 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c265d0 0x5574a3c30ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd6b7fe640 0 client.26723 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 >> 172.21.15.37:0/399159224 conn(0x5574a3bef310 msgr2=0x5574a3c28960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 shutdown_connections 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 1 -- 172.21.15.37:0/399159224 wait complete. 2024-09-19T06:48:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.441+0000 7efd946a3740 20 client.26723 trim_cache size 0 max 16384 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 Processor -- start 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 -- start start 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c3d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1b0250 con 0x5574a3c27b40 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1adaf0 con 0x5574a3c265d0 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.454+0000 7efd946a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1ab960 con 0x5574a3c31370 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c29120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49226/0 (socket says 172.21.15.37:49226) 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 -- 172.21.15.37:0/2431147241 learned_addr learned my addr 172.21.15.37:0/2431147241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8bfff640 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c3d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd90867640 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 -- 172.21.15.37:0/2431147241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 msgr2=0x5574a3c29660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c29660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 -- 172.21.15.37:0/2431147241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 msgr2=0x5574a3c3d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c3d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 -- 172.21.15.37:0/2431147241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574a3c23960 con 0x5574a3c27b40 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8bfff640 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c3d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c29120 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efd80000c00 tx=0x7efd80010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b5ab8e874d91c7a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd71ffb640 1 -- 172.21.15.37:0/2431147241 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd80029070 con 0x5574a3c27b40 2024-09-19T06:48:34.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd71ffb640 1 -- 172.21.15.37:0/2431147241 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd800035f0 con 0x5574a3c27b40 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.455+0000 7efd71ffb640 1 -- 172.21.15.37:0/2431147241 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd8000fdd0 con 0x5574a3c27b40 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 -- 172.21.15.37:0/2431147241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 msgr2=0x5574a3c29120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c29120 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7efd80000c00 tx=0x7efd80010640 comp rx=0 tx=0).stop 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 -- 172.21.15.37:0/2431147241 shutdown_connections 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c31370 0x5574a3c29660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c27b40 0x5574a3c29120 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 --2- 172.21.15.37:0/2431147241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c3d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 -- 172.21.15.37:0/2431147241 >> 172.21.15.37:0/2431147241 conn(0x5574a3bef310 msgr2=0x5574a3c2d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 -- 172.21.15.37:0/2431147241 shutdown_connections 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.456+0000 7efd946a3740 1 -- 172.21.15.37:0/2431147241 wait complete. 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 -- start start 2024-09-19T06:48:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 Processor -- start 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c39d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c27b40 0x5574a3c2aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c31370 0x5574a3c2b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1abfb0 con 0x5574a3c31370 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1ad420 con 0x5574a3c265d0 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd8c1ac680 con 0x5574a3c27b40 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd946a3740 10 client.0 _ll_register_callbacks cb 0x7efd800097a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd90867640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c31370 0x5574a3c2b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd8bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c39d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd90867640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c31370 0x5574a3c2b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49234/0 (socket says 172.21.15.37:49234) 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.457+0000 7efd90867640 1 -- 172.21.15.37:0/4018040388 learned_addr learned my addr 172.21.15.37:0/4018040388 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c27b40 0x5574a3c2aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd8bfff640 1 -- 172.21.15.37:0/4018040388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c27b40 msgr2=0x5574a3c2aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd8bfff640 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c27b40 0x5574a3c2aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd8bfff640 1 -- 172.21.15.37:0/4018040388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c31370 msgr2=0x5574a3c2b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd8bfff640 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c31370 0x5574a3c2b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd8bfff640 1 -- 172.21.15.37:0/4018040388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574a3caf0e0 con 0x5574a3c265d0 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd90867640 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c31370 0x5574a3c2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd8bfff640 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c39d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0bc690 tx=0x7efd7c0f0810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cdc2853e846ce31d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd6b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd6b7fe640 1 -- 172.21.15.37:0/4018040388 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7c049040 con 0x5574a3c265d0 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd800035f0 con 0x5574a3c265d0 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd6b7fe640 1 -- 172.21.15.37:0/4018040388 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7efd7c03b460 con 0x5574a3c265d0 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd6b7fe640 1 -- 172.21.15.37:0/4018040388 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efd7c0da050 con 0x5574a3c265d0 2024-09-19T06:48:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd946a3740 10 client.26729 fetch_fsmap learned FSMap version 33 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.458+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5574a3cb0300 con 0x5574a3c265d0 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd6b7fe640 1 -- 172.21.15.37:0/4018040388 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd800035f0 con 0x5574a3c265d0 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd6b7fe640 1 -- 172.21.15.37:0/4018040388 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7efd7c0eddf0 con 0x5574a3c265d0 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd946a3740 10 client.26729 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd946a3740 10 client.26729 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd946a3740 20 client.26729 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd946a3740 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3cb0a80 0x5574a3cb2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd946a3740 4 client.26729 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5574a3b93d70 con 0x5574a3cb0a80 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3cb0a80 0x5574a3cb2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd8b7fe640 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3cb0a80 0x5574a3cb2ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53d060d9dbcce996 server_cookie=22bdd802306a1d83 in_seq=0 out_seq=0 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.459+0000 7efd6b7fe640 10 client.26729 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:48:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd6b7fe640 1 -- 172.21.15.37:0/4018040388 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5574a3b93d70 con 0x5574a3cb0a80 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd6b7fe640 10 client.26729 handle_command_reply: tid=0 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 client.26729 shutdown 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 msgr2=0x5574a3c39d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c39d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efd7c0bc690 tx=0x7efd7c0f0810 comp rx=0 tx=0).stop 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 shutdown_connections 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x5574a3cb0a80 0x5574a3cb2ea0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5574a3c31370 0x5574a3c2b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5574a3c27b40 0x5574a3c2aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd6b7fe640 0 client.26729 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 --2- 172.21.15.37:0/4018040388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5574a3c265d0 0x5574a3c39d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 >> 172.21.15.37:0/4018040388 conn(0x5574a3bef310 msgr2=0x5574a3c2f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 shutdown_connections 2024-09-19T06:48:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 1 -- 172.21.15.37:0/4018040388 wait complete. 2024-09-19T06:48:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.460+0000 7efd946a3740 20 client.26729 trim_cache size 0 max 16384 2024-09-19T06:48:34.470 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd740a33b0 msgr2=0x7efd740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd740a33b0 0x7efd740a5870 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efd84018000 tx=0x7efd8401be90 comp rx=0 tx=0).stop 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 msgr2=0x7efd8c198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c198500 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7efd7c006010 tx=0x7efd7c003c10 comp rx=0 tx=0).stop 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 shutdown_connections 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd8c102ef0 0x7efd8c19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd8c102570 0x7efd8c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd740a33b0 0x7efd740a5870 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 --2- 172.21.15.37:0/3231720156 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd8c100e20 0x7efd8c198500 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 >> 172.21.15.37:0/3231720156 conn(0x7efd8c0fc7d0 msgr2=0x7efd8c06a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 shutdown_connections 2024-09-19T06:48:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:34.461+0000 7efd922ca640 1 -- 172.21.15.37:0/3231720156 wait complete. 2024-09-19T06:48:34.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:34 smithi096 ceph-mon[35004]: pgmap v209: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-19T06:48:34.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:34 smithi151 ceph-mon[35236]: pgmap v209: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 722 KiB/s rd, 3.7 MiB/s wr, 295 op/s 2024-09-19T06:48:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.620+0000 7fde5f313640 1 -- 172.21.15.37:0/80394314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 msgr2=0x7fde580749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.620+0000 7fde5f313640 1 --2- 172.21.15.37:0/80394314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde580749a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fde54009900 tx=0x7fde5402a480 comp rx=0 tx=0).stop 2024-09-19T06:48:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.620+0000 7fde5f313640 1 -- 172.21.15.37:0/80394314 shutdown_connections 2024-09-19T06:48:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.620+0000 7fde5f313640 1 --2- 172.21.15.37:0/80394314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde5810e370 0x7fde58110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.620+0000 7fde5f313640 1 --2- 172.21.15.37:0/80394314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde58074ee0 0x7fde5810de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.623 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.620+0000 7fde5f313640 1 --2- 172.21.15.37:0/80394314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde580749a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.620+0000 7fde5f313640 1 -- 172.21.15.37:0/80394314 >> 172.21.15.37:0/80394314 conn(0x7fde580fd7d0 msgr2=0x7fde580ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 -- 172.21.15.37:0/80394314 shutdown_connections 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 -- 172.21.15.37:0/80394314 wait complete. 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 Processor -- start 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 -- start start 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde5819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde58074ee0 0x7fde5819aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde5810e370 0x7fde5819ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde58114160 con 0x7fde580745c0 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde58113fe0 con 0x7fde58074ee0 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.621+0000 7fde5f313640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde581142e0 con 0x7fde5810e370 2024-09-19T06:48:34.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde5819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde5819a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49248/0 (socket says 172.21.15.37:49248) 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 -- 172.21.15.37:0/3767957766 learned_addr learned my addr 172.21.15.37:0/3767957766 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde58074ee0 0x7fde5819aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde5810e370 0x7fde5819ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 -- 172.21.15.37:0/3767957766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde5810e370 msgr2=0x7fde5819ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde5810e370 0x7fde5819ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 -- 172.21.15.37:0/3767957766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde58074ee0 msgr2=0x7fde5819aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde58074ee0 0x7fde5819aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 -- 172.21.15.37:0/3767957766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5819f580 con 0x7fde580745c0 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5d0af640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde5819a590 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fde54004770 tx=0x7fde54003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde4e7fc640 1 -- 172.21.15.37:0/3767957766 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5403f070 con 0x7fde580745c0 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5819f810 con 0x7fde580745c0 2024-09-19T06:48:34.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde4e7fc640 1 -- 172.21.15.37:0/3767957766 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde54031ca0 con 0x7fde580745c0 2024-09-19T06:48:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde4e7fc640 1 -- 172.21.15.37:0/3767957766 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54035030 con 0x7fde580745c0 2024-09-19T06:48:34.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.622+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde581a7100 con 0x7fde580745c0 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 Processor -- start 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde4e7fc640 1 -- 172.21.15.37:0/3767957766 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fde54042070 con 0x7fde580745c0 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 -- start start 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8bbed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bc410 0x55bdca8c6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdca8202f0 con 0x55bdca8bbaf0 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bdca7c8270 con 0x55bdca8bc410 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde620a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdca81c2b0 con 0x55bdca8c6cb0 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5d8b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8bbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5c8ae640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5d8b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8bbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49258/0 (socket says 172.21.15.37:49258) 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5d8b0640 1 -- 172.21.15.37:0/2760436511 learned_addr learned my addr 172.21.15.37:0/2760436511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5d0af640 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bc410 0x55bdca8c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5c8ae640 1 -- 172.21.15.37:0/2760436511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bc410 msgr2=0x55bdca8c6770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bc410 0x55bdca8c6770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5c8ae640 1 -- 172.21.15.37:0/2760436511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 msgr2=0x55bdca8bbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8bbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.626+0000 7fde5c8ae640 1 -- 172.21.15.37:0/2760436511 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdca7c7340 con 0x55bdca8c6cb0 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.627+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c90a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fde48009bf0 tx=0x7fde48011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdaa848915208295 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.627+0000 7fde437fe640 1 -- 172.21.15.37:0/2760436511 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48004050 con 0x55bdca8c6cb0 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.627+0000 7fde437fe640 1 -- 172.21.15.37:0/2760436511 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde48005500 con 0x55bdca8c6cb0 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.627+0000 7fde437fe640 1 -- 172.21.15.37:0/2760436511 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48004e40 con 0x55bdca8c6cb0 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.627+0000 7fde4e7fc640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde440a3460 0x7fde440a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.627+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde440a3460 0x7fde440a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.627+0000 7fde4e7fc640 1 -- 172.21.15.37:0/3767957766 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7fde540f0310 con 0x7fde580745c0 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 -- 172.21.15.37:0/2760436511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8c6cb0 msgr2=0x55bdca8c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c90a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fde48009bf0 tx=0x7fde48011810 comp rx=0 tx=0).stop 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde440a3460 0x7fde440a5920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde50011630 tx=0x7fde50009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 -- 172.21.15.37:0/2760436511 shutdown_connections 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c90a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bc410 0x55bdca8c6770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 --2- 172.21.15.37:0/2760436511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8bbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 -- 172.21.15.37:0/2760436511 >> 172.21.15.37:0/2760436511 conn(0x55bdca8b8060 msgr2=0x55bdca8b8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 -- 172.21.15.37:0/2760436511 shutdown_connections 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 -- 172.21.15.37:0/2760436511 wait complete. 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 -- start start 2024-09-19T06:48:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.628+0000 7fde620a4740 1 Processor -- start 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55bdca8c9880 con 0x55bdca8bbaf0 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bdca7a9290 con 0x55bdca8c6cb0 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55bdca8c9a00 con 0x55bdca8bc410 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 10 client.0 _ll_register_callbacks cb 0x55bdca7da140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49260/0 (socket says 172.21.15.37:49260) 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 -- 172.21.15.37:0/2155753896 learned_addr learned my addr 172.21.15.37:0/2155753896 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 -- 172.21.15.37:0/2155753896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 msgr2=0x55bdca8c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 -- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8c6cb0 msgr2=0x55bdca8c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 -- 172.21.15.37:0/2155753896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdca533210 con 0x55bdca8bbaf0 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde5d0af640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c3b70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fde540bc6c0 tx=0x7fde54031f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f419f8c9d9eae11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde40ff9640 1 -- 172.21.15.37:0/2155753896 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54049070 con 0x55bdca8bbaf0 2024-09-19T06:48:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde40ff9640 1 -- 172.21.15.37:0/2155753896 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7fde540b8da0 con 0x55bdca8bbaf0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde48005500 con 0x55bdca8bbaf0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde40ff9640 1 -- 172.21.15.37:0/2155753896 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde540bc500 con 0x55bdca8bbaf0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 10 client.17346 fetch_fsmap learned FSMap version 33 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.629+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bdca5ff620 con 0x55bdca8bbaf0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde40ff9640 1 -- 172.21.15.37:0/2155753896 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde48005500 con 0x55bdca8bbaf0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde40ff9640 1 -- 172.21.15.37:0/2155753896 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fde5403b4f0 con 0x55bdca8bbaf0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde620a4740 10 client.17346 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde620a4740 10 client.17346 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde620a4740 20 client.17346 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde620a4740 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca8d6bc0 0x55bdca8d9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde620a4740 4 client.17346 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdca82da20 con 0x55bdca8d6bc0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.630+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca8d6bc0 0x55bdca8d9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca8d6bc0 0x55bdca8d9010 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=787f54f4fd49b1aa server_cookie=97ad95812bb0e310 in_seq=0 out_seq=0 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde40ff9640 10 client.17346 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde40ff9640 1 -- 172.21.15.37:0/2155753896 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55bdca82da20 con 0x55bdca8d6bc0 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde40ff9640 10 client.17346 handle_command_reply: tid=0 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 client.17346 shutdown 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 msgr2=0x55bdca8c3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c3b70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fde540bc6c0 tx=0x7fde54031f10 comp rx=0 tx=0).stop 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 shutdown_connections 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca8d6bc0 0x55bdca8d9010 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 --2- 172.21.15.37:0/2155753896 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c3b70 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde40ff9640 0 client.17346 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.631+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 >> 172.21.15.37:0/2155753896 conn(0x55bdca8b8060 msgr2=0x55bdca8bddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.632+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 shutdown_connections 2024-09-19T06:48:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.632+0000 7fde620a4740 1 -- 172.21.15.37:0/2155753896 wait complete. 2024-09-19T06:48:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.632+0000 7fde620a4740 20 client.17346 trim_cache size 0 max 16384 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 Processor -- start 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 -- start start 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8de8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8dee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde581b2430 con 0x55bdca8c6cb0 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde581afcd0 con 0x55bdca8bbaf0 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde620a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde581adb40 con 0x55bdca8bc410 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde5d8b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8dee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde5d8b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8dee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49264/0 (socket says 172.21.15.37:49264) 2024-09-19T06:48:34.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde5d0af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde5d8b0640 1 -- 172.21.15.37:0/2114270655 learned_addr learned my addr 172.21.15.37:0/2114270655 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.646+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8de8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde5d8b0640 1 -- 172.21.15.37:0/2114270655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 msgr2=0x55bdca8de8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8de8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde5d8b0640 1 -- 172.21.15.37:0/2114270655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 msgr2=0x55bdca8c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde5d8b0640 1 -- 172.21.15.37:0/2114270655 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdca8bcc90 con 0x55bdca8c6cb0 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde5d0af640 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8dee30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fde50003e60 tx=0x7fde50019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c5adf81f2a4e2c9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde42ffd640 1 -- 172.21.15.37:0/2114270655 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5000f4e0 con 0x55bdca8c6cb0 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde42ffd640 1 -- 172.21.15.37:0/2114270655 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5000fb60 con 0x55bdca8c6cb0 2024-09-19T06:48:34.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.647+0000 7fde42ffd640 1 -- 172.21.15.37:0/2114270655 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50029db0 con 0x55bdca8c6cb0 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 -- 172.21.15.37:0/2114270655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 msgr2=0x55bdca8dee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8dee30 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fde50003e60 tx=0x7fde50019000 comp rx=0 tx=0).stop 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 -- 172.21.15.37:0/2114270655 shutdown_connections 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8dee30 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8de8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 --2- 172.21.15.37:0/2114270655 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 -- 172.21.15.37:0/2114270655 >> 172.21.15.37:0/2114270655 conn(0x55bdca8b8060 msgr2=0x55bdca8c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 -- 172.21.15.37:0/2114270655 shutdown_connections 2024-09-19T06:48:34.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 -- 172.21.15.37:0/2114270655 wait complete. 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 -- start start 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.648+0000 7fde620a4740 1 Processor -- start 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde620a4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde620a4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde620a4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde620a4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ae190 con 0x55bdca8c6cb0 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde620a4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde581af600 con 0x55bdca8bbaf0 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde620a4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde581ae860 con 0x55bdca8bc410 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde620a4740 10 client.0 _ll_register_callbacks cb 0x7fde540f3b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d0af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49280/0 (socket says 172.21.15.37:49280) 2024-09-19T06:48:34.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d0af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43802/0 (socket says 172.21.15.37:43802) 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 -- 172.21.15.37:0/4225320745 learned_addr learned my addr 172.21.15.37:0/4225320745 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5c8ae640 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 -- 172.21.15.37:0/4225320745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 msgr2=0x55bdca8c1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 -- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 msgr2=0x55bdca8c0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 -- 172.21.15.37:0/4225320745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdca940970 con 0x55bdca8c6cb0 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d0af640 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde5d8b0640 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c16a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fde500048b0 tx=0x7fde50009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602918a5d37f7b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.649+0000 7fde40ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde40ff9640 1 -- 172.21.15.37:0/4225320745 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50004c40 con 0x55bdca8c6cb0 2024-09-19T06:48:34.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde40ff9640 1 -- 172.21.15.37:0/4225320745 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8571+0+0 (secure 0 0 0) 0x7fde50004de0 con 0x55bdca8c6cb0 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde5000fb60 con 0x55bdca8c6cb0 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde40ff9640 1 -- 172.21.15.37:0/4225320745 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fde500299f0 con 0x55bdca8c6cb0 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 10 client.17352 fetch_fsmap learned FSMap version 33 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bdca941c60 con 0x55bdca8c6cb0 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde40ff9640 1 -- 172.21.15.37:0/4225320745 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde5000fb60 con 0x55bdca8c6cb0 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde40ff9640 1 -- 172.21.15.37:0/4225320745 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 33) ==== 8354+0+0 (secure 0 0 0) 0x7fde50030070 con 0x55bdca8c6cb0 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 10 client.17352 fetch_fsmap finished waiting for FSMap version 33 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 10 client.17352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 20 client.17352 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca9422d0 0x55bdca9446f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 4 client.17352 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55bdca827d70 con 0x55bdca9422d0 2024-09-19T06:48:34.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.650+0000 7fde5d0af640 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca9422d0 0x55bdca9446f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.651+0000 7fde40ff9640 10 client.17352 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:34.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.651+0000 7fde5d0af640 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca9422d0 0x55bdca9446f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79ea145f061845c2 server_cookie=9f4c21dc0157cc89 in_seq=0 out_seq=0 2024-09-19T06:48:34.765 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde40ff9640 1 -- 172.21.15.37:0/4225320745 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55bdca827d70 con 0x55bdca9422d0 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde40ff9640 10 client.17352 handle_command_reply: tid=0 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 client.17352 shutdown 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 msgr2=0x55bdca8c16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c16a0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fde500048b0 tx=0x7fde50009f20 comp rx=0 tx=0).stop 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 shutdown_connections 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55bdca9422d0 0x55bdca9446f0 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55bdca8c6cb0 0x55bdca8c16a0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde40ff9640 0 client.17352 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55bdca8bc410 0x55bdca8c1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 --2- 172.21.15.37:0/4225320745 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bdca8bbaf0 0x55bdca8c0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 >> 172.21.15.37:0/4225320745 conn(0x55bdca8b8060 msgr2=0x55bdca8b69a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 shutdown_connections 2024-09-19T06:48:34.766 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 1 -- 172.21.15.37:0/4225320745 wait complete. 2024-09-19T06:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.764+0000 7fde620a4740 20 client.17352 trim_cache size 0 max 16384 2024-09-19T06:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:48:34.767 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde440a3460 msgr2=0x7fde440a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde440a3460 0x7fde440a5920 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fde50011630 tx=0x7fde50009500 comp rx=0 tx=0).stop 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 msgr2=0x7fde5819a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde5819a590 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fde54004770 tx=0x7fde54003c10 comp rx=0 tx=0).stop 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 shutdown_connections 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde5810e370 0x7fde5819ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde58074ee0 0x7fde5819aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.768 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde440a3460 0x7fde440a5920 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 --2- 172.21.15.37:0/3767957766 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde580745c0 0x7fde5819a590 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 >> 172.21.15.37:0/3767957766 conn(0x7fde580fd7d0 msgr2=0x7fde580fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 shutdown_connections 2024-09-19T06:48:34.769 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:34.765+0000 7fde5f313640 1 -- 172.21.15.37:0/3767957766 wait complete. 2024-09-19T06:48:35.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 484628 7.78 MB/sec execute 182 sec latency 63.215 ms 2024-09-19T06:48:35.286 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:48:35.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:35 smithi151 ceph-mon[35236]: daemon mds.i restarted 2024-09-19T06:48:35.363 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:48:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setuser ceph since I am not root 2024-09-19T06:48:35.363 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:48:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setgroup ceph since I am not root 2024-09-19T06:48:35.363 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:48:35 smithi151 ceph-mds[66270]: -- 172.21.15.151:0/847828879 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f1aece7860 con 0x55f1aecc5000 2024-09-19T06:48:35.363 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:48:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: starting mds.i at 2024-09-19T06:48:35.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:35 smithi096 ceph-mon[35004]: daemon mds.i restarted 2024-09-19T06:48:35.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:35 smithi037 ceph-mon[31211]: daemon mds.i restarted 2024-09-19T06:48:35.559 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 -- 172.21.15.37:0/3530139359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 msgr2=0x7fbb0c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3530139359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c1021c0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fbb00009900 tx=0x7fbb0002a410 comp rx=0 tx=0).stop 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 -- 172.21.15.37:0/3530139359 shutdown_connections 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3530139359 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb0c106f10 0x7fbb0c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3530139359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb0c102790 0x7fbb0c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3530139359 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c1021c0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 -- 172.21.15.37:0/3530139359 >> 172.21.15.37:0/3530139359 conn(0x7fbb0c0fd8c0 msgr2=0x7fbb0c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 -- 172.21.15.37:0/3530139359 shutdown_connections 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.080+0000 7fbb12a98640 1 -- 172.21.15.37:0/3530139359 wait complete. 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 Processor -- start 2024-09-19T06:48:36.085 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 -- start start 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb0c102790 0x7fbb0c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb0c106f10 0x7fbb0c19ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c111f30 con 0x7fbb0c101de0 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c111db0 con 0x7fbb0c102790 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb12a98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1120b0 con 0x7fbb0c106f10 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb10834640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb10834640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c198400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49304/0 (socket says 172.21.15.37:49304) 2024-09-19T06:48:36.086 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb10834640 1 -- 172.21.15.37:0/3186322651 learned_addr learned my addr 172.21.15.37:0/3186322651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.081+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb0c102790 0x7fbb0c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb11035640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb0c106f10 0x7fbb0c19ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb10834640 1 -- 172.21.15.37:0/3186322651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb0c106f10 msgr2=0x7fbb0c19ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb10834640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb0c106f10 0x7fbb0c19ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb10834640 1 -- 172.21.15.37:0/3186322651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb0c102790 msgr2=0x7fbb0c198940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb10834640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb0c102790 0x7fbb0c198940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb10834640 1 -- 172.21.15.37:0/3186322651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb0c19d3b0 con 0x7fbb0c101de0 2024-09-19T06:48:36.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb0c102790 0x7fbb0c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:36.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb10834640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c198400 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006010 tx=0x7fbb00004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb09ffb640 1 -- 172.21.15.37:0/3186322651 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0003f030 con 0x7fbb0c101de0 2024-09-19T06:48:36.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb09ffb640 1 -- 172.21.15.37:0/3186322651 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb00004440 con 0x7fbb0c101de0 2024-09-19T06:48:36.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb09ffb640 1 -- 172.21.15.37:0/3186322651 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb00035070 con 0x7fbb0c101de0 2024-09-19T06:48:36.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.082+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb0c19d640 con 0x7fbb0c101de0 2024-09-19T06:48:36.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.083+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb0c19dad0 con 0x7fbb0c101de0 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 Processor -- start 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 -- start start 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb09ffb640 1 -- 172.21.15.37:0/3186322651 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbb00042030 con 0x7fbb0c101de0 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839fb40 0x5576b839e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b839ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83ab760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5576b83042f0 con 0x5576b839fb40 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5576b82abcf0 con 0x5576b83a9370 2024-09-19T06:48:36.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb14e71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5576b83002b0 con 0x5576b839e5d0 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839fb40 0x5576b839e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb10834640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b839ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839fb40 0x5576b839e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49316/0 (socket says 172.21.15.37:49316) 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb11035640 1 -- 172.21.15.37:0/4191896490 learned_addr learned my addr 172.21.15.37:0/4191896490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.086+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83ab760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb11035640 1 -- 172.21.15.37:0/4191896490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 msgr2=0x5576b839ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb11035640 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b839ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb11035640 1 -- 172.21.15.37:0/4191896490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 msgr2=0x5576b83ab760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb11035640 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83ab760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb11035640 1 -- 172.21.15.37:0/4191896490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b82903a0 con 0x5576b839fb40 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb10834640 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b839ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:36.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb11035640 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839fb40 0x5576b839e090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009f50 tx=0x7fbb040118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515ff636a0bc6952 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbaf6ffd640 1 -- 172.21.15.37:0/4191896490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04004d20 con 0x5576b839fb40 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbaf6ffd640 1 -- 172.21.15.37:0/4191896490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04004050 con 0x5576b839fb40 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbaf6ffd640 1 -- 172.21.15.37:0/4191896490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb04005240 con 0x5576b839fb40 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb09ffb640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbafc0a3430 0x7fbafc0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb10834640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbafc0a3430 0x7fbafc0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.087+0000 7fbb09ffb640 1 -- 172.21.15.37:0/3186322651 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbb000efc80 con 0x7fbb0c101de0 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 -- 172.21.15.37:0/4191896490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839fb40 msgr2=0x5576b839e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb10834640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbafc0a3430 0x7fbafc0a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ed480 tx=0x7fbb000f37d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839fb40 0x5576b839e090 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009f50 tx=0x7fbb040118d0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 -- 172.21.15.37:0/4191896490 shutdown_connections 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83ab760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b839ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 --2- 172.21.15.37:0/4191896490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839fb40 0x5576b839e090 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 -- 172.21.15.37:0/4191896490 >> 172.21.15.37:0/4191896490 conn(0x5576b8367310 msgr2=0x5576b839ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 -- 172.21.15.37:0/4191896490 shutdown_connections 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 -- 172.21.15.37:0/4191896490 wait complete. 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.088+0000 7fbb14e71740 1 -- start start 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 1 Processor -- start 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83a8d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83a6020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b83a9370 0x5576b83a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5576b83b0480 con 0x5576b839e5d0 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5576b82bf060 con 0x5576b839fb40 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5576b83b0600 con 0x5576b83a9370 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb14e71740 10 client.0 _ll_register_callbacks cb 0x5576b80128c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83a8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83a8d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49326/0 (socket says 172.21.15.37:49326) 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 -- 172.21.15.37:0/3437260283 learned_addr learned my addr 172.21.15.37:0/3437260283 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b83a9370 0x5576b83a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 -- 172.21.15.37:0/3437260283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b83a9370 msgr2=0x5576b83a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b83a9370 0x5576b83a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 -- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 msgr2=0x5576b83a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.089+0000 7fbb11035640 1 -- 172.21.15.37:0/3437260283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b80753c0 con 0x5576b839e5d0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbb11035640 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83a8d70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009730 tx=0x7fbb04005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6de0569d94fcc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbaebfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbaebfff640 1 -- 172.21.15.37:0/3437260283 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402b020 con 0x5576b839e5d0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbaebfff640 1 -- 172.21.15.37:0/3437260283 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbb04005040 con 0x5576b839e5d0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb04004050 con 0x5576b839e5d0 2024-09-19T06:48:36.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbaebfff640 1 -- 172.21.15.37:0/3437260283 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbb04004e10 con 0x5576b839e5d0 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbb14e71740 10 client.17370 fetch_fsmap learned FSMap version 35 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5576b83bbf20 con 0x5576b839e5d0 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.090+0000 7fbaebfff640 1 -- 172.21.15.37:0/3437260283 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04004050 con 0x5576b839e5d0 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbaebfff640 1 -- 172.21.15.37:0/3437260283 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fbb04027990 con 0x5576b839e5d0 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb14e71740 10 client.17370 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb14e71740 10 client.17370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb14e71740 20 client.17370 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b83bd380 0x5576b83bf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb14e71740 4 client.17370 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576b8311860 con 0x5576b83bd380 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b83bd380 0x5576b83bf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b83bd380 0x5576b83bf770 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c65f06c277f0d7b1 server_cookie=6d6d0fcd709e7264 in_seq=0 out_seq=0 2024-09-19T06:48:36.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.091+0000 7fbaebfff640 10 client.17370 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.092+0000 7fbaebfff640 1 -- 172.21.15.37:0/3437260283 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5576b8311860 con 0x5576b83bd380 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.092+0000 7fbaebfff640 10 client.17370 handle_command_reply: tid=0 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.092+0000 7fbb14e71740 1 client.17370 shutdown 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.092+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 msgr2=0x5576b83a8d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83a8d70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009730 tx=0x7fbb04005bd0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 shutdown_connections 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b83bd380 0x5576b83bf770 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b83a9370 0x5576b83a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbaebfff640 0 client.17370 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3437260283 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83a8d70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 >> 172.21.15.37:0/3437260283 conn(0x5576b8367310 msgr2=0x5576b83a0960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:36.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 shutdown_connections 2024-09-19T06:48:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 1 -- 172.21.15.37:0/3437260283 wait complete. 2024-09-19T06:48:36.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.093+0000 7fbb14e71740 20 client.17370 trim_cache size 0 max 16384 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.106+0000 7fbb14e71740 1 Processor -- start 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.106+0000 7fbb14e71740 1 -- start start 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.106+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83b9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.106+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839fb40 0x5576b83a5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.106+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83c4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb14e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ae010 con 0x5576b839e5d0 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb14e71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ab8b0 con 0x5576b83a9370 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb14e71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1a9700 con 0x5576b839fb40 2024-09-19T06:48:36.112 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83b9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83b9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49330/0 (socket says 172.21.15.37:49330) 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 -- 172.21.15.37:0/1379069440 learned_addr learned my addr 172.21.15.37:0/1379069440 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83c4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839fb40 0x5576b83a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 -- 172.21.15.37:0/1379069440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839fb40 msgr2=0x5576b83a5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839fb40 0x5576b83a5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 -- 172.21.15.37:0/1379069440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 msgr2=0x5576b83c4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83c4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 -- 172.21.15.37:0/1379069440 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b83a4570 con 0x5576b839e5d0 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbb11035640 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83b9720 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009c30 tx=0x7fbb04009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ffe7cf319ab3a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbaf67fc640 1 -- 172.21.15.37:0/1379069440 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402b040 con 0x5576b839e5d0 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbaf67fc640 1 -- 172.21.15.37:0/1379069440 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04009e20 con 0x5576b839e5d0 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.107+0000 7fbaf67fc640 1 -- 172.21.15.37:0/1379069440 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb040276d0 con 0x5576b839e5d0 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 -- 172.21.15.37:0/1379069440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 msgr2=0x5576b83b9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83b9720 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009c30 tx=0x7fbb04009230 comp rx=0 tx=0).stop 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 -- 172.21.15.37:0/1379069440 shutdown_connections 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b83a9370 0x5576b83c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839fb40 0x5576b83a5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 --2- 172.21.15.37:0/1379069440 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b839e5d0 0x5576b83b9720 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 -- 172.21.15.37:0/1379069440 >> 172.21.15.37:0/1379069440 conn(0x5576b8367310 msgr2=0x5576b83a48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 -- 172.21.15.37:0/1379069440 shutdown_connections 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 -- 172.21.15.37:0/1379069440 wait complete. 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.108+0000 7fbb14e71740 1 -- start start 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 1 Processor -- start 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b83b5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83b6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b83a9370 0x5576b83b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1a9d70 con 0x5576b83a9370 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1ab1e0 con 0x5576b839fb40 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbb0c1aa440 con 0x5576b839e5d0 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb14e71740 10 client.0 _ll_register_callbacks cb 0x7fbaf8009b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:36.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83b6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b83a9370 0x5576b83b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb0bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83b6190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43854/0 (socket says 172.21.15.37:43854) 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb0bfff640 1 -- 172.21.15.37:0/3201644047 learned_addr learned my addr 172.21.15.37:0/3201644047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb11035640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b83a9370 0x5576b83b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49342/0 (socket says 172.21.15.37:49342) 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.109+0000 7fbb11035640 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b83b5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb0bfff640 1 -- 172.21.15.37:0/3201644047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 msgr2=0x5576b83b5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b83b5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb0bfff640 1 -- 172.21.15.37:0/3201644047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b83a9370 msgr2=0x5576b83b11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b83a9370 0x5576b83b11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb0bfff640 1 -- 172.21.15.37:0/3201644047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b83b19d0 con 0x5576b839fb40 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb11035640 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b83b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb0bfff640 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83b6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8012620 tx=0x7fbaf8012b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c1f97d7abd78611 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbaebfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:48:36.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbaebfff640 1 -- 172.21.15.37:0/3201644047 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8022070 con 0x5576b839fb40 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb04009e20 con 0x5576b839fb40 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbaebfff640 1 -- 172.21.15.37:0/3201644047 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fbaf80109c0 con 0x5576b839fb40 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbaebfff640 1 -- 172.21.15.37:0/3201644047 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7fbaf8010bc0 con 0x5576b839fb40 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb14e71740 10 client.26771 fetch_fsmap learned FSMap version 35 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x5576b8427ab0 con 0x5576b839fb40 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.110+0000 7fbaebfff640 1 -- 172.21.15.37:0/3201644047 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb04009e20 con 0x5576b839fb40 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbaebfff640 1 -- 172.21.15.37:0/3201644047 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 35) ==== 8354+0+0 (secure 0 0 0) 0x7fbaf801e3f0 con 0x5576b839fb40 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb14e71740 10 client.26771 fetch_fsmap finished waiting for FSMap version 35 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb14e71740 10 client.26771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb14e71740 20 client.26771 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b8428290 0x5576b842a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb14e71740 4 client.26771 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:48:36.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576b830bd70 con 0x5576b8428290 2024-09-19T06:48:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb11035640 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b8428290 0x5576b842a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbaebfff640 10 client.26771 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:36.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.111+0000 7fbb11035640 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b8428290 0x5576b842a6b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7a5b7bc2316355 server_cookie=48cc28a583d1c19c in_seq=0 out_seq=0 2024-09-19T06:48:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbaebfff640 1 -- 172.21.15.37:0/3201644047 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576b830bd70 con 0x5576b8428290 2024-09-19T06:48:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbaebfff640 10 client.26771 handle_command_reply: tid=0 2024-09-19T06:48:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 client.26771 shutdown 2024-09-19T06:48:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 msgr2=0x5576b83b6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83b6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8012620 tx=0x7fbaf8012b00 comp rx=0 tx=0).stop 2024-09-19T06:48:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 shutdown_connections 2024-09-19T06:48:36.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5576b8428290 0x5576b842a6b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5576b83a9370 0x5576b83b11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5576b839fb40 0x5576b83b6190 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbaebfff640 0 client.26771 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 --2- 172.21.15.37:0/3201644047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5576b839e5d0 0x5576b83b5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 >> 172.21.15.37:0/3201644047 conn(0x5576b8367310 msgr2=0x5576b83a5110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 shutdown_connections 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 1 -- 172.21.15.37:0/3201644047 wait complete. 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.113+0000 7fbb14e71740 20 client.26771 trim_cache size 0 max 16384 2024-09-19T06:48:36.120 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbafc0a3430 msgr2=0x7fbafc0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbafc0a3430 0x7fbafc0a58f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ed480 tx=0x7fbb000f37d0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 msgr2=0x7fbb0c198400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c198400 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7fbb00006010 tx=0x7fbb00004040 comp rx=0 tx=0).stop 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 shutdown_connections 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbb0c106f10 0x7fbb0c19ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb0c102790 0x7fbb0c198940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbafc0a3430 0x7fbafc0a58f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 --2- 172.21.15.37:0/3186322651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbb0c101de0 0x7fbb0c198400 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 >> 172.21.15.37:0/3186322651 conn(0x7fbb0c0fd8c0 msgr2=0x7fbb0c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 shutdown_connections 2024-09-19T06:48:36.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:36.114+0000 7fbb12a98640 1 -- 172.21.15.37:0/3186322651 wait complete. 2024-09-19T06:48:36.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 487422 7.82 MB/sec execute 183 sec latency 12.143 ms 2024-09-19T06:48:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:36 smithi096 ceph-mon[35004]: pgmap v210: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-19T06:48:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:36 smithi096 ceph-mon[35004]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:48:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:36 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6838/2867779560,v1:172.21.15.151:6839/2867779560] up:boot 2024-09-19T06:48:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:36 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:48:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:36 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:48:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:36 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:36 smithi037 ceph-mon[31211]: pgmap v210: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-19T06:48:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:36 smithi037 ceph-mon[31211]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:48:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:36 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6838/2867779560,v1:172.21.15.151:6839/2867779560] up:boot 2024-09-19T06:48:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:36 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:48:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:36 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:48:36.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:36 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:36.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:36 smithi151 ceph-mon[35236]: pgmap v210: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 556 KiB/s rd, 2.8 MiB/s wr, 143 op/s 2024-09-19T06:48:36.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:36 smithi151 ceph-mon[35236]: osdmap e88: 12 total, 12 up, 12 in 2024-09-19T06:48:36.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:36 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6838/2867779560,v1:172.21.15.151:6839/2867779560] up:boot 2024-09-19T06:48:36.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:36 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:48:36.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:36 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:48:36.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:36 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:37.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 490571 7.88 MB/sec execute 184 sec latency 11.333 ms 2024-09-19T06:48:37.880 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:48:37.880 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:48:38.141 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:38.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 493208 7.90 MB/sec execute 185 sec latency 15.220 ms 2024-09-19T06:48:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:38 smithi096 ceph-mon[35004]: pgmap v212: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.3 MiB/s wr, 171 op/s 2024-09-19T06:48:38.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:38 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:38.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:38 smithi037 ceph-mon[31211]: pgmap v212: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.3 MiB/s wr, 171 op/s 2024-09-19T06:48:38.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:38 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 -- 172.21.15.37:0/290724643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 msgr2=0x7ffb10100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 --2- 172.21.15.37:0/290724643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 0x7ffb10100ee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009930 tx=0x7ffaf802a420 comp rx=0 tx=0).stop 2024-09-19T06:48:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 -- 172.21.15.37:0/290724643 shutdown_connections 2024-09-19T06:48:38.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 --2- 172.21.15.37:0/290724643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101420 0x7ffb1010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 --2- 172.21.15.37:0/290724643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 0x7ffb10100ee0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 --2- 172.21.15.37:0/290724643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb10106a80 0x7ffb10106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 -- 172.21.15.37:0/290724643 >> 172.21.15.37:0/290724643 conn(0x7ffb100fc7a0 msgr2=0x7ffb100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 -- 172.21.15.37:0/290724643 shutdown_connections 2024-09-19T06:48:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.578+0000 7ffb16424640 1 -- 172.21.15.37:0/290724643 wait complete. 2024-09-19T06:48:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 Processor -- start 2024-09-19T06:48:38.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 -- start start 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 0x7ffb101984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101420 0x7ffb101989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb10106a80 0x7ffb1019cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10110e40 con 0x7ffb10101420 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10110cc0 con 0x7ffb10106a80 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101420 0x7ffb101989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb16424640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb10110fc0 con 0x7ffb10100a80 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb149c1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb10106a80 0x7ffb1019cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101420 0x7ffb101989e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49368/0 (socket says 172.21.15.37:49368) 2024-09-19T06:48:38.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 0x7ffb101984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb07fff640 1 -- 172.21.15.37:0/22374805 learned_addr learned my addr 172.21.15.37:0/22374805 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb149c1640 1 -- 172.21.15.37:0/22374805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 msgr2=0x7ffb101984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb149c1640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 0x7ffb101984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb149c1640 1 -- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101420 msgr2=0x7ffb101989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.579+0000 7ffb149c1640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101420 0x7ffb101989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.580+0000 7ffb149c1640 1 -- 172.21.15.37:0/22374805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb1019d450 con 0x7ffb10106a80 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.580+0000 7ffb149c1640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb10106a80 0x7ffb1019cd70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800ec50 tx=0x7ffb0800c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.580+0000 7ffb0dffb640 1 -- 172.21.15.37:0/22374805 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08003b70 con 0x7ffb10106a80 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.580+0000 7ffb0dffb640 1 -- 172.21.15.37:0/22374805 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb08003d10 con 0x7ffb10106a80 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.580+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb1019d740 con 0x7ffb10106a80 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.580+0000 7ffb0dffb640 1 -- 172.21.15.37:0/22374805 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb08010890 con 0x7ffb10106a80 2024-09-19T06:48:38.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.580+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb101a5020 con 0x7ffb10106a80 2024-09-19T06:48:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.582+0000 7ffb187fd740 1 Processor -- start 2024-09-19T06:48:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.582+0000 7ffb0dffb640 1 -- 172.21.15.37:0/22374805 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffb08003fc0 con 0x7ffb10106a80 2024-09-19T06:48:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb187fd740 1 -- start start 2024-09-19T06:48:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bafb60 0x55b6f2baff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bbad40 0x55b6f2bbd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb187fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6f2b15280 con 0x55b6f2bbad40 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb187fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6f2abcb10 con 0x55b6f2bb0480 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb187fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b6f2b11240 con 0x55b6f2bafb60 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bbad40 0x55b6f2bbd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bba800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43892/0 (socket says 172.21.15.37:43892) 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 -- 172.21.15.37:0/763068195 learned_addr learned my addr 172.21.15.37:0/763068195 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb07fff640 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bafb60 0x55b6f2baff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 -- 172.21.15.37:0/763068195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bafb60 msgr2=0x55b6f2baff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bafb60 0x55b6f2baff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 -- 172.21.15.37:0/763068195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bbad40 msgr2=0x55b6f2bbd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bbad40 0x55b6f2bbd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.583+0000 7ffb0ffff640 1 -- 172.21.15.37:0/763068195 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f2ac91f0 con 0x55b6f2bb0480 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.584+0000 7ffb0dffb640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffafc0a33b0 0x7ffafc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.584+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bba800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009c70 tx=0x7ffb00011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64ed8cd2bbbc1c1e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.584+0000 7ffb07fff640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffafc0a33b0 0x7ffafc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.584+0000 7ffb0dffb640 1 -- 172.21.15.37:0/22374805 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ffb080d02e0 con 0x7ffb10106a80 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.584+0000 7ffb067fc640 1 -- 172.21.15.37:0/763068195 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0001e070 con 0x55b6f2bb0480 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.584+0000 7ffb067fc640 1 -- 172.21.15.37:0/763068195 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb00009e20 con 0x55b6f2bb0480 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.584+0000 7ffb067fc640 1 -- 172.21.15.37:0/763068195 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00019690 con 0x55b6f2bb0480 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 -- 172.21.15.37:0/763068195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 msgr2=0x55b6f2bba800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb07fff640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffafc0a33b0 0x7ffafc0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb10105190 tx=0x7ffaf8041040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bba800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009c70 tx=0x7ffb00011810 comp rx=0 tx=0).stop 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 -- 172.21.15.37:0/763068195 shutdown_connections 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bbad40 0x55b6f2bbd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.589 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bba800 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 --2- 172.21.15.37:0/763068195 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bafb60 0x55b6f2baff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 -- 172.21.15.37:0/763068195 >> 172.21.15.37:0/763068195 conn(0x55b6f2bac0f0 msgr2=0x55b6f2bac4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 -- 172.21.15.37:0/763068195 shutdown_connections 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 -- 172.21.15.37:0/763068195 wait complete. 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 -- start start 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.585+0000 7ffb187fd740 1 Processor -- start 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb187fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b6f2bbd910 con 0x55b6f2bafb60 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb187fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b6f2acff10 con 0x55b6f2bb0480 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb187fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b6f2bbda90 con 0x55b6f2bbad40 2024-09-19T06:48:38.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb187fd740 10 client.0 _ll_register_callbacks cb 0x55b6f2a98a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49386/0 (socket says 172.21.15.37:49386) 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 -- 172.21.15.37:0/3353791437 learned_addr learned my addr 172.21.15.37:0/3353791437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb07fff640 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb149c1640 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 -- 172.21.15.37:0/3353791437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 msgr2=0x55b6f2bb8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 -- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 msgr2=0x55b6f2bb8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 -- 172.21.15.37:0/3353791437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f28866a0 con 0x55b6f2bafb60 2024-09-19T06:48:38.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb7bc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ffb00014040 tx=0x7ffb00005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529ac6d9147fe10e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.586+0000 7ffaef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffaef7fe640 1 -- 172.21.15.37:0/3353791437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0001e040 con 0x55b6f2bafb60 2024-09-19T06:48:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb00009e20 con 0x55b6f2bafb60 2024-09-19T06:48:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffaef7fe640 1 -- 172.21.15.37:0/3353791437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ffb000052c0 con 0x55b6f2bafb60 2024-09-19T06:48:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffaef7fe640 1 -- 172.21.15.37:0/3353791437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffb00030050 con 0x55b6f2bafb60 2024-09-19T06:48:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 10 client.17388 fetch_fsmap learned FSMap version 36 2024-09-19T06:48:38.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b6f2bc94c0 con 0x55b6f2bafb60 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffaef7fe640 1 -- 172.21.15.37:0/3353791437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb00009e20 con 0x55b6f2bafb60 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffaef7fe640 1 -- 172.21.15.37:0/3353791437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7ffb00021070 con 0x55b6f2bafb60 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 10 client.17388 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 10 client.17388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 20 client.17388 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2bcccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 4 client.17388 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.587+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6f2b227f0 con 0x55b6f2bca8c0 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.588+0000 7ffb07fff640 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2bcccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.588+0000 7ffb07fff640 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2bcccb0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caae1e84aff60030 server_cookie=371d14a7cffaac65 in_seq=0 out_seq=0 2024-09-19T06:48:38.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.588+0000 7ffaef7fe640 10 client.17388 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffaef7fe640 1 -- 172.21.15.37:0/3353791437 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b6f2b227f0 con 0x55b6f2bca8c0 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffaef7fe640 10 client.17388 handle_command_reply: tid=0 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 client.17388 shutdown 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 msgr2=0x55b6f2bb7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb7bc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7ffb00014040 tx=0x7ffb00005850 comp rx=0 tx=0).stop 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 shutdown_connections 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2bcccb0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffaef7fe640 0 client.17388 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 --2- 172.21.15.37:0/3353791437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb7bc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 >> 172.21.15.37:0/3353791437 conn(0x55b6f2bac0f0 msgr2=0x55b6f2bb1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 shutdown_connections 2024-09-19T06:48:38.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 1 -- 172.21.15.37:0/3353791437 wait complete. 2024-09-19T06:48:38.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.590+0000 7ffb187fd740 20 client.17388 trim_cache size 0 max 16384 2024-09-19T06:48:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 Processor -- start 2024-09-19T06:48:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 -- start start 2024-09-19T06:48:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bb0480 0x55b6f2bd2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bbad40 0x55b6f2bd2a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101b02e0 con 0x55b6f2bafb60 2024-09-19T06:48:38.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101adb50 con 0x55b6f2bbad40 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.608+0000 7ffb187fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101ab9a0 con 0x55b6f2bb0480 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb9a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49390/0 (socket says 172.21.15.37:49390) 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 -- 172.21.15.37:0/2137280200 learned_addr learned my addr 172.21.15.37:0/2137280200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb149c1640 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bbad40 0x55b6f2bd2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb07fff640 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bb0480 0x55b6f2bd2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 -- 172.21.15.37:0/2137280200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bb0480 msgr2=0x55b6f2bd2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bb0480 0x55b6f2bd2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 -- 172.21.15.37:0/2137280200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bbad40 msgr2=0x55b6f2bd2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bbad40 0x55b6f2bd2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 -- 172.21.15.37:0/2137280200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f2bba490 con 0x55b6f2bafb60 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb9a20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009820 tx=0x7ffb00005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5289cd3d5fdd278a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb05ffb640 1 -- 172.21.15.37:0/2137280200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb0001e030 con 0x55b6f2bafb60 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb05ffb640 1 -- 172.21.15.37:0/2137280200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb00027070 con 0x55b6f2bafb60 2024-09-19T06:48:38.611 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.609+0000 7ffb05ffb640 1 -- 172.21.15.37:0/2137280200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb00022860 con 0x55b6f2bafb60 2024-09-19T06:48:38.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 -- 172.21.15.37:0/2137280200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 msgr2=0x55b6f2bb9a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb9a20 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7ffb00009820 tx=0x7ffb00005f90 comp rx=0 tx=0).stop 2024-09-19T06:48:38.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 -- 172.21.15.37:0/2137280200 shutdown_connections 2024-09-19T06:48:38.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bbad40 0x55b6f2bd2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bb0480 0x55b6f2bd2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 --2- 172.21.15.37:0/2137280200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bafb60 0x55b6f2bb9a20 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 -- 172.21.15.37:0/2137280200 >> 172.21.15.37:0/2137280200 conn(0x55b6f2bac0f0 msgr2=0x55b6f2bb9310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 -- 172.21.15.37:0/2137280200 shutdown_connections 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.610+0000 7ffb187fd740 1 -- 172.21.15.37:0/2137280200 wait complete. 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 -- start start 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 Processor -- start 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bafb60 0x55b6f2bc35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb5230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101ac010 con 0x55b6f2bb0480 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101ad480 con 0x55b6f2bafb60 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffb101ac6e0 con 0x55b6f2bbad40 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb187fd740 10 client.0 _ll_register_callbacks cb 0x7ffb0001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:38.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb0ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bafb60 0x55b6f2bc35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:49398/0 (socket says 172.21.15.37:49398) 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb07fff640 1 -- 172.21.15.37:0/1879583118 learned_addr learned my addr 172.21.15.37:0/1879583118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.611+0000 7ffb149c1640 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb07fff640 1 -- 172.21.15.37:0/1879583118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 msgr2=0x55b6f2bb5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb07fff640 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb07fff640 1 -- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bafb60 msgr2=0x55b6f2bc35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb07fff640 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bafb60 0x55b6f2bc35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb07fff640 1 -- 172.21.15.37:0/1879583118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f2c344e0 con 0x55b6f2bb0480 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bafb60 0x55b6f2bc35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb07fff640 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb4aa0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009230 tx=0x7ffaf8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88f34da2174e7c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffaef7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffaef7fe640 1 -- 172.21.15.37:0/1879583118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaf8005220 con 0x55b6f2bb0480 2024-09-19T06:48:38.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffaef7fe640 1 -- 172.21.15.37:0/1879583118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7ffaf80053c0 con 0x55b6f2bb0480 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb00027070 con 0x55b6f2bb0480 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffaef7fe640 1 -- 172.21.15.37:0/1879583118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ffaf80055c0 con 0x55b6f2bb0480 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb187fd740 10 client.17394 fetch_fsmap learned FSMap version 36 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b6f2c35710 con 0x55b6f2bb0480 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.612+0000 7ffaef7fe640 1 -- 172.21.15.37:0/1879583118 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb00027070 con 0x55b6f2bb0480 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffaef7fe640 1 -- 172.21.15.37:0/1879583118 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7ffaf8004b10 con 0x55b6f2bb0480 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb187fd740 10 client.17394 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb187fd740 10 client.17394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:38.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb187fd740 20 client.17394 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb187fd740 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2c37e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb187fd740 4 client.17394 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:48:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b6f2b1cd00 con 0x55b6f2bca8c0 2024-09-19T06:48:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2c37e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffaef7fe640 10 client.17394 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:38.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.613+0000 7ffb0ffff640 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2c37e60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63c6a75f7f13eeb2 server_cookie=4ffbf4bd8d130feb in_seq=0 out_seq=0 2024-09-19T06:48:38.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:38 smithi151 ceph-mon[35236]: pgmap v212: 97 pgs: 97 active+clean; 51 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 3.3 MiB/s wr, 171 op/s 2024-09-19T06:48:38.619 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:38 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:48:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffaef7fe640 1 -- 172.21.15.37:0/1879583118 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b6f2b1cd00 con 0x55b6f2bca8c0 2024-09-19T06:48:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffaef7fe640 10 client.17394 handle_command_reply: tid=0 2024-09-19T06:48:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 client.17394 shutdown 2024-09-19T06:48:38.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bb0480 msgr2=0x55b6f2bb4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb4aa0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009230 tx=0x7ffaf8009260 comp rx=0 tx=0).stop 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 shutdown_connections 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b6f2bca8c0 0x55b6f2c37e60 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffaef7fe640 0 client.17394 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b6f2bbad40 0x55b6f2bb5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b6f2bb0480 0x55b6f2bb4aa0 secure :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7ffaf8009230 tx=0x7ffaf8009260 comp rx=0 tx=0).stop 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 --2- 172.21.15.37:0/1879583118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b6f2bafb60 0x55b6f2bc35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 >> 172.21.15.37:0/1879583118 conn(0x55b6f2bac0f0 msgr2=0x55b6f2bb9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.617+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 shutdown_connections 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb187fd740 1 -- 172.21.15.37:0/1879583118 wait complete. 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb187fd740 20 client.17394 trim_cache size 0 max 16384 2024-09-19T06:48:38.620 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:48:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:48:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "24670327-0db3-42b6-a911-26ff1ee78a7b", 2024-09-19T06:48:38.621 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:48:38.621 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:48:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffafc0a33b0 msgr2=0x7ffafc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffafc0a33b0 0x7ffafc0a5870 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb10105190 tx=0x7ffaf8041040 comp rx=0 tx=0).stop 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb10106a80 msgr2=0x7ffb1019cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb10106a80 0x7ffb1019cd70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800ec50 tx=0x7ffb0800c6a0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 shutdown_connections 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffb10106a80 0x7ffb1019cd70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffafc0a33b0 0x7ffafc0a5870 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffb10101420 0x7ffb101989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.618+0000 7ffb16424640 1 --2- 172.21.15.37:0/22374805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb10100a80 0x7ffb101984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.619+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 >> 172.21.15.37:0/22374805 conn(0x7ffb100fc7a0 msgr2=0x7ffb1006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.619+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 shutdown_connections 2024-09-19T06:48:38.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:38.619+0000 7ffb16424640 1 -- 172.21.15.37:0/22374805 wait complete. 2024-09-19T06:48:39.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 495036 7.91 MB/sec execute 186 sec latency 63.846 ms 2024-09-19T06:48:39.497 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:48:39.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:39 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:48:39.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:39 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:39 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:48:39.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:39 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:48:39.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:39 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.520 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:39 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:39 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:48:39.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:39 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:39 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:48:39.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:39 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:48:39.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:39 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:39 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:39 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:48:39.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:39 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:39 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:48:39.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:39 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:48:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:39 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:39 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:48:39.759 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:48:40.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 497822 7.96 MB/sec execute 187 sec latency 36.748 ms 2024-09-19T06:48:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 -- 172.21.15.37:0/1640160805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0864102d40 msgr2=0x7f0864103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 --2- 172.21.15.37:0/1640160805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0864102d40 0x7f0864103120 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f0860009900 tx=0x7f086002a480 comp rx=0 tx=0).stop 2024-09-19T06:48:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 -- 172.21.15.37:0/1640160805 shutdown_connections 2024-09-19T06:48:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 --2- 172.21.15.37:0/1640160805 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f086410c170 0x7f086410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 --2- 172.21.15.37:0/1640160805 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0864103660 0x7f086410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 --2- 172.21.15.37:0/1640160805 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0864102d40 0x7f0864103120 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 -- 172.21.15.37:0/1640160805 >> 172.21.15.37:0/1640160805 conn(0x7f08640fc7a0 msgr2=0x7f08640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 -- 172.21.15.37:0/1640160805 shutdown_connections 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 -- 172.21.15.37:0/1640160805 wait complete. 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.210+0000 7f086bf68640 1 Processor -- start 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086bf68640 1 -- start start 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086bf68640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0864102d40 0x7f08641984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086bf68640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0864103660 0x7f0864198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086bf68640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f086410c170 0x7f086419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086bf68640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0864111f70 con 0x7f086410c170 2024-09-19T06:48:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086bf68640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0864111df0 con 0x7f0864102d40 2024-09-19T06:48:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086bf68640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f08641120f0 con 0x7f0864103660 2024-09-19T06:48:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f0869d04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0864102d40 0x7f08641984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086a505640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f086410c170 0x7f086419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f0869503640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0864103660 0x7f0864198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f0869d04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0864102d40 0x7f08641984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38688/0 (socket says 172.21.15.37:38688) 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f0869d04640 1 -- 172.21.15.37:0/482008751 learned_addr learned my addr 172.21.15.37:0/482008751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086a505640 1 -- 172.21.15.37:0/482008751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0864103660 msgr2=0x7f0864198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086a505640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0864103660 0x7f0864198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086a505640 1 -- 172.21.15.37:0/482008751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0864102d40 msgr2=0x7f08641984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086a505640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0864102d40 0x7f08641984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f086a505640 1 -- 172.21.15.37:0/482008751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f086419d490 con 0x7f086410c170 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.211+0000 7f0869d04640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0864102d40 0x7f08641984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:40.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.212+0000 7f086a505640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f086410c170 0x7f086419cdb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f085c009fd0 tx=0x7f085c00ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.212+0000 7f085affd640 1 -- 172.21.15.37:0/482008751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c00ce20 con 0x7f086410c170 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.212+0000 7f085affd640 1 -- 172.21.15.37:0/482008751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085c005020 con 0x7f086410c170 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.212+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f086419d780 con 0x7f086410c170 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.212+0000 7f085affd640 1 -- 172.21.15.37:0/482008751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085c010720 con 0x7f086410c170 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.212+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08641a5060 con 0x7f086410c170 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.214+0000 7f085affd640 1 -- 172.21.15.37:0/482008751 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f085c005540 con 0x7f086410c170 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 Processor -- start 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 -- start start 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x55942585f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585fc80 0x55942586e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942586ea90 0x559425870e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5594257c52e0 con 0x55942586ea90 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55942576cce0 con 0x55942585fc80 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086e341740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5594257c12a0 con 0x55942585f360 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f085affd640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f08500a3430 0x7f08500a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f0869503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942586ea90 0x559425870e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f0869d04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585fc80 0x55942586e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f0869503640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x55942585f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f0869d04640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585fc80 0x55942586e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38696/0 (socket says 172.21.15.37:38696) 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f0869d04640 1 -- 172.21.15.37:0/1773724275 learned_addr learned my addr 172.21.15.37:0/1773724275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:40.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.215+0000 7f086a505640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f08500a3430 0x7f08500a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f0869503640 1 -- 172.21.15.37:0/1773724275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585fc80 msgr2=0x55942586e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f0869503640 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585fc80 0x55942586e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f0869503640 1 -- 172.21.15.37:0/1773724275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942586ea90 msgr2=0x559425870e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f0869503640 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942586ea90 0x559425870e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f0869503640 1 -- 172.21.15.37:0/1773724275 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559425751440 con 0x55942585f360 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f0869d04640 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585fc80 0x55942586e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f085affd640 1 -- 172.21.15.37:0/482008751 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f085c0d02e0 con 0x7f086410c170 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f0869503640 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x55942585f740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0854005fd0 tx=0x7f085400b2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7e0ca4ace81f68c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f084ffff640 1 -- 172.21.15.37:0/1773724275 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085400f760 con 0x55942585f360 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f086a505640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f08500a3430 0x7f08500a58f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f085c0cdae0 tx=0x7f085c0054b0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:40.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f084ffff640 1 -- 172.21.15.37:0/1773724275 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085400fde0 con 0x55942585f360 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.216+0000 7f084ffff640 1 -- 172.21.15.37:0/1773724275 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085401fdc0 con 0x55942585f360 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 -- 172.21.15.37:0/1773724275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 msgr2=0x55942585f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x55942585f740 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f0854005fd0 tx=0x7f085400b2e0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 -- 172.21.15.37:0/1773724275 shutdown_connections 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942586ea90 0x559425870e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585fc80 0x55942586e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 --2- 172.21.15.37:0/1773724275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x55942585f740 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 -- 172.21.15.37:0/1773724275 >> 172.21.15.37:0/1773724275 conn(0x55942585ba30 msgr2=0x55942585be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 -- 172.21.15.37:0/1773724275 shutdown_connections 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 -- 172.21.15.37:0/1773724275 wait complete. 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.217+0000 7f086e341740 1 -- start start 2024-09-19T06:48:40.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 1 Processor -- start 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585f360 0x55942586e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x55942586b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942586ea90 0x55942586bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559425871660 con 0x55942585f360 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559425780050 con 0x55942586ea90 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5594258717e0 con 0x55942585fc80 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f086e341740 10 client.0 _ll_register_callbacks cb 0x5594257c0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585f360 0x55942586e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585f360 0x55942586e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36582/0 (socket says 172.21.15.37:36582) 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 -- 172.21.15.37:0/2066838952 learned_addr learned my addr 172.21.15.37:0/2066838952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869d04640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x55942586b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942586ea90 0x55942586bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 -- 172.21.15.37:0/2066838952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 msgr2=0x55942586b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x55942586b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 -- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942586ea90 msgr2=0x55942586bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942586ea90 0x55942586bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869503640 1 -- 172.21.15.37:0/2066838952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594255363c0 con 0x55942585f360 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.218+0000 7f0869d04640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x55942586b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f0869503640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585f360 0x55942586e470 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f085400b7e0 tx=0x7f085401fa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef86a183219480b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:40.232 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f084d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f084d7fa640 1 -- 172.21.15.37:0/2066838952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854026070 con 0x55942585f360 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f085400fde0 con 0x55942585f360 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f084d7fa640 1 -- 172.21.15.37:0/2066838952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f0854012020 con 0x55942585f360 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f084d7fa640 1 -- 172.21.15.37:0/2066838952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0854012220 con 0x55942585f360 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f086e341740 10 client.17406 fetch_fsmap learned FSMap version 36 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55942587d0e0 con 0x55942585f360 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f084d7fa640 1 -- 172.21.15.37:0/2066838952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085400fde0 con 0x55942585f360 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f084d7fa640 1 -- 172.21.15.37:0/2066838952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f08540046c0 con 0x55942585f360 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f086e341740 10 client.17406 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f086e341740 10 client.17406 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f086e341740 20 client.17406 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:40.233 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.219+0000 7f086e341740 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942587e490 0x559425880880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.220+0000 7f086e341740 4 client.17406 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.220+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5594257d2850 con 0x55942587e490 2024-09-19T06:48:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.220+0000 7f0869d04640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942587e490 0x559425880880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.220+0000 7f0869d04640 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942587e490 0x559425880880 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bc8022440ea5ebc server_cookie=1b2b98e329d6f9c9 in_seq=0 out_seq=0 2024-09-19T06:48:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.220+0000 7f084d7fa640 10 client.17406 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f084d7fa640 1 -- 172.21.15.37:0/2066838952 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5594257d2850 con 0x55942587e490 2024-09-19T06:48:40.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f084d7fa640 10 client.17406 handle_command_reply: tid=0 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 client.17406 shutdown 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585f360 msgr2=0x55942586e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585f360 0x55942586e470 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f085400b7e0 tx=0x7f085401fa60 comp rx=0 tx=0).stop 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 shutdown_connections 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942587e490 0x559425880880 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942586ea90 0x55942586bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x55942586b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 --2- 172.21.15.37:0/2066838952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585f360 0x55942586e470 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f084d7fa640 0 client.17406 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 >> 172.21.15.37:0/2066838952 conn(0x55942585ba30 msgr2=0x559425861790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 shutdown_connections 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 1 -- 172.21.15.37:0/2066838952 wait complete. 2024-09-19T06:48:40.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.221+0000 7f086e341740 20 client.17406 trim_cache size 0 max 16384 2024-09-19T06:48:40.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 Processor -- start 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 -- start start 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585ba30 0x55942587a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585f360 0x55942585aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x559425885fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ae030 con 0x55942585ba30 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ab8a0 con 0x55942585f360 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f086e341740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f08641a96f0 con 0x55942585fc80 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f0869503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585ba30 0x55942587a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f0869503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585ba30 0x55942587a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36588/0 (socket says 172.21.15.37:36588) 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f0869503640 1 -- 172.21.15.37:0/3365898335 learned_addr learned my addr 172.21.15.37:0/3365898335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.240+0000 7f0869d04640 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585f360 0x55942585aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f0869503640 1 -- 172.21.15.37:0/3365898335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 msgr2=0x559425885fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f0869503640 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x559425885fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f0869503640 1 -- 172.21.15.37:0/3365898335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585f360 msgr2=0x55942585aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f0869503640 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585f360 0x55942585aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f0869503640 1 -- 172.21.15.37:0/3365898335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559425869c70 con 0x55942585ba30 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f0869503640 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585ba30 0x55942587a900 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f0854006010 tx=0x7f085400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3261b6b010bb95da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f084f7fe640 1 -- 172.21.15.37:0/3365898335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854026040 con 0x55942585ba30 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f084f7fe640 1 -- 172.21.15.37:0/3365898335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085400bb80 con 0x55942585ba30 2024-09-19T06:48:40.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.241+0000 7f084f7fe640 1 -- 172.21.15.37:0/3365898335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f085400f730 con 0x55942585ba30 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 -- 172.21.15.37:0/3365898335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585ba30 msgr2=0x55942587a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585ba30 0x55942587a900 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f0854006010 tx=0x7f085400b3b0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 -- 172.21.15.37:0/3365898335 shutdown_connections 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585fc80 0x559425885fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585f360 0x55942585aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 --2- 172.21.15.37:0/3365898335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585ba30 0x55942587a900 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 -- 172.21.15.37:0/3365898335 >> 172.21.15.37:0/3365898335 conn(0x559425867110 msgr2=0x55942585a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 -- 172.21.15.37:0/3365898335 shutdown_connections 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 -- 172.21.15.37:0/3365898335 wait complete. 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 -- start start 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.242+0000 7f086e341740 1 Processor -- start 2024-09-19T06:48:40.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f086e341740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585ba30 0x559425872780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f086e341740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x559425872cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f086e341740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585fc80 0x559425873200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f086e341740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f08641a9d60 con 0x55942585fc80 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f086e341740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f08641ab1d0 con 0x55942585ba30 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f086e341740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f08641aa430 con 0x55942585f360 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f086e341740 10 client.0 _ll_register_callbacks cb 0x7f085400be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585ba30 0x559425872780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585fc80 0x559425873200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585fc80 0x559425873200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36594/0 (socket says 172.21.15.37:36594) 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 -- 172.21.15.37:0/1297431524 learned_addr learned my addr 172.21.15.37:0/1297431524 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869d04640 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x559425872cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 -- 172.21.15.37:0/1297431524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 msgr2=0x559425872cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x559425872cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 -- 172.21.15.37:0/1297431524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585ba30 msgr2=0x559425872780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585ba30 0x559425872780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 -- 172.21.15.37:0/1297431524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5594258e7f70 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f0869503640 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585fc80 0x559425873200 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0854004c80 tx=0x7f0854000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e259ae26520d834c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f084d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.243+0000 7f084d7fa640 1 -- 172.21.15.37:0/1297431524 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0854005200 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f084d7fa640 1 -- 172.21.15.37:0/1297431524 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7f08540053a0 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f085400bb80 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f084d7fa640 1 -- 172.21.15.37:0/1297431524 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f085401f990 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 10 client.17412 fetch_fsmap learned FSMap version 36 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5594258e9210 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f084d7fa640 1 -- 172.21.15.37:0/1297431524 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f085400bb80 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f084d7fa640 1 -- 172.21.15.37:0/1297431524 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f0854020040 con 0x55942585fc80 2024-09-19T06:48:40.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 10 client.17412 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 10 client.17412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 20 client.17412 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942586ea90 0x5594258eb8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 4 client.17412 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5594257ccd60 con 0x55942586ea90 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.244+0000 7f0869d04640 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942586ea90 0x5594258eb8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f084d7fa640 10 client.17412 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f0869d04640 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942586ea90 0x5594258eb8d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f49e50ebb950036 server_cookie=323178f2d3320607 in_seq=0 out_seq=0 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f084d7fa640 1 -- 172.21.15.37:0/1297431524 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5594257ccd60 con 0x55942586ea90 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f084d7fa640 10 client.17412 handle_command_reply: tid=0 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 client.17412 shutdown 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585fc80 msgr2=0x559425873200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585fc80 0x559425873200 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f0854004c80 tx=0x7f0854000bc0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 shutdown_connections 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55942586ea90 0x5594258eb8d0 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55942585fc80 0x559425873200 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f084d7fa640 0 client.17412 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55942585f360 0x559425872cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 --2- 172.21.15.37:0/1297431524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55942585ba30 0x559425872780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 >> 172.21.15.37:0/1297431524 conn(0x559425867110 msgr2=0x55942585ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 shutdown_connections 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 1 -- 172.21.15.37:0/1297431524 wait complete. 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.245+0000 7f086e341740 20 client.17412 trim_cache size 0 max 16384 2024-09-19T06:48:40.251 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "24670327-0db3-42b6-a911-26ff1ee78a7b": { 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "24670327-0db3-42b6-a911-26ff1ee78a7b", 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:48:40.252 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f08500a3430 msgr2=0x7f08500a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f08500a3430 0x7f08500a58f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f085c0cdae0 tx=0x7f085c0054b0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f086410c170 msgr2=0x7f086419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f086410c170 0x7f086419cdb0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f085c009fd0 tx=0x7f085c00ef10 comp rx=0 tx=0).stop 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 shutdown_connections 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f08500a3430 0x7f08500a58f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f086410c170 0x7f086419cdb0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0864103660 0x7f0864198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 --2- 172.21.15.37:0/482008751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0864102d40 0x7f08641984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:40.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 >> 172.21.15.37:0/482008751 conn(0x7f08640fc7a0 msgr2=0x7f08640fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 shutdown_connections 2024-09-19T06:48:40.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:48:40.246+0000 7f086bf68640 1 -- 172.21.15.37:0/482008751 wait complete. 2024-09-19T06:48:40.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:40 smithi096 ceph-mon[35004]: pgmap v213: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 352 op/s 2024-09-19T06:48:40.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:40 smithi037 ceph-mon[31211]: pgmap v213: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 352 op/s 2024-09-19T06:48:40.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:40 smithi151 ceph-mon[35236]: pgmap v213: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 352 op/s 2024-09-19T06:48:40.999 INFO:tasks.cephfs.filesystem:scrub status for tag:24670327-0db3-42b6-a911-26ff1ee78a7b - {'path': '/', 'tag': '24670327-0db3-42b6-a911-26ff1ee78a7b', 'options': 'recursive,force'} 2024-09-19T06:48:41.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 500758 7.99 MB/sec execute 188 sec latency 11.618 ms 2024-09-19T06:48:42.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 503032 8.00 MB/sec execute 189 sec latency 11.746 ms 2024-09-19T06:48:42.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:42 smithi096 ceph-mon[35004]: pgmap v214: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 352 op/s 2024-09-19T06:48:42.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:42 smithi037 ceph-mon[31211]: pgmap v214: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 352 op/s 2024-09-19T06:48:42.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:42 smithi151 ceph-mon[35236]: pgmap v214: 97 pgs: 97 active+clean; 48 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 352 op/s 2024-09-19T06:48:43.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 505615 8.04 MB/sec execute 190 sec latency 18.233 ms 2024-09-19T06:48:44.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 508410 8.09 MB/sec execute 191 sec latency 47.907 ms 2024-09-19T06:48:44.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:44 smithi096 ceph-mon[35004]: pgmap v215: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-09-19T06:48:44.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:44 smithi037 ceph-mon[31211]: pgmap v215: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-09-19T06:48:44.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:44 smithi151 ceph-mon[35236]: pgmap v215: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-09-19T06:48:45.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 511000 8.11 MB/sec execute 192 sec latency 19.692 ms 2024-09-19T06:48:46.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 513470 8.13 MB/sec execute 193 sec latency 18.379 ms 2024-09-19T06:48:46.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:46 smithi096 ceph-mon[35004]: pgmap v216: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-09-19T06:48:46.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:46 smithi037 ceph-mon[31211]: pgmap v216: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-09-19T06:48:46.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:46 smithi151 ceph-mon[35236]: pgmap v216: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 356 op/s 2024-09-19T06:48:47.212 INFO:tasks.workunit.client.0.smithi037.stdout: 1 516264 8.17 MB/sec execute 194 sec latency 16.676 ms 2024-09-19T06:48:47.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:47 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:48:47.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:47 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:48:47.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:47 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:48:47.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:47 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:48:47.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:47 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:48:47.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:47 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:48:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.859+0000 7fcdbcae6640 1 -- 172.21.15.37:0/1715425956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8109090 msgr2=0x7fcdb8110be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.859+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/1715425956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8109090 0x7fcdb8110be0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009be0 tx=0x7fcdb002a6c0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 -- 172.21.15.37:0/1715425956 shutdown_connections 2024-09-19T06:48:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/1715425956 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb8109090 0x7fcdb8110be0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/1715425956 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb81048f0 0x7fcdb8108930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/1715425956 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdb8103f40 0x7fcdb8104320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 -- 172.21.15.37:0/1715425956 >> 172.21.15.37:0/1715425956 conn(0x7fcdb80ff970 msgr2=0x7fcdb8101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 -- 172.21.15.37:0/1715425956 shutdown_connections 2024-09-19T06:48:47.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 -- 172.21.15.37:0/1715425956 wait complete. 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.860+0000 7fcdbcae6640 1 Processor -- start 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdbcae6640 1 -- start start 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdbcae6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdb8103f40 0x7fcdb819a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdbcae6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8109090 0x7fcdb819aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdbcae6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb819ee60 0x7fcdb819f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdbcae6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81140d0 con 0x7fcdb819ee60 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdbcae6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8113f50 con 0x7fcdb8109090 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdbcae6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb8114250 con 0x7fcdb8103f40 2024-09-19T06:48:47.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdb6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb819ee60 0x7fcdb819f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8109090 0x7fcdb819aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8109090 0x7fcdb819aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38750/0 (socket says 172.21.15.37:38750) 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdadd9b640 1 -- 172.21.15.37:0/3546506888 learned_addr learned my addr 172.21.15.37:0/3546506888 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdb659c640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdb8103f40 0x7fcdb819a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdb6d9d640 1 -- 172.21.15.37:0/3546506888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdb8103f40 msgr2=0x7fcdb819a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdb6d9d640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdb8103f40 0x7fcdb819a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdb6d9d640 1 -- 172.21.15.37:0/3546506888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8109090 msgr2=0x7fcdb819aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdb6d9d640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8109090 0x7fcdb819aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.861+0000 7fcdb6d9d640 1 -- 172.21.15.37:0/3546506888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdb819f950 con 0x7fcdb819ee60 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.862+0000 7fcdb6d9d640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb819ee60 0x7fcdb819f310 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009b30 tx=0x7fcdb00305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.862+0000 7fcdaffff640 1 -- 172.21.15.37:0/3546506888 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0041020 con 0x7fcdb819ee60 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.862+0000 7fcdaffff640 1 -- 172.21.15.37:0/3546506888 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcdb0003e50 con 0x7fcdb819ee60 2024-09-19T06:48:47.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.862+0000 7fcdaffff640 1 -- 172.21.15.37:0/3546506888 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0036040 con 0x7fcdb819ee60 2024-09-19T06:48:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.862+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdb819fbe0 con 0x7fcdb819ee60 2024-09-19T06:48:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.862+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdb81a7340 con 0x7fcdb819ee60 2024-09-19T06:48:47.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.865+0000 7fcdaffff640 1 -- 172.21.15.37:0/3546506888 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcdb0044020 con 0x7fcdb819ee60 2024-09-19T06:48:47.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.865+0000 7fcdbf87e740 1 Processor -- start 2024-09-19T06:48:47.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.865+0000 7fcdbf87e740 1 -- start start 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f1492ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14a00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5601f13f72f0 con 0x5601f1493410 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5601f139f270 con 0x5601f1492af0 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5601f13f32b0 con 0x5601f149dcb0 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f1492ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdaffff640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcda40a3460 0x7fcda40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36632/0 (socket says 172.21.15.37:36632) 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 -- 172.21.15.37:0/2259103023 learned_addr learned my addr 172.21.15.37:0/2259103023 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:47.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f1492ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38756/0 (socket says 172.21.15.37:38756) 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14a00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb6d9d640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcda40a3460 0x7fcda40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 -- 172.21.15.37:0/2259103023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 msgr2=0x5601f14a00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14a00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 -- 172.21.15.37:0/2259103023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 msgr2=0x5601f1492ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f1492ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdb659c640 1 -- 172.21.15.37:0/2259103023 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f139e340 con 0x5601f1493410 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f1492ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.866+0000 7fcdaffff640 1 -- 172.21.15.37:0/3546506888 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fcdb00f08c0 con 0x7fcdb819ee60 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.867+0000 7fcdb659c640 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149d770 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b760 tx=0x7fcda800bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0c31128ea3043e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.867+0000 7fcdb6d9d640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcda40a3460 0x7fcda40a5920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00ee610 tx=0x7fcdb0009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:47.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.867+0000 7fcdacd99640 1 -- 172.21.15.37:0/2259103023 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8003dd0 con 0x5601f1493410 2024-09-19T06:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.867+0000 7fcdacd99640 1 -- 172.21.15.37:0/2259103023 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8015070 con 0x5601f1493410 2024-09-19T06:48:47.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.867+0000 7fcdacd99640 1 -- 172.21.15.37:0/2259103023 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8004e10 con 0x5601f1493410 2024-09-19T06:48:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2259103023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 msgr2=0x5601f149d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149d770 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b760 tx=0x7fcda800bc20 comp rx=0 tx=0).stop 2024-09-19T06:48:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2259103023 shutdown_connections 2024-09-19T06:48:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14a00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149d770 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2259103023 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f1492ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.875 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2259103023 >> 172.21.15.37:0/2259103023 conn(0x5601f148f060 msgr2=0x5601f148f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2259103023 shutdown_connections 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.868+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2259103023 wait complete. 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 -- start start 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 Processor -- start 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1492af0 0x5601f149ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f149dcb0 0x5601f149b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5601f14a0880 con 0x5601f1493410 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5601f1380290 con 0x5601f149dcb0 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5601f14a0a00 con 0x5601f1492af0 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdbf87e740 10 client.0 _ll_register_callbacks cb 0x5601f13b1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:47.876 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36636/0 (socket says 172.21.15.37:36636) 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1492af0 0x5601f149ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 -- 172.21.15.37:0/116990438 learned_addr learned my addr 172.21.15.37:0/116990438 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f149dcb0 0x5601f149b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 -- 172.21.15.37:0/116990438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1492af0 msgr2=0x5601f149ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1492af0 0x5601f149ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 -- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f149dcb0 msgr2=0x5601f149b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f149dcb0 0x5601f149b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.869+0000 7fcdb659c640 1 -- 172.21.15.37:0/116990438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f110a210 con 0x5601f1493410 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f149dcb0 0x5601f149b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:47.877 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcdb659c640 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149b150 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcda8006010 tx=0x7fcda8004840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3aea25b6d92d883 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcd9dffb640 1 -- 172.21.15.37:0/116990438 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8028060 con 0x5601f1493410 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcd9dffb640 1 -- 172.21.15.37:0/116990438 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fcda8004b90 con 0x5601f1493410 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda8015070 con 0x5601f1493410 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcd9dffb640 1 -- 172.21.15.37:0/116990438 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcda8004d90 con 0x5601f1493410 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcdbf87e740 10 client.17424 fetch_fsmap learned FSMap version 36 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5601f11d6620 con 0x5601f1493410 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.870+0000 7fcd9dffb640 1 -- 172.21.15.37:0/116990438 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8015070 con 0x5601f1493410 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcd9dffb640 1 -- 172.21.15.37:0/116990438 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fcda8022070 con 0x5601f1493410 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdbf87e740 10 client.17424 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdbf87e740 10 client.17424 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdbf87e740 20 client.17424 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:47.878 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f14adc70 0x5601f14b0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdbf87e740 4 client.17424 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601f1404a20 con 0x5601f14adc70 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f14adc70 0x5601f14b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcd9dffb640 10 client.17424 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.871+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f14adc70 0x5601f14b0060 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=438bb20c027ee75a server_cookie=658c980acc7302f1 in_seq=0 out_seq=0 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.872+0000 7fcd9dffb640 1 -- 172.21.15.37:0/116990438 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5601f1404a20 con 0x5601f14adc70 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.872+0000 7fcd9dffb640 10 client.17424 handle_command_reply: tid=0 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.872+0000 7fcdbf87e740 1 client.17424 shutdown 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 msgr2=0x5601f149b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149b150 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fcda8006010 tx=0x7fcda8004840 comp rx=0 tx=0).stop 2024-09-19T06:48:47.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 shutdown_connections 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f14adc70 0x5601f14b0060 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f149dcb0 0x5601f149b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149b150 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcd9dffb640 0 client.17424 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/116990438 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1492af0 0x5601f149ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 >> 172.21.15.37:0/116990438 conn(0x5601f148f060 msgr2=0x5601f1494dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 shutdown_connections 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 1 -- 172.21.15.37:0/116990438 wait complete. 2024-09-19T06:48:47.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.873+0000 7fcdbf87e740 20 client.17424 trim_cache size 0 max 16384 2024-09-19T06:48:47.895 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 Processor -- start 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 -- start start 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f149b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14b56c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81b22f0 con 0x5601f1493410 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81afb60 con 0x5601f1492af0 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81ad9b0 con 0x5601f149dcb0 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.896 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f149b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149cb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36644/0 (socket says 172.21.15.37:36644) 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 -- 172.21.15.37:0/2812110102 learned_addr learned my addr 172.21.15.37:0/2812110102 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14b56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 -- 172.21.15.37:0/2812110102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 msgr2=0x5601f14b56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14b56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 -- 172.21.15.37:0/2812110102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 msgr2=0x5601f149b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f149b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.889+0000 7fcdb659c640 1 -- 172.21.15.37:0/2812110102 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5601f14a0400 con 0x5601f1493410 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.890+0000 7fcdb659c640 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149cb80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b760 tx=0x7fcda800b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfcd49259dc8bafb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.890+0000 7fcd9ffff640 1 -- 172.21.15.37:0/2812110102 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8003ea0 con 0x5601f1493410 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.890+0000 7fcd9ffff640 1 -- 172.21.15.37:0/2812110102 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8015030 con 0x5601f1493410 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.890+0000 7fcd9ffff640 1 -- 172.21.15.37:0/2812110102 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda8005190 con 0x5601f1493410 2024-09-19T06:48:47.897 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2812110102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 msgr2=0x5601f149cb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149cb80 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fcda800b760 tx=0x7fcda800b390 comp rx=0 tx=0).stop 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2812110102 shutdown_connections 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f149dcb0 0x5601f14b56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f1493410 0x5601f149cb80 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/2812110102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f149b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2812110102 >> 172.21.15.37:0/2812110102 conn(0x5601f148f060 msgr2=0x5601f149c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2812110102 shutdown_connections 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- 172.21.15.37:0/2812110102 wait complete. 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- start start 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 Processor -- start 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f14a1400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1493410 0x5601f14a1940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.898 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f149dcb0 0x5601f15106c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81ae020 con 0x5601f149dcb0 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81af490 con 0x5601f1492af0 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcdb81ae6f0 con 0x5601f1493410 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdbf87e740 10 client.0 _ll_register_callbacks cb 0x7fcda0009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f149dcb0 0x5601f15106c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.891+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f14a1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f14a1400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38788/0 (socket says 172.21.15.37:38788) 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 -- 172.21.15.37:0/3755424009 learned_addr learned my addr 172.21.15.37:0/3755424009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdb659c640 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1493410 0x5601f14a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 -- 172.21.15.37:0/3755424009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1493410 msgr2=0x5601f14a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1493410 0x5601f14a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 -- 172.21.15.37:0/3755424009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f149dcb0 msgr2=0x5601f15106c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:48:47.899 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f149dcb0 0x5601f15106c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 -- 172.21.15.37:0/3755424009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601f1510d60 con 0x5601f1492af0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdb659c640 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1493410 0x5601f14a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdadd9b640 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f14a1400 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5601f1495690 tx=0x7fcda0000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1714e814dece2e66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcd9dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcd9dffb640 1 -- 172.21.15.37:0/3755424009 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcda0014e20 con 0x5601f1492af0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcda8015030 con 0x5601f1492af0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcd9dffb640 1 -- 172.21.15.37:0/3755424009 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fcda0023040 con 0x5601f1492af0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcd9dffb640 1 -- 172.21.15.37:0/3755424009 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcda0013020 con 0x5601f1492af0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdbf87e740 10 client.26825 fetch_fsmap learned FSMap version 36 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.892+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5601f15181d0 con 0x5601f1492af0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcd9dffb640 1 -- 172.21.15.37:0/3755424009 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcda8015030 con 0x5601f1492af0 2024-09-19T06:48:47.900 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcd9dffb640 1 -- 172.21.15.37:0/3755424009 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fcda0005340 con 0x5601f1492af0 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdbf87e740 10 client.26825 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdbf87e740 10 client.26825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdbf87e740 20 client.26825 populate_metadata read hostname 'smithi037' 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f1518910 0x5601f151ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdbf87e740 4 client.26825 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5601f13fed70 con 0x5601f1518910 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdb659c640 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f1518910 0x5601f151ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcdb659c640 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f1518910 0x5601f151ad30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c9139ff32a9f851 server_cookie=e85bc92aa27ca643 in_seq=0 out_seq=0 2024-09-19T06:48:47.901 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:47.893+0000 7fcd9dffb640 10 client.26825 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.049+0000 7fcd9dffb640 1 -- 172.21.15.37:0/3755424009 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5601f13fed70 con 0x5601f1518910 2024-09-19T06:48:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.049+0000 7fcd9dffb640 10 client.26825 handle_command_reply: tid=0 2024-09-19T06:48:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.049+0000 7fcdbf87e740 1 client.26825 shutdown 2024-09-19T06:48:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 msgr2=0x5601f14a1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f14a1400 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x5601f1495690 tx=0x7fcda0000bc0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 shutdown_connections 2024-09-19T06:48:48.052 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5601f1518910 0x5601f151ad30 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcd9dffb640 0 client.26825 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5601f149dcb0 0x5601f15106c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5601f1493410 0x5601f14a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 --2- 172.21.15.37:0/3755424009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5601f1492af0 0x5601f14a1400 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 >> 172.21.15.37:0/3755424009 conn(0x5601f148f060 msgr2=0x5601f1499080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 shutdown_connections 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 1 -- 172.21.15.37:0/3755424009 wait complete. 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.050+0000 7fcdbf87e740 20 client.26825 trim_cache size 0 max 16384 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:48:48.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:48:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:48:48.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:48:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.051+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcda40a3460 msgr2=0x7fcda40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.051+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcda40a3460 0x7fcda40a5920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcdb00ee610 tx=0x7fcdb0009500 comp rx=0 tx=0).stop 2024-09-19T06:48:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.051+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb819ee60 msgr2=0x7fcdb819f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:48:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.051+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb819ee60 0x7fcdb819f310 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0009b30 tx=0x7fcdb00305d0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 shutdown_connections 2024-09-19T06:48:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcda40a3460 0x7fcda40a5920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcdb819ee60 0x7fcdb819f310 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcdb8109090 0x7fcdb819aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 --2- 172.21.15.37:0/3546506888 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcdb8103f40 0x7fcdb819a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:48:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 >> 172.21.15.37:0/3546506888 conn(0x7fcdb80ff970 msgr2=0x7fcdb8100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:48:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 shutdown_connections 2024-09-19T06:48:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:48:48.052+0000 7fcdbcae6640 1 -- 172.21.15.37:0/3546506888 wait complete. 2024-09-19T06:48:48.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 518946 8.22 MB/sec execute 195 sec latency 33.935 ms 2024-09-19T06:48:48.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:48 smithi151 ceph-mon[35236]: pgmap v217: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-09-19T06:48:48.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:48 smithi096 ceph-mon[35004]: pgmap v217: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-09-19T06:48:48.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:48 smithi037 ceph-mon[31211]: pgmap v217: 97 pgs: 97 active+clean; 55 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 311 op/s 2024-09-19T06:48:49.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 521074 8.21 MB/sec execute 196 sec latency 87.478 ms 2024-09-19T06:48:50.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 523216 8.24 MB/sec execute 197 sec latency 15.722 ms 2024-09-19T06:48:50.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:50 smithi151 ceph-mon[35236]: pgmap v218: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 408 op/s 2024-09-19T06:48:50.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:50 smithi096 ceph-mon[35004]: pgmap v218: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 408 op/s 2024-09-19T06:48:50.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:50 smithi037 ceph-mon[31211]: pgmap v218: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.0 MiB/s wr, 408 op/s 2024-09-19T06:48:51.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 526196 8.29 MB/sec execute 198 sec latency 10.885 ms 2024-09-19T06:48:52.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 529134 8.32 MB/sec execute 199 sec latency 16.089 ms 2024-09-19T06:48:52.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:52 smithi151 ceph-mon[35236]: pgmap v219: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-09-19T06:48:52.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:52 smithi096 ceph-mon[35004]: pgmap v219: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-09-19T06:48:52.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:52 smithi037 ceph-mon[31211]: pgmap v219: 97 pgs: 97 active+clean; 63 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.5 MiB/s wr, 223 op/s 2024-09-19T06:48:53.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 531482 8.33 MB/sec execute 200 sec latency 13.320 ms 2024-09-19T06:48:54.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 533831 8.36 MB/sec execute 201 sec latency 17.193 ms 2024-09-19T06:48:54.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:54 smithi151 ceph-mon[35236]: pgmap v220: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 456 op/s 2024-09-19T06:48:54.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:54 smithi096 ceph-mon[35004]: pgmap v220: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 456 op/s 2024-09-19T06:48:54.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:54 smithi037 ceph-mon[31211]: pgmap v220: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 6.8 MiB/s wr, 456 op/s 2024-09-19T06:48:55.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 536797 8.41 MB/sec execute 202 sec latency 11.703 ms 2024-09-19T06:48:56.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 539439 8.43 MB/sec execute 203 sec latency 12.975 ms 2024-09-19T06:48:56.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:56 smithi037 ceph-mon[31211]: pgmap v221: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-09-19T06:48:56.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:56 smithi151 ceph-mon[35236]: pgmap v221: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-09-19T06:48:56.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:56 smithi096 ceph-mon[35004]: pgmap v221: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-09-19T06:48:57.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 541647 8.45 MB/sec execute 204 sec latency 12.635 ms 2024-09-19T06:48:58.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 544242 8.47 MB/sec execute 205 sec latency 14.097 ms 2024-09-19T06:48:58.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:48:58 smithi151 ceph-mon[35236]: pgmap v222: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-09-19T06:48:58.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:48:58 smithi096 ceph-mon[35004]: pgmap v222: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-09-19T06:48:58.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:48:58 smithi037 ceph-mon[31211]: pgmap v222: 97 pgs: 97 active+clean; 70 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.1 MiB/s wr, 345 op/s 2024-09-19T06:48:59.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 547155 8.52 MB/sec execute 206 sec latency 29.178 ms 2024-09-19T06:49:00.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 549408 8.52 MB/sec execute 207 sec latency 16.619 ms 2024-09-19T06:49:00.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:00 smithi151 ceph-mon[35236]: pgmap v223: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 431 op/s 2024-09-19T06:49:00.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:00 smithi096 ceph-mon[35004]: pgmap v223: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 431 op/s 2024-09-19T06:49:00.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:00 smithi037 ceph-mon[31211]: pgmap v223: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 6.1 MiB/s wr, 431 op/s 2024-09-19T06:49:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.147+0000 7fb6285fd640 1 -- 172.21.15.37:0/3819224973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620106b70 msgr2=0x7fb620106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.147+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3819224973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620106b70 0x7fb620106f50 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009900 tx=0x7fb61c02a410 comp rx=0 tx=0).stop 2024-09-19T06:49:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 -- 172.21.15.37:0/3819224973 shutdown_connections 2024-09-19T06:49:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3819224973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 0x7fb62010d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3819224973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620101740 0x7fb620101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.151 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3819224973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620106b70 0x7fb620106f50 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 -- 172.21.15.37:0/3819224973 >> 172.21.15.37:0/3819224973 conn(0x7fb620075ef0 msgr2=0x7fb620076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 -- 172.21.15.37:0/3819224973 shutdown_connections 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 -- 172.21.15.37:0/3819224973 wait complete. 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 Processor -- start 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 -- start start 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620101740 0x7fb6201983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 0x7fb620198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.148+0000 7fb6285fd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620106b70 0x7fb62019cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb6285fd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb620110f20 con 0x7fb620101740 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb6285fd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb620110da0 con 0x7fb6201020c0 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb6285fd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201110a0 con 0x7fb620106b70 2024-09-19T06:49:01.152 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb626399640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620101740 0x7fb6201983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 0x7fb620198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 0x7fb620198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47676/0 (socket says 172.21.15.37:47676) 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb626399640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620101740 0x7fb6201983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40730/0 (socket says 172.21.15.37:40730) 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620106b70 0x7fb62019cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 -- 172.21.15.37:0/3753310583 learned_addr learned my addr 172.21.15.37:0/3753310583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 -- 172.21.15.37:0/3753310583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620106b70 msgr2=0x7fb62019cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620106b70 0x7fb62019cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 -- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620101740 msgr2=0x7fb6201983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620101740 0x7fb6201983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 -- 172.21.15.37:0/3753310583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62019d3b0 con 0x7fb6201020c0 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb626399640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620101740 0x7fb6201983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb625b98640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 0x7fb620198900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb61000d770 tx=0x7fb61000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb6177fe640 1 -- 172.21.15.37:0/3753310583 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610017070 con 0x7fb6201020c0 2024-09-19T06:49:01.153 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb6177fe640 1 -- 172.21.15.37:0/3753310583 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb610005500 con 0x7fb6201020c0 2024-09-19T06:49:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb62019d6a0 con 0x7fb6201020c0 2024-09-19T06:49:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.149+0000 7fb6177fe640 1 -- 172.21.15.37:0/3753310583 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb610004e00 con 0x7fb6201020c0 2024-09-19T06:49:01.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.150+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6201a4f30 con 0x7fb6201020c0 2024-09-19T06:49:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.152+0000 7fb6177fe640 1 -- 172.21.15.37:0/3753310583 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb610004fa0 con 0x7fb6201020c0 2024-09-19T06:49:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.152+0000 7fb62b47f740 1 Processor -- start 2024-09-19T06:49:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb62b47f740 1 -- start start 2024-09-19T06:49:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a638adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6399b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb6177fe640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb60c0a34e0 0x7fb60c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a639c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a638adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb625b98640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb60c0a34e0 0x7fb60c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a638adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40732/0 (socket says 172.21.15.37:40732) 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb626b9a640 1 -- 172.21.15.37:0/494120940 learned_addr learned my addr 172.21.15.37:0/494120940 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5600a62ef2f0 con 0x5600a638a9d0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600a6297270 con 0x5600a638b2f0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5600a62eb2b0 con 0x5600a639a0c0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb626b9a640 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a639c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.153+0000 7fb626399640 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6399b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb6177fe640 1 -- 172.21.15.37:0/3753310583 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb6100cf620 con 0x7fb6201020c0 2024-09-19T06:49:01.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb626b9a640 1 -- 172.21.15.37:0/494120940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 msgr2=0x5600a639c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb626b9a640 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a639c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb626b9a640 1 -- 172.21.15.37:0/494120940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 msgr2=0x5600a6399b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb626b9a640 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6399b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb626b9a640 1 -- 172.21.15.37:0/494120940 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a6296340 con 0x5600a638a9d0 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb626399640 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6399b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb625b98640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb60c0a34e0 0x7fb60c0a59a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb610098380 tx=0x7fb61006e7e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb626b9a640 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a638adb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb618013b50 tx=0x7fb618009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58d6a97d86c99880 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb614bfe640 1 -- 172.21.15.37:0/494120940 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61800fd10 con 0x5600a638a9d0 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb614bfe640 1 -- 172.21.15.37:0/494120940 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb618013130 con 0x5600a638a9d0 2024-09-19T06:49:01.158 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.154+0000 7fb614bfe640 1 -- 172.21.15.37:0/494120940 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618003bb0 con 0x5600a638a9d0 2024-09-19T06:49:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 msgr2=0x5600a638adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a638adb0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb618013b50 tx=0x7fb618009b30 comp rx=0 tx=0).stop 2024-09-19T06:49:01.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 shutdown_connections 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a639c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6399b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 --2- 172.21.15.37:0/494120940 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a638adb0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 >> 172.21.15.37:0/494120940 conn(0x5600a6387060 msgr2=0x5600a6387440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 shutdown_connections 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.155+0000 7fb62b47f740 1 -- 172.21.15.37:0/494120940 wait complete. 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 -- start start 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 Processor -- start 2024-09-19T06:49:01.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a6396fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a6397a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5600a639cc90 con 0x5600a639a0c0 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600a6278290 con 0x5600a638b2f0 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5600a639ce10 con 0x5600a638a9d0 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb62b47f740 10 client.0 _ll_register_callbacks cb 0x5600a62a9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb626399640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a6396fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb626399640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47702/0 (socket says 172.21.15.37:47702) 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb626399640 1 -- 172.21.15.37:0/2730664578 learned_addr learned my addr 172.21.15.37:0/2730664578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.156+0000 7fb626b9a640 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a6397a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb626399640 1 -- 172.21.15.37:0/2730664578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 msgr2=0x5600a6396fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb626399640 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a6396fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb626399640 1 -- 172.21.15.37:0/2730664578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 msgr2=0x5600a6397a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb626399640 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a6397a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb626399640 1 -- 172.21.15.37:0/2730664578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a6002210 con 0x5600a638b2f0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb626399640 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009900 tx=0x7fb61c005270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a246cf7624a40dc1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb609ffb640 1 -- 172.21.15.37:0/2730664578 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb61c035040 con 0x5600a638b2f0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb609ffb640 1 -- 172.21.15.37:0/2730664578 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb61c004760 con 0x5600a638b2f0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb618013130 con 0x5600a638b2f0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb609ffb640 1 -- 172.21.15.37:0/2730664578 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb61c004960 con 0x5600a638b2f0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb62b47f740 10 client.26837 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.157+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5600a60ce620 con 0x5600a638b2f0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb609ffb640 1 -- 172.21.15.37:0/2730664578 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb618013130 con 0x5600a638b2f0 2024-09-19T06:49:01.165 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb609ffb640 1 -- 172.21.15.37:0/2730664578 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb61c044020 con 0x5600a638b2f0 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb62b47f740 10 client.26837 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb62b47f740 10 client.26837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb62b47f740 20 client.26837 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a63ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb62b47f740 4 client.26837 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600a62fca20 con 0x5600a63aa0c0 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.158+0000 7fb626b9a640 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a63ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.159+0000 7fb609ffb640 10 client.26837 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.159+0000 7fb626b9a640 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a63ac4e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f33643151578e server_cookie=61dc0d91648ecb89 in_seq=0 out_seq=0 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb609ffb640 1 -- 172.21.15.37:0/2730664578 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5600a62fca20 con 0x5600a63aa0c0 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb609ffb640 10 client.26837 handle_command_reply: tid=0 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 client.26837 shutdown 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 msgr2=0x5600a6397520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb61c009900 tx=0x7fb61c005270 comp rx=0 tx=0).stop 2024-09-19T06:49:01.166 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 shutdown_connections 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a63ac4e0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb609ffb640 0 client.26837 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a6397a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2730664578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a6396fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.160+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 >> 172.21.15.37:0/2730664578 conn(0x5600a6387060 msgr2=0x5600a638cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.161+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 shutdown_connections 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.161+0000 7fb62b47f740 1 -- 172.21.15.37:0/2730664578 wait complete. 2024-09-19T06:49:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.161+0000 7fb62b47f740 20 client.26837 trim_cache size 0 max 16384 2024-09-19T06:49:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 Processor -- start 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 -- start start 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a6395890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a63b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201adea0 con 0x5600a638a9d0 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ab710 con 0x5600a638b2f0 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb62b47f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a9560 con 0x5600a639a0c0 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a6395890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a6395890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40744/0 (socket says 172.21.15.37:40744) 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 -- 172.21.15.37:0/2146277786 learned_addr learned my addr 172.21.15.37:0/2146277786 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626399640 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a63b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 -- 172.21.15.37:0/2146277786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 msgr2=0x5600a63b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a63b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 -- 172.21.15.37:0/2146277786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 msgr2=0x5600a6397890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.177+0000 7fb626b9a640 1 -- 172.21.15.37:0/2146277786 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600a63ac850 con 0x5600a638a9d0 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb626b9a640 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a6395890 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb618013430 tx=0x7fb618009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36e24f19ac0f723 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb60bfff640 1 -- 172.21.15.37:0/2146277786 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618003970 con 0x5600a638a9d0 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb60bfff640 1 -- 172.21.15.37:0/2146277786 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61802f3c0 con 0x5600a638a9d0 2024-09-19T06:49:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb60bfff640 1 -- 172.21.15.37:0/2146277786 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6180050c0 con 0x5600a638a9d0 2024-09-19T06:49:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb62b47f740 1 -- 172.21.15.37:0/2146277786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 msgr2=0x5600a6395890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a6395890 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb618013430 tx=0x7fb618009cd0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb62b47f740 1 -- 172.21.15.37:0/2146277786 shutdown_connections 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a639a0c0 0x5600a63b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a6397890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb62b47f740 1 --2- 172.21.15.37:0/2146277786 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a638a9d0 0x5600a6395890 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.178+0000 7fb62b47f740 1 -- 172.21.15.37:0/2146277786 >> 172.21.15.37:0/2146277786 conn(0x5600a6387060 msgr2=0x5600a6399060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 -- 172.21.15.37:0/2146277786 shutdown_connections 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 -- 172.21.15.37:0/2146277786 wait complete. 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 -- start start 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 Processor -- start 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a63a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a639d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a639de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201a9bd0 con 0x5600a639a0c0 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201ab040 con 0x5600a638b2f0 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb6201aa2a0 con 0x5600a638a9d0 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb62b47f740 10 client.0 _ll_register_callbacks cb 0x7fb61c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a639de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb626b9a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a639de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40748/0 (socket says 172.21.15.37:40748) 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb626b9a640 1 -- 172.21.15.37:0/1024222242 learned_addr learned my addr 172.21.15.37:0/1024222242 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626b9a640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a63a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.179+0000 7fb626399640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a639d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626b9a640 1 -- 172.21.15.37:0/1024222242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 msgr2=0x5600a63a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626b9a640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a63a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626b9a640 1 -- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 msgr2=0x5600a639d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626b9a640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a639d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626b9a640 1 -- 172.21.15.37:0/1024222242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600a638b120 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626399640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a639d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb626b9a640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a639de90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb618009fd0 tx=0x7fb618015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=189db9acd39b8de9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb609ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb609ffb640 1 -- 172.21.15.37:0/1024222242 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb618004bb0 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb609ffb640 1 -- 172.21.15.37:0/1024222242 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8527+0+0 (secure 0 0 0) 0x7fb618004d50 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb61802f3c0 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb609ffb640 1 -- 172.21.15.37:0/1024222242 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb61802f570 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb62b47f740 10 client.17448 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.180+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5600a6414a10 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb609ffb640 1 -- 172.21.15.37:0/1024222242 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb61802f3c0 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb609ffb640 1 -- 172.21.15.37:0/1024222242 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb618021b80 con 0x5600a639a0c0 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb62b47f740 10 client.17448 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb62b47f740 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb62b47f740 20 client.17448 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb62b47f740 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a6416ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb62b47f740 4 client.17448 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:49:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600a62f6d70 con 0x5600a63aa0c0 2024-09-19T06:49:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb626399640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a6416ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb609ffb640 10 client.17448 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.181+0000 7fb626399640 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a6416ea0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f36178bb3d793b server_cookie=5b0892f4b3af3dcc in_seq=0 out_seq=0 2024-09-19T06:49:01.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 551365 8.54 MB/sec execute 208 sec latency 92.051 ms 2024-09-19T06:49:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb609ffb640 1 -- 172.21.15.37:0/1024222242 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600a62f6d70 con 0x5600a63aa0c0 2024-09-19T06:49:01.341 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb609ffb640 10 client.17448 handle_command_reply: tid=0 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 client.17448 shutdown 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 msgr2=0x5600a639de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a639de90 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fb618009fd0 tx=0x7fb618015040 comp rx=0 tx=0).stop 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 shutdown_connections 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5600a63aa0c0 0x5600a6416ea0 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5600a639a0c0 0x5600a639de90 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600a638b2f0 0x5600a639d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb609ffb640 0 client.17448 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.339+0000 7fb62b47f740 1 --2- 172.21.15.37:0/1024222242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5600a638a9d0 0x5600a63a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.340+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 >> 172.21.15.37:0/1024222242 conn(0x5600a6387060 msgr2=0x5600a638c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.340+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 shutdown_connections 2024-09-19T06:49:01.342 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.340+0000 7fb62b47f740 1 -- 172.21.15.37:0/1024222242 wait complete. 2024-09-19T06:49:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.340+0000 7fb62b47f740 20 client.17448 trim_cache size 0 max 16384 2024-09-19T06:49:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:49:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:49:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:49:01.343 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb60c0a34e0 msgr2=0x7fb60c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb60c0a34e0 0x7fb60c0a59a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb610098380 tx=0x7fb61006e7e0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 msgr2=0x7fb620198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 0x7fb620198900 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb61000d770 tx=0x7fb61000dc30 comp rx=0 tx=0).stop 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 shutdown_connections 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb620106b70 0x7fb62019cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb6201020c0 0x7fb620198900 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fb61000d770 tx=0x7fb61000dc30 comp rx=0 tx=0).stop 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb60c0a34e0 0x7fb60c0a59a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 --2- 172.21.15.37:0/3753310583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb620101740 0x7fb6201983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 >> 172.21.15.37:0/3753310583 conn(0x7fb620075ef0 msgr2=0x7fb620104d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:01.344 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.341+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 shutdown_connections 2024-09-19T06:49:01.345 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:01.342+0000 7fb6285fd640 1 -- 172.21.15.37:0/3753310583 wait complete. 2024-09-19T06:49:01.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:01 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:01.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:01 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:01.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:01 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:49:02.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 552992 8.53 MB/sec execute 209 sec latency 86.984 ms 2024-09-19T06:49:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:02 smithi151 ceph-mon[35236]: pgmap v224: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 319 op/s 2024-09-19T06:49:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:02 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:02 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:02.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:02 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:49:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:02 smithi096 ceph-mon[35004]: pgmap v224: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 319 op/s 2024-09-19T06:49:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:02 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:02 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:02 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:49:02.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:02 smithi037 ceph-mon[31211]: pgmap v224: 97 pgs: 97 active+clean; 80 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 4.2 MiB/s wr, 319 op/s 2024-09-19T06:49:02.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:02 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:49:02.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:02 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:49:02.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:02 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:49:02.997 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.5) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:49:02.998 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T06:49:02.998 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:03.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 554291 8.54 MB/sec execute 210 sec latency 27.724 ms 2024-09-19T06:49:03.441 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:03.573 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:03 smithi037 ceph-mon[31211]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:49:03.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:03 smithi151 ceph-mon[35236]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:49:03.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:03 smithi096 ceph-mon[35004]: osdmap e89: 12 total, 12 up, 12 in 2024-09-19T06:49:03.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.930+0000 7f8920667640 1 -- 172.21.15.37:0/4249546133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918103740 msgr2=0x7f8918107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.930+0000 7f8920667640 1 --2- 172.21.15.37:0/4249546133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918103740 0x7f8918107b80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8908009900 tx=0x7f890802a410 comp rx=0 tx=0).stop 2024-09-19T06:49:03.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 -- 172.21.15.37:0/4249546133 shutdown_connections 2024-09-19T06:49:03.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 --2- 172.21.15.37:0/4249546133 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8918108290 0x7f891810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 --2- 172.21.15.37:0/4249546133 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918103740 0x7f8918107b80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 --2- 172.21.15.37:0/4249546133 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918102d90 0x7f8918103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 -- 172.21.15.37:0/4249546133 >> 172.21.15.37:0/4249546133 conn(0x7f89180fc7d0 msgr2=0x7f89180febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:03.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 -- 172.21.15.37:0/4249546133 shutdown_connections 2024-09-19T06:49:03.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 -- 172.21.15.37:0/4249546133 wait complete. 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 Processor -- start 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.931+0000 7f8920667640 1 -- start start 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f8920667640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8918102d90 0x7f89181985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f8920667640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918103740 0x7f8918198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f8920667640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918108290 0x7f891819ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f8920667640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f89181120a0 con 0x7f8918102d90 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f8920667640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8918111f20 con 0x7f8918103740 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891dc02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918103740 0x7f8918198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891dc02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918103740 0x7f8918198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47742/0 (socket says 172.21.15.37:47742) 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891dc02640 1 -- 172.21.15.37:0/1447495660 learned_addr learned my addr 172.21.15.37:0/1447495660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:03.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8918112220 con 0x7f8918108290 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891ec04640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918108290 0x7f891819ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891e403640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8918102d90 0x7f89181985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891ec04640 1 -- 172.21.15.37:0/1447495660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918103740 msgr2=0x7f8918198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891ec04640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918103740 0x7f8918198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891ec04640 1 -- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8918102d90 msgr2=0x7f89181985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891ec04640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8918102d90 0x7f89181985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891ec04640 1 -- 172.21.15.37:0/1447495660 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f891819d560 con 0x7f8918108290 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.932+0000 7f891dc02640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918103740 0x7f8918198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.933+0000 7f891ec04640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918108290 0x7f891819ce80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8910009fd0 tx=0x7f891000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.933+0000 7f890f7fe640 1 -- 172.21.15.37:0/1447495660 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891000ce20 con 0x7f8918108290 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.933+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f891819d850 con 0x7f8918108290 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.933+0000 7f890f7fe640 1 -- 172.21.15.37:0/1447495660 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8910005020 con 0x7f8918108290 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.933+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8918100920 con 0x7f8918108290 2024-09-19T06:49:03.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.933+0000 7f890f7fe640 1 -- 172.21.15.37:0/1447495660 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8910010720 con 0x7f8918108290 2024-09-19T06:49:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.934+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88fc005290 con 0x7f8918108290 2024-09-19T06:49:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.935+0000 7f890f7fe640 1 -- 172.21.15.37:0/1447495660 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8910004a40 con 0x7f8918108290 2024-09-19T06:49:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.936+0000 7f890f7fe640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f89040a3430 0x7f89040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.936+0000 7f891e403640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f89040a3430 0x7f89040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.936+0000 7f890f7fe640 1 -- 172.21.15.37:0/1447495660 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f89100d0220 con 0x7f8918108290 2024-09-19T06:49:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.937+0000 7f891e403640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f89040a3430 0x7f89040a58f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8914009bd0 tx=0x7f8914005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:03.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:03.941+0000 7f890f7fe640 1 -- 172.21.15.37:0/1447495660 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f891009cba0 con 0x7f8918108290 2024-09-19T06:49:04.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.208+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88fc005850 con 0x7f8918108290 2024-09-19T06:49:04.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.209+0000 7f890f7fe640 1 -- 172.21.15.37:0/1447495660 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11907 (secure 0 0 0) 0x7f891009c4a0 con 0x7f8918108290 2024-09-19T06:49:04.211 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:49:04.211 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"btime":"2024-09-19T06:48:37:250513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:48:36.625010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:49:04.211 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f89040a3430 msgr2=0x7f89040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f89040a3430 0x7f89040a58f0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8914009bd0 tx=0x7f8914005f90 comp rx=0 tx=0).stop 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918108290 msgr2=0x7f891819ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918108290 0x7f891819ce80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f8910009fd0 tx=0x7f891000ef10 comp rx=0 tx=0).stop 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 shutdown_connections 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8918108290 0x7f891819ce80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8918103740 0x7f8918198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f89040a3430 0x7f89040a58f0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:04.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 --2- 172.21.15.37:0/1447495660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8918102d90 0x7f89181985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:04.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 >> 172.21.15.37:0/1447495660 conn(0x7f89180fc7d0 msgr2=0x7f8918106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:04.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 shutdown_connections 2024-09-19T06:49:04.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:04.211+0000 7f8920667640 1 -- 172.21.15.37:0/1447495660 wait complete. 2024-09-19T06:49:04.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 555382 8.51 MB/sec execute 211 sec latency 69.573 ms 2024-09-19T06:49:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:04 smithi037 ceph-mon[31211]: pgmap v226: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:04.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:04 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1447495660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:04 smithi151 ceph-mon[35236]: pgmap v226: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:04 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1447495660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:04.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:04 smithi096 ceph-mon[35004]: pgmap v226: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:04.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:04 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1447495660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:04.939 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T06:49:04.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T06:49:05.213 INFO:tasks.workunit.client.0.smithi037.stdout: 1 558116 8.55 MB/sec execute 212 sec latency 23.268 ms 2024-09-19T06:49:05.225 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:06.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.54 MB/sec execute 213 sec latency 303.380 ms 2024-09-19T06:49:06.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:06 smithi151 ceph-mon[35236]: pgmap v227: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:06 smithi096 ceph-mon[35004]: pgmap v227: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:06.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:06 smithi037 ceph-mon[31211]: pgmap v227: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 -- 172.21.15.37:0/237924524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 msgr2=0x7f061410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 --2- 172.21.15.37:0/237924524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061410ea90 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f060c009ca0 tx=0x7f060c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 -- 172.21.15.37:0/237924524 shutdown_connections 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 --2- 172.21.15.37:0/237924524 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061410ea90 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 --2- 172.21.15.37:0/237924524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0614102790 0x7f06141067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 --2- 172.21.15.37:0/237924524 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0614101de0 0x7f06141021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 -- 172.21.15.37:0/237924524 >> 172.21.15.37:0/237924524 conn(0x7f06140fd8e0 msgr2=0x7f06140ffd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 -- 172.21.15.37:0/237924524 shutdown_connections 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.775+0000 7f061bd92640 1 -- 172.21.15.37:0/237924524 wait complete. 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 Processor -- start 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 -- start start 2024-09-19T06:49:06.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0614101de0 0x7f06141983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0614102790 0x7f0614198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0614111f70 con 0x7f0614106f10 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0614111df0 con 0x7f0614102790 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061bd92640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06141120f0 con 0x7f0614101de0 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061a32f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061932d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0614102790 0x7f0614198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061a32f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40798/0 (socket says 172.21.15.37:40798) 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.776+0000 7f061a32f640 1 -- 172.21.15.37:0/126844768 learned_addr learned my addr 172.21.15.37:0/126844768 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f0619b2e640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0614101de0 0x7f06141983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061a32f640 1 -- 172.21.15.37:0/126844768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0614101de0 msgr2=0x7f06141983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061a32f640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0614101de0 0x7f06141983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061a32f640 1 -- 172.21.15.37:0/126844768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0614102790 msgr2=0x7f0614198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061a32f640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0614102790 0x7f0614198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061a32f640 1 -- 172.21.15.37:0/126844768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f061419d3a0 con 0x7f0614106f10 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061932d640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0614102790 0x7f0614198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061a32f640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061419ccc0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f060c031a20 tx=0x7f060c031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f060affd640 1 -- 172.21.15.37:0/126844768 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c003b70 con 0x7f0614106f10 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f060affd640 1 -- 172.21.15.37:0/126844768 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c003d10 con 0x7f0614106f10 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f061419d630 con 0x7f0614106f10 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f060affd640 1 -- 172.21.15.37:0/126844768 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c02fc40 con 0x7f0614106f10 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.777+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0614074420 con 0x7f0614106f10 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.779+0000 7f060affd640 1 -- 172.21.15.37:0/126844768 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f060c004050 con 0x7f0614106f10 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 Processor -- start 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 -- start start 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f060affd640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e75fcc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e76074f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e7607a30 0x55c7e7609e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c7e7561160 con 0x55c7e7607a30 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c7e7508720 con 0x55c7e75fd170 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.780+0000 7f061e16b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c7e755d120 con 0x55c7e75fc850 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e7607a30 0x55c7e7609e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f0619b2e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e75fcc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f060affd640 1 -- 172.21.15.37:0/126844768 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f060c0f0d60 con 0x7f0614106f10 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e7607a30 0x55c7e7609e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40814/0 (socket says 172.21.15.37:40814) 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 -- 172.21.15.37:0/4101651315 learned_addr learned my addr 172.21.15.37:0/4101651315 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061932d640 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e76074f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061932d640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 -- 172.21.15.37:0/4101651315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 msgr2=0x55c7e75fcc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e75fcc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 -- 172.21.15.37:0/4101651315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 msgr2=0x55c7e76074f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e76074f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 -- 172.21.15.37:0/4101651315 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e72d2ba0 con 0x55c7e7607a30 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f0619b2e640 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e75fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061a32f640 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e7607a30 0x55c7e7609e20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f060c0b95e0 tx=0x7f060c0b9420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=231744dc8cf76488 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.781+0000 7f061932d640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06000a33b0 0x7f06000a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0604009fd0 tx=0x7f0604005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f05fffff640 1 -- 172.21.15.37:0/4101651315 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c0a0d40 con 0x55c7e7607a30 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f05fffff640 1 -- 172.21.15.37:0/4101651315 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c036040 con 0x55c7e7607a30 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f05fffff640 1 -- 172.21.15.37:0/4101651315 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060c0bb480 con 0x55c7e7607a30 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f061e16b740 1 -- 172.21.15.37:0/4101651315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e7607a30 msgr2=0x55c7e7609e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f061e16b740 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e7607a30 0x55c7e7609e20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f060c0b95e0 tx=0x7f060c0b9420 comp rx=0 tx=0).stop 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f061e16b740 1 -- 172.21.15.37:0/4101651315 shutdown_connections 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f061e16b740 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e7607a30 0x55c7e7609e20 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f061e16b740 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e76074f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f061e16b740 1 --2- 172.21.15.37:0/4101651315 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e75fcc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.782+0000 7f061e16b740 1 -- 172.21.15.37:0/4101651315 >> 172.21.15.37:0/4101651315 conn(0x55c7e75f8de0 msgr2=0x55c7e75f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 -- 172.21.15.37:0/4101651315 shutdown_connections 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 -- 172.21.15.37:0/4101651315 wait complete. 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 -- start start 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 Processor -- start 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7604910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e7604e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e7605390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c7e7561160 con 0x55c7e75fd170 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c7e74e9900 con 0x55c7e7607a30 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c7e760a3c0 con 0x55c7e75fc850 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061e16b740 10 client.0 _ll_register_callbacks cb 0x55c7e74e7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:06.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.783+0000 7f061932d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e7604e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061a32f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e7605390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e7604e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40824/0 (socket says 172.21.15.37:40824) 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 -- 172.21.15.37:0/2866875338 learned_addr learned my addr 172.21.15.37:0/2866875338 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f0619b2e640 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7604910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 -- 172.21.15.37:0/2866875338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 msgr2=0x55c7e7604910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7604910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 -- 172.21.15.37:0/2866875338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 msgr2=0x55c7e7605390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e7605390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 -- 172.21.15.37:0/2866875338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e7529c50 con 0x55c7e75fd170 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061932d640 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e7604e50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0604011700 tx=0x7f0604004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ce5bf574b68ef3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f05fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f05fd7fa640 1 -- 172.21.15.37:0/2866875338 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060402f060 con 0x55c7e75fd170 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f05fd7fa640 1 -- 172.21.15.37:0/2866875338 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f060401fc70 con 0x55c7e75fd170 2024-09-19T06:49:06.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f060c036040 con 0x55c7e75fd170 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f05fd7fa640 1 -- 172.21.15.37:0/2866875338 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f060402a420 con 0x55c7e75fd170 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061e16b740 10 client.17466 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.784+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c7e7615e60 con 0x55c7e75fd170 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f05fd7fa640 1 -- 172.21.15.37:0/2866875338 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f060c036040 con 0x55c7e75fd170 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f05fd7fa640 1 -- 172.21.15.37:0/2866875338 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f060401d070 con 0x55c7e75fd170 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f061e16b740 10 client.17466 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f061e16b740 10 client.17466 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f061e16b740 20 client.17466 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f061e16b740 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e7617410 0x55c7e7619830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f061e16b740 4 client.17466 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7e756e890 con 0x55c7e7617410 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f0619b2e640 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e7617410 0x55c7e7619830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f0619b2e640 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e7617410 0x55c7e7619830 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd30b8651e7513b0 server_cookie=89301c4846635d13 in_seq=0 out_seq=0 2024-09-19T06:49:06.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.785+0000 7f05fd7fa640 10 client.17466 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f05fd7fa640 1 -- 172.21.15.37:0/2866875338 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c7e756e890 con 0x55c7e7617410 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f05fd7fa640 10 client.17466 handle_command_reply: tid=0 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 client.17466 shutdown 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 msgr2=0x55c7e7604e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e7604e50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0604011700 tx=0x7f0604004730 comp rx=0 tx=0).stop 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 shutdown_connections 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e7617410 0x55c7e7619830 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e7605390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e7604e50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 --2- 172.21.15.37:0/2866875338 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7604910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f05fd7fa640 0 client.17466 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 >> 172.21.15.37:0/2866875338 conn(0x55c7e75f8de0 msgr2=0x55c7e75feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 shutdown_connections 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 1 -- 172.21.15.37:0/2866875338 wait complete. 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.787+0000 7f061e16b740 20 client.17466 trim_cache size 0 max 16384 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.805+0000 7f061e16b740 1 Processor -- start 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061e16b740 1 -- start start 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061e16b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7606880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061e16b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e761f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061e16b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e761f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061e16b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06141b2520 con 0x55c7e75fd170 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061e16b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06141afd90 con 0x55c7e7607a30 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061e16b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06141adc00 con 0x55c7e75fc850 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e761f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e761f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40832/0 (socket says 172.21.15.37:40832) 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 -- 172.21.15.37:0/1946546966 learned_addr learned my addr 172.21.15.37:0/1946546966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061a32f640 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e761f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f0619b2e640 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7606880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 -- 172.21.15.37:0/1946546966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 msgr2=0x55c7e7606880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7606880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 -- 172.21.15.37:0/1946546966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 msgr2=0x55c7e761f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e761f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 -- 172.21.15.37:0/1946546966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7e75f43f0 con 0x55c7e75fd170 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061a32f640 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e761f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.806+0000 7f061932d640 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e761f0e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0604032830 tx=0x7f0604009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3bb05aa87f99e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f05ff7fe640 1 -- 172.21.15.37:0/1946546966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060402f070 con 0x55c7e75fd170 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f05ff7fe640 1 -- 172.21.15.37:0/1946546966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0604032410 con 0x55c7e75fd170 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f05ff7fe640 1 -- 172.21.15.37:0/1946546966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f060400e3b0 con 0x55c7e75fd170 2024-09-19T06:49:06.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f061e16b740 1 -- 172.21.15.37:0/1946546966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 msgr2=0x55c7e761f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f061e16b740 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e761f0e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0604032830 tx=0x7f0604009c20 comp rx=0 tx=0).stop 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f061e16b740 1 -- 172.21.15.37:0/1946546966 shutdown_connections 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f061e16b740 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e7607a30 0x55c7e761f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f061e16b740 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fd170 0x55c7e761f0e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.807+0000 7f061e16b740 1 --2- 172.21.15.37:0/1946546966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e75fc850 0x55c7e7606880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 -- 172.21.15.37:0/1946546966 >> 172.21.15.37:0/1946546966 conn(0x55c7e75f8de0 msgr2=0x55c7e7606170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 -- 172.21.15.37:0/1946546966 shutdown_connections 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 -- 172.21.15.37:0/1946546966 wait complete. 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 -- start start 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 Processor -- start 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fc850 0x55c7e760ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e760bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e7607a30 0x55c7e76810b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f06141ae250 con 0x55c7e75fc850 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f06141af6c0 con 0x55c7e75fd170 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f06141ae920 con 0x55c7e7607a30 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f061e16b740 10 client.0 _ll_register_callbacks cb 0x7f0604009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.808+0000 7f0619b2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fc850 0x55c7e760ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fc850 0x55c7e760ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40838/0 (socket says 172.21.15.37:40838) 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f061932d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e760bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 -- 172.21.15.37:0/264048435 learned_addr learned my addr 172.21.15.37:0/264048435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f061a32f640 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e7607a30 0x55c7e76810b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 -- 172.21.15.37:0/264048435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e7607a30 msgr2=0x55c7e76810b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e7607a30 0x55c7e76810b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 -- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 msgr2=0x55c7e760bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e760bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 -- 172.21.15.37:0/264048435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7e76816a0 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f061932d640 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e760bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f0619b2e640 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fc850 0x55c7e760ba90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55c7e761a020 tx=0x7f0610010c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=890d68f67a98d2e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f05fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f05fd7fa640 1 -- 172.21.15.37:0/264048435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0610022030 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0604032410 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f05fd7fa640 1 -- 172.21.15.37:0/264048435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8615+0+0 (secure 0 0 0) 0x7f0610004260 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f05fd7fa640 1 -- 172.21.15.37:0/264048435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f0610004460 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f061e16b740 10 client.17472 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.809+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c7e76829a0 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f05fd7fa640 1 -- 172.21.15.37:0/264048435 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0604032410 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f05fd7fa640 1 -- 172.21.15.37:0/264048435 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f0610003c20 con 0x55c7e75fc850 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061e16b740 10 client.17472 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061e16b740 10 client.17472 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:49:06.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061e16b740 20 client.17472 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061e16b740 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e76831c0 0x55c7e76855e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061e16b740 4 client.17472 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x55c7e7568be0 con 0x55c7e76831c0 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061932d640 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e76831c0 0x55c7e76855e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f05fd7fa640 10 client.17472 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.810+0000 7f061932d640 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e76831c0 0x55c7e76855e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c0f16c7b5070760 server_cookie=9032aef41e2fcdfc in_seq=0 out_seq=0 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.901+0000 7f05fd7fa640 1 -- 172.21.15.37:0/264048435 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c7e7568be0 con 0x55c7e76831c0 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.901+0000 7f05fd7fa640 10 client.17472 handle_command_reply: tid=0 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.901+0000 7f061e16b740 1 client.17472 shutdown 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fc850 msgr2=0x55c7e760ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fc850 0x55c7e760ba90 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x55c7e761a020 tx=0x7f0610010c30 comp rx=0 tx=0).stop 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 shutdown_connections 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c7e76831c0 0x55c7e76855e0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c7e7607a30 0x55c7e76810b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c7e75fd170 0x55c7e760bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 --2- 172.21.15.37:0/264048435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c7e75fc850 0x55c7e760ba90 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f05fd7fa640 0 client.17472 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 >> 172.21.15.37:0/264048435 conn(0x55c7e75f8de0 msgr2=0x55c7e75feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 shutdown_connections 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 1 -- 172.21.15.37:0/264048435 wait complete. 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.902+0000 7f061e16b740 20 client.17472 trim_cache size 0 max 16384 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 36, 2024-09-19T06:49:06.831 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 26, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "d016b263": { 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 26, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 145.8, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:49:06.832 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06000a33b0 msgr2=0x7f06000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06000a33b0 0x7f06000a5870 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0604009fd0 tx=0x7f0604005f90 comp rx=0 tx=0).stop 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 msgr2=0x7f061419ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061419ccc0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f060c031a20 tx=0x7f060c031a50 comp rx=0 tx=0).stop 2024-09-19T06:49:06.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 shutdown_connections 2024-09-19T06:49:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06000a33b0 0x7f06000a5870 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0614106f10 0x7f061419ccc0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0614102790 0x7f0614198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 --2- 172.21.15.37:0/126844768 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0614101de0 0x7f06141983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 >> 172.21.15.37:0/126844768 conn(0x7f06140fd8e0 msgr2=0x7f06140fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 shutdown_connections 2024-09-19T06:49:06.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:05.903+0000 7f061bd92640 1 -- 172.21.15.37:0/126844768 wait complete. 2024-09-19T06:49:07.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.50 MB/sec execute 214 sec latency 1303.474 ms 2024-09-19T06:49:07.487 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd016b263', quiesce took 4.5 seconds. Will release after: 21.25 2024-09-19T06:49:08.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.46 MB/sec execute 215 sec latency 2303.583 ms 2024-09-19T06:49:08.576 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:08 smithi096 ceph-mon[35004]: pgmap v228: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:08.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:08 smithi151 ceph-mon[35236]: pgmap v228: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:08.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:08 smithi037 ceph-mon[31211]: pgmap v228: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 68 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.4 MiB/s wr, 385 op/s 2024-09-19T06:49:09.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.42 MB/sec execute 216 sec latency 3303.638 ms 2024-09-19T06:49:10.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.38 MB/sec execute 217 sec latency 4303.756 ms 2024-09-19T06:49:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:10 smithi151 ceph-mon[35236]: pgmap v229: 97 pgs: 97 active+clean; 71 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-09-19T06:49:10.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:10 smithi151 ceph-mon[35236]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:49:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:10 smithi096 ceph-mon[35004]: pgmap v229: 97 pgs: 97 active+clean; 71 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-09-19T06:49:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:10 smithi096 ceph-mon[35004]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:49:10.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:10 smithi037 ceph-mon[31211]: pgmap v229: 97 pgs: 97 active+clean; 71 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 2.9 MiB/s wr, 376 op/s 2024-09-19T06:49:10.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:10 smithi037 ceph-mon[31211]: osdmap e90: 12 total, 12 up, 12 in 2024-09-19T06:49:11.000 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:49:11.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.34 MB/sec execute 218 sec latency 5303.817 ms 2024-09-19T06:49:11.270 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 -- 172.21.15.37:0/2337456797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 msgr2=0x7f7678103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 --2- 172.21.15.37:0/2337456797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678103120 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f7674009900 tx=0x7f767402a480 comp rx=0 tx=0).stop 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 -- 172.21.15.37:0/2337456797 shutdown_connections 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 --2- 172.21.15.37:0/2337456797 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f767810c170 0x7f767810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 --2- 172.21.15.37:0/2337456797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103660 0x7f767810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 --2- 172.21.15.37:0/2337456797 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678103120 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 -- 172.21.15.37:0/2337456797 >> 172.21.15.37:0/2337456797 conn(0x7f76780fc7a0 msgr2=0x7f76780febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 -- 172.21.15.37:0/2337456797 shutdown_connections 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.702+0000 7f767ff73640 1 -- 172.21.15.37:0/2337456797 wait complete. 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 Processor -- start 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 -- start start 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103660 0x7f76781989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f767810c170 0x7f767819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7678111fb0 con 0x7f7678102d40 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7678111e30 con 0x7f7678103660 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767ff73640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7678112130 con 0x7f767810c170 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767dd0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.703+0000 7f767d50e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103660 0x7f76781989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767dd0f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60888/0 (socket says 172.21.15.37:60888) 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767dd0f640 1 -- 172.21.15.37:0/2827117289 learned_addr learned my addr 172.21.15.37:0/2827117289 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767e510640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f767810c170 0x7f767819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767d50e640 1 -- 172.21.15.37:0/2827117289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f767810c170 msgr2=0x7f767819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767d50e640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f767810c170 0x7f767819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767d50e640 1 -- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 msgr2=0x7f7678198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767d50e640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767d50e640 1 -- 172.21.15.37:0/2827117289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f767819d430 con 0x7f7678103660 2024-09-19T06:49:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767dd0f640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f767d50e640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103660 0x7f76781989c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f766800d680 tx=0x7f766800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.704+0000 7f766effd640 1 -- 172.21.15.37:0/2827117289 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668017070 con 0x7f7678103660 2024-09-19T06:49:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.705+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f767819d720 con 0x7f7678103660 2024-09-19T06:49:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.705+0000 7f766effd640 1 -- 172.21.15.37:0/2827117289 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76680097c0 con 0x7f7678103660 2024-09-19T06:49:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.705+0000 7f766effd640 1 -- 172.21.15.37:0/2827117289 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668004c60 con 0x7f7678103660 2024-09-19T06:49:11.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.705+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76781a5000 con 0x7f7678103660 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.707+0000 7f766effd640 1 -- 172.21.15.37:0/2827117289 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7668009ce0 con 0x7f7678103660 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f766effd640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f76640a3430 0x7f76640a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 Processor -- start 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f767dd0f640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f76640a3430 0x7f76640a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f766effd640 1 -- 172.21.15.37:0/2827117289 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f76680cf1f0 con 0x7f7678103660 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 -- start start 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607411780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c607420550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c607422e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c6073772e0 con 0x55c607420a90 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c60731ece0 con 0x55c607411cc0 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f768234c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c6073732a0 con 0x55c6074113a0 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f767dd0f640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f76640a3430 0x7f76640a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f76740096c0 tx=0x7f7674005a20 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f767e510640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c607422e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f767d50e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c607420550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f767e510640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c607422e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60904/0 (socket says 172.21.15.37:60904) 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.708+0000 7f767e510640 1 -- 172.21.15.37:0/1691183241 learned_addr learned my addr 172.21.15.37:0/1691183241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f767dd0f640 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607411780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f767d50e640 1 -- 172.21.15.37:0/1691183241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 msgr2=0x55c607411780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f767d50e640 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607411780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f767d50e640 1 -- 172.21.15.37:0/1691183241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 msgr2=0x55c607422e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f767d50e640 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c607422e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f767d50e640 1 -- 172.21.15.37:0/1691183241 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c607303440 con 0x55c607411cc0 2024-09-19T06:49:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f767d50e640 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c607420550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f76680cc8b0 tx=0x7f766809b5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4940e1376001d248 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f7663fff640 1 -- 172.21.15.37:0/1691183241 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76680b4070 con 0x55c607411cc0 2024-09-19T06:49:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f7663fff640 1 -- 172.21.15.37:0/1691183241 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76680997f0 con 0x55c607411cc0 2024-09-19T06:49:11.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.709+0000 7f7663fff640 1 -- 172.21.15.37:0/1691183241 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668020040 con 0x55c607411cc0 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 -- 172.21.15.37:0/1691183241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 msgr2=0x55c607420550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c607420550 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f76680cc8b0 tx=0x7f766809b5b0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 -- 172.21.15.37:0/1691183241 shutdown_connections 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c607422e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c607420550 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 --2- 172.21.15.37:0/1691183241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607411780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 -- 172.21.15.37:0/1691183241 >> 172.21.15.37:0/1691183241 conn(0x55c60740da30 msgr2=0x55c60740de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 -- 172.21.15.37:0/1691183241 shutdown_connections 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 -- 172.21.15.37:0/1691183241 wait complete. 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 -- start start 2024-09-19T06:49:11.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.710+0000 7f768234c740 1 Processor -- start 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f768234c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607420490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f768234c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c60741d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f768234c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c60741dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f768234c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c6074236c0 con 0x55c607420a90 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f768234c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c607332050 con 0x55c607411cc0 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f768234c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c607423840 con 0x55c6074113a0 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f768234c740 10 client.0 _ll_register_callbacks cb 0x55c607372160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767e510640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c60741dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c60741d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c60741d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:49050/0 (socket says 172.21.15.37:49050) 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 -- 172.21.15.37:0/2519027518 learned_addr learned my addr 172.21.15.37:0/2519027518 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767e510640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c60741dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60916/0 (socket says 172.21.15.37:60916) 2024-09-19T06:49:11.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767dd0f640 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607420490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 -- 172.21.15.37:0/2519027518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 msgr2=0x55c607420490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607420490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 -- 172.21.15.37:0/2519027518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 msgr2=0x55c60741dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c60741dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.711+0000 7f767d50e640 1 -- 172.21.15.37:0/2519027518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6070e83c0 con 0x55c607411cc0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f767d50e640 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c60741d740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f766809b1f0 tx=0x7f76680a4170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bde096b2d032700 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f76617fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f76617fa640 1 -- 172.21.15.37:0/2519027518 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668020040 con 0x55c607411cc0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f76617fa640 1 -- 172.21.15.37:0/2519027518 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f76680b41e0 con 0x55c607411cc0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76680997f0 con 0x55c607411cc0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f76617fa640 1 -- 172.21.15.37:0/2519027518 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f76680b4040 con 0x55c607411cc0 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 10 client.26879 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:11.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c60742f160 con 0x55c607411cc0 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f76617fa640 1 -- 172.21.15.37:0/2519027518 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f76680997f0 con 0x55c607411cc0 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f76617fa640 1 -- 172.21.15.37:0/2519027518 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f766809a440 con 0x55c607411cc0 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 10 client.26879 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 10 client.26879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 20 client.26879 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c6074304b0 0x55c6074328a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 4 client.26879 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.712+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c607384850 con 0x55c6074304b0 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.713+0000 7f767dd0f640 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c6074304b0 0x55c6074328a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.713+0000 7f767dd0f640 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c6074304b0 0x55c6074328a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27e1e4511f221e9f server_cookie=18660833c3b155b6 in_seq=0 out_seq=0 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.713+0000 7f76617fa640 10 client.26879 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f76617fa640 1 -- 172.21.15.37:0/2519027518 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c607384850 con 0x55c6074304b0 2024-09-19T06:49:11.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f76617fa640 10 client.26879 handle_command_reply: tid=0 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 client.26879 shutdown 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 msgr2=0x55c60741d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c60741d740 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f766809b1f0 tx=0x7f76680a4170 comp rx=0 tx=0).stop 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 shutdown_connections 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c6074304b0 0x55c6074328a0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607420a90 0x55c60741dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607411cc0 0x55c60741d740 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f76617fa640 0 client.26879 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 --2- 172.21.15.37:0/2519027518 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607420490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 >> 172.21.15.37:0/2519027518 conn(0x55c60740da30 msgr2=0x55c607413790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 shutdown_connections 2024-09-19T06:49:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 1 -- 172.21.15.37:0/2519027518 wait complete. 2024-09-19T06:49:11.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.714+0000 7f768234c740 20 client.26879 trim_cache size 0 max 16384 2024-09-19T06:49:11.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.731+0000 7f768234c740 1 Processor -- start 2024-09-19T06:49:11.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.731+0000 7f768234c740 1 -- start start 2024-09-19T06:49:11.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f768234c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6074113a0 0x55c60741aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f768234c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c60741af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.741 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f768234c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c607420a90 0x55c6074184f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f768234c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76781ae1c0 con 0x55c607411cc0 2024-09-19T06:49:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f768234c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76781aba30 con 0x55c6074113a0 2024-09-19T06:49:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f768234c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a98a0 con 0x55c607420a90 2024-09-19T06:49:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c60741af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c60741af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60932/0 (socket says 172.21.15.37:60932) 2024-09-19T06:49:11.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 -- 172.21.15.37:0/914499568 learned_addr learned my addr 172.21.15.37:0/914499568 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767dd0f640 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6074113a0 0x55c60741aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767e510640 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c607420a90 0x55c6074184f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 -- 172.21.15.37:0/914499568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c607420a90 msgr2=0x55c6074184f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c607420a90 0x55c6074184f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 -- 172.21.15.37:0/914499568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6074113a0 msgr2=0x55c60741aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6074113a0 0x55c60741aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 -- 172.21.15.37:0/914499568 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6074231f0 con 0x55c607411cc0 2024-09-19T06:49:11.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f767d50e640 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c60741af40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7668097950 tx=0x7f766809b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba8eb434f3aabc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:11.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f76637fe640 1 -- 172.21.15.37:0/914499568 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668020070 con 0x55c607411cc0 2024-09-19T06:49:11.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f76637fe640 1 -- 172.21.15.37:0/914499568 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7668097040 con 0x55c607411cc0 2024-09-19T06:49:11.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.732+0000 7f76637fe640 1 -- 172.21.15.37:0/914499568 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766809a940 con 0x55c607411cc0 2024-09-19T06:49:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 -- 172.21.15.37:0/914499568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 msgr2=0x55c60741af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c60741af40 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f7668097950 tx=0x7f766809b5b0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 -- 172.21.15.37:0/914499568 shutdown_connections 2024-09-19T06:49:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c607420a90 0x55c6074184f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.745 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c60741af40 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 --2- 172.21.15.37:0/914499568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c6074113a0 0x55c60741aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 -- 172.21.15.37:0/914499568 >> 172.21.15.37:0/914499568 conn(0x55c60740da30 msgr2=0x55c60741c870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 -- 172.21.15.37:0/914499568 shutdown_connections 2024-09-19T06:49:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.733+0000 7f768234c740 1 -- 172.21.15.37:0/914499568 wait complete. 2024-09-19T06:49:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 -- start start 2024-09-19T06:49:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 Processor -- start 2024-09-19T06:49:11.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607435d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c6074330d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607420a90 0x55c607433610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f76781a9ef0 con 0x55c607411cc0 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76781ab360 con 0x55c607420a90 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f76781aa5c0 con 0x55c6074113a0 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f768234c740 10 client.0 _ll_register_callbacks cb 0x7f76680b2070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f767e510640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607420a90 0x55c607433610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f767d50e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c6074330d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.734+0000 7f767dd0f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607435d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767e510640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607420a90 0x55c607433610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:49068/0 (socket says 172.21.15.37:49068) 2024-09-19T06:49:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767e510640 1 -- 172.21.15.37:0/1939155207 learned_addr learned my addr 172.21.15.37:0/1939155207 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767dd0f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607435d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:48334/0 (socket says 172.21.15.37:48334) 2024-09-19T06:49:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767d50e640 1 -- 172.21.15.37:0/1939155207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 msgr2=0x55c607435d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767d50e640 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607435d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767d50e640 1 -- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607420a90 msgr2=0x55c607433610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767d50e640 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607420a90 0x55c607433610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767d50e640 1 -- 172.21.15.37:0/1939155207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c607433b80 con 0x55c607411cc0 2024-09-19T06:49:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767e510640 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607420a90 0x55c607433610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f767d50e640 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c6074330d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f766809e000 tx=0x7f7668009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49afbbca39e805d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f76617fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:11.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f76617fa640 1 -- 172.21.15.37:0/1939155207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7668020040 con 0x55c607411cc0 2024-09-19T06:49:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f76617fa640 1 -- 172.21.15.37:0/1939155207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f76680b43e0 con 0x55c607411cc0 2024-09-19T06:49:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c60737ed60 con 0x55c607411cc0 2024-09-19T06:49:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f76617fa640 1 -- 172.21.15.37:0/1939155207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f766809a5c0 con 0x55c607411cc0 2024-09-19T06:49:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f768234c740 10 client.17490 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.735+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c607433b80 con 0x55c607411cc0 2024-09-19T06:49:11.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f76617fa640 1 -- 172.21.15.37:0/1939155207 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55c60737ed60 con 0x55c607411cc0 2024-09-19T06:49:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f76617fa640 1 -- 172.21.15.37:0/1939155207 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f76680a0980 con 0x55c607411cc0 2024-09-19T06:49:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f768234c740 10 client.17490 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f768234c740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f768234c740 20 client.17490 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f768234c740 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c60749aef0 0x55c60749d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:11.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f768234c740 4 client.17490 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:49:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6072fb210 con 0x55c60749aef0 2024-09-19T06:49:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f767dd0f640 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c60749aef0 0x55c60749d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f767dd0f640 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c60749aef0 0x55c60749d310 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=630663ba64e2af95 server_cookie=24b9e08dbba3fa5f in_seq=0 out_seq=0 2024-09-19T06:49:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f76617fa640 10 client.17490 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f76617fa640 1 -- 172.21.15.37:0/1939155207 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c6072fb210 con 0x55c60749aef0 2024-09-19T06:49:11.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f76617fa640 10 client.17490 handle_command_reply: tid=0 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.736+0000 7f768234c740 1 client.17490 shutdown 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 msgr2=0x55c6074330d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c6074330d0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f766809e000 tx=0x7f7668009c20 comp rx=0 tx=0).stop 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 shutdown_connections 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c60749aef0 0x55c60749d310 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c607420a90 0x55c607433610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c607411cc0 0x55c6074330d0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f76617fa640 0 client.17490 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 --2- 172.21.15.37:0/1939155207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c6074113a0 0x55c607435d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 >> 172.21.15.37:0/1939155207 conn(0x55c60740da30 msgr2=0x55c60740c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:11.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 shutdown_connections 2024-09-19T06:49:11.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 1 -- 172.21.15.37:0/1939155207 wait complete. 2024-09-19T06:49:11.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f768234c740 20 client.17490 trim_cache size 0 max 16384 2024-09-19T06:49:11.754 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:49:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:49:11.754 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:49:11.754 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:49:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f76640a3430 msgr2=0x7f76640a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f76640a3430 0x7f76640a58f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f76740096c0 tx=0x7f7674005a20 comp rx=0 tx=0).stop 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103660 msgr2=0x7f76781989c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103660 0x7f76781989c0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f766800d680 tx=0x7f766800db40 comp rx=0 tx=0).stop 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 shutdown_connections 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f767810c170 0x7f767819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7678103660 0x7f76781989c0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f76640a3430 0x7f76640a58f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.737+0000 7f767ff73640 1 --2- 172.21.15.37:0/2827117289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7678102d40 0x7f7678198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.738+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 >> 172.21.15.37:0/2827117289 conn(0x7f76780fc7a0 msgr2=0x7f76780fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.738+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 shutdown_connections 2024-09-19T06:49:11.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:11.738+0000 7f767ff73640 1 -- 172.21.15.37:0/2827117289 wait complete. 2024-09-19T06:49:12.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.30 MB/sec execute 219 sec latency 6303.920 ms 2024-09-19T06:49:12.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:12 smithi151 ceph-mon[35236]: pgmap v231: 97 pgs: 97 active+clean; 71 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 448 op/s 2024-09-19T06:49:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:12 smithi096 ceph-mon[35004]: pgmap v231: 97 pgs: 97 active+clean; 71 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 448 op/s 2024-09-19T06:49:12.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:12 smithi037 ceph-mon[31211]: pgmap v231: 97 pgs: 97 active+clean; 71 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.5 MiB/s wr, 448 op/s 2024-09-19T06:49:12.945 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:49:12.946 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:13.205 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:13.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.27 MB/sec execute 220 sec latency 7303.960 ms 2024-09-19T06:49:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 -- 172.21.15.37:0/2862394623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8106f30 msgr2=0x7f1ea810eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 --2- 172.21.15.37:0/2862394623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8106f30 0x7f1ea810eab0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f1ea0009ca0 tx=0x7f1ea002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 -- 172.21.15.37:0/2862394623 shutdown_connections 2024-09-19T06:49:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 --2- 172.21.15.37:0/2862394623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8106f30 0x7f1ea810eab0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 --2- 172.21.15.37:0/2862394623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ea8100cc0 0x7f1ea8106800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 --2- 172.21.15.37:0/2862394623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ea8100310 0x7f1ea81006f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 -- 172.21.15.37:0/2862394623 >> 172.21.15.37:0/2862394623 conn(0x7f1ea8077210 msgr2=0x7f1ea80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 -- 172.21.15.37:0/2862394623 shutdown_connections 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.672+0000 7f1eae63e640 1 -- 172.21.15.37:0/2862394623 wait complete. 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 Processor -- start 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 -- start start 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ea8100310 0x7f1ea8101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8100cc0 0x7f1ea81017d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ea8106f30 0x7f1ea8101d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8111f90 con 0x7f1ea8100cc0 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8111e10 con 0x7f1ea8100310 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eae63e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1ea8112110 con 0x7f1ea8106f30 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8100cc0 0x7f1ea81017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:13.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8100cc0 0x7f1ea81017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60956/0 (socket says 172.21.15.37:60956) 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 -- 172.21.15.37:0/583171767 learned_addr learned my addr 172.21.15.37:0/583171767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea7fff640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ea8100310 0x7f1ea8101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1eacbdb640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ea8106f30 0x7f1ea8101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 -- 172.21.15.37:0/583171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ea8106f30 msgr2=0x7f1ea8101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ea8106f30 0x7f1ea8101d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 -- 172.21.15.37:0/583171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ea8100310 msgr2=0x7f1ea8101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ea8100310 0x7f1ea8101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.673+0000 7f1ea77fe640 1 -- 172.21.15.37:0/583171767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ea81a6870 con 0x7f1ea8100cc0 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.674+0000 7f1ea77fe640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8100cc0 0x7f1ea81017d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c920 tx=0x7f1e9c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.674+0000 7f1ea57fa640 1 -- 172.21.15.37:0/583171767 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c007c70 con 0x7f1ea8100cc0 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.674+0000 7f1ea57fa640 1 -- 172.21.15.37:0/583171767 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e9c007e10 con 0x7f1ea8100cc0 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.674+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ea81a6b60 con 0x7f1ea8100cc0 2024-09-19T06:49:13.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.674+0000 7f1ea57fa640 1 -- 172.21.15.37:0/583171767 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e9c018890 con 0x7f1ea8100cc0 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.674+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ea81a70a0 con 0x7f1ea8100cc0 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.675+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ea81006f0 con 0x7f1ea8100cc0 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.676+0000 7f1ea57fa640 1 -- 172.21.15.37:0/583171767 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1e9c018a70 con 0x7f1ea8100cc0 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.677+0000 7f1ea57fa640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e900a33b0 0x7f1e900a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.677+0000 7f1ea7fff640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e900a33b0 0x7f1e900a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.677+0000 7f1ea57fa640 1 -- 172.21.15.37:0/583171767 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f1e9c0cfe20 con 0x7f1ea8100cc0 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.677+0000 7f1ea7fff640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e900a33b0 0x7f1e900a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e980058c0 tx=0x7f1e9800a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:13.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.681+0000 7f1ea57fa640 1 -- 172.21.15.37:0/583171767 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e9c014030 con 0x7f1ea8100cc0 2024-09-19T06:49:13.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.897+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1ea81027c0 con 0x7f1ea8100cc0 2024-09-19T06:49:13.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.898+0000 7f1ea57fa640 1 -- 172.21.15.37:0/583171767 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11907 (secure 0 0 0) 0x7f1e9c09c7d0 con 0x7f1ea8100cc0 2024-09-19T06:49:13.900 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:49:13.900 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"btime":"2024-09-19T06:48:37:250513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:48:36.625010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:49:13.900 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-09-19T06:49:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e900a33b0 msgr2=0x7f1e900a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e900a33b0 0x7f1e900a5870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e980058c0 tx=0x7f1e9800a2d0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8100cc0 msgr2=0x7f1ea81017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8100cc0 0x7f1ea81017d0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c00c920 tx=0x7f1e9c00cde0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 shutdown_connections 2024-09-19T06:49:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1ea8106f30 0x7f1ea8101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e900a33b0 0x7f1e900a5870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1ea8100cc0 0x7f1ea81017d0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 --2- 172.21.15.37:0/583171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1ea8100310 0x7f1ea8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:13.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 >> 172.21.15.37:0/583171767 conn(0x7f1ea8077210 msgr2=0x7f1ea8104660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 shutdown_connections 2024-09-19T06:49:13.904 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:13.900+0000 7f1eae63e640 1 -- 172.21.15.37:0/583171767 wait complete. 2024-09-19T06:49:14.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.23 MB/sec execute 221 sec latency 8304.062 ms 2024-09-19T06:49:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.435+0000 7f1c1d576640 1 -- 172.21.15.37:0/4239965620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c181069c0 msgr2=0x7f1c18106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.435+0000 7f1c1d576640 1 --2- 172.21.15.37:0/4239965620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c181069c0 0x7f1c18106da0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009930 tx=0x7f1c0802a420 comp rx=0 tx=0).stop 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 -- 172.21.15.37:0/4239965620 shutdown_connections 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 --2- 172.21.15.37:0/4239965620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c18101350 0x7f1c1810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 --2- 172.21.15.37:0/4239965620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c181009b0 0x7f1c18100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 --2- 172.21.15.37:0/4239965620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c181069c0 0x7f1c18106da0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 -- 172.21.15.37:0/4239965620 >> 172.21.15.37:0/4239965620 conn(0x7f1c18077990 msgr2=0x7f1c180fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 -- 172.21.15.37:0/4239965620 shutdown_connections 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 -- 172.21.15.37:0/4239965620 wait complete. 2024-09-19T06:49:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 Processor -- start 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.436+0000 7f1c1d576640 1 -- start start 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c181009b0 0x7f1c181991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c18101350 0x7f1c18199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c181069c0 0x7f1c181a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18110e50 con 0x7f1c18101350 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18110cd0 con 0x7f1c181009b0 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c18110fd0 con 0x7f1c181069c0 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c18101350 0x7f1c18199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.440 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c18101350 0x7f1c18199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60960/0 (socket says 172.21.15.37:60960) 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c16ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c181009b0 0x7f1c181991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 -- 172.21.15.37:0/2949660130 learned_addr learned my addr 172.21.15.37:0/2949660130 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 -- 172.21.15.37:0/2949660130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c181069c0 msgr2=0x7f1c181a4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c181069c0 0x7f1c181a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 -- 172.21.15.37:0/2949660130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c181009b0 msgr2=0x7f1c181991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c181009b0 0x7f1c181991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 -- 172.21.15.37:0/2949660130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c181a47a0 con 0x7f1c18101350 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c167fc640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c18101350 0x7f1c18199720 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00d950 tx=0x7f1c0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1bfffff640 1 -- 172.21.15.37:0/2949660130 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c017070 con 0x7f1c18101350 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1bfffff640 1 -- 172.21.15.37:0/2949660130 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0c009c30 con 0x7f1c18101350 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c181a4970 con 0x7f1c18101350 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1bfffff640 1 -- 172.21.15.37:0/2949660130 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c00f6b0 con 0x7f1c18101350 2024-09-19T06:49:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.437+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181a4e90 con 0x7f1c18101350 2024-09-19T06:49:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.440+0000 7f1bfffff640 1 -- 172.21.15.37:0/2949660130 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1c0c0109d0 con 0x7f1c18101350 2024-09-19T06:49:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.440+0000 7f1c203ed740 1 Processor -- start 2024-09-19T06:49:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.440+0000 7f1c203ed740 1 -- start start 2024-09-19T06:49:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82aaf0 0x555cff82aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff835770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff835cb0 0x555cff8380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c203ed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555cff78f2f0 con 0x555cff82aaf0 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c203ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555cff737270 con 0x555cff82b410 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c203ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555cff78b2b0 con 0x555cff835cb0 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82aaf0 0x555cff82aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82aaf0 0x555cff82aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60970/0 (socket says 172.21.15.37:60970) 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 -- 172.21.15.37:0/2381765522 learned_addr learned my addr 172.21.15.37:0/2381765522 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1bfffff640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1c000a3460 0x7f1c000a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c16ffd640 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff835770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c167fc640 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff835cb0 0x555cff8380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 -- 172.21.15.37:0/2381765522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff835cb0 msgr2=0x555cff8380a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff835cb0 0x555cff8380a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 -- 172.21.15.37:0/2381765522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 msgr2=0x555cff835770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff835770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 -- 172.21.15.37:0/2381765522 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cff736340 con 0x555cff82aaf0 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c16ffd640 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff835770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1bfffff640 1 -- 172.21.15.37:0/2949660130 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f1c0c0cfa00 con 0x7f1c18101350 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1c000a3460 0x7f1c000a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.441+0000 7f1c177fe640 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82aaf0 0x555cff82aed0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ec0 tx=0x7f1c1000b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6977ac4cbdb57d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1bfd7fa640 1 -- 172.21.15.37:0/2381765522 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10020640 con 0x555cff82aaf0 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1bfd7fa640 1 -- 172.21.15.37:0/2381765522 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c10020cc0 con 0x555cff82aaf0 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1bfd7fa640 1 -- 172.21.15.37:0/2381765522 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10014ad0 con 0x555cff82aaf0 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 -- 172.21.15.37:0/2381765522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82aaf0 msgr2=0x555cff82aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c177fe640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1c000a3460 0x7f1c000a5920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1c1001d000 tx=0x7f1c1001e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82aaf0 0x555cff82aed0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f1c10009ec0 tx=0x7f1c1000b420 comp rx=0 tx=0).stop 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 -- 172.21.15.37:0/2381765522 shutdown_connections 2024-09-19T06:49:14.450 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff835cb0 0x555cff8380a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff835770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2381765522 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82aaf0 0x555cff82aed0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 -- 172.21.15.37:0/2381765522 >> 172.21.15.37:0/2381765522 conn(0x555cff827060 msgr2=0x555cff827440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 -- 172.21.15.37:0/2381765522 shutdown_connections 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.442+0000 7f1c203ed740 1 -- 172.21.15.37:0/2381765522 wait complete. 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 -- start start 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 Processor -- start 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff832b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff833080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff8335c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555cff838880 con 0x555cff82b410 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555cff718290 con 0x555cff835cb0 2024-09-19T06:49:14.451 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555cff838a00 con 0x555cff82aaf0 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c203ed740 10 client.0 _ll_register_callbacks cb 0x555cff749140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff833080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff833080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60972/0 (socket says 172.21.15.37:60972) 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 -- 172.21.15.37:0/2384135417 learned_addr learned my addr 172.21.15.37:0/2384135417 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c16ffd640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff832b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c177fe640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff8335c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 -- 172.21.15.37:0/2384135417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 msgr2=0x555cff832b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff832b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 -- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 msgr2=0x555cff8335c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff8335c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c167fc640 1 -- 172.21.15.37:0/2384135417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cff4a2210 con 0x555cff82b410 2024-09-19T06:49:14.452 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.443+0000 7f1c177fe640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff8335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c167fc640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff833080 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c098620 tx=0x7f1c0c06d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de4ac9ea871b077 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1bfaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1bfaff5640 1 -- 172.21.15.37:0/2384135417 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c020040 con 0x555cff82b410 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1bfaff5640 1 -- 172.21.15.37:0/2384135417 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f1c0c0cd340 con 0x555cff82b410 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c10020cc0 con 0x555cff82b410 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1bfaff5640 1 -- 172.21.15.37:0/2384135417 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c0b1070 con 0x555cff82b410 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 10 client.17508 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555cff56e620 con 0x555cff82b410 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1bfaff5640 1 -- 172.21.15.37:0/2384135417 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c10020cc0 con 0x555cff82b410 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1bfaff5640 1 -- 172.21.15.37:0/2384135417 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f1c0c0860c0 con 0x555cff82b410 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 10 client.17508 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 10 client.17508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:14.453 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 20 client.17508 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff845c40 0x555cff848030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 4 client.17508 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.444+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cff79ca20 con 0x555cff845c40 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.445+0000 7f1c16ffd640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff845c40 0x555cff848030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.445+0000 7f1c16ffd640 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff845c40 0x555cff848030 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6bc5ce475fbdedf6 server_cookie=560cb289df0998c1 in_seq=0 out_seq=0 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.445+0000 7f1bfaff5640 10 client.17508 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1bfaff5640 1 -- 172.21.15.37:0/2384135417 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555cff79ca20 con 0x555cff845c40 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1bfaff5640 10 client.17508 handle_command_reply: tid=0 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 client.17508 shutdown 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 msgr2=0x555cff833080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff833080 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c098620 tx=0x7f1c0c06d9d0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 shutdown_connections 2024-09-19T06:49:14.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff845c40 0x555cff848030 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff8335c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1bfaff5640 0 client.17508 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff833080 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 --2- 172.21.15.37:0/2384135417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff832b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 >> 172.21.15.37:0/2384135417 conn(0x555cff827060 msgr2=0x555cff82cdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 shutdown_connections 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 1 -- 172.21.15.37:0/2384135417 wait complete. 2024-09-19T06:49:14.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.446+0000 7f1c203ed740 20 client.17508 trim_cache size 0 max 16384 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 Processor -- start 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 -- start start 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff8333f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff84d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff835cb0 0x555cff84de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181b0110 con 0x555cff835cb0 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ad980 con 0x555cff82b410 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.460+0000 7f1c203ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ab7f0 con 0x555cff82aaf0 2024-09-19T06:49:14.463 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff835cb0 0x555cff84de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff835cb0 0x555cff84de40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60978/0 (socket says 172.21.15.37:60978) 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 -- 172.21.15.37:0/3022935981 learned_addr learned my addr 172.21.15.37:0/3022935981 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c167fc640 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff84d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 -- 172.21.15.37:0/3022935981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 msgr2=0x555cff8333f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff8333f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 -- 172.21.15.37:0/3022935981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 msgr2=0x555cff84d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff84d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 -- 172.21.15.37:0/3022935981 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cff826e90 con 0x555cff835cb0 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1c177fe640 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff835cb0 0x555cff84de40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1c100094d0 tx=0x7f1c10009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586bd6bcbe102aee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1bfcff9640 1 -- 172.21.15.37:0/3022935981 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10014ab0 con 0x555cff835cb0 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1bfcff9640 1 -- 172.21.15.37:0/3022935981 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c100148c0 con 0x555cff835cb0 2024-09-19T06:49:14.464 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.461+0000 7f1bfcff9640 1 -- 172.21.15.37:0/3022935981 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c10020ab0 con 0x555cff835cb0 2024-09-19T06:49:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 -- 172.21.15.37:0/3022935981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff835cb0 msgr2=0x555cff84de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.467 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff835cb0 0x555cff84de40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f1c100094d0 tx=0x7f1c10009500 comp rx=0 tx=0).stop 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 -- 172.21.15.37:0/3022935981 shutdown_connections 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff835cb0 0x555cff84de40 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff82b410 0x555cff84d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 --2- 172.21.15.37:0/3022935981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff8333f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 -- 172.21.15.37:0/3022935981 >> 172.21.15.37:0/3022935981 conn(0x555cff827060 msgr2=0x555cff834bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 -- 172.21.15.37:0/3022935981 shutdown_connections 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 -- 172.21.15.37:0/3022935981 wait complete. 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 -- start start 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 Processor -- start 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.462+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff83e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff82f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff82fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.468 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181abe40 con 0x555cff82b410 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ad2b0 con 0x555cff835cb0 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1c181ac510 con 0x555cff82aaf0 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 10 client.0 _ll_register_callbacks cb 0x7f1c0c084f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff82f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff82f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60988/0 (socket says 172.21.15.37:60988) 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 -- 172.21.15.37:0/1030163172 learned_addr learned my addr 172.21.15.37:0/1030163172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c177fe640 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff82fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 -- 172.21.15.37:0/1030163172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 msgr2=0x555cff83e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff83e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 -- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 msgr2=0x555cff82fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff82fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 -- 172.21.15.37:0/1030163172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cff8222d0 con 0x555cff82b410 2024-09-19T06:49:14.469 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c167fc640 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff82f8d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c086460 tx=0x7f1c0c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334b6619e90b896e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1bfaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1bfaff5640 1 -- 172.21.15.37:0/1030163172 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0c020070 con 0x555cff82b410 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1bfaff5640 1 -- 172.21.15.37:0/1030163172 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f1c0c0979a0 con 0x555cff82b410 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c100148c0 con 0x555cff82b410 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1bfaff5640 1 -- 172.21.15.37:0/1030163172 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f1c0c09ba20 con 0x555cff82b410 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 10 client.17514 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.463+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x555cff82d940 con 0x555cff82b410 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1bfaff5640 1 -- 172.21.15.37:0/1030163172 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1c0c0b4e50 con 0x555cff82b410 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1bfaff5640 1 -- 172.21.15.37:0/1030163172 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f1c0c097ba0 con 0x555cff82b410 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c203ed740 10 client.17514 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c203ed740 10 client.17514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c203ed740 20 client.17514 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c203ed740 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff8b0dc0 0x555cff8b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c203ed740 4 client.17514 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:49:14.470 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555cff796d70 con 0x555cff8b0dc0 2024-09-19T06:49:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c16ffd640 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff8b0dc0 0x555cff8b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1bfaff5640 10 client.17514 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:14.471 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.464+0000 7f1c16ffd640 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff8b0dc0 0x555cff8b31e0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=899a95c2e1546c0f server_cookie=dbd268e8b6850053 in_seq=0 out_seq=0 2024-09-19T06:49:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:14 smithi037 ceph-mon[31211]: pgmap v232: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:14.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:14 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/583171767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1bfaff5640 1 -- 172.21.15.37:0/1030163172 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555cff796d70 con 0x555cff8b0dc0 2024-09-19T06:49:14.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1bfaff5640 10 client.17514 handle_command_reply: tid=0 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 client.17514 shutdown 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 msgr2=0x555cff82f8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff82f8d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c086460 tx=0x7f1c0c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 shutdown_connections 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555cff8b0dc0 0x555cff8b31e0 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555cff835cb0 0x555cff82fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555cff82b410 0x555cff82f8d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 --2- 172.21.15.37:0/1030163172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555cff82aaf0 0x555cff83e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 >> 172.21.15.37:0/1030163172 conn(0x555cff827060 msgr2=0x555cff8259a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1bfaff5640 0 client.17514 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 shutdown_connections 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 1 -- 172.21.15.37:0/1030163172 wait complete. 2024-09-19T06:49:14.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.579+0000 7f1c203ed740 20 client.17514 trim_cache size 0 max 16384 2024-09-19T06:49:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:49:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:49:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:49:14.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1c000a3460 msgr2=0x7f1c000a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1c000a3460 0x7f1c000a5920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1c1001d000 tx=0x7f1c1001e000 comp rx=0 tx=0).stop 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c18101350 msgr2=0x7f1c18199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c18101350 0x7f1c18199720 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c00d950 tx=0x7f1c0c00de10 comp rx=0 tx=0).stop 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 shutdown_connections 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c181069c0 0x7f1c181a4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1c000a3460 0x7f1c000a5920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1c18101350 0x7f1c18199720 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 --2- 172.21.15.37:0/2949660130 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c181009b0 0x7f1c181991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:14.584 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 >> 172.21.15.37:0/2949660130 conn(0x7f1c18077990 msgr2=0x7f1c180fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 shutdown_connections 2024-09-19T06:49:14.585 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:14.580+0000 7f1c1d576640 1 -- 172.21.15.37:0/2949660130 wait complete. 2024-09-19T06:49:14.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:14 smithi151 ceph-mon[35236]: pgmap v232: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:14.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:14 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/583171767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:14 smithi096 ceph-mon[35004]: pgmap v232: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:14 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/583171767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:15.043 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:49:15.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.19 MB/sec execute 222 sec latency 9304.098 ms 2024-09-19T06:49:15.301 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:15.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.803+0000 7f2e95caf640 1 -- 172.21.15.37:0/423293767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 msgr2=0x7f2e90100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.803+0000 7f2e95caf640 1 --2- 172.21.15.37:0/423293767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e90100f00 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f2e80009ca0 tx=0x7f2e8002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 -- 172.21.15.37:0/423293767 shutdown_connections 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 --2- 172.21.15.37:0/423293767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e9010e500 0x7f2e90110910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 --2- 172.21.15.37:0/423293767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e90101440 0x7f2e9010df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 --2- 172.21.15.37:0/423293767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e90100f00 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 -- 172.21.15.37:0/423293767 >> 172.21.15.37:0/423293767 conn(0x7f2e900fc7d0 msgr2=0x7f2e900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 -- 172.21.15.37:0/423293767 shutdown_connections 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 -- 172.21.15.37:0/423293767 wait complete. 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 Processor -- start 2024-09-19T06:49:15.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.804+0000 7f2e95caf640 1 -- start start 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e95caf640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e901a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e95caf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e90101440 0x7f2e901a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e95caf640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e9010e500 0x7f2e9019a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e95caf640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e90114260 con 0x7f2e90100b20 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e95caf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901140e0 con 0x7f2e90101440 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e95caf640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901143e0 con 0x7f2e9010e500 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e901a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e901a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32776/0 (socket says 172.21.15.37:32776) 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/2283795831 learned_addr learned my addr 172.21.15.37:0/2283795831 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e9010e500 0x7f2e9019a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8effd640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e90101440 0x7f2e901a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/2283795831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e9010e500 msgr2=0x7f2e9019a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e9010e500 0x7f2e9019a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/2283795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e90101440 msgr2=0x7f2e901a0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e90101440 0x7f2e901a0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/2283795831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e9019ace0 con 0x7f2e90100b20 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.805+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e9010e500 0x7f2e9019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.806+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e901a03f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2e8002ade0 tx=0x7f2e800319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.806+0000 7f2e8cff9640 1 -- 172.21.15.37:0/2283795831 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e8002fa20 con 0x7f2e90100b20 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.806+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e9019af70 con 0x7f2e90100b20 2024-09-19T06:49:15.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.806+0000 7f2e8cff9640 1 -- 172.21.15.37:0/2283795831 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e8002fbc0 con 0x7f2e90100b20 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.806+0000 7f2e8cff9640 1 -- 172.21.15.37:0/2283795831 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800467b0 con 0x7f2e90100b20 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.806+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e901a72c0 con 0x7f2e90100b20 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.808+0000 7f2e8cff9640 1 -- 172.21.15.37:0/2283795831 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2e80046950 con 0x7f2e90100b20 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.809+0000 7f2e8cff9640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2e7c0a33b0 0x7f2e7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.809+0000 7f2e98088740 1 Processor -- start 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.809+0000 7f2e98088740 1 -- start start 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.809+0000 7f2e8effd640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2e7c0a33b0 0x7f2e7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8cff9640 1 -- 172.21.15.37:0/2283795831 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f2e800f11f0 con 0x7f2e90100b20 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e98088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6509090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e98088740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6509a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e98088740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb6516760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e98088740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abb646f2f0 con 0x55abb6514370 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e98088740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55abb6416cf0 con 0x55abb650ab40 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e98088740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abb646b2b0 con 0x55abb65095d0 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb6516760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb6516760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32792/0 (socket says 172.21.15.37:32792) 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 -- 172.21.15.37:0/428074751 learned_addr learned my addr 172.21.15.37:0/428074751 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8effd640 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6509a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8effd640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2e7c0a33b0 0x7f2e7c0a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e780058c0 tx=0x7f2e78005850 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6509090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 -- 172.21.15.37:0/428074751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 msgr2=0x55abb6509a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6509a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 -- 172.21.15.37:0/428074751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 msgr2=0x55abb6509090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6509090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.817 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.810+0000 7f2e8ffff640 1 -- 172.21.15.37:0/428074751 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb63fb3a0 con 0x55abb6514370 2024-09-19T06:49:15.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.811+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb6516760 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800bc40 tx=0x7f2e8800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4eec93efe18724 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.811+0000 7f2e85ffb640 1 -- 172.21.15.37:0/428074751 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88015070 con 0x55abb6514370 2024-09-19T06:49:15.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.811+0000 7f2e85ffb640 1 -- 172.21.15.37:0/428074751 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e88005500 con 0x55abb6514370 2024-09-19T06:49:15.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.811+0000 7f2e85ffb640 1 -- 172.21.15.37:0/428074751 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e880116a0 con 0x55abb6514370 2024-09-19T06:49:15.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.811+0000 7f2e98088740 1 -- 172.21.15.37:0/428074751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 msgr2=0x55abb6516760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb6516760 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800bc40 tx=0x7f2e8800f730 comp rx=0 tx=0).stop 2024-09-19T06:49:15.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 -- 172.21.15.37:0/428074751 shutdown_connections 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb6516760 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6509a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 --2- 172.21.15.37:0/428074751 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6509090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 -- 172.21.15.37:0/428074751 >> 172.21.15.37:0/428074751 conn(0x55abb64d2310 msgr2=0x55abb6505c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 -- 172.21.15.37:0/428074751 shutdown_connections 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 -- 172.21.15.37:0/428074751 wait complete. 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 -- start start 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 Processor -- start 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb6513d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6511000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb6514370 0x55abb65115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55abb6517070 con 0x55abb65095d0 2024-09-19T06:49:15.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.812+0000 7f2e98088740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55abb642a060 con 0x55abb650ab40 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e98088740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55abb65171f0 con 0x55abb6514370 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e98088740 10 client.0 _ll_register_callbacks cb 0x55abb617d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb6513d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb6513d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32800/0 (socket says 172.21.15.37:32800) 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/3429231612 learned_addr learned my addr 172.21.15.37:0/3429231612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb6514370 0x55abb65115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8effd640 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6511000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/3429231612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb6514370 msgr2=0x55abb65115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb6514370 0x55abb65115d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 msgr2=0x55abb6511000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6511000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/3429231612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb61e03c0 con 0x55abb65095d0 2024-09-19T06:49:15.823 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb6513d50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e800b9aa0 tx=0x7f2e800454a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e901cfeb48056b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e6effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e6effd640 1 -- 172.21.15.37:0/3429231612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800d6060 con 0x55abb65095d0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e6effd640 1 -- 172.21.15.37:0/3429231612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f2e80046c40 con 0x55abb65095d0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.813+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e88005500 con 0x55abb65095d0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e6effd640 1 -- 172.21.15.37:0/3429231612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2e80046e40 con 0x55abb65095d0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 10 client.17526 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55abb6517070 con 0x55abb65095d0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e6effd640 1 -- 172.21.15.37:0/3429231612 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e88005500 con 0x55abb65095d0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e6effd640 1 -- 172.21.15.37:0/3429231612 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f2e8004e030 con 0x55abb65095d0 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 10 client.17526 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 10 client.17526 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:49:15.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 20 client.17526 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb6523f80 0x55abb65263b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 4 client.17526 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.814+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abb647c860 con 0x55abb6523f80 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.815+0000 7f2e8effd640 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb6523f80 0x55abb65263b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.815+0000 7f2e6effd640 10 client.17526 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.815+0000 7f2e8effd640 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb6523f80 0x55abb65263b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a1591eef3ebd545 server_cookie=ba2f27c7f2ffa3fb in_seq=0 out_seq=0 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e6effd640 1 -- 172.21.15.37:0/3429231612 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55abb647c860 con 0x55abb6523f80 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e6effd640 10 client.17526 handle_command_reply: tid=0 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 client.17526 shutdown 2024-09-19T06:49:15.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 msgr2=0x55abb6513d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb6513d50 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f2e800b9aa0 tx=0x7f2e800454a0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 shutdown_connections 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb6523f80 0x55abb65263b0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb6514370 0x55abb65115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb6511000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 --2- 172.21.15.37:0/3429231612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb6513d50 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e6effd640 0 client.17526 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 >> 172.21.15.37:0/3429231612 conn(0x55abb64d2310 msgr2=0x55abb650b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 shutdown_connections 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 1 -- 172.21.15.37:0/3429231612 wait complete. 2024-09-19T06:49:15.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.816+0000 7f2e98088740 20 client.17526 trim_cache size 0 max 16384 2024-09-19T06:49:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.829+0000 7f2e98088740 1 Processor -- start 2024-09-19T06:49:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.829+0000 7f2e98088740 1 -- start start 2024-09-19T06:49:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e98088740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6520340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e98088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb65108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e98088740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb652bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e98088740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901b2410 con 0x55abb6514370 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e98088740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901afc80 con 0x55abb650ab40 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e98088740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901adaf0 con 0x55abb65095d0 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb652bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb652bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32816/0 (socket says 172.21.15.37:32816) 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 -- 172.21.15.37:0/2098022847 learned_addr learned my addr 172.21.15.37:0/2098022847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8effd640 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb65108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6520340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 -- 172.21.15.37:0/2098022847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 msgr2=0x55abb6520340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6520340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 -- 172.21.15.37:0/2098022847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 msgr2=0x55abb65108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb65108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 -- 172.21.15.37:0/2098022847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abb650f550 con 0x55abb6514370 2024-09-19T06:49:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.830+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb652bc90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x55abb650a070 tx=0x7f2e8800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6378f37fa5bc5e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e857fa640 1 -- 172.21.15.37:0/2098022847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88029070 con 0x55abb6514370 2024-09-19T06:49:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e857fa640 1 -- 172.21.15.37:0/2098022847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e88004c90 con 0x55abb6514370 2024-09-19T06:49:15.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e857fa640 1 -- 172.21.15.37:0/2098022847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e88015050 con 0x55abb6514370 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e98088740 1 -- 172.21.15.37:0/2098022847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 msgr2=0x55abb652bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e98088740 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb652bc90 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x55abb650a070 tx=0x7f2e8800b8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e98088740 1 -- 172.21.15.37:0/2098022847 shutdown_connections 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e98088740 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb6514370 0x55abb652bc90 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e98088740 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb650ab40 0x55abb65108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e98088740 1 --2- 172.21.15.37:0/2098022847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb65095d0 0x55abb6520340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.831+0000 7f2e98088740 1 -- 172.21.15.37:0/2098022847 >> 172.21.15.37:0/2098022847 conn(0x55abb64d2310 msgr2=0x55abb650f8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.832+0000 7f2e98088740 1 -- 172.21.15.37:0/2098022847 shutdown_connections 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.832+0000 7f2e98088740 1 -- 172.21.15.37:0/2098022847 wait complete. 2024-09-19T06:49:15.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.832+0000 7f2e98088740 1 -- start start 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.832+0000 7f2e98088740 1 Processor -- start 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e98088740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb651cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e98088740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb650ab40 0x55abb650dc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e98088740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb6514370 0x55abb650e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e98088740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901ae140 con 0x55abb65095d0 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e98088740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901af5b0 con 0x55abb6514370 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e98088740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2e901ae810 con 0x55abb650ab40 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e98088740 10 client.0 _ll_register_callbacks cb 0x7f2e800bd500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb651cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb651cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32830/0 (socket says 172.21.15.37:32830) 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/4275078698 learned_addr learned my addr 172.21.15.37:0/4275078698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb6514370 0x55abb650e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8effd640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb650ab40 0x55abb650dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/4275078698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb650ab40 msgr2=0x55abb650dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb650ab40 0x55abb650dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb6514370 msgr2=0x55abb650e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb6514370 0x55abb650e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 -- 172.21.15.37:0/4275078698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abb650e9d0 con 0x55abb65095d0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8ffff640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb6514370 0x55abb650e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e8f7fe640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb651cbc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e800ba010 tx=0x7f2e8002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2816ab4ab0354f74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e6effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.833+0000 7f2e6effd640 1 -- 172.21.15.37:0/4275078698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e800bcfa0 con 0x55abb65095d0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e6effd640 1 -- 172.21.15.37:0/4275078698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f2e80004d50 con 0x55abb65095d0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e88004c90 con 0x55abb65095d0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e6effd640 1 -- 172.21.15.37:0/4275078698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2e800b98e0 con 0x55abb65095d0 2024-09-19T06:49:15.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 10 client.17532 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55abb658ec00 con 0x55abb65095d0 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e6effd640 1 -- 172.21.15.37:0/4275078698 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2e88004c90 con 0x55abb65095d0 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e6effd640 1 -- 172.21.15.37:0/4275078698 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f2e8004e070 con 0x55abb65095d0 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 10 client.17532 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 10 client.17532 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 20 client.17532 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb658f1b0 0x55abb65915d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 4 client.17532 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.834+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55abb6476d70 con 0x55abb658f1b0 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e8effd640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb658f1b0 0x55abb65915d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e8effd640 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb658f1b0 0x55abb65915d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cf0da361d48fc534 server_cookie=40385499e3944957 in_seq=0 out_seq=0 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e6effd640 10 client.17532 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:15.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e6effd640 1 -- 172.21.15.37:0/4275078698 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55abb6476d70 con 0x55abb658f1b0 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e6effd640 10 client.17532 handle_command_reply: tid=0 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 client.17532 shutdown 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 msgr2=0x55abb651cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb651cbc0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f2e800ba010 tx=0x7f2e8002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 shutdown_connections 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55abb658f1b0 0x55abb65915d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55abb6514370 0x55abb650e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55abb650ab40 0x55abb650dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 --2- 172.21.15.37:0/4275078698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55abb65095d0 0x55abb651cbc0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 >> 172.21.15.37:0/4275078698 conn(0x55abb64d2310 msgr2=0x55abb65044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.835+0000 7f2e6effd640 0 client.17532 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 shutdown_connections 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e98088740 1 -- 172.21.15.37:0/4275078698 wait complete. 2024-09-19T06:49:15.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e98088740 20 client.17532 trim_cache size 0 max 16384 2024-09-19T06:49:15.843 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:49:15.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2e7c0a33b0 msgr2=0x7f2e7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2e7c0a33b0 0x7f2e7c0a5870 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2e780058c0 tx=0x7f2e78005850 comp rx=0 tx=0).stop 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 msgr2=0x7f2e901a03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e901a03f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f2e8002ade0 tx=0x7f2e800319f0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 shutdown_connections 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2e9010e500 0x7f2e9019a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e90101440 0x7f2e901a0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2e7c0a33b0 0x7f2e7c0a5870 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 --2- 172.21.15.37:0/2283795831 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2e90100b20 0x7f2e901a03f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 >> 172.21.15.37:0/2283795831 conn(0x7f2e900fc7d0 msgr2=0x7f2e900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.836+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 shutdown_connections 2024-09-19T06:49:15.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:15.837+0000 7f2e95caf640 1 -- 172.21.15.37:0/2283795831 wait complete. 2024-09-19T06:49:16.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.15 MB/sec execute 223 sec latency 10304.195 ms 2024-09-19T06:49:16.581 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:49:16.607 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:16 smithi037 ceph-mon[31211]: pgmap v233: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:16.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:16 smithi151 ceph-mon[35236]: pgmap v233: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:16.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:16 smithi096 ceph-mon[35004]: pgmap v233: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:16.845 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:17.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.12 MB/sec execute 224 sec latency 11304.302 ms 2024-09-19T06:49:17.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 -- 172.21.15.37:0/3450380102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c102d40 msgr2=0x7f8e3c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 --2- 172.21.15.37:0/3450380102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c102d40 0x7f8e3c103120 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009900 tx=0x7f8e2c02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 -- 172.21.15.37:0/3450380102 shutdown_connections 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 --2- 172.21.15.37:0/3450380102 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3c10c170 0x7f8e3c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 --2- 172.21.15.37:0/3450380102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e3c103660 0x7f8e3c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 --2- 172.21.15.37:0/3450380102 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c102d40 0x7f8e3c103120 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 -- 172.21.15.37:0/3450380102 >> 172.21.15.37:0/3450380102 conn(0x7f8e3c0fc7a0 msgr2=0x7f8e3c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 -- 172.21.15.37:0/3450380102 shutdown_connections 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.312+0000 7f8e427df640 1 -- 172.21.15.37:0/3450380102 wait complete. 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 Processor -- start 2024-09-19T06:49:17.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 -- start start 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e3c102d40 0x7f8e3c198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c103660 0x7f8e3c198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3c10c170 0x7f8e3c19cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c111f70 con 0x7f8e3c103660 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c111df0 con 0x7f8e3c102d40 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e427df640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1120f0 con 0x7f8e3c10c170 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.313+0000 7f8e3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c103660 0x7f8e3c198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e3c102d40 0x7f8e3c198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c103660 0x7f8e3c198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32852/0 (socket says 172.21.15.37:32852) 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3374237554 learned_addr learned my addr 172.21.15.37:0/3374237554 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3c10c170 0x7f8e3c19cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3374237554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3c10c170 msgr2=0x7f8e3c19cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3c10c170 0x7f8e3c19cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3374237554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e3c102d40 msgr2=0x7f8e3c198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e3c102d40 0x7f8e3c198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3374237554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e3c19d400 con 0x7f8e3c103660 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3c10c170 0x7f8e3c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c103660 0x7f8e3c198990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000d740 tx=0x7f8e3000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e397fa640 1 -- 172.21.15.37:0/3374237554 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30017070 con 0x7f8e3c103660 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e3c19d6f0 con 0x7f8e3c103660 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e397fa640 1 -- 172.21.15.37:0/3374237554 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e3000de50 con 0x7f8e3c103660 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e397fa640 1 -- 172.21.15.37:0/3374237554 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3000f620 con 0x7f8e3c103660 2024-09-19T06:49:17.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.314+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e3c1008d0 con 0x7f8e3c103660 2024-09-19T06:49:17.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e397fa640 1 -- 172.21.15.37:0/3374237554 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8e30009c30 con 0x7f8e3c103660 2024-09-19T06:49:17.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 Processor -- start 2024-09-19T06:49:17.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 -- start start 2024-09-19T06:49:17.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e397fa640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8e240a33b0 0x7f8e240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.320 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb46090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb46a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb53760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55772caac2f0 con 0x55772cb51370 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55772ca53cf0 con 0x55772cb47b40 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.317+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55772caa82b0 con 0x55772cb465d0 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e397fa640 1 -- 172.21.15.37:0/3374237554 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f8e300cf9b0 con 0x7f8e3c103660 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb46090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb46090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:49190/0 (socket says 172.21.15.37:49190) 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2282645760 learned_addr learned my addr 172.21.15.37:0/2282645760 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb53760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8e240a33b0 0x7f8e240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb46a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.321 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2282645760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 msgr2=0x55772cb46a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb46a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2282645760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 msgr2=0x55772cb53760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb53760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2282645760 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55772ca383a0 con 0x55772cb47b40 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb46090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009900 tx=0x7f8e2c005e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ed2ebb7379bf82 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.318+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8e240a33b0 0x7f8e240a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8e34005c40 tx=0x7f8e34009af0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e227fc640 1 -- 172.21.15.37:0/2282645760 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c040070 con 0x55772cb47b40 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e227fc640 1 -- 172.21.15.37:0/2282645760 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c031830 con 0x55772cb47b40 2024-09-19T06:49:17.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e227fc640 1 -- 172.21.15.37:0/2282645760 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c035040 con 0x55772cb47b40 2024-09-19T06:49:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2282645760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 msgr2=0x55772cb46090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb46090 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009900 tx=0x7f8e2c005e00 comp rx=0 tx=0).stop 2024-09-19T06:49:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2282645760 shutdown_connections 2024-09-19T06:49:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb53760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb46a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.319+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2282645760 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb46090 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2282645760 >> 172.21.15.37:0/2282645760 conn(0x55772cb0f310 msgr2=0x55772cb42c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2282645760 shutdown_connections 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2282645760 wait complete. 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 -- start start 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 Processor -- start 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb50d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb4dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb4e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55772cb54010 con 0x55772cb51370 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55772ca67060 con 0x55772cb47b40 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55772cb54190 con 0x55772cb465d0 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.320+0000 7f8e44bb8740 10 client.0 _ll_register_callbacks cb 0x55772c7ba8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e40d7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb4e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb4dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb4dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:49204/0 (socket says 172.21.15.37:49204) 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3072438882 learned_addr learned my addr 172.21.15.37:0/3072438882 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e40d7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb4e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32868/0 (socket says 172.21.15.37:32868) 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb50d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3072438882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 msgr2=0x55772cb50d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb50d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3072438882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 msgr2=0x55772cb4e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb4e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 -- 172.21.15.37:0/3072438882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55772c81d3c0 con 0x55772cb47b40 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb4e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb4dff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e300058c0 tx=0x7f8e3000b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d0f049795dd96f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/3072438882 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e30020070 con 0x55772cb47b40 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e2c031830 con 0x55772cb47b40 2024-09-19T06:49:17.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.321+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/3072438882 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f8e300860b0 con 0x55772cb47b40 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/3072438882 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8e300a4870 con 0x55772cb47b40 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 10 client.26933 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55772cb5fb30 con 0x55772cb47b40 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/3072438882 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e2c031830 con 0x55772cb47b40 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/3072438882 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f8e3009aac0 con 0x55772cb47b40 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 10 client.26933 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 10 client.26933 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 20 client.26933 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cb60f40 0x55772cb63380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 4 client.26933 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55772cab9860 con 0x55772cb60f40 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.322+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cb60f40 0x55772cb63380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.323+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cb60f40 0x55772cb63380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c0d9f02944e7bd28 server_cookie=50e72fc0b044a8ba in_seq=0 out_seq=0 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.323+0000 7f8e1b7fe640 10 client.26933 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.323+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/3072438882 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55772cab9860 con 0x55772cb60f40 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.323+0000 7f8e1b7fe640 10 client.26933 handle_command_reply: tid=0 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.323+0000 7f8e44bb8740 1 client.26933 shutdown 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 msgr2=0x55772cb4dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb4dff0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f8e300058c0 tx=0x7f8e3000b480 comp rx=0 tx=0).stop 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 shutdown_connections 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cb60f40 0x55772cb63380 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb4e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e1b7fe640 0 client.26933 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb4dff0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3072438882 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb465d0 0x55772cb50d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 >> 172.21.15.37:0/3072438882 conn(0x55772cb0f310 msgr2=0x55772cb48960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 shutdown_connections 2024-09-19T06:49:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3072438882 wait complete. 2024-09-19T06:49:17.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.324+0000 7f8e44bb8740 20 client.26933 trim_cache size 0 max 16384 2024-09-19T06:49:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 Processor -- start 2024-09-19T06:49:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 -- start start 2024-09-19T06:49:17.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb465d0 0x55772cb5d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb47b40 0x55772cb4d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb68c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1b25b0 con 0x55772cb51370 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1afe20 con 0x55772cb465d0 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1adc90 con 0x55772cb47b40 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e40d7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb68c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e40d7c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb68c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32876/0 (socket says 172.21.15.37:32876) 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e40d7c640 1 -- 172.21.15.37:0/3966288558 learned_addr learned my addr 172.21.15.37:0/3966288558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.337+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb465d0 0x55772cb5d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb47b40 0x55772cb4d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e40d7c640 1 -- 172.21.15.37:0/3966288558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb47b40 msgr2=0x55772cb4d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb47b40 0x55772cb4d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e40d7c640 1 -- 172.21.15.37:0/3966288558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb465d0 msgr2=0x55772cb5d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb465d0 0x55772cb5d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e40d7c640 1 -- 172.21.15.37:0/3966288558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55772cb49120 con 0x55772cb51370 2024-09-19T06:49:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb465d0 0x55772cb5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:17.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb68c50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009e20 tx=0x7f8e3401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcb296bf6ec76732 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.338+0000 7f8e21ffb640 1 -- 172.21.15.37:0/3966288558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402f070 con 0x55772cb51370 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e21ffb640 1 -- 172.21.15.37:0/3966288558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e34021df0 con 0x55772cb51370 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e21ffb640 1 -- 172.21.15.37:0/3966288558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3402ac20 con 0x55772cb51370 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3966288558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 msgr2=0x55772cb68c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb68c50 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f8e34009e20 tx=0x7f8e3401f000 comp rx=0 tx=0).stop 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3966288558 shutdown_connections 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb51370 0x55772cb68c50 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb47b40 0x55772cb4d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/3966288558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb465d0 0x55772cb5d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3966288558 >> 172.21.15.37:0/3966288558 conn(0x55772cb0f310 msgr2=0x55772cb4c8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3966288558 shutdown_connections 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.339+0000 7f8e44bb8740 1 -- 172.21.15.37:0/3966288558 wait complete. 2024-09-19T06:49:17.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 -- start start 2024-09-19T06:49:17.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 Processor -- start 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb465d0 0x55772cb59810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb59d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb51370 0x55772cb54d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1ae2e0 con 0x55772cb465d0 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1af750 con 0x55772cb47b40 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8e3c1ae9b0 con 0x55772cb51370 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.340+0000 7f8e44bb8740 10 client.0 _ll_register_callbacks cb 0x7f8e3009b560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb465d0 0x55772cb59810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb465d0 0x55772cb59810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32880/0 (socket says 172.21.15.37:32880) 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2593278123 learned_addr learned my addr 172.21.15.37:0/2593278123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:17.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb59d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e40d7c640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb51370 0x55772cb54d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2593278123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb51370 msgr2=0x55772cb54d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb51370 0x55772cb54d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 msgr2=0x55772cb59d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb59d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 -- 172.21.15.37:0/2593278123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55772cb555c0 con 0x55772cb465d0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb59d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e3bfff640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb465d0 0x55772cb59810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0039a0 tx=0x7f8e2c031b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ecd20597edae045 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e1b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/2593278123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c054070 con 0x55772cb465d0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/2593278123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f8e2c005240 con 0x55772cb465d0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e34021df0 con 0x55772cb465d0 2024-09-19T06:49:17.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/2593278123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8e2c05d050 con 0x55772cb465d0 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e44bb8740 10 client.17550 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.341+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55772cbcb8d0 con 0x55772cb465d0 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/2593278123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e34021df0 con 0x55772cb465d0 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/2593278123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f8e2c057020 con 0x55772cb465d0 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e44bb8740 10 client.17550 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e44bb8740 10 client.17550 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e44bb8740 20 client.17550 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cbcbcd0 0x55772cbce0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e44bb8740 4 client.17550 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55772cab3d70 con 0x55772cbcbcd0 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.342+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cbcbcd0 0x55772cbce0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e3b7fe640 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cbcbcd0 0x55772cbce0c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=449fc6a918d934d7 server_cookie=f939256f48b0b282 in_seq=0 out_seq=0 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e1b7fe640 10 client.17550 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:49:17.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e1b7fe640 1 -- 172.21.15.37:0/2593278123 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55772cab3d70 con 0x55772cbcbcd0 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e1b7fe640 10 client.17550 handle_command_reply: tid=0 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 client.17550 shutdown 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb465d0 msgr2=0x55772cb59810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb465d0 0x55772cb59810 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0039a0 tx=0x7f8e2c031b00 comp rx=0 tx=0).stop 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 shutdown_connections 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55772cbcbcd0 0x55772cbce0c0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e1b7fe640 0 client.17550 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55772cb51370 0x55772cb54d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55772cb47b40 0x55772cb59d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 --2- 172.21.15.37:0/2593278123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55772cb465d0 0x55772cb59810 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 >> 172.21.15.37:0/2593278123 conn(0x55772cb0f310 msgr2=0x55772cb4f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 shutdown_connections 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 1 -- 172.21.15.37:0/2593278123 wait complete. 2024-09-19T06:49:17.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.343+0000 7f8e44bb8740 20 client.17550 trim_cache size 0 max 16384 2024-09-19T06:49:17.353 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:49:17.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8e240a33b0 msgr2=0x7f8e240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8e240a33b0 0x7f8e240a5870 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8e34005c40 tx=0x7f8e34009af0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c103660 msgr2=0x7f8e3c198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c103660 0x7f8e3c198990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f8e3000d740 tx=0x7f8e3000dc00 comp rx=0 tx=0).stop 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 shutdown_connections 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8e3c10c170 0x7f8e3c19cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8e240a33b0 0x7f8e240a5870 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8e3c103660 0x7f8e3c198990 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 --2- 172.21.15.37:0/3374237554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e3c102d40 0x7f8e3c198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 >> 172.21.15.37:0/3374237554 conn(0x7f8e3c0fc7a0 msgr2=0x7f8e3c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 shutdown_connections 2024-09-19T06:49:17.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:17.344+0000 7f8e427df640 1 -- 172.21.15.37:0/3374237554 wait complete. 2024-09-19T06:49:18.005 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:49:18.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.08 MB/sec execute 225 sec latency 12304.343 ms 2024-09-19T06:49:18.303 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:18.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:18 smithi151 ceph-mon[35236]: pgmap v234: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:18.680 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:18 smithi037 ceph-mon[31211]: pgmap v234: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:18.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 -- 172.21.15.37:0/1558613183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 msgr2=0x7f8570100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 --2- 172.21.15.37:0/1558613183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f8570100cb0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f8560009900 tx=0x7f856002a410 comp rx=0 tx=0).stop 2024-09-19T06:49:18.764 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 -- 172.21.15.37:0/1558613183 shutdown_connections 2024-09-19T06:49:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 --2- 172.21.15.37:0/1558613183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f857010e480 0x7f8570110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 --2- 172.21.15.37:0/1558613183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8570101280 0x7f857010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 --2- 172.21.15.37:0/1558613183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f8570100cb0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 -- 172.21.15.37:0/1558613183 >> 172.21.15.37:0/1558613183 conn(0x7f85700fc7a0 msgr2=0x7f85700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 -- 172.21.15.37:0/1558613183 shutdown_connections 2024-09-19T06:49:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.762+0000 7f85761ee640 1 -- 172.21.15.37:0/1558613183 wait complete. 2024-09-19T06:49:18.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 Processor -- start 2024-09-19T06:49:18.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 -- start start 2024-09-19T06:49:18.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f857019a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8570101280 0x7f857019ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f857010e480 0x7f857019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8570114200 con 0x7f85701008d0 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8570114080 con 0x7f857010e480 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.763+0000 7f85761ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8570114380 con 0x7f8570101280 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f857019a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f857019a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32902/0 (socket says 172.21.15.37:32902) 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 -- 172.21.15.37:0/1732870358 learned_addr learned my addr 172.21.15.37:0/1732870358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856effd640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8570101280 0x7f857019ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856ffff640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f857010e480 0x7f857019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 -- 172.21.15.37:0/1732870358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8570101280 msgr2=0x7f857019ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8570101280 0x7f857019ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 -- 172.21.15.37:0/1732870358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f857010e480 msgr2=0x7f857019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f857010e480 0x7f857019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 -- 172.21.15.37:0/1732870358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f857019f690 con 0x7f85701008d0 2024-09-19T06:49:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856f7fe640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f857019a6e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8560004770 tx=0x7f8560003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856cff9640 1 -- 172.21.15.37:0/1732870358 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856003f070 con 0x7f85701008d0 2024-09-19T06:49:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f857019f920 con 0x7f85701008d0 2024-09-19T06:49:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856cff9640 1 -- 172.21.15.37:0/1732870358 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8560031ca0 con 0x7f85701008d0 2024-09-19T06:49:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f856cff9640 1 -- 172.21.15.37:0/1732870358 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560035030 con 0x7f85701008d0 2024-09-19T06:49:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.764+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85701a7260 con 0x7f85701008d0 2024-09-19T06:49:18.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f856cff9640 1 -- 172.21.15.37:0/1732870358 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8560042070 con 0x7f85701008d0 2024-09-19T06:49:18.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 Processor -- start 2024-09-19T06:49:18.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 -- start start 2024-09-19T06:49:18.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345cf5d0 0x5652345cfa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345dc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5652345352f0 con 0x5652345da370 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5652344dccf0 con 0x5652345cf5d0 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.767+0000 7f85785c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5652345312b0 con 0x5652345d0b40 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856cff9640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f85580a3430 0x7f85580a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345dc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345dc760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32908/0 (socket says 172.21.15.37:32908) 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 -- 172.21.15.37:0/4110110528 learned_addr learned my addr 172.21.15.37:0/4110110528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856ffff640 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856ffff640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f85580a3430 0x7f85580a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856cff9640 1 -- 172.21.15.37:0/1732870358 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f85600f0450 con 0x7f85701008d0 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856f7fe640 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345cf5d0 0x5652345cfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 -- 172.21.15.37:0/4110110528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 msgr2=0x5652345cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 -- 172.21.15.37:0/4110110528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345cf5d0 msgr2=0x5652345cfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345cf5d0 0x5652345cfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 -- 172.21.15.37:0/4110110528 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652344c13a0 con 0x5652345da370 2024-09-19T06:49:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856effd640 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345dc760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f856400abe0 tx=0x7f856400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=510f55e8c7666a19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.768+0000 7f856ffff640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f85580a3430 0x7f85580a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8568009fd0 tx=0x7f8568009600 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f8555ffb640 1 -- 172.21.15.37:0/4110110528 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f856401d070 con 0x5652345da370 2024-09-19T06:49:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f8555ffb640 1 -- 172.21.15.37:0/4110110528 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8564018b20 con 0x5652345da370 2024-09-19T06:49:18.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f8555ffb640 1 -- 172.21.15.37:0/4110110528 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8564010a00 con 0x5652345da370 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 -- 172.21.15.37:0/4110110528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 msgr2=0x5652345dc760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345dc760 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f856400abe0 tx=0x7f856400f790 comp rx=0 tx=0).stop 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 -- 172.21.15.37:0/4110110528 shutdown_connections 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345dc760 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345cf5d0 0x5652345cfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 --2- 172.21.15.37:0/4110110528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 -- 172.21.15.37:0/4110110528 >> 172.21.15.37:0/4110110528 conn(0x565234598310 msgr2=0x5652345cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 -- 172.21.15.37:0/4110110528 shutdown_connections 2024-09-19T06:49:18.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.769+0000 7f85785c7740 1 -- 172.21.15.37:0/4110110528 wait complete. 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 -- start start 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 Processor -- start 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345d9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345d6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 0x5652345d7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5652345dd070 con 0x5652345cf5d0 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5652344f0060 con 0x5652345da370 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5652345dd1f0 con 0x5652345d0b40 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f85785c7740 10 client.0 _ll_register_callbacks cb 0x5652342438c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345d9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345d9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32918/0 (socket says 172.21.15.37:32918) 2024-09-19T06:49:18.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.770+0000 7f856f7fe640 1 -- 172.21.15.37:0/3649378466 learned_addr learned my addr 172.21.15.37:0/3649378466 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856effd640 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345d6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856ffff640 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 0x5652345d7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856f7fe640 1 -- 172.21.15.37:0/3649378466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 msgr2=0x5652345d6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856f7fe640 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345d6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856f7fe640 1 -- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 msgr2=0x5652345d7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856f7fe640 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 0x5652345d7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856f7fe640 1 -- 172.21.15.37:0/3649378466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652342a63c0 con 0x5652345cf5d0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f856f7fe640 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345d9d20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f85600bc450 tx=0x7f85600bc5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea52d57e1f34f2ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f854effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f854effd640 1 -- 172.21.15.37:0/3649378466 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8560049040 con 0x5652345cf5d0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f854effd640 1 -- 172.21.15.37:0/3649378466 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f85600edd90 con 0x5652345cf5d0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8564018b20 con 0x5652345cf5d0 2024-09-19T06:49:18.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f854effd640 1 -- 172.21.15.37:0/3649378466 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8560031e50 con 0x5652345cf5d0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f85785c7740 10 client.17562 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.771+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5652345e8ac0 con 0x5652345cf5d0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f854effd640 1 -- 172.21.15.37:0/3649378466 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8564018b20 con 0x5652345cf5d0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f854effd640 1 -- 172.21.15.37:0/3649378466 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f85600bc9b0 con 0x5652345cf5d0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f85785c7740 10 client.17562 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f85785c7740 10 client.17562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f85785c7740 20 client.17562 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f85785c7740 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652345e9eb0 0x5652345ec300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f85785c7740 4 client.17562 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565234542860 con 0x5652345e9eb0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.772+0000 7f856effd640 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652345e9eb0 0x5652345ec300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.773+0000 7f854effd640 10 client.17562 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.773+0000 7f856effd640 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652345e9eb0 0x5652345ec300 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf68a041de545ac server_cookie=5c1618186cb8e5a3 in_seq=0 out_seq=0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.773+0000 7f854effd640 1 -- 172.21.15.37:0/3649378466 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x565234542860 con 0x5652345e9eb0 2024-09-19T06:49:18.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.773+0000 7f854effd640 10 client.17562 handle_command_reply: tid=0 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.773+0000 7f85785c7740 1 client.17562 shutdown 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 msgr2=0x5652345d9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345d9d20 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f85600bc450 tx=0x7f85600bc5e0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 shutdown_connections 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652345e9eb0 0x5652345ec300 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 0x5652345d7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345d6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 --2- 172.21.15.37:0/3649378466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345d9d20 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 >> 172.21.15.37:0/3649378466 conn(0x565234598310 msgr2=0x5652345d1960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f854effd640 0 client.17562 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 shutdown_connections 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 1 -- 172.21.15.37:0/3649378466 wait complete. 2024-09-19T06:49:18.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.774+0000 7f85785c7740 20 client.17562 trim_cache size 0 max 16384 2024-09-19T06:49:18.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:18 smithi096 ceph-mon[35004]: pgmap v234: 97 pgs: 97 active+clean; 72 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.0 MiB/s wr, 137 op/s 2024-09-19T06:49:18.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 Processor -- start 2024-09-19T06:49:18.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 -- start start 2024-09-19T06:49:18.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345e6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345d8f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 0x5652345d9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f85701b24c0 con 0x5652345cf5d0 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85701afd30 con 0x5652345da370 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f85785c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85701adba0 con 0x5652345d0b40 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345e6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32924/0 (socket says 172.21.15.37:32924) 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 -- 172.21.15.37:0/1593089933 learned_addr learned my addr 172.21.15.37:0/1593089933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 -- 172.21.15.37:0/1593089933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 msgr2=0x5652345d8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 --2- 172.21.15.37:0/1593089933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345d8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 -- 172.21.15.37:0/1593089933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 msgr2=0x5652345d9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 --2- 172.21.15.37:0/1593089933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 0x5652345d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.787+0000 7f856f7fe640 1 -- 172.21.15.37:0/1593089933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652345cc960 con 0x5652345cf5d0 2024-09-19T06:49:18.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f856f7fe640 1 --2- 172.21.15.37:0/1593089933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345e6310 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f85600c4de0 tx=0x7f85600bc650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a038b10352fae8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85557fa640 1 -- 172.21.15.37:0/1593089933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600bc750 con 0x5652345cf5d0 2024-09-19T06:49:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85557fa640 1 -- 172.21.15.37:0/1593089933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8560031e50 con 0x5652345cf5d0 2024-09-19T06:49:18.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85557fa640 1 -- 172.21.15.37:0/1593089933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85600bbba0 con 0x5652345cf5d0 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 -- 172.21.15.37:0/1593089933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 msgr2=0x5652345e6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 --2- 172.21.15.37:0/1593089933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345e6310 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f85600c4de0 tx=0x7f85600bc650 comp rx=0 tx=0).stop 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 -- 172.21.15.37:0/1593089933 shutdown_connections 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 --2- 172.21.15.37:0/1593089933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345da370 0x5652345d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 --2- 172.21.15.37:0/1593089933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345d0b40 0x5652345d8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 --2- 172.21.15.37:0/1593089933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345cf5d0 0x5652345e6310 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 -- 172.21.15.37:0/1593089933 >> 172.21.15.37:0/1593089933 conn(0x565234598310 msgr2=0x5652345d8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.788+0000 7f85785c7740 1 -- 172.21.15.37:0/1593089933 shutdown_connections 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 -- 172.21.15.37:0/1593089933 wait complete. 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 -- start start 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 Processor -- start 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345cf5d0 0x5652345e2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345d0b40 0x5652345df450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345df990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f85701ae1f0 con 0x5652345da370 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f85701af660 con 0x5652345d0b40 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f85701ae8c0 con 0x5652345cf5d0 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f85785c7740 10 client.0 _ll_register_callbacks cb 0x7f85600b8ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f856effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345d0b40 0x5652345df450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f856ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345df990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f856effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345d0b40 0x5652345df450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:49260/0 (socket says 172.21.15.37:49260) 2024-09-19T06:49:18.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.789+0000 7f856ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345df990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:32940/0 (socket says 172.21.15.37:32940) 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856effd640 1 -- 172.21.15.37:0/3433650807 learned_addr learned my addr 172.21.15.37:0/3433650807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856f7fe640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345cf5d0 0x5652345e2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856ffff640 1 -- 172.21.15.37:0/3433650807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345cf5d0 msgr2=0x5652345e2900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856ffff640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345cf5d0 0x5652345e2900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856ffff640 1 -- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345d0b40 msgr2=0x5652345df450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856ffff640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345d0b40 0x5652345df450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856ffff640 1 -- 172.21.15.37:0/3433650807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652345dff00 con 0x5652345da370 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856f7fe640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345cf5d0 0x5652345e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856effd640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345d0b40 0x5652345df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f856ffff640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345df990 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x5652345e6fb0 tx=0x7f8568004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee10d210ec5f323f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f854effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f854effd640 1 -- 172.21.15.37:0/3433650807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8568004b20 con 0x5652345da370 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8560031e50 con 0x5652345da370 2024-09-19T06:49:18.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f854effd640 1 -- 172.21.15.37:0/3433650807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f856801b070 con 0x5652345da370 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f854effd640 1 -- 172.21.15.37:0/3433650807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f8568004cc0 con 0x5652345da370 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f85785c7740 10 client.17568 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.790+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x565234655980 con 0x5652345da370 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f854effd640 1 -- 172.21.15.37:0/3433650807 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8560031e50 con 0x5652345da370 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f854effd640 1 -- 172.21.15.37:0/3433650807 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f8568027070 con 0x5652345da370 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f85785c7740 10 client.17568 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f85785c7740 10 client.17568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f85785c7740 20 client.17568 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f85785c7740 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652346561e0 0x565234658600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f85785c7740 4 client.17568 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56523453cd70 con 0x5652346561e0 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f856f7fe640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652346561e0 0x565234658600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:18.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f856f7fe640 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652346561e0 0x565234658600 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ecc7dddcb85bfe3 server_cookie=1772727a57aa209d in_seq=0 out_seq=0 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.791+0000 7f854effd640 10 client.17568 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f854effd640 1 -- 172.21.15.37:0/3433650807 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56523453cd70 con 0x5652346561e0 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f854effd640 10 client.17568 handle_command_reply: tid=0 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 client.17568 shutdown 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 msgr2=0x5652345df990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345df990 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x5652345e6fb0 tx=0x7f8568004a40 comp rx=0 tx=0).stop 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 shutdown_connections 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652346561e0 0x565234658600 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652345da370 0x5652345df990 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652345d0b40 0x5652345df450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 --2- 172.21.15.37:0/3433650807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652345cf5d0 0x5652345e2900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 >> 172.21.15.37:0/3433650807 conn(0x565234598310 msgr2=0x5652345d8830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f854effd640 0 client.17568 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:18.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 shutdown_connections 2024-09-19T06:49:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 1 -- 172.21.15.37:0/3433650807 wait complete. 2024-09-19T06:49:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.792+0000 7f85785c7740 20 client.17568 trim_cache size 0 max 16384 2024-09-19T06:49:18.799 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:49:18.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f85580a3430 msgr2=0x7f85580a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f85580a3430 0x7f85580a58f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8568009fd0 tx=0x7f8568009600 comp rx=0 tx=0).stop 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 msgr2=0x7f857019a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f857019a6e0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f8560004770 tx=0x7f8560003c10 comp rx=0 tx=0).stop 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 shutdown_connections 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f857010e480 0x7f857019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8570101280 0x7f857019ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f85580a3430 0x7f85580a58f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 --2- 172.21.15.37:0/1732870358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f85701008d0 0x7f857019a6e0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 >> 172.21.15.37:0/1732870358 conn(0x7f85700fc7a0 msgr2=0x7f85700feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 shutdown_connections 2024-09-19T06:49:18.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:18.793+0000 7f85761ee640 1 -- 172.21.15.37:0/1732870358 wait complete. 2024-09-19T06:49:19.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.05 MB/sec execute 226 sec latency 13304.376 ms 2024-09-19T06:49:20.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 8.01 MB/sec execute 227 sec latency 14304.482 ms 2024-09-19T06:49:20.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:20 smithi151 ceph-mon[35236]: pgmap v235: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-19T06:49:20.615 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:49:20.615 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:49:20.643 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:20 smithi037 ceph-mon[31211]: pgmap v235: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-19T06:49:20.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:20 smithi096 ceph-mon[35004]: pgmap v235: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.6 MiB/s wr, 141 op/s 2024-09-19T06:49:20.882 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:21.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.98 MB/sec execute 228 sec latency 15304.547 ms 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 -- 172.21.15.37:0/175681197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568101420 msgr2=0x7f556810d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 --2- 172.21.15.37:0/175681197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568101420 0x7f556810d950 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f5560009cd0 tx=0x7f556002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 -- 172.21.15.37:0/175681197 shutdown_connections 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 --2- 172.21.15.37:0/175681197 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568101420 0x7f556810d950 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 --2- 172.21.15.37:0/175681197 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5568100a80 0x7f5568100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 --2- 172.21.15.37:0/175681197 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5568106a80 0x7f5568106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 -- 172.21.15.37:0/175681197 >> 172.21.15.37:0/175681197 conn(0x7f55680fc7a0 msgr2=0x7f55680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 -- 172.21.15.37:0/175681197 shutdown_connections 2024-09-19T06:49:21.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 -- 172.21.15.37:0/175681197 wait complete. 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.341+0000 7f556dac5640 1 Processor -- start 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f556dac5640 1 -- start start 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f556dac5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568100a80 0x7f556819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f556dac5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5568101420 0x7f556819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f556dac5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5568106a80 0x7f556819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f556dac5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5568110ea0 con 0x7f5568100a80 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f556dac5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5568110d20 con 0x7f5568106a80 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f556dac5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5568111020 con 0x7f5568101420 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568100a80 0x7f556819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568100a80 0x7f556819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41762/0 (socket says 172.21.15.37:41762) 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 -- 172.21.15.37:0/1142916037 learned_addr learned my addr 172.21.15.37:0/1142916037 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f5567fff640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5568106a80 0x7f556819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f5566ffd640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5568101420 0x7f556819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 -- 172.21.15.37:0/1142916037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5568101420 msgr2=0x7f556819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5568101420 0x7f556819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 -- 172.21.15.37:0/1142916037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5568106a80 msgr2=0x7f556819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5568106a80 0x7f556819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f55677fe640 1 -- 172.21.15.37:0/1142916037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556819f5f0 con 0x7f5568100a80 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.342+0000 7f5566ffd640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5568101420 0x7f556819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.343+0000 7f55677fe640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568100a80 0x7f556819a640 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f555800d950 tx=0x7f555800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.343+0000 7f5564ff9640 1 -- 172.21.15.37:0/1142916037 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558017070 con 0x7f5568100a80 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.343+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f556819f8e0 con 0x7f5568100a80 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.343+0000 7f5564ff9640 1 -- 172.21.15.37:0/1142916037 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5558009c30 con 0x7f5568100a80 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.343+0000 7f5564ff9640 1 -- 172.21.15.37:0/1142916037 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800f6b0 con 0x7f5568100a80 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.343+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681a71c0 con 0x7f5568100a80 2024-09-19T06:49:21.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f5564ff9640 1 -- 172.21.15.37:0/1142916037 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f55580109d0 con 0x7f5568100a80 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 Processor -- start 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 -- start start 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 0x55cf20586f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20587480 0x55cf20591800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf20594130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf204ec280 con 0x55cf20591d40 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf20493b10 con 0x55cf20587480 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f556fe9e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cf204e8240 con 0x55cf20586b60 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f5566ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf20594130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f55677fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20587480 0x55cf20591800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f5566ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf20594130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41778/0 (socket says 172.21.15.37:41778) 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.346+0000 7f5566ffd640 1 -- 172.21.15.37:0/2083623082 learned_addr learned my addr 172.21.15.37:0/2083623082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5567fff640 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 0x55cf20586f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5564ff9640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55500a33b0 0x7f55500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5567fff640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55500a33b0 0x7f55500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5566ffd640 1 -- 172.21.15.37:0/2083623082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 msgr2=0x55cf20586f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5566ffd640 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 0x55cf20586f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5566ffd640 1 -- 172.21.15.37:0/2083623082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20587480 msgr2=0x55cf20591800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5566ffd640 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20587480 0x55cf20591800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5566ffd640 1 -- 172.21.15.37:0/2083623082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf204a01f0 con 0x55cf20591d40 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5564ff9640 1 -- 172.21.15.37:0/1142916037 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f55580cfa20 con 0x7f5568100a80 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5566ffd640 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf20594130 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f555c007800 tx=0x7f555c007cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1e1c532733aeae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f554dffb640 1 -- 172.21.15.37:0/2083623082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c00d950 con 0x55cf20591d40 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f554dffb640 1 -- 172.21.15.37:0/2083623082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c018070 con 0x55cf20591d40 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f5567fff640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55500a33b0 0x7f55500a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5560004ab0 tx=0x7f556002a8e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.347+0000 7f554dffb640 1 -- 172.21.15.37:0/2083623082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c0123e0 con 0x55cf20591d40 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 -- 172.21.15.37:0/2083623082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 msgr2=0x55cf20594130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf20594130 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f555c007800 tx=0x7f555c007cc0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 -- 172.21.15.37:0/2083623082 shutdown_connections 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf20594130 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20587480 0x55cf20591800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2083623082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 0x55cf20586f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 -- 172.21.15.37:0/2083623082 >> 172.21.15.37:0/2083623082 conn(0x55cf205830f0 msgr2=0x55cf205834d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 -- 172.21.15.37:0/2083623082 shutdown_connections 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 -- 172.21.15.37:0/2083623082 wait complete. 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.348+0000 7f556fe9e740 1 -- start start 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 1 Processor -- start 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20586b60 0x55cf2058eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2058f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf2058f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55cf20594910 con 0x55cf20591d40 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cf204a6f10 con 0x55cf20586b60 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55cf20594a90 con 0x55cf20587480 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f556fe9e740 10 client.0 _ll_register_callbacks cb 0x55cf2046fa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf2058f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf2058f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41780/0 (socket says 172.21.15.37:41780) 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 -- 172.21.15.37:0/1399225004 learned_addr learned my addr 172.21.15.37:0/1399225004 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f55677fe640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20586b60 0x55cf2058eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5566ffd640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2058f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 -- 172.21.15.37:0/1399225004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 msgr2=0x55cf2058f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2058f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 -- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20586b60 msgr2=0x55cf2058eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20586b60 0x55cf2058eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5567fff640 1 -- 172.21.15.37:0/1399225004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2025d6a0 con 0x55cf20591d40 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.349+0000 7f5566ffd640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2058f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f5567fff640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf2058f600 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f5560009cd0 tx=0x7f55600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5757285fd8dc2bf0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f5546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f5546ffd640 1 -- 172.21.15.37:0/1399225004 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5560004790 con 0x55cf20591d40 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f5546ffd640 1 -- 172.21.15.37:0/1399225004 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f55600305a0 con 0x55cf20591d40 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555c018070 con 0x55cf20591d40 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f5546ffd640 1 -- 172.21.15.37:0/1399225004 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f55600307a0 con 0x55cf20591d40 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 10 client.17580 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cf205a0480 con 0x55cf20591d40 2024-09-19T06:49:21.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f5546ffd640 1 -- 172.21.15.37:0/1399225004 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c018070 con 0x55cf20591d40 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f5546ffd640 1 -- 172.21.15.37:0/1399225004 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f556004f030 con 0x55cf20591d40 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 10 client.17580 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 10 client.17580 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 20 client.17580 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf205a19b0 0x55cf205a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 4 client.17580 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.350+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cf204f97f0 con 0x55cf205a19b0 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.351+0000 7f55677fe640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf205a19b0 0x55cf205a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.351+0000 7f55677fe640 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf205a19b0 0x55cf205a3e50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3054f4c571d08677 server_cookie=cc922961ed707ea8 in_seq=0 out_seq=0 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.351+0000 7f5546ffd640 10 client.17580 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f5546ffd640 1 -- 172.21.15.37:0/1399225004 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cf204f97f0 con 0x55cf205a19b0 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f5546ffd640 10 client.17580 handle_command_reply: tid=0 2024-09-19T06:49:21.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 client.17580 shutdown 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 msgr2=0x55cf2058f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf2058f600 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f5560009cd0 tx=0x7f55600095c0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 shutdown_connections 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf205a19b0 0x55cf205a3e50 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20591d40 0x55cf2058f600 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2058f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 --2- 172.21.15.37:0/1399225004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20586b60 0x55cf2058eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f5546ffd640 0 client.17580 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:21.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 >> 172.21.15.37:0/1399225004 conn(0x55cf205830f0 msgr2=0x55cf20588dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 shutdown_connections 2024-09-19T06:49:21.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 1 -- 172.21.15.37:0/1399225004 wait complete. 2024-09-19T06:49:21.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.352+0000 7f556fe9e740 20 client.17580 trim_cache size 0 max 16384 2024-09-19T06:49:21.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.369+0000 7f556fe9e740 1 Processor -- start 2024-09-19T06:49:21.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.369+0000 7f556fe9e740 1 -- start start 2024-09-19T06:49:21.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 0x55cf20588cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20587480 0x55cf205a9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf205a9ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f556fe9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55681b2480 con 0x55cf20587480 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f556fe9e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55681afcf0 con 0x55cf20591d40 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f556fe9e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55681adbc0 con 0x55cf20586b60 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20587480 0x55cf205a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20587480 0x55cf205a9760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41786/0 (socket says 172.21.15.37:41786) 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 -- 172.21.15.37:0/986889029 learned_addr learned my addr 172.21.15.37:0/986889029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5567fff640 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf205a9ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 -- 172.21.15.37:0/986889029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 msgr2=0x55cf20588cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 0x55cf20588cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 -- 172.21.15.37:0/986889029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 msgr2=0x55cf205a9ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf205a9ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 -- 172.21.15.37:0/986889029 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cf205872b0 con 0x55cf20587480 2024-09-19T06:49:21.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.370+0000 7f5566ffd640 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20587480 0x55cf205a9760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f555c004810 tx=0x7f555c00db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144ec02bab1a258e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f554d7fa640 1 -- 172.21.15.37:0/986889029 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c013480 con 0x55cf20587480 2024-09-19T06:49:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f554d7fa640 1 -- 172.21.15.37:0/986889029 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c018040 con 0x55cf20587480 2024-09-19T06:49:21.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f554d7fa640 1 -- 172.21.15.37:0/986889029 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555c027a50 con 0x55cf20587480 2024-09-19T06:49:21.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 -- 172.21.15.37:0/986889029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20587480 msgr2=0x55cf205a9760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20587480 0x55cf205a9760 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f555c004810 tx=0x7f555c00db60 comp rx=0 tx=0).stop 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 -- 172.21.15.37:0/986889029 shutdown_connections 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf205a9ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20587480 0x55cf205a9760 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 --2- 172.21.15.37:0/986889029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20586b60 0x55cf20588cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 -- 172.21.15.37:0/986889029 >> 172.21.15.37:0/986889029 conn(0x55cf205830f0 msgr2=0x55cf205885b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 -- 172.21.15.37:0/986889029 shutdown_connections 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.371+0000 7f556fe9e740 1 -- 172.21.15.37:0/986889029 wait complete. 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 -- start start 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 Processor -- start 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20586b60 0x55cf20599ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2059a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf20596cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f55681ae1b0 con 0x55cf20586b60 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f55681af620 con 0x55cf20591d40 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f55681ae880 con 0x55cf20587480 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f556fe9e740 10 client.0 _ll_register_callbacks cb 0x7f55600098f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f55677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20586b60 0x55cf20599ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f55677fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20586b60 0x55cf20599ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41802/0 (socket says 172.21.15.37:41802) 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f55677fe640 1 -- 172.21.15.37:0/2560209226 learned_addr learned my addr 172.21.15.37:0/2560209226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.372+0000 7f5567fff640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf20596cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f5566ffd640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2059a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f55677fe640 1 -- 172.21.15.37:0/2560209226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 msgr2=0x55cf2059a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f55677fe640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2059a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f55677fe640 1 -- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 msgr2=0x55cf20596cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:21.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f55677fe640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf20596cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f55677fe640 1 -- 172.21.15.37:0/2560209226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cf2059a810 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f5567fff640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf20596cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f55677fe640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20586b60 0x55cf20599ce0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f555809bc00 tx=0x7f5558098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7645e7a8f5c2336a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f5546ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f5546ffd640 1 -- 172.21.15.37:0/2560209226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55580d5020 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f5546ffd640 1 -- 172.21.15.37:0/2560209226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f55580cd360 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f555c018040 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f5546ffd640 1 -- 172.21.15.37:0/2560209226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f5558020040 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f556fe9e740 10 client.17586 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55cf2060ccf0 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.373+0000 7f5546ffd640 1 -- 172.21.15.37:0/2560209226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f555c018040 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f5546ffd640 1 -- 172.21.15.37:0/2560209226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f55580a4b30 con 0x55cf20586b60 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f556fe9e740 10 client.17586 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:21.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f556fe9e740 10 client.17586 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:21.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f556fe9e740 20 client.17586 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:21.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf2060d470 0x55cf2060f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:21.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f556fe9e740 4 client.17586 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:49:21.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55cf204f3d00 con 0x55cf2060d470 2024-09-19T06:49:21.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f5566ffd640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf2060d470 0x55cf2060f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:21.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f5566ffd640 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf2060d470 0x55cf2060f890 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8eaaa96b92095975 server_cookie=aeea788e8b7a854d in_seq=0 out_seq=0 2024-09-19T06:49:21.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.374+0000 7f5546ffd640 10 client.17586 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:21.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f5546ffd640 1 -- 172.21.15.37:0/2560209226 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55cf204f3d00 con 0x55cf2060d470 2024-09-19T06:49:21.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f5546ffd640 10 client.17586 handle_command_reply: tid=0 2024-09-19T06:49:21.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 client.17586 shutdown 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20586b60 msgr2=0x55cf20599ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20586b60 0x55cf20599ce0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f555809bc00 tx=0x7f5558098040 comp rx=0 tx=0).stop 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 shutdown_connections 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55cf2060d470 0x55cf2060f890 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cf20591d40 0x55cf20596cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55cf20587480 0x55cf2059a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f5546ffd640 0 client.17586 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 --2- 172.21.15.37:0/2560209226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55cf20586b60 0x55cf20599ce0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 >> 172.21.15.37:0/2560209226 conn(0x55cf205830f0 msgr2=0x55cf20588dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 shutdown_connections 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 1 -- 172.21.15.37:0/2560209226 wait complete. 2024-09-19T06:49:21.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.376+0000 7f556fe9e740 20 client.17586 trim_cache size 0 max 16384 2024-09-19T06:49:21.383 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:49:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:49:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "4e8cf6d6-fc5a-46ca-9950-6478551890a6", 2024-09-19T06:49:21.383 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:49:21.383 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:49:21.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55500a33b0 msgr2=0x7f55500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55500a33b0 0x7f55500a5870 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f5560004ab0 tx=0x7f556002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568100a80 msgr2=0x7f556819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568100a80 0x7f556819a640 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f555800d950 tx=0x7f555800de10 comp rx=0 tx=0).stop 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 shutdown_connections 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5568106a80 0x7f556819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5568101420 0x7f556819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f55500a33b0 0x7f55500a5870 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 --2- 172.21.15.37:0/1142916037 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5568100a80 0x7f556819a640 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 >> 172.21.15.37:0/1142916037 conn(0x7f55680fc7a0 msgr2=0x7f556806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 shutdown_connections 2024-09-19T06:49:21.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:21.377+0000 7f556dac5640 1 -- 172.21.15.37:0/1142916037 wait complete. 2024-09-19T06:49:22.130 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:49:22.214 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.94 MB/sec execute 229 sec latency 16304.595 ms 2024-09-19T06:49:22.448 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:22 smithi151 ceph-mon[35236]: pgmap v236: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-09-19T06:49:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:22 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:49:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:22 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:22 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:49:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:22 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:49:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:22 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:22 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.708 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:22 smithi037 ceph-mon[31211]: pgmap v236: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-09-19T06:49:22.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:22 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:49:22.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:22 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:22 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:49:22.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:22 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:49:22.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:22 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.709 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:22 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:22 smithi096 ceph-mon[35004]: pgmap v236: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 2.4 MiB/s wr, 129 op/s 2024-09-19T06:49:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:22 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:49:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:22 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:22 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:49:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:22 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:49:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:22 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:22 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:49:22.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 -- 172.21.15.37:0/2520590638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec1082a0 msgr2=0x7fbeec10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 --2- 172.21.15.37:0/2520590638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec1082a0 0x7fbeec10ec40 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009cd0 tx=0x7fbee402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 -- 172.21.15.37:0/2520590638 shutdown_connections 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 --2- 172.21.15.37:0/2520590638 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec1082a0 0x7fbeec10ec40 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 --2- 172.21.15.37:0/2520590638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeec1036e0 0x7fbeec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 --2- 172.21.15.37:0/2520590638 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeec102d30 0x7fbeec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 -- 172.21.15.37:0/2520590638 >> 172.21.15.37:0/2520590638 conn(0x7fbeec0fc7d0 msgr2=0x7fbeec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 -- 172.21.15.37:0/2520590638 shutdown_connections 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 -- 172.21.15.37:0/2520590638 wait complete. 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.926+0000 7fbef20e3640 1 Processor -- start 2024-09-19T06:49:22.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbef20e3640 1 -- start start 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbef20e3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec102d30 0x7fbeec100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbef20e3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeec1036e0 0x7fbeec100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbef20e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeec1082a0 0x7fbeec101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbef20e3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1120a0 con 0x7fbeec102d30 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbef20e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec111f20 con 0x7fbeec1082a0 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbef20e3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec112220 con 0x7fbeec1036e0 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec102d30 0x7fbeec100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec102d30 0x7fbeec100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41826/0 (socket says 172.21.15.37:41826) 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/172894977 learned_addr learned my addr 172.21.15.37:0/172894977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeebfff640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeec1082a0 0x7fbeec101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeec1036e0 0x7fbeec100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/172894977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeec1036e0 msgr2=0x7fbeec100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeec1036e0 0x7fbeec100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/172894977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeec1082a0 msgr2=0x7fbeec101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeec1082a0 0x7fbeec101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.927+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/172894977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbeec1a6a30 con 0x7fbeec102d30 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.928+0000 7fbeebfff640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeec1082a0 0x7fbeec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.928+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec102d30 0x7fbeec100990 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d770 tx=0x7fbedc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.928+0000 7fbee8ff9640 1 -- 172.21.15.37:0/172894977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc017070 con 0x7fbeec102d30 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.928+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbeec1a6d20 con 0x7fbeec102d30 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.928+0000 7fbee8ff9640 1 -- 172.21.15.37:0/172894977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc005500 con 0x7fbeec102d30 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.928+0000 7fbee8ff9640 1 -- 172.21.15.37:0/172894977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc004e00 con 0x7fbeec102d30 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.928+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbeec1a7200 con 0x7fbeec102d30 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.930+0000 7fbee8ff9640 1 -- 172.21.15.37:0/172894977 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbedc004fa0 con 0x7fbeec102d30 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.930+0000 7fbef44bc740 1 Processor -- start 2024-09-19T06:49:22.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbef44bc740 1 -- start start 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e7134a0 0x55d04e713880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71e140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e71e680 0x55d04e720a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbef44bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d04e6792e0 con 0x55d04e71e680 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbef44bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d04e620ce0 con 0x55d04e713dc0 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbef44bc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d04e6752a0 con 0x55d04e7134a0 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e71e680 0x55d04e720a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e71e680 0x55d04e720a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41830/0 (socket says 172.21.15.37:41830) 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 -- 172.21.15.37:0/2959973694 learned_addr learned my addr 172.21.15.37:0/2959973694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeebfff640 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e7134a0 0x55d04e713880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbee8ff9640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbed40a33b0 0x7fbed40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeebfff640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbed40a33b0 0x7fbed40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 -- 172.21.15.37:0/2959973694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e7134a0 msgr2=0x55d04e713880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e7134a0 0x55d04e713880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 -- 172.21.15.37:0/2959973694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 msgr2=0x55d04e71e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbeeaffd640 1 -- 172.21.15.37:0/2959973694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04e605440 con 0x55d04e71e680 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.931+0000 7fbee8ff9640 1 -- 172.21.15.37:0/172894977 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fbedc0cf890 con 0x7fbeec102d30 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.932+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e71e680 0x55d04e720a70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbee000caa0 tx=0x7fbee000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e75dd9a1e7828e4a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.932+0000 7fbeebfff640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbed40a33b0 0x7fbed40a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006010 tx=0x7fbee402a8e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.932+0000 7fbed1ffb640 1 -- 172.21.15.37:0/2959973694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee0010430 con 0x55d04e71e680 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.932+0000 7fbed1ffb640 1 -- 172.21.15.37:0/2959973694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee0010ab0 con 0x55d04e71e680 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.932+0000 7fbed1ffb640 1 -- 172.21.15.37:0/2959973694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee000fac0 con 0x55d04e71e680 2024-09-19T06:49:22.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.932+0000 7fbef44bc740 1 -- 172.21.15.37:0/2959973694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e71e680 msgr2=0x55d04e720a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.932+0000 7fbef44bc740 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e71e680 0x55d04e720a70 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fbee000caa0 tx=0x7fbee000cf60 comp rx=0 tx=0).stop 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- 172.21.15.37:0/2959973694 shutdown_connections 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e71e680 0x55d04e720a70 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 --2- 172.21.15.37:0/2959973694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e7134a0 0x55d04e713880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- 172.21.15.37:0/2959973694 >> 172.21.15.37:0/2959973694 conn(0x55d04e70fa30 msgr2=0x55d04e70fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- 172.21.15.37:0/2959973694 shutdown_connections 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- 172.21.15.37:0/2959973694 wait complete. 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- start start 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 Processor -- start 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e71dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 0x55d04e71b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d04e721250 con 0x55d04e7134a0 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d04e634050 con 0x55d04e713dc0 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.933+0000 7fbef44bc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d04e7213d0 con 0x55d04e71e680 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbef44bc740 10 client.0 _ll_register_callbacks cb 0x55d04e674160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e71dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e71dfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41838/0 (socket says 172.21.15.37:41838) 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1661218132 learned_addr learned my addr 172.21.15.37:0/1661218132 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1661218132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 msgr2=0x55d04e71b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 0x55d04e71b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 msgr2=0x55d04e71b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1661218132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04e3ea3c0 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e71dfd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbedc09ba70 tx=0x7fbedc06b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ce2a9b81689cc4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbecaffd640 1 -- 172.21.15.37:0/1661218132 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc020040 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbecaffd640 1 -- 172.21.15.37:0/1661218132 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fbedc084de0 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbee0010ab0 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbecaffd640 1 -- 172.21.15.37:0/1661218132 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbedc066930 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbef44bc740 10 client.17598 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.934+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d04e72cc80 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbecaffd640 1 -- 172.21.15.37:0/1661218132 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee0010ab0 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbecaffd640 1 -- 172.21.15.37:0/1661218132 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fbedc0860b0 con 0x55d04e7134a0 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbef44bc740 10 client.17598 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:22.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbef44bc740 10 client.17598 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbef44bc740 20 client.17598 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e730500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbef44bc740 4 client.17598 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d04e686850 con 0x55d04e72e040 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.935+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e730500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e730500 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8258ecd1aee5fa4e server_cookie=72a6fc06173da995 in_seq=0 out_seq=0 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbecaffd640 10 client.17598 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbecaffd640 1 -- 172.21.15.37:0/1661218132 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d04e686850 con 0x55d04e72e040 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbecaffd640 10 client.17598 handle_command_reply: tid=0 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 client.17598 shutdown 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 msgr2=0x55d04e71dfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e71dfd0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fbedc09ba70 tx=0x7fbedc06b700 comp rx=0 tx=0).stop 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 shutdown_connections 2024-09-19T06:49:22.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e730500 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 0x55d04e71b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbecaffd640 0 client.17598 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e713dc0 0x55d04e71b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.936+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1661218132 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e71dfd0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.937+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 >> 172.21.15.37:0/1661218132 conn(0x55d04e70fa30 msgr2=0x55d04e715710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.937+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 shutdown_connections 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.937+0000 7fbef44bc740 1 -- 172.21.15.37:0/1661218132 wait complete. 2024-09-19T06:49:22.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.937+0000 7fbef44bc740 20 client.17598 trim_cache size 0 max 16384 2024-09-19T06:49:22.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 Processor -- start 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 -- start start 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e719b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e713dc0 0x55d04e735e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e71e680 0x55d04e736360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1b24a0 con 0x55d04e7134a0 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1afd10 con 0x55d04e71e680 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbef44bc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1adb80 con 0x55d04e713dc0 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e719b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e719b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41854/0 (socket says 172.21.15.37:41854) 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1142485794 learned_addr learned my addr 172.21.15.37:0/1142485794 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeebfff640 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e71e680 0x55d04e736360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e713dc0 0x55d04e735e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1142485794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e713dc0 msgr2=0x55d04e735e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e713dc0 0x55d04e735e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1142485794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e71e680 msgr2=0x55d04e736360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e71e680 0x55d04e736360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 -- 172.21.15.37:0/1142485794 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d04e71a230 con 0x55d04e7134a0 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.955+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e719b50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbedc09b7e0 tx=0x7fbedc099ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25a62e556bbfcf22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbed17fa640 1 -- 172.21.15.37:0/1142485794 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc020070 con 0x55d04e7134a0 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbed17fa640 1 -- 172.21.15.37:0/1142485794 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbedc0cd3c0 con 0x55d04e7134a0 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbed17fa640 1 -- 172.21.15.37:0/1142485794 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbedc0860b0 con 0x55d04e7134a0 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbef44bc740 1 -- 172.21.15.37:0/1142485794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 msgr2=0x55d04e719b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e719b50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fbedc09b7e0 tx=0x7fbedc099ff0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbef44bc740 1 -- 172.21.15.37:0/1142485794 shutdown_connections 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e71e680 0x55d04e736360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e713dc0 0x55d04e735e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbef44bc740 1 --2- 172.21.15.37:0/1142485794 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e7134a0 0x55d04e719b50 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.956+0000 7fbef44bc740 1 -- 172.21.15.37:0/1142485794 >> 172.21.15.37:0/1142485794 conn(0x55d04e70fa30 msgr2=0x55d04e70eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 -- 172.21.15.37:0/1142485794 shutdown_connections 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 -- 172.21.15.37:0/1142485794 wait complete. 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 -- start start 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 Processor -- start 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e7134a0 0x55d04e724460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e713dc0 0x55d04e7249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 0x55d04e724ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1ae1d0 con 0x55d04e713dc0 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1af640 con 0x55d04e7134a0 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbeec1ae8a0 con 0x55d04e71e680 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.957+0000 7fbef44bc740 10 client.0 _ll_register_callbacks cb 0x7fbedc00de80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e713dc0 0x55d04e7249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e713dc0 0x55d04e7249a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41870/0 (socket says 172.21.15.37:41870) 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 -- 172.21.15.37:0/454842176 learned_addr learned my addr 172.21.15.37:0/454842176 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:22.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e7134a0 0x55d04e724460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeebfff640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 0x55d04e724ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 -- 172.21.15.37:0/454842176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 msgr2=0x55d04e724ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 0x55d04e724ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 -- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e7134a0 msgr2=0x55d04e724460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e7134a0 0x55d04e724460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 -- 172.21.15.37:0/454842176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d04e713bf0 con 0x55d04e713dc0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e7134a0 0x55d04e724460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbeeaffd640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e713dc0 0x55d04e7249a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbee000f670 tx=0x7fbee000f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dbc428bce7b3d66 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbecaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbecaffd640 1 -- 172.21.15.37:0/454842176 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbee002b070 con 0x55d04e713dc0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbecaffd640 1 -- 172.21.15.37:0/454842176 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fbee0005450 con 0x55d04e713dc0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.958+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d04e680d60 con 0x55d04e713dc0 2024-09-19T06:49:22.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbecaffd640 1 -- 172.21.15.37:0/454842176 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbee0005650 con 0x55d04e713dc0 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 10 client.17604 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d04e798f80 con 0x55d04e713dc0 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbecaffd640 1 -- 172.21.15.37:0/454842176 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbee000ee70 con 0x55d04e713dc0 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbecaffd640 1 -- 172.21.15.37:0/454842176 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fbee002e020 con 0x55d04e713dc0 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 10 client.17604 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 10 client.17604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 20 client.17604 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e79b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 4 client.17604 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d04e680d60 con 0x55d04e72e040 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.959+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e79b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbeeb7fe640 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e79b560 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=418f3ad8be8515ed server_cookie=11969bca2741ba15 in_seq=0 out_seq=0 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbecaffd640 10 client.17604 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:22.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbecaffd640 1 -- 172.21.15.37:0/454842176 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d04e680d60 con 0x55d04e72e040 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbecaffd640 10 client.17604 handle_command_reply: tid=0 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 client.17604 shutdown 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e713dc0 msgr2=0x55d04e7249a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e713dc0 0x55d04e7249a0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fbee000f670 tx=0x7fbee000f600 comp rx=0 tx=0).stop 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 shutdown_connections 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d04e72e040 0x55d04e79b560 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d04e71e680 0x55d04e724ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d04e713dc0 0x55d04e7249a0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbecaffd640 0 client.17604 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 --2- 172.21.15.37:0/454842176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d04e7134a0 0x55d04e724460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 >> 172.21.15.37:0/454842176 conn(0x55d04e70fa30 msgr2=0x55d04e70eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 shutdown_connections 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.960+0000 7fbef44bc740 1 -- 172.21.15.37:0/454842176 wait complete. 2024-09-19T06:49:22.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef44bc740 20 client.17604 trim_cache size 0 max 16384 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: "4e8cf6d6-fc5a-46ca-9950-6478551890a6": { 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "4e8cf6d6-fc5a-46ca-9950-6478551890a6", 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:22.972 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:49:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbed40a33b0 msgr2=0x7fbed40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbed40a33b0 0x7fbed40a5870 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fbee4006010 tx=0x7fbee402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec102d30 msgr2=0x7fbeec100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec102d30 0x7fbeec100990 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fbedc00d770 tx=0x7fbedc00dc30 comp rx=0 tx=0).stop 2024-09-19T06:49:22.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 shutdown_connections 2024-09-19T06:49:22.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbeec1082a0 0x7fbeec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbeec1036e0 0x7fbeec100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbed40a33b0 0x7fbed40a5870 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 --2- 172.21.15.37:0/172894977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbeec102d30 0x7fbeec100990 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:22.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 >> 172.21.15.37:0/172894977 conn(0x7fbeec0fc7d0 msgr2=0x7fbeec105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:22.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 shutdown_connections 2024-09-19T06:49:22.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:22.961+0000 7fbef20e3640 1 -- 172.21.15.37:0/172894977 wait complete. 2024-09-19T06:49:23.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.91 MB/sec execute 230 sec latency 17304.679 ms 2024-09-19T06:49:23.749 INFO:tasks.cephfs.filesystem:scrub status for tag:4e8cf6d6-fc5a-46ca-9950-6478551890a6 - {'path': '/', 'tag': '4e8cf6d6-fc5a-46ca-9950-6478551890a6', 'options': 'recursive,force'} 2024-09-19T06:49:24.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.87 MB/sec execute 231 sec latency 18304.779 ms 2024-09-19T06:49:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:24 smithi151 ceph-mon[35236]: pgmap v237: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.1 MiB/s wr, 162 op/s 2024-09-19T06:49:24.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:24 smithi096 ceph-mon[35004]: pgmap v237: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.1 MiB/s wr, 162 op/s 2024-09-19T06:49:24.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:24 smithi037 ceph-mon[31211]: pgmap v237: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 207 KiB/s rd, 3.1 MiB/s wr, 162 op/s 2024-09-19T06:49:25.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.84 MB/sec execute 232 sec latency 19304.903 ms 2024-09-19T06:49:26.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.80 MB/sec execute 233 sec latency 20305.022 ms 2024-09-19T06:49:26.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:26 smithi151 ceph-mon[35236]: pgmap v238: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:49:26.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:26 smithi096 ceph-mon[35004]: pgmap v238: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:49:26.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:26 smithi037 ceph-mon[31211]: pgmap v238: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:49:27.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.77 MB/sec execute 234 sec latency 21305.146 ms 2024-09-19T06:49:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:27 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:49:27.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:27 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:49:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3148855839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 msgr2=0x7f9dec106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3148855839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec106da0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc009930 tx=0x7f9ddc02a420 comp rx=0 tx=0).stop 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3148855839 shutdown_connections 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3148855839 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dec101350 0x7f9dec10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3148855839 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dec1009b0 0x7f9dec100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3148855839 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec106da0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3148855839 >> 172.21.15.37:0/3148855839 conn(0x7f9dec077990 msgr2=0x7f9dec0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3148855839 shutdown_connections 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.676+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3148855839 wait complete. 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 Processor -- start 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 -- start start 2024-09-19T06:49:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dec1009b0 0x7f9dec1991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dec101350 0x7f9dec199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec1a4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec110e00 con 0x7f9dec1069c0 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec110c80 con 0x7f9dec1009b0 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df2ad0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec110f80 con 0x7f9dec101350 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df106d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec1a4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df106d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec1a4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41894/0 (socket says 172.21.15.37:41894) 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df106d640 1 -- 172.21.15.37:0/3235164174 learned_addr learned my addr 172.21.15.37:0/3235164174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.677+0000 7f9df086c640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dec1009b0 0x7f9dec1991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9debfff640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dec101350 0x7f9dec199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df106d640 1 -- 172.21.15.37:0/3235164174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dec101350 msgr2=0x7f9dec199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df106d640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dec101350 0x7f9dec199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df106d640 1 -- 172.21.15.37:0/3235164174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dec1009b0 msgr2=0x7f9dec1991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df106d640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dec1009b0 0x7f9dec1991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df106d640 1 -- 172.21.15.37:0/3235164174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dec1a4790 con 0x7f9dec1069c0 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9debfff640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dec101350 0x7f9dec199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df106d640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec1a4250 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9de400eb10 tx=0x7f9de400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9de9ffb640 1 -- 172.21.15.37:0/3235164174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4004b20 con 0x7f9dec1069c0 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9de9ffb640 1 -- 172.21.15.37:0/3235164174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de4004cc0 con 0x7f9dec1069c0 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dec1a4960 con 0x7f9dec1069c0 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9de9ffb640 1 -- 172.21.15.37:0/3235164174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de4010760 con 0x7f9dec1069c0 2024-09-19T06:49:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.678+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dec1a4e30 con 0x7f9dec1069c0 2024-09-19T06:49:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.680+0000 7f9de9ffb640 1 -- 172.21.15.37:0/3235164174 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9de4005020 con 0x7f9dec1069c0 2024-09-19T06:49:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 Processor -- start 2024-09-19T06:49:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 -- start start 2024-09-19T06:49:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.689 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563fec3262f0 con 0x563fec3c2410 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563fec2ce270 con 0x563fec3cccb0 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df5867740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563fec3222b0 con 0x563fec3c1af0 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9de9ffb640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9dd40a7870 0x7f9dd40a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df086c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df106d640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9dd40a7870 0x7f9dd40a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9debfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9de9ffb640 1 -- 172.21.15.37:0/3235164174 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f9de40d02f0 con 0x7f9dec1069c0 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df086c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41910/0 (socket says 172.21.15.37:41910) 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9df086c640 1 -- 172.21.15.37:0/3274805804 learned_addr learned my addr 172.21.15.37:0/3274805804 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.681+0000 7f9debfff640 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9df086c640 1 -- 172.21.15.37:0/3274805804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 msgr2=0x563fec3c1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9df086c640 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9df086c640 1 -- 172.21.15.37:0/3274805804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 msgr2=0x563fec3cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9df086c640 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9df086c640 1 -- 172.21.15.37:0/3274805804 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fec2cd340 con 0x563fec3c2410 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9debfff640 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9df106d640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9dd40a7870 0x7f9dd40a9d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9de409c2a0 tx=0x7f9de4091a30 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9df086c640 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3cc770 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc02a940 tx=0x7f9ddc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5fe65676970a200 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9dd2ffd640 1 -- 172.21.15.37:0/3274805804 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc040070 con 0x563fec3c2410 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9dd2ffd640 1 -- 172.21.15.37:0/3274805804 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ddc0318f0 con 0x563fec3c2410 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.682+0000 7f9dd2ffd640 1 -- 172.21.15.37:0/3274805804 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc035040 con 0x563fec3c2410 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 -- 172.21.15.37:0/3274805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 msgr2=0x563fec3cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3cc770 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc02a940 tx=0x7f9ddc005c00 comp rx=0 tx=0).stop 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 -- 172.21.15.37:0/3274805804 shutdown_connections 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3cc770 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 --2- 172.21.15.37:0/3274805804 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 -- 172.21.15.37:0/3274805804 >> 172.21.15.37:0/3274805804 conn(0x563fec3be060 msgr2=0x563fec3be440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 -- 172.21.15.37:0/3274805804 shutdown_connections 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 -- 172.21.15.37:0/3274805804 wait complete. 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 -- start start 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.683+0000 7f9df5867740 1 Processor -- start 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df5867740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df5867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df5867740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3ca610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df5867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563fec3cf880 con 0x563fec3c2410 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df5867740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563fec2af290 con 0x563fec3cccb0 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df5867740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563fec3cfa00 con 0x563fec3c1af0 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df5867740 10 client.0 _ll_register_callbacks cb 0x563fec2e0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3ca0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41924/0 (socket says 172.21.15.37:41924) 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 -- 172.21.15.37:0/2561697237 learned_addr learned my addr 172.21.15.37:0/2561697237 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9debfff640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9debfff640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 -- 172.21.15.37:0/2561697237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 msgr2=0x563fec3c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 -- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 msgr2=0x563fec3ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 -- 172.21.15.37:0/2561697237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fec039210 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9debfff640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9df086c640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3ca0d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005280 tx=0x7f9ddc0054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b02e24a08a20738 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.684+0000 7f9dcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9dcbfff640 1 -- 172.21.15.37:0/2561697237 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ddc04d060 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9dcbfff640 1 -- 172.21.15.37:0/2561697237 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f9ddc004240 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ddc0318f0 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9dcbfff640 1 -- 172.21.15.37:0/2561697237 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9ddc040070 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 10 client.17616 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563fec105620 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9dcbfff640 1 -- 172.21.15.37:0/2561697237 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9ddc0318f0 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9dcbfff640 1 -- 172.21.15.37:0/2561697237 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f9ddc0037b0 con 0x563fec3c2410 2024-09-19T06:49:27.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 10 client.17616 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 10 client.17616 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 20 client.17616 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec3dcab0 0x563fec3deea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 4 client.17616 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fec333a20 con 0x563fec3dcab0 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.685+0000 7f9debfff640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec3dcab0 0x563fec3deea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.686+0000 7f9debfff640 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec3dcab0 0x563fec3deea0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d18d20dbca9f7b08 server_cookie=7461ce8db0fa1eac in_seq=0 out_seq=0 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.686+0000 7f9dcbfff640 10 client.17616 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.686+0000 7f9dcbfff640 1 -- 172.21.15.37:0/2561697237 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563fec333a20 con 0x563fec3dcab0 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.686+0000 7f9dcbfff640 10 client.17616 handle_command_reply: tid=0 2024-09-19T06:49:27.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.686+0000 7f9df5867740 1 client.17616 shutdown 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 msgr2=0x563fec3ca0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3ca0d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f9ddc005280 tx=0x7f9ddc0054b0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 shutdown_connections 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec3dcab0 0x563fec3deea0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3ca610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c2410 0x563fec3ca0d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9dcbfff640 0 client.17616 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 --2- 172.21.15.37:0/2561697237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c1af0 0x563fec3c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 >> 172.21.15.37:0/2561697237 conn(0x563fec3be060 msgr2=0x563fec3c3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 shutdown_connections 2024-09-19T06:49:27.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 1 -- 172.21.15.37:0/2561697237 wait complete. 2024-09-19T06:49:27.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.687+0000 7f9df5867740 20 client.17616 trim_cache size 0 max 16384 2024-09-19T06:49:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.701+0000 7f9df5867740 1 Processor -- start 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.701+0000 7f9df5867740 1 -- start start 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df5867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3ca440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df5867740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3c2410 0x563fec3e4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df5867740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3cccb0 0x563fec3e4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df5867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1adff0 con 0x563fec3c1af0 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df5867740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1ab860 con 0x563fec3c2410 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df5867740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1a96d0 con 0x563fec3cccb0 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3ca440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3ca440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41928/0 (socket says 172.21.15.37:41928) 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 -- 172.21.15.37:0/1535890086 learned_addr learned my addr 172.21.15.37:0/1535890086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df086c640 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3c2410 0x563fec3e4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 -- 172.21.15.37:0/1535890086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3cccb0 msgr2=0x563fec3e4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3cccb0 0x563fec3e4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 -- 172.21.15.37:0/1535890086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3c2410 msgr2=0x563fec3e4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3c2410 0x563fec3e4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 -- 172.21.15.37:0/1535890086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fec3c1870 con 0x563fec3c1af0 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9df086c640 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3c2410 0x563fec3e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9debfff640 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3ca440 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005d00 tx=0x7f9de000b750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336961c1497cfe78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9dd27fc640 1 -- 172.21.15.37:0/1535890086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de001b040 con 0x563fec3c1af0 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9dd27fc640 1 -- 172.21.15.37:0/1535890086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9de000be80 con 0x563fec3c1af0 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.702+0000 7f9dd27fc640 1 -- 172.21.15.37:0/1535890086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de00128b0 con 0x563fec3c1af0 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 -- 172.21.15.37:0/1535890086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 msgr2=0x563fec3ca440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3ca440 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005d00 tx=0x7f9de000b750 comp rx=0 tx=0).stop 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 -- 172.21.15.37:0/1535890086 shutdown_connections 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3cccb0 0x563fec3e4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3c2410 0x563fec3e4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 --2- 172.21.15.37:0/1535890086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3ca440 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 -- 172.21.15.37:0/1535890086 >> 172.21.15.37:0/1535890086 conn(0x563fec3be060 msgr2=0x563fec3cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 -- 172.21.15.37:0/1535890086 shutdown_connections 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 -- 172.21.15.37:0/1535890086 wait complete. 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 -- start start 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.703+0000 7f9df5867740 1 Processor -- start 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df5867740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3d0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df5867740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c2410 0x563fec3d0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df5867740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df5867740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1a9d20 con 0x563fec3c1af0 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df5867740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1ab190 con 0x563fec3cccb0 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df5867740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9dec1aa3f0 con 0x563fec3c2410 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df5867740 10 client.0 _ll_register_callbacks cb 0x7f9de000b5b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:27.712 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3d0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3d0510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41942/0 (socket says 172.21.15.37:41942) 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 -- 172.21.15.37:0/3496570845 learned_addr learned my addr 172.21.15.37:0/3496570845 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9df086c640 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c2410 0x563fec3d0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 -- 172.21.15.37:0/3496570845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c2410 msgr2=0x563fec3d0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c2410 0x563fec3d0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 -- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 msgr2=0x563fec3d4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3d4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.704+0000 7f9debfff640 1 -- 172.21.15.37:0/3496570845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fec3d51d0 con 0x563fec3c1af0 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9debfff640 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3d0510 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9de0021970 tx=0x7f9de000b380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c364cf209124d9cf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9dcbfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:27.713 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9dcbfff640 1 -- 172.21.15.37:0/3496570845 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9de0021ad0 con 0x563fec3c1af0 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9dcbfff640 1 -- 172.21.15.37:0/3496570845 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f9de0012450 con 0x563fec3c1af0 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563fec32dd70 con 0x563fec3c1af0 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9dcbfff640 1 -- 172.21.15.37:0/3496570845 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9de0012650 con 0x563fec3c1af0 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 10 client.17622 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x563fec447010 con 0x563fec3c1af0 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9dcbfff640 1 -- 172.21.15.37:0/3496570845 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x563fec32dd70 con 0x563fec3c1af0 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9dcbfff640 1 -- 172.21.15.37:0/3496570845 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f9de003c020 con 0x563fec3c1af0 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 10 client.17622 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 10 client.17622 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 20 client.17622 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec447630 0x563fec449a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 4 client.17622 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:49:27.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.705+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fec2aa900 con 0x563fec447630 2024-09-19T06:49:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.706+0000 7f9df086c640 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec447630 0x563fec449a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.706+0000 7f9df086c640 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec447630 0x563fec449a50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da2731331daab60 server_cookie=edfddae23266b195 in_seq=0 out_seq=0 2024-09-19T06:49:27.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.706+0000 7f9dcbfff640 10 client.17622 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9dcbfff640 1 -- 172.21.15.37:0/3496570845 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fec2aa900 con 0x563fec447630 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9dcbfff640 10 client.17622 handle_command_reply: tid=0 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 client.17622 shutdown 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 msgr2=0x563fec3d0510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3d0510 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9de0021970 tx=0x7f9de000b380 comp rx=0 tx=0).stop 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 shutdown_connections 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563fec447630 0x563fec449a50 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563fec3cccb0 0x563fec3d4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563fec3c2410 0x563fec3d0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.749 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9dcbfff640 0 client.17622 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 --2- 172.21.15.37:0/3496570845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563fec3c1af0 0x563fec3d0510 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.747+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 >> 172.21.15.37:0/3496570845 conn(0x563fec3be060 msgr2=0x563fec3bc9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.748+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 shutdown_connections 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.748+0000 7f9df5867740 1 -- 172.21.15.37:0/3496570845 wait complete. 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.748+0000 7f9df5867740 20 client.17622 trim_cache size 0 max 16384 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:49:27.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:49:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.748+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9dd40a7870 msgr2=0x7f9dd40a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.748+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9dd40a7870 0x7f9dd40a9d30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f9de409c2a0 tx=0x7f9de4091a30 comp rx=0 tx=0).stop 2024-09-19T06:49:27.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.748+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 msgr2=0x7f9dec1a4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.748+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec1a4250 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f9de400eb10 tx=0x7f9de400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 shutdown_connections 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9dd40a7870 0x7f9dd40a9d30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9dec1069c0 0x7f9dec1a4250 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9dec101350 0x7f9dec199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 --2- 172.21.15.37:0/3235164174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9dec1009b0 0x7f9dec1991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 >> 172.21.15.37:0/3235164174 conn(0x7f9dec077990 msgr2=0x7f9dec103750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 shutdown_connections 2024-09-19T06:49:27.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:27.749+0000 7f9df2ad0640 1 -- 172.21.15.37:0/3235164174 wait complete. 2024-09-19T06:49:27.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:27 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:49:27.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:27 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:49:27.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:27 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:49:27.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:27 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:49:28.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.74 MB/sec execute 235 sec latency 22305.272 ms 2024-09-19T06:49:28.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:28 smithi151 ceph-mon[35236]: pgmap v239: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:49:28.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:28.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:28 smithi037 ceph-mon[31211]: pgmap v239: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:49:28.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:28 smithi096 ceph-mon[35004]: pgmap v239: 97 pgs: 97 active+clean; 67 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 164 KiB/s rd, 2.0 MiB/s wr, 126 op/s 2024-09-19T06:49:29.008 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:29.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.71 MB/sec execute 236 sec latency 23305.364 ms 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 -- 172.21.15.37:0/1257938098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 msgr2=0x7f5084107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1257938098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f5084107ed0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f5078009930 tx=0x7f507802a420 comp rx=0 tx=0).stop 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 -- 172.21.15.37:0/1257938098 shutdown_connections 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1257938098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5084102380 0x7f508410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1257938098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50841019e0 0x7f5084101e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1257938098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f5084107ed0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 -- 172.21.15.37:0/1257938098 >> 172.21.15.37:0/1257938098 conn(0x7f50840fd700 msgr2=0x7f50840ffb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.467+0000 7f5089fa0640 1 -- 172.21.15.37:0/1257938098 shutdown_connections 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 -- 172.21.15.37:0/1257938098 wait complete. 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 Processor -- start 2024-09-19T06:49:29.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 -- start start 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50841019e0 0x7f508419a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5084102380 0x7f508419ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f508419f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5084111f90 con 0x7f5084107af0 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5084111e10 con 0x7f50841019e0 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f5089fa0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5084112110 con 0x7f5084102380 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.468+0000 7f508979f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f508419f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f508419f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57798/0 (socket says 172.21.15.37:57798) 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 -- 172.21.15.37:0/1184251398 learned_addr learned my addr 172.21.15.37:0/1184251398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f5088f9e640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50841019e0 0x7f508419a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f5083fff640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5084102380 0x7f508419ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 -- 172.21.15.37:0/1184251398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5084102380 msgr2=0x7f508419ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5084102380 0x7f508419ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 -- 172.21.15.37:0/1184251398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50841019e0 msgr2=0x7f508419a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50841019e0 0x7f508419a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 -- 172.21.15.37:0/1184251398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508419f700 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f508979f640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f508419f020 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f507c00b970 tx=0x7f507c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f5081ffb640 1 -- 172.21.15.37:0/1184251398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c019070 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f5081ffb640 1 -- 172.21.15.37:0/1184251398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c014440 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508419f9f0 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f5081ffb640 1 -- 172.21.15.37:0/1184251398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c0075d0 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.469+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50841a72d0 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.470+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5084101ec0 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.472+0000 7f5081ffb640 1 -- 172.21.15.37:0/1184251398 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f507c01c070 con 0x7f5084107af0 2024-09-19T06:49:29.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.472+0000 7f5081ffb640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f50740a33b0 0x7f50740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:29.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.472+0000 7f5088f9e640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f50740a33b0 0x7f50740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:29.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.472+0000 7f5081ffb640 1 -- 172.21.15.37:0/1184251398 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f507c0cf140 con 0x7f5084107af0 2024-09-19T06:49:29.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.473+0000 7f5088f9e640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f50740a33b0 0x7f50740a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5078009900 tx=0x7f50780058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:29.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.477+0000 7f5081ffb640 1 -- 172.21.15.37:0/1184251398 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f507c09b650 con 0x7f5084107af0 2024-09-19T06:49:29.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.722+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50841a74a0 con 0x7f5084107af0 2024-09-19T06:49:29.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.723+0000 7f5081ffb640 1 -- 172.21.15.37:0/1184251398 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11907 (secure 0 0 0) 0x7f507c09f950 con 0x7f5084107af0 2024-09-19T06:49:29.724 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:49:29.724 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"btime":"2024-09-19T06:48:37:250513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:48:36.625010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:49:29.724 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-09-19T06:49:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f50740a33b0 msgr2=0x7f50740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f50740a33b0 0x7f50740a5870 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5078009900 tx=0x7f50780058d0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 msgr2=0x7f508419f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f508419f020 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f507c00b970 tx=0x7f507c00bd70 comp rx=0 tx=0).stop 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 shutdown_connections 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f50740a33b0 0x7f50740a5870 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5084107af0 0x7f508419f020 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5084102380 0x7f508419ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 --2- 172.21.15.37:0/1184251398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f50841019e0 0x7f508419a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 >> 172.21.15.37:0/1184251398 conn(0x7f50840fd700 msgr2=0x7f508410d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 shutdown_connections 2024-09-19T06:49:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:29.726+0000 7f5089fa0640 1 -- 172.21.15.37:0/1184251398 wait complete. 2024-09-19T06:49:30.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559327 7.67 MB/sec execute 237 sec latency 24305.461 ms 2024-09-19T06:49:30.387 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id d016b263 --release --await' 2024-09-19T06:49:30.388 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id d016b263 --release --await 2024-09-19T06:49:30.451 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:30 smithi037 ceph-mon[31211]: pgmap v240: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-19T06:49:30.451 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:30 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1184251398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:30.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:30 smithi151 ceph-mon[35236]: pgmap v240: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-19T06:49:30.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:30 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1184251398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:30.685 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:30.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:30 smithi096 ceph-mon[35004]: pgmap v240: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 208 KiB/s rd, 2.8 MiB/s wr, 161 op/s 2024-09-19T06:49:30.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:30 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1184251398' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 -- 172.21.15.37:0/3701409506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 msgr2=0x7fcf5410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/3701409506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf5410bb50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009900 tx=0x7fcf4402a480 comp rx=0 tx=0).stop 2024-09-19T06:49:31.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 -- 172.21.15.37:0/3701409506 shutdown_connections 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/3701409506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5410c170 0x7fcf5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/3701409506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf5410bb50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/3701409506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf54102d40 0x7fcf54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 -- 172.21.15.37:0/3701409506 >> 172.21.15.37:0/3701409506 conn(0x7fcf540fc7a0 msgr2=0x7fcf540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 -- 172.21.15.37:0/3701409506 shutdown_connections 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.133+0000 7fcf5bd39640 1 -- 172.21.15.37:0/3701409506 wait complete. 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 Processor -- start 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 -- start start 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf54102d40 0x7fcf54198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf54198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5410c170 0x7fcf5419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf54111f70 con 0x7fcf54103660 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf54111df0 con 0x7fcf5410c170 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5bd39640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541120f0 con 0x7fcf54102d40 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf592d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf54198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf592d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf54198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57806/0 (socket says 172.21.15.37:57806) 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf592d4640 1 -- 172.21.15.37:0/97131182 learned_addr learned my addr 172.21.15.37:0/97131182 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf54102d40 0x7fcf54198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.134+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5410c170 0x7fcf5419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf592d4640 1 -- 172.21.15.37:0/97131182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf54102d40 msgr2=0x7fcf54198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf592d4640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf54102d40 0x7fcf54198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf592d4640 1 -- 172.21.15.37:0/97131182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5410c170 msgr2=0x7fcf5419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf592d4640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5410c170 0x7fcf5419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf592d4640 1 -- 172.21.15.37:0/97131182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf5419d400 con 0x7fcf54103660 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf592d4640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf54198990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcf44006010 tx=0x7fcf440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf4affd640 1 -- 172.21.15.37:0/97131182 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4403f030 con 0x7fcf54103660 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf4affd640 1 -- 172.21.15.37:0/97131182 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf44003ce0 con 0x7fcf54103660 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf5419d690 con 0x7fcf54103660 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf4affd640 1 -- 172.21.15.37:0/97131182 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44035070 con 0x7fcf54103660 2024-09-19T06:49:31.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.135+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf541a4fd0 con 0x7fcf54103660 2024-09-19T06:49:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.138+0000 7fcf4affd640 1 -- 172.21.15.37:0/97131182 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcf44042030 con 0x7fcf54103660 2024-09-19T06:49:31.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.138+0000 7fcf5e112740 1 Processor -- start 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5e112740 1 -- start start 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4ac340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4bb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4bda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5e112740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d24b411620 con 0x55d24b4bb650 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5e112740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d24b3b8e50 con 0x55d24b4abf60 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5e112740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d24b40d5e0 con 0x55d24b4ac880 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf592d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4bda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf592d4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4bda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57814/0 (socket says 172.21.15.37:57814) 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf592d4640 1 -- 172.21.15.37:0/4059238298 learned_addr learned my addr 172.21.15.37:0/4059238298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4bb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4ac340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.139+0000 7fcf4affd640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcf400a33b0 0x7fcf400a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcf400a33b0 0x7fcf400a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf4affd640 1 -- 172.21.15.37:0/97131182 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fcf440f02e0 con 0x7fcf54103660 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf592d4640 1 -- 172.21.15.37:0/4059238298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 msgr2=0x55d24b4bb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf592d4640 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4bb110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf592d4640 1 -- 172.21.15.37:0/4059238298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 msgr2=0x55d24b4ac340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf592d4640 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4ac340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf592d4640 1 -- 172.21.15.37:0/4059238298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d24b3a9d40 con 0x55d24b4bb650 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:31.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf592d4640 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4bda40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcf440bc630 tx=0x7fcf4402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4503e99673aad665 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcf400a33b0 0x7fcf400a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c004ab0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf3ffff640 1 -- 172.21.15.37:0/4059238298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf44049070 con 0x55d24b4bb650 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf3ffff640 1 -- 172.21.15.37:0/4059238298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf440c5260 con 0x55d24b4bb650 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.140+0000 7fcf3ffff640 1 -- 172.21.15.37:0/4059238298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf440dc030 con 0x55d24b4bb650 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 -- 172.21.15.37:0/4059238298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 msgr2=0x55d24b4bda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4bda40 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcf440bc630 tx=0x7fcf4402a480 comp rx=0 tx=0).stop 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 -- 172.21.15.37:0/4059238298 shutdown_connections 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4bda40 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4bb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4059238298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4ac340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 -- 172.21.15.37:0/4059238298 >> 172.21.15.37:0/4059238298 conn(0x55d24b4a85f0 msgr2=0x55d24b4a89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 -- 172.21.15.37:0/4059238298 shutdown_connections 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 -- 172.21.15.37:0/4059238298 wait complete. 2024-09-19T06:49:31.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 -- start start 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.141+0000 7fcf5e112740 1 Processor -- start 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4b8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4b90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5e112740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d24b4be040 con 0x55d24b4bb650 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5e112740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d24b399f10 con 0x55d24b4abf60 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5e112740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d24b4be1c0 con 0x55d24b4ac880 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5e112740 10 client.0 _ll_register_callbacks cb 0x55d24b3bddb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5a2d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4b90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5a2d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4b90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57816/0 (socket says 172.21.15.37:57816) 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/4159725533 learned_addr learned my addr 172.21.15.37:0/4159725533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:31.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf592d4640 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4b8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/4159725533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 msgr2=0x55d24b4b8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4b8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 msgr2=0x55d24b4b8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.142+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4b8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/4159725533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d24b120580 con 0x55d24b4bb650 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4b90d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c012c80 tx=0x7fcf4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c0006fb1bad309 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/4159725533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c025070 con 0x55d24b4bb650 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/4159725533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fcf4c014230 con 0x55d24b4bb650 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf440c5260 con 0x55d24b4bb650 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/4159725533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf4c01a070 con 0x55d24b4bb650 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf5e112740 10 client.17640 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:31.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d24b1ea960 con 0x55d24b4bb650 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.143+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/4159725533 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf440c5260 con 0x55d24b4bb650 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/4159725533 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fcf4c00f040 con 0x55d24b4bb650 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf5e112740 10 client.17640 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf5e112740 10 client.17640 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf5e112740 20 client.17640 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b4cb030 0x55d24b4cd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf5e112740 4 client.17640 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d24b41eb90 con 0x55d24b4cb030 2024-09-19T06:49:31.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.144+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b4cb030 0x55d24b4cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.145+0000 7fcf3d7fa640 10 client.17640 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.145+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b4cb030 0x55d24b4cd490 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3586730145f2f9f server_cookie=8a927eb58531daa6 in_seq=0 out_seq=0 2024-09-19T06:49:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.145+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/4159725533 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d24b41eb90 con 0x55d24b4cb030 2024-09-19T06:49:31.151 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.145+0000 7fcf3d7fa640 10 client.17640 handle_command_reply: tid=0 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.145+0000 7fcf5e112740 1 client.17640 shutdown 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.145+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 msgr2=0x55d24b4b90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.145+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4b90d0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c012c80 tx=0x7fcf4c004a40 comp rx=0 tx=0).stop 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 shutdown_connections 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b4cb030 0x55d24b4cd490 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4b90d0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4b8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf3d7fa640 0 client.17640 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 --2- 172.21.15.37:0/4159725533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4b8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 >> 172.21.15.37:0/4159725533 conn(0x55d24b4a85f0 msgr2=0x55d24b4ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 shutdown_connections 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 1 -- 172.21.15.37:0/4159725533 wait complete. 2024-09-19T06:49:31.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.146+0000 7fcf5e112740 20 client.17640 trim_cache size 0 max 16384 2024-09-19T06:49:31.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 Processor -- start 2024-09-19T06:49:31.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 -- start start 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4ae240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541ae0d0 con 0x55d24b4bb650 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541ab940 con 0x55d24b4abf60 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5e112740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541a97b0 con 0x55d24b4ac880 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5a2d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5a2d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4d2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57822/0 (socket says 172.21.15.37:57822) 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.163+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/727042097 learned_addr learned my addr 172.21.15.37:0/727042097 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:31.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/727042097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 msgr2=0x55d24b4ae240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4ae240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/727042097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 msgr2=0x55d24b4a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/727042097 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d24b4b8cf0 con 0x55d24b4bb650 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4d2d60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c000f80 tx=0x7fcf4c011f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb43b8b26941585 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf3f7fe640 1 -- 172.21.15.37:0/727042097 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c025040 con 0x55d24b4bb650 2024-09-19T06:49:31.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf3f7fe640 1 -- 172.21.15.37:0/727042097 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4c014220 con 0x55d24b4bb650 2024-09-19T06:49:31.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.164+0000 7fcf3f7fe640 1 -- 172.21.15.37:0/727042097 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c02a6a0 con 0x55d24b4bb650 2024-09-19T06:49:31.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 -- 172.21.15.37:0/727042097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 msgr2=0x55d24b4d2d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4d2d60 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c000f80 tx=0x7fcf4c011f00 comp rx=0 tx=0).stop 2024-09-19T06:49:31.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 -- 172.21.15.37:0/727042097 shutdown_connections 2024-09-19T06:49:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4d2d60 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4ac880 0x55d24b4ae240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 --2- 172.21.15.37:0/727042097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4abf60 0x55d24b4a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 -- 172.21.15.37:0/727042097 >> 172.21.15.37:0/727042097 conn(0x55d24b4a85f0 msgr2=0x55d24b4adb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 -- 172.21.15.37:0/727042097 shutdown_connections 2024-09-19T06:49:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 -- 172.21.15.37:0/727042097 wait complete. 2024-09-19T06:49:31.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 -- start start 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.165+0000 7fcf5e112740 1 Processor -- start 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4abf60 0x55d24b4c3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4ac880 0x55d24b4c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5e112740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5e112740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541a9e00 con 0x55d24b4bb650 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5e112740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541ab270 con 0x55d24b4ac880 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5e112740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcf541aa4d0 con 0x55d24b4abf60 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5e112740 10 client.0 _ll_register_callbacks cb 0x7fcf4c014510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:31.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4c0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57832/0 (socket says 172.21.15.37:57832) 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/852517472 learned_addr learned my addr 172.21.15.37:0/852517472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf592d4640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4ac880 0x55d24b4c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4abf60 0x55d24b4c3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/852517472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4abf60 msgr2=0x55d24b4c3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4abf60 0x55d24b4c3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4ac880 msgr2=0x55d24b4c37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4ac880 0x55d24b4c37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 -- 172.21.15.37:0/852517472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d24b4c0830 con 0x55d24b4bb650 2024-09-19T06:49:31.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4abf60 0x55d24b4c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf5a2d6640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4c0240 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0214d0 tx=0x7fcf4c02b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452d021a871778b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf3d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.166+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/852517472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf4c025040 con 0x55d24b4bb650 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/852517472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fcf4c02bc80 con 0x55d24b4bb650 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcf4c014220 con 0x55d24b4bb650 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/852517472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fcf4c04d050 con 0x55d24b4bb650 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 10 client.17646 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55d24b5365b0 con 0x55d24b4bb650 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/852517472 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf4c014220 con 0x55d24b4bb650 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/852517472 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fcf4c03e070 con 0x55d24b4bb650 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 10 client.17646 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 10 client.17646 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:49:31.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 20 client.17646 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b536c90 0x55d24b5390b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 4 client.17646 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d016b263", "release": true, "await": true}] 2024-09-19T06:49:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d016b263", "release": true, "await": true}) -- 0x55d24b4b9460 con 0x55d24b536c90 2024-09-19T06:49:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.167+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b536c90 0x55d24b5390b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.168+0000 7fcf59ad5640 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b536c90 0x55d24b5390b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba656cbc47f8b060 server_cookie=86373eff113707ed in_seq=0 out_seq=0 2024-09-19T06:49:31.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.168+0000 7fcf3d7fa640 10 client.17646 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.177+0000 7fcf3d7fa640 1 -- 172.21.15.37:0/852517472 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55d24b4b9460 con 0x55d24b536c90 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.177+0000 7fcf3d7fa640 10 client.17646 handle_command_reply: tid=0 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.177+0000 7fcf5e112740 1 client.17646 shutdown 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 msgr2=0x55d24b4c0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4c0240 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c0214d0 tx=0x7fcf4c02b680 comp rx=0 tx=0).stop 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 shutdown_connections 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d24b536c90 0x55d24b5390b0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d24b4bb650 0x55d24b4c0240 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d24b4ac880 0x55d24b4c37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf3d7fa640 0 client.17646 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 --2- 172.21.15.37:0/852517472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d24b4abf60 0x55d24b4c3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 >> 172.21.15.37:0/852517472 conn(0x55d24b4a85f0 msgr2=0x55d24b4ae350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 shutdown_connections 2024-09-19T06:49:31.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 1 -- 172.21.15.37:0/852517472 wait complete. 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.178+0000 7fcf5e112740 20 client.17646 trim_cache size 0 max 16384 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 36, 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 28, 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "d016b263": { 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 28, 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:49:31.180 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:49:31.181 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:49:31.181 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:49:31.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 145.8, 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:49:31.185 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:31.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:31.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:31.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:31.186 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:31.186 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcf400a33b0 msgr2=0x7fcf400a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcf400a33b0 0x7fcf400a5870 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcf4c005fd0 tx=0x7fcf4c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 msgr2=0x7fcf54198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf54198990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fcf44006010 tx=0x7fcf440042f0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 shutdown_connections 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf5410c170 0x7fcf5419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcf400a33b0 0x7fcf400a5870 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcf54103660 0x7fcf54198990 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 --2- 172.21.15.37:0/97131182 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcf54102d40 0x7fcf54198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 >> 172.21.15.37:0/97131182 conn(0x7fcf540fc7a0 msgr2=0x7fcf540fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:31.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 shutdown_connections 2024-09-19T06:49:31.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:31.179+0000 7fcf5bd39640 1 -- 172.21.15.37:0/97131182 wait complete. 2024-09-19T06:49:31.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 559354 7.64 MB/sec execute 238 sec latency 25271.683 ms 2024-09-19T06:49:31.915 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd016b263', total seconds elapsed: 28.9 2024-09-19T06:49:31.916 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.75 2024-09-19T06:49:31.916 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:32.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 561787 7.67 MB/sec execute 239 sec latency 27.058 ms 2024-09-19T06:49:32.240 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:32.698 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:32 smithi037 ceph-mon[31211]: pgmap v241: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-19T06:49:32.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:32 smithi096 ceph-mon[35004]: pgmap v241: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-19T06:49:32.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.784+0000 7fb52c64c640 1 -- 172.21.15.37:0/313417144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52410e500 msgr2=0x7fb524110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.784+0000 7fb52c64c640 1 --2- 172.21.15.37:0/313417144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52410e500 0x7fb524110910 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fb51c009ca0 tx=0x7fb51c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- 172.21.15.37:0/313417144 shutdown_connections 2024-09-19T06:49:32.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 --2- 172.21.15.37:0/313417144 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb52410e500 0x7fb524110910 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 --2- 172.21.15.37:0/313417144 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb524101440 0x7fb52410df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 --2- 172.21.15.37:0/313417144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb524100b20 0x7fb524100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- 172.21.15.37:0/313417144 >> 172.21.15.37:0/313417144 conn(0x7fb5240fc7d0 msgr2=0x7fb5240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- 172.21.15.37:0/313417144 shutdown_connections 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- 172.21.15.37:0/313417144 wait complete. 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 Processor -- start 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- start start 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb524100b20 0x7fb52419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb524101440 0x7fb52419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb52410e500 0x7fb52419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb524114250 con 0x7fb524100b20 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241140d0 con 0x7fb52410e500 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.785+0000 7fb52c64c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb5241143d0 con 0x7fb524101440 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb524100b20 0x7fb52419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb524100b20 0x7fb52419a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57850/0 (socket says 172.21.15.37:57850) 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52abe9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb52410e500 0x7fb52419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 -- 172.21.15.37:0/3735520517 learned_addr learned my addr 172.21.15.37:0/3735520517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb529be7640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb524101440 0x7fb52419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 -- 172.21.15.37:0/3735520517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb524101440 msgr2=0x7fb52419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb524101440 0x7fb52419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 -- 172.21.15.37:0/3735520517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb52410e500 msgr2=0x7fb52419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:32.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb52410e500 0x7fb52419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 -- 172.21.15.37:0/3735520517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb52419f6c0 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52a3e8640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb524100b20 0x7fb52419a710 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9f0 tx=0x7fb52000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb51b7fe640 1 -- 172.21.15.37:0/3735520517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb52000cd80 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb51b7fe640 1 -- 172.21.15.37:0/3735520517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb520010910 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb51b7fe640 1 -- 172.21.15.37:0/3735520517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb520018990 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb52419f9b0 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.786+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5241a7290 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.787+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb524100f00 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.789+0000 7fb51b7fe640 1 -- 172.21.15.37:0/3735520517 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb520010ac0 con 0x7fb524100b20 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.790+0000 7fb51b7fe640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb5100a33b0 0x7fb5100a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.790+0000 7fb529be7640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb5100a33b0 0x7fb5100a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:32.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.790+0000 7fb51b7fe640 1 -- 172.21.15.37:0/3735520517 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb5200d04a0 con 0x7fb524100b20 2024-09-19T06:49:32.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.790+0000 7fb529be7640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb5100a33b0 0x7fb5100a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb52410ee00 tx=0x7fb514005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:32.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:32.793+0000 7fb51b7fe640 1 -- 172.21.15.37:0/3735520517 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb52009c800 con 0x7fb524100b20 2024-09-19T06:49:32.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:32 smithi151 ceph-mon[35236]: pgmap v241: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.8 MiB/s wr, 79 op/s 2024-09-19T06:49:33.016 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.015+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb524109600 con 0x7fb524100b20 2024-09-19T06:49:33.017 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.016+0000 7fb51b7fe640 1 -- 172.21.15.37:0/3735520517 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11907 (secure 0 0 0) 0x7fb5200a0b00 con 0x7fb524100b20 2024-09-19T06:49:33.017 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:49:33.018 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"btime":"2024-09-19T06:48:37:250513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:48:36.625010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:49:33.018 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb5100a33b0 msgr2=0x7fb5100a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb5100a33b0 0x7fb5100a5870 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb52410ee00 tx=0x7fb514005f90 comp rx=0 tx=0).stop 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb524100b20 msgr2=0x7fb52419a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb524100b20 0x7fb52419a710 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fb52000e9f0 tx=0x7fb52000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 shutdown_connections 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb52410e500 0x7fb52419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb524101440 0x7fb52419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb5100a33b0 0x7fb5100a5870 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 --2- 172.21.15.37:0/3735520517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb524100b20 0x7fb52419a710 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:33.020 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 >> 172.21.15.37:0/3735520517 conn(0x7fb5240fc7d0 msgr2=0x7fb5240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:33.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 shutdown_connections 2024-09-19T06:49:33.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:33.019+0000 7fb52c64c640 1 -- 172.21.15.37:0/3735520517 wait complete. 2024-09-19T06:49:33.215 INFO:tasks.workunit.client.0.smithi037.stdout: 1 564555 7.69 MB/sec execute 240 sec latency 42.212 ms 2024-09-19T06:49:33.671 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:33 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3735520517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:33.698 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0' 2024-09-19T06:49:33.698 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 145.75 --await-for 45.0 2024-09-19T06:49:33.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:33 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3735520517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:33.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:33 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3735520517' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:33.972 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:34.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 566620 7.70 MB/sec execute 241 sec latency 14.348 ms 2024-09-19T06:49:34.454 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 -- 172.21.15.37:0/471951153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 msgr2=0x7effd4100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 --2- 172.21.15.37:0/471951153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd4100c90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7effc4009900 tx=0x7effc402a410 comp rx=0 tx=0).stop 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 -- 172.21.15.37:0/471951153 shutdown_connections 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 --2- 172.21.15.37:0/471951153 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effd410e460 0x7effd4110850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 --2- 172.21.15.37:0/471951153 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd4101260 0x7effd410df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 --2- 172.21.15.37:0/471951153 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd4100c90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 -- 172.21.15.37:0/471951153 >> 172.21.15.37:0/471951153 conn(0x7effd40fc780 msgr2=0x7effd40feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 -- 172.21.15.37:0/471951153 shutdown_connections 2024-09-19T06:49:34.455 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 -- 172.21.15.37:0/471951153 wait complete. 2024-09-19T06:49:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.454+0000 7effd9377640 1 Processor -- start 2024-09-19T06:49:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd9377640 1 -- start start 2024-09-19T06:49:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd9377640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd419a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd9377640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effd4101260 0x7effd419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.457 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd9377640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd410e460 0x7effd419ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd9377640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effd41141c0 con 0x7effd41008b0 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd9377640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effd4114040 con 0x7effd410e460 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd9377640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effd4114340 con 0x7effd4101260 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd410e460 0x7effd419ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd419a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd410e460 0x7effd419ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43614/0 (socket says 172.21.15.37:43614) 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd419a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57868/0 (socket says 172.21.15.37:57868) 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 -- 172.21.15.37:0/2673307500 learned_addr learned my addr 172.21.15.37:0/2673307500 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd27fc640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effd4101260 0x7effd419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 -- 172.21.15.37:0/2673307500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effd4101260 msgr2=0x7effd419ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.458 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effd4101260 0x7effd419ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 -- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 msgr2=0x7effd419a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd419a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd37fe640 1 -- 172.21.15.37:0/2673307500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effd419f5f0 con 0x7effd410e460 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.455+0000 7effd2ffd640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.456+0000 7effd37fe640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd410e460 0x7effd419ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7effcc009fd0 tx=0x7effcc00ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.456+0000 7effbbfff640 1 -- 172.21.15.37:0/2673307500 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc004d20 con 0x7effd410e460 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.456+0000 7effbbfff640 1 -- 172.21.15.37:0/2673307500 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effcc005020 con 0x7effd410e460 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.456+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effd419f8e0 con 0x7effd410e460 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.456+0000 7effbbfff640 1 -- 172.21.15.37:0/2673307500 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effcc0116d0 con 0x7effd410e460 2024-09-19T06:49:34.459 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.456+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effd41a71c0 con 0x7effd410e460 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.458+0000 7effbbfff640 1 -- 172.21.15.37:0/2673307500 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7effcc0051d0 con 0x7effd410e460 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 Processor -- start 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 -- start start 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d30cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d3174f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d319e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55870d271160 con 0x55870d317a30 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55870d218720 con 0x55870d30c850 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effdb750740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55870d26d120 con 0x55870d30d170 2024-09-19T06:49:34.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d30cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effbbfff640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effbc0a33b0 0x7effbc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d30cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43620/0 (socket says 172.21.15.37:43620) 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 -- 172.21.15.37:0/2986783689 learned_addr learned my addr 172.21.15.37:0/2986783689 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d319e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd2ffd640 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d3174f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 -- 172.21.15.37:0/2986783689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 msgr2=0x55870d3174f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effbbfff640 1 -- 172.21.15.37:0/2673307500 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7effcc0d0260 con 0x7effd410e460 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d3174f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 -- 172.21.15.37:0/2986783689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 msgr2=0x55870d319e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d319e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.466 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd37fe640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effbc0a33b0 0x7effbc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd27fc640 1 -- 172.21.15.37:0/2986783689 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55870cfe2ba0 con 0x55870d30c850 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.459+0000 7effd2ffd640 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d3174f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.460+0000 7effd27fc640 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d30cc30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7effc8009fd0 tx=0x7effc8005850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2fd41f3a5f4caac5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.460+0000 7effb97fa640 1 -- 172.21.15.37:0/2986783689 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8004030 con 0x55870d30c850 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.460+0000 7effb97fa640 1 -- 172.21.15.37:0/2986783689 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc8005500 con 0x55870d30c850 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.460+0000 7effb97fa640 1 -- 172.21.15.37:0/2986783689 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc8004d90 con 0x55870d30c850 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.460+0000 7effd37fe640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effbc0a33b0 0x7effbc0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7effcc0989c0 tx=0x7effcc005840 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 -- 172.21.15.37:0/2986783689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 msgr2=0x55870d30cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d30cc30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7effc8009fd0 tx=0x7effc8005850 comp rx=0 tx=0).stop 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 -- 172.21.15.37:0/2986783689 shutdown_connections 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d319e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.467 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d3174f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 --2- 172.21.15.37:0/2986783689 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d30cc30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 -- 172.21.15.37:0/2986783689 >> 172.21.15.37:0/2986783689 conn(0x55870d308de0 msgr2=0x55870d3091c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 -- 172.21.15.37:0/2986783689 shutdown_connections 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 -- 172.21.15.37:0/2986783689 wait complete. 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 -- start start 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.461+0000 7effdb750740 1 Processor -- start 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effdb750740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d3148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effdb750740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d30d170 0x55870d314e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effdb750740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d317a30 0x55870d315350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effdb750740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55870d31a600 con 0x55870d30d170 2024-09-19T06:49:34.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effdb750740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55870d1f9900 con 0x55870d30c850 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effdb750740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55870d31a780 con 0x55870d317a30 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effdb750740 10 client.0 _ll_register_callbacks cb 0x55870d1f7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d30d170 0x55870d314e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d3148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d3148d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43622/0 (socket says 172.21.15.37:43622) 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 -- 172.21.15.37:0/2357210549 learned_addr learned my addr 172.21.15.37:0/2357210549 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d317a30 0x55870d315350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 -- 172.21.15.37:0/2357210549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d317a30 msgr2=0x55870d315350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d317a30 0x55870d315350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 -- 172.21.15.37:0/2357210549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d30d170 msgr2=0x55870d314e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d30d170 0x55870d314e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 -- 172.21.15.37:0/2357210549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870d239c50 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd2ffd640 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d30d170 0x55870d314e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effd27fc640 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d3148d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7effc8009800 tx=0x7effc8004970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=482f07ffeeb1a0a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.462+0000 7effb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effb6ff5640 1 -- 172.21.15.37:0/2357210549 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc800f790 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effb6ff5640 1 -- 172.21.15.37:0/2357210549 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7effc800f930 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc8005500 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effb6ff5640 1 -- 172.21.15.37:0/2357210549 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7effc8035050 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effdb750740 10 client.27032 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55870d3261d0 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effb6ff5640 1 -- 172.21.15.37:0/2357210549 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc8005500 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effb6ff5640 1 -- 172.21.15.37:0/2357210549 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7effc802d020 con 0x55870d30c850 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effdb750740 10 client.27032 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:34.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effdb750740 10 client.27032 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:49:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effdb750740 20 client.27032 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.463+0000 7effdb750740 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d3275d0 0x55870d3299c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.464+0000 7effdb750740 4 client.27032 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.464+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55870d27e890 con 0x55870d3275d0 2024-09-19T06:49:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.464+0000 7effd2ffd640 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d3275d0 0x55870d3299c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.464+0000 7effd2ffd640 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d3275d0 0x55870d3299c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f4c1cb1beb025ae server_cookie=35d9e9ae1392d43 in_seq=0 out_seq=0 2024-09-19T06:49:34.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.464+0000 7effb6ff5640 10 client.27032 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.465+0000 7effb6ff5640 1 -- 172.21.15.37:0/2357210549 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55870d27e890 con 0x55870d3275d0 2024-09-19T06:49:34.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.465+0000 7effb6ff5640 10 client.27032 handle_command_reply: tid=0 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.465+0000 7effdb750740 1 client.27032 shutdown 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.465+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 msgr2=0x55870d3148d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.465+0000 7effdb750740 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d3148d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7effc8009800 tx=0x7effc8004970 comp rx=0 tx=0).stop 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 shutdown_connections 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d3275d0 0x55870d3299c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d317a30 0x55870d315350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d30d170 0x55870d314e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effb6ff5640 0 client.27032 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 --2- 172.21.15.37:0/2357210549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d3148d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 >> 172.21.15.37:0/2357210549 conn(0x55870d308de0 msgr2=0x55870d30eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 shutdown_connections 2024-09-19T06:49:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 1 -- 172.21.15.37:0/2357210549 wait complete. 2024-09-19T06:49:34.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.466+0000 7effdb750740 20 client.27032 trim_cache size 0 max 16384 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 Processor -- start 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 -- start start 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d32f150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d32f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d311b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effd41b25d0 con 0x55870d317a30 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effd41afe40 con 0x55870d30c850 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effdb750740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effd41adcb0 con 0x55870d30d170 2024-09-19T06:49:34.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.483+0000 7effd27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d32f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d311b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d32f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d311b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57890/0 (socket says 172.21.15.37:57890) 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd27fc640 1 -- 172.21.15.37:0/197268267 learned_addr learned my addr 172.21.15.37:0/197268267 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd2ffd640 1 -- 172.21.15.37:0/197268267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 msgr2=0x55870d32f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd2ffd640 1 --2- 172.21.15.37:0/197268267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d32f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd2ffd640 1 -- 172.21.15.37:0/197268267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 msgr2=0x55870d311b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd2ffd640 1 --2- 172.21.15.37:0/197268267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d311b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd2ffd640 1 -- 172.21.15.37:0/197268267 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55870d30cfa0 con 0x55870d30d170 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effd2ffd640 1 --2- 172.21.15.37:0/197268267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d32f690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effc402aa00 tx=0x7effc4031510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b39e2af95bb8de0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effb8ff9640 1 -- 172.21.15.37:0/197268267 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc404a070 con 0x55870d30d170 2024-09-19T06:49:34.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effb8ff9640 1 -- 172.21.15.37:0/197268267 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc4004760 con 0x55870d30d170 2024-09-19T06:49:34.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.484+0000 7effb8ff9640 1 -- 172.21.15.37:0/197268267 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc4035040 con 0x55870d30d170 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 -- 172.21.15.37:0/197268267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 msgr2=0x55870d32f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 --2- 172.21.15.37:0/197268267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d32f690 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7effc402aa00 tx=0x7effc4031510 comp rx=0 tx=0).stop 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 -- 172.21.15.37:0/197268267 shutdown_connections 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 --2- 172.21.15.37:0/197268267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d311b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 --2- 172.21.15.37:0/197268267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d32f690 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 --2- 172.21.15.37:0/197268267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d32f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 -- 172.21.15.37:0/197268267 >> 172.21.15.37:0/197268267 conn(0x55870d308de0 msgr2=0x55870d313070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 -- 172.21.15.37:0/197268267 shutdown_connections 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.485+0000 7effdb750740 1 -- 172.21.15.37:0/197268267 wait complete. 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 -- start start 2024-09-19T06:49:34.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 Processor -- start 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d31b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d31bca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d31c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7effd41ae300 con 0x55870d317a30 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7effd41af770 con 0x55870d30c850 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7effd41ae9d0 con 0x55870d30d170 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effdb750740 10 client.0 _ll_register_callbacks cb 0x7effc8009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d31c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d31c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57892/0 (socket says 172.21.15.37:57892) 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 -- 172.21.15.37:0/1952718748 learned_addr learned my addr 172.21.15.37:0/1952718748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d31b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd2ffd640 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d31bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 -- 172.21.15.37:0/1952718748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 msgr2=0x55870d31bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d31bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 -- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 msgr2=0x55870d31b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d31b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.486+0000 7effd27fc640 1 -- 172.21.15.37:0/1952718748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55870d391170 con 0x55870d317a30 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effd27fc640 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d31c1e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7effc8001eb0 tx=0x7effc80048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64dec6e1282c4fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effb6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effb6ff5640 1 -- 172.21.15.37:0/1952718748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effc80112f0 con 0x55870d317a30 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effb6ff5640 1 -- 172.21.15.37:0/1952718748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7effc801f440 con 0x55870d317a30 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7effc4004760 con 0x55870d317a30 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effb6ff5640 1 -- 172.21.15.37:0/1952718748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7effc801f640 con 0x55870d317a30 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 10 client.17670 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:34.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55870d392410 con 0x55870d317a30 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effb6ff5640 1 -- 172.21.15.37:0/1952718748 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effc4004760 con 0x55870d317a30 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effb6ff5640 1 -- 172.21.15.37:0/1952718748 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7effc802d040 con 0x55870d317a30 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 10 client.17670 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 10 client.17670 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 20 client.17670 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d392b50 0x55870d394f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 4 client.17670 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}] 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.487+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 145.75, "await_for": 45.0}) -- 0x55870d278be0 con 0x55870d392b50 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.488+0000 7effd2ffd640 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d392b50 0x55870d394f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.488+0000 7effd2ffd640 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d392b50 0x55870d394f90 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58e3a1cc2d99cc64 server_cookie=c443a0b556cc6d20 in_seq=0 out_seq=0 2024-09-19T06:49:34.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.488+0000 7effb6ff5640 10 client.17670 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effb6ff5640 1 -- 172.21.15.37:0/1952718748 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55870d278be0 con 0x55870d392b50 2024-09-19T06:49:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effb6ff5640 10 client.17670 handle_command_reply: tid=0 2024-09-19T06:49:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 client.17670 shutdown 2024-09-19T06:49:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 msgr2=0x55870d31c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d31c1e0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7effc8001eb0 tx=0x7effc80048d0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 shutdown_connections 2024-09-19T06:49:34.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55870d392b50 0x55870d394f90 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55870d317a30 0x55870d31c1e0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55870d30d170 0x55870d31bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effb6ff5640 0 client.17670 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 --2- 172.21.15.37:0/1952718748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55870d30c850 0x55870d31b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 >> 172.21.15.37:0/1952718748 conn(0x55870d308de0 msgr2=0x55870d30e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 shutdown_connections 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 1 -- 172.21.15.37:0/1952718748 wait complete. 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.589+0000 7effdb750740 20 client.17670 trim_cache size 0 max 16384 2024-09-19T06:49:34.591 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 36, 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 30, 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "506b7504": { 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 30, 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:49:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 145.8, 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:49:34.594 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:49:34.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effbc0a33b0 msgr2=0x7effbc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effbc0a33b0 0x7effbc0a5870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7effcc0989c0 tx=0x7effcc005840 comp rx=0 tx=0).stop 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd410e460 msgr2=0x7effd419ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd410e460 0x7effd419ef10 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7effcc009fd0 tx=0x7effcc00ee60 comp rx=0 tx=0).stop 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 shutdown_connections 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7effd410e460 0x7effd419ef10 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7effd4101260 0x7effd419ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7effbc0a33b0 0x7effbc0a5870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 --2- 172.21.15.37:0/2673307500 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7effd41008b0 0x7effd419a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 >> 172.21.15.37:0/2673307500 conn(0x7effd40fc780 msgr2=0x7effd40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 shutdown_connections 2024-09-19T06:49:34.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:34.590+0000 7effd9377640 1 -- 172.21.15.37:0/2673307500 wait complete. 2024-09-19T06:49:34.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:34 smithi037 ceph-mon[31211]: pgmap v242: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.0 MiB/s wr, 152 op/s 2024-09-19T06:49:34.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:34 smithi096 ceph-mon[35004]: pgmap v242: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.0 MiB/s wr, 152 op/s 2024-09-19T06:49:34.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:34 smithi151 ceph-mon[35236]: pgmap v242: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 3.0 MiB/s wr, 152 op/s 2024-09-19T06:49:35.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.70 MB/sec execute 242 sec latency 620.313 ms 2024-09-19T06:49:35.379 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '506b7504', quiesce took 3.5 seconds. Will release after: 22.25 2024-09-19T06:49:36.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.67 MB/sec execute 243 sec latency 1620.363 ms 2024-09-19T06:49:36.655 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:36 smithi037 ceph-mon[31211]: pgmap v243: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:49:36.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:36 smithi096 ceph-mon[35004]: pgmap v243: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:49:36.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:36 smithi151 ceph-mon[35236]: pgmap v243: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:49:37.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.63 MB/sec execute 244 sec latency 2620.704 ms 2024-09-19T06:49:38.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.60 MB/sec execute 245 sec latency 3620.862 ms 2024-09-19T06:49:38.665 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:38 smithi151 ceph-mon[35236]: pgmap v244: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:49:38.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:38 smithi096 ceph-mon[35004]: pgmap v244: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:49:38.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:38 smithi037 ceph-mon[31211]: pgmap v244: 97 pgs: 97 active+clean; 63 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 2.1 MiB/s wr, 107 op/s 2024-09-19T06:49:39.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.57 MB/sec execute 246 sec latency 4620.992 ms 2024-09-19T06:49:40.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.54 MB/sec execute 247 sec latency 5621.117 ms 2024-09-19T06:49:40.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:40 smithi096 ceph-mon[35004]: pgmap v245: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 163 op/s 2024-09-19T06:49:40.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:40 smithi037 ceph-mon[31211]: pgmap v245: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 163 op/s 2024-09-19T06:49:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/194464611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 msgr2=0x7fb7b8101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/194464611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8101f30 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fb7ac009930 tx=0x7fb7ac02a420 comp rx=0 tx=0).stop 2024-09-19T06:49:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/194464611 shutdown_connections 2024-09-19T06:49:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/194464611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7b8102470 0x7fb7b810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/194464611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8101f30 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/194464611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7b8107b70 0x7fb7b8107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/194464611 >> 172.21.15.37:0/194464611 conn(0x7fb7b80fd7d0 msgr2=0x7fb7b80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/194464611 shutdown_connections 2024-09-19T06:49:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.851+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/194464611 wait complete. 2024-09-19T06:49:40.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 Processor -- start 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 -- start start 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7b8102470 0x7fb7b8198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7b8107b70 0x7fb7b819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8111ee0 con 0x7fb7b8101ad0 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8111d60 con 0x7fb7b8102470 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7bdfc0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b8112060 con 0x7fb7b8107b70 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34198/0 (socket says 172.21.15.37:34198) 2024-09-19T06:49:40.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7b77fe640 1 -- 172.21.15.37:0/2780092563 learned_addr learned my addr 172.21.15.37:0/2780092563 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7b6ffd640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7b8102470 0x7fb7b8198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.852+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7b8107b70 0x7fb7b819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b77fe640 1 -- 172.21.15.37:0/2780092563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7b8107b70 msgr2=0x7fb7b819cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7b8107b70 0x7fb7b819cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b77fe640 1 -- 172.21.15.37:0/2780092563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7b8102470 msgr2=0x7fb7b8198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7b8102470 0x7fb7b8198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b77fe640 1 -- 172.21.15.37:0/2780092563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7b819d320 con 0x7fb7b8101ad0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8198330 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800ebc0 tx=0x7fb7a800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b4ff9640 1 -- 172.21.15.37:0/2780092563 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8010630 con 0x7fb7b8101ad0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b4ff9640 1 -- 172.21.15.37:0/2780092563 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a8010cb0 con 0x7fb7b8101ad0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7b819d610 con 0x7fb7b8101ad0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7b4ff9640 1 -- 172.21.15.37:0/2780092563 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a8018bd0 con 0x7fb7b8101ad0 2024-09-19T06:49:40.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.853+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7b81a4ea0 con 0x7fb7b8101ad0 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.855+0000 7fb7b4ff9640 1 -- 172.21.15.37:0/2780092563 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb7a80107d0 con 0x7fb7b8101ad0 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 Processor -- start 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 -- start start 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4902770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49050a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564bf485c2f0 con 0x564bf48f7af0 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bf4804270 con 0x564bf4902cb0 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bf48582b0 con 0x564bf48f8410 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34212/0 (socket says 172.21.15.37:34212) 2024-09-19T06:49:40.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b7fff640 1 -- 172.21.15.37:0/249174357 learned_addr learned my addr 172.21.15.37:0/249174357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b4ff9640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7a00a34e0 0x7fb7a00a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4902770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b6ffd640 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49050a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b7fff640 1 -- 172.21.15.37:0/249174357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 msgr2=0x564bf4902770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4902770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.856+0000 7fb7b7fff640 1 -- 172.21.15.37:0/249174357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 msgr2=0x564bf49050a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49050a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb7b7fff640 1 -- 172.21.15.37:0/249174357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf4803340 con 0x564bf48f7af0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4902770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb7b4ff9640 1 -- 172.21.15.37:0/2780092563 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb7a80d02c0 con 0x7fb7b8101ad0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7a00a34e0 0x7fb7a00a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48f7ed0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b880 tx=0x7fb7b000bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a5b38e22c7c70ca server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb79dffb640 1 -- 172.21.15.37:0/249174357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0024070 con 0x564bf48f7af0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb79dffb640 1 -- 172.21.15.37:0/249174357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b0010b20 con 0x564bf48f7af0 2024-09-19T06:49:40.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.857+0000 7fb79dffb640 1 -- 172.21.15.37:0/249174357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00118e0 con 0x564bf48f7af0 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/249174357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 msgr2=0x564bf48f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48f7ed0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b880 tx=0x7fb7b000bd40 comp rx=0 tx=0).stop 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7a00a34e0 0x7fb7a00a59a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0021000 tx=0x7fb7b001e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/249174357 shutdown_connections 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49050a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4902770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/249174357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48f7ed0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/249174357 >> 172.21.15.37:0/249174357 conn(0x564bf48f4060 msgr2=0x564bf48f4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/249174357 shutdown_connections 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/249174357 wait complete. 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 -- start start 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.858+0000 7fb7c0e2e740 1 Processor -- start 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48ffb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4900090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49005d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564bf4905880 con 0x564bf48f7af0 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564bf47e5290 con 0x564bf4902cb0 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564bf4905a00 con 0x564bf48f8410 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7c0e2e740 10 client.0 _ll_register_callbacks cb 0x564bf4816140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48ffb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48ffb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34216/0 (socket says 172.21.15.37:34216) 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 -- 172.21.15.37:0/1701912134 learned_addr learned my addr 172.21.15.37:0/1701912134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49005d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b6ffd640 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4900090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 -- 172.21.15.37:0/1701912134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 msgr2=0x564bf4900090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4900090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 -- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 msgr2=0x564bf49005d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49005d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 -- 172.21.15.37:0/1701912134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf456f210 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48ffb50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8098a20 tx=0x7fb7a809c390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50fa78559428401 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.859+0000 7fb796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb796ffd640 1 -- 172.21.15.37:0/1701912134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a80d0030 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb796ffd640 1 -- 172.21.15.37:0/1701912134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb7a809ba10 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7b0010b20 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb796ffd640 1 -- 172.21.15.37:0/1701912134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb7a80cde00 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 10 client.17682 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bf463b620 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb796ffd640 1 -- 172.21.15.37:0/1701912134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7b0010b20 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb796ffd640 1 -- 172.21.15.37:0/1701912134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb7a809b480 con 0x564bf48f7af0 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 10 client.17682 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:40.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 10 client.17682 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 20 client.17682 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf4912b30 0x564bf4914f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 4 client.17682 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bf4869a20 con 0x564bf4912b30 2024-09-19T06:49:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.860+0000 7fb7b6ffd640 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf4912b30 0x564bf4914f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.861+0000 7fb7b6ffd640 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf4912b30 0x564bf4914f20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8074ce35674b09a0 server_cookie=c6928fe766046b98 in_seq=0 out_seq=0 2024-09-19T06:49:40.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.861+0000 7fb796ffd640 10 client.17682 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:40.867 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:40 smithi151 ceph-mon[35236]: pgmap v245: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.8 MiB/s wr, 163 op/s 2024-09-19T06:49:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb796ffd640 1 -- 172.21.15.37:0/1701912134 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564bf4869a20 con 0x564bf4912b30 2024-09-19T06:49:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb796ffd640 10 client.17682 handle_command_reply: tid=0 2024-09-19T06:49:40.868 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 client.17682 shutdown 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 msgr2=0x564bf48ffb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48ffb50 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8098a20 tx=0x7fb7a809c390 comp rx=0 tx=0).stop 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 shutdown_connections 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf4912b30 0x564bf4914f20 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf4902cb0 0x564bf49005d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f8410 0x564bf4900090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb796ffd640 0 client.17682 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/1701912134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf48ffb50 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 >> 172.21.15.37:0/1701912134 conn(0x564bf48f4060 msgr2=0x564bf48f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 shutdown_connections 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/1701912134 wait complete. 2024-09-19T06:49:40.869 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.862+0000 7fb7c0e2e740 20 client.17682 trim_cache size 0 max 16384 2024-09-19T06:49:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 Processor -- start 2024-09-19T06:49:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 -- start start 2024-09-19T06:49:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf4900400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf491a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf4902cb0 0x564bf491ad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81b0270 con 0x564bf48f7af0 2024-09-19T06:49:40.879 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81adae0 con 0x564bf48f8410 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81ab950 con 0x564bf4902cb0 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf4900400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf4900400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34220/0 (socket says 172.21.15.37:34220) 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 -- 172.21.15.37:0/4204603326 learned_addr learned my addr 172.21.15.37:0/4204603326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf4902cb0 0x564bf491ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 -- 172.21.15.37:0/4204603326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf4902cb0 msgr2=0x564bf491ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b6ffd640 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf491a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf4902cb0 0x564bf491ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 -- 172.21.15.37:0/4204603326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 msgr2=0x564bf491a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf491a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 -- 172.21.15.37:0/4204603326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bf48fa940 con 0x564bf48f7af0 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf4902cb0 0x564bf491ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:40.880 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.877+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf4900400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009a80 tx=0x7fb7a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ca98c7d0c2ae8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb79d7fa640 1 -- 172.21.15.37:0/4204603326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a809c100 con 0x564bf48f7af0 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb79d7fa640 1 -- 172.21.15.37:0/4204603326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a809c5c0 con 0x564bf48f7af0 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb79d7fa640 1 -- 172.21.15.37:0/4204603326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7a80a1b70 con 0x564bf48f7af0 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/4204603326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 msgr2=0x564bf4900400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf4900400 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fb7a8009a80 tx=0x7fb7a800cf90 comp rx=0 tx=0).stop 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/4204603326 shutdown_connections 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf4902cb0 0x564bf491ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf491a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/4204603326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf48f7af0 0x564bf4900400 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/4204603326 >> 172.21.15.37:0/4204603326 conn(0x564bf48f4060 msgr2=0x564bf4901bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/4204603326 shutdown_connections 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.878+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/4204603326 wait complete. 2024-09-19T06:49:40.881 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 -- start start 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 Processor -- start 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f7af0 0x564bf490b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf49065f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf4902cb0 0x564bf4906b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81abfa0 con 0x564bf4902cb0 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81ad410 con 0x564bf48f8410 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb7b81ac670 con 0x564bf48f7af0 2024-09-19T06:49:40.884 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7c0e2e740 10 client.0 _ll_register_callbacks cb 0x7fb7a80099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.879+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf4902cb0 0x564bf4906b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf4902cb0 0x564bf4906b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34232/0 (socket says 172.21.15.37:34232) 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 -- 172.21.15.37:0/3864817200 learned_addr learned my addr 172.21.15.37:0/3864817200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b6ffd640 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf49065f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f7af0 0x564bf490b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 -- 172.21.15.37:0/3864817200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f7af0 msgr2=0x564bf490b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f7af0 0x564bf490b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 -- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 msgr2=0x564bf49065f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf49065f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 -- 172.21.15.37:0/3864817200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bf48ef2d0 con 0x564bf4902cb0 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7b7fff640 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf4902cb0 0x564bf4906b30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b880 tx=0x7fb7b0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e26ad0c21bce2b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:40.885 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb796ffd640 1 -- 172.21.15.37:0/3864817200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7b0024040 con 0x564bf4902cb0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb796ffd640 1 -- 172.21.15.37:0/3864817200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb7b0005a70 con 0x564bf4902cb0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb7a809c5c0 con 0x564bf4902cb0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb796ffd640 1 -- 172.21.15.37:0/3864817200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb7b0004160 con 0x564bf4902cb0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7c0e2e740 10 client.17688 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.880+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564bf497d490 con 0x564bf4902cb0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb796ffd640 1 -- 172.21.15.37:0/3864817200 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb7a809c5c0 con 0x564bf4902cb0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb796ffd640 1 -- 172.21.15.37:0/3864817200 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb7b00048c0 con 0x564bf4902cb0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7c0e2e740 10 client.17688 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7c0e2e740 10 client.17688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7c0e2e740 20 client.17688 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf497d9d0 0x564bf497fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7c0e2e740 4 client.17688 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564bf4863d70 con 0x564bf497d9d0 2024-09-19T06:49:40.886 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf497d9d0 0x564bf497fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb796ffd640 10 client.17688 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:40.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.881+0000 7fb7b77fe640 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf497d9d0 0x564bf497fdf0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a345c7afea0698 server_cookie=73094316fe03e7e4 in_seq=0 out_seq=0 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb796ffd640 1 -- 172.21.15.37:0/3864817200 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564bf4863d70 con 0x564bf497d9d0 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb796ffd640 10 client.17688 handle_command_reply: tid=0 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 client.17688 shutdown 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf4902cb0 msgr2=0x564bf4906b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf4902cb0 0x564bf4906b30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fb7b000b880 tx=0x7fb7b0011440 comp rx=0 tx=0).stop 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 shutdown_connections 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564bf497d9d0 0x564bf497fdf0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564bf4902cb0 0x564bf4906b30 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb796ffd640 0 client.17688 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564bf48f8410 0x564bf49065f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 --2- 172.21.15.37:0/3864817200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564bf48f7af0 0x564bf490b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 >> 172.21.15.37:0/3864817200 conn(0x564bf48f4060 msgr2=0x564bf48f9550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 shutdown_connections 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 1 -- 172.21.15.37:0/3864817200 wait complete. 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.954+0000 7fb7c0e2e740 20 client.17688 trim_cache size 0 max 16384 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:49:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:49:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.955+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7a00a34e0 msgr2=0x7fb7a00a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.955+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7a00a34e0 0x7fb7a00a59a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0021000 tx=0x7fb7b001e000 comp rx=0 tx=0).stop 2024-09-19T06:49:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.955+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 msgr2=0x7fb7b8198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.955+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8198330 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800ebc0 tx=0x7fb7a800c6a0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 shutdown_connections 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb7b8107b70 0x7fb7b819cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7b8102470 0x7fb7b8198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7a00a34e0 0x7fb7a00a59a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 --2- 172.21.15.37:0/2780092563 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb7b8101ad0 0x7fb7b8198330 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 >> 172.21.15.37:0/2780092563 conn(0x7fb7b80fd7d0 msgr2=0x7fb7b80fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 shutdown_connections 2024-09-19T06:49:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:40.956+0000 7fb7bdfc0640 1 -- 172.21.15.37:0/2780092563 wait complete. 2024-09-19T06:49:41.216 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.51 MB/sec execute 248 sec latency 6621.217 ms 2024-09-19T06:49:42.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.48 MB/sec execute 249 sec latency 7621.305 ms 2024-09-19T06:49:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:42 smithi096 ceph-mon[35004]: pgmap v246: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-09-19T06:49:42.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:42 smithi037 ceph-mon[31211]: pgmap v246: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-09-19T06:49:42.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:42 smithi151 ceph-mon[35236]: pgmap v246: 97 pgs: 97 active+clean; 65 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 129 op/s 2024-09-19T06:49:43.217 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.45 MB/sec execute 250 sec latency 8621.424 ms 2024-09-19T06:49:44.217 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.42 MB/sec execute 251 sec latency 9621.557 ms 2024-09-19T06:49:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:44 smithi096 ceph-mon[35004]: pgmap v247: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 175 op/s 2024-09-19T06:49:44.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:44 smithi037 ceph-mon[31211]: pgmap v247: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 175 op/s 2024-09-19T06:49:44.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:44 smithi151 ceph-mon[35236]: pgmap v247: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.2 MiB/s wr, 175 op/s 2024-09-19T06:49:45.217 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.39 MB/sec execute 252 sec latency 10621.688 ms 2024-09-19T06:49:46.217 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.36 MB/sec execute 253 sec latency 11621.788 ms 2024-09-19T06:49:46.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:46 smithi037 ceph-mon[31211]: pgmap v248: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 938 KiB/s wr, 102 op/s 2024-09-19T06:49:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:46 smithi096 ceph-mon[35004]: pgmap v248: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 938 KiB/s wr, 102 op/s 2024-09-19T06:49:46.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:46 smithi151 ceph-mon[35236]: pgmap v248: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 938 KiB/s wr, 102 op/s 2024-09-19T06:49:47.217 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.33 MB/sec execute 254 sec latency 12621.892 ms 2024-09-19T06:49:48.217 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.30 MB/sec execute 255 sec latency 13621.942 ms 2024-09-19T06:49:48.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:48 smithi096 ceph-mon[35004]: pgmap v249: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 938 KiB/s wr, 102 op/s 2024-09-19T06:49:48.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:48 smithi037 ceph-mon[31211]: pgmap v249: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 938 KiB/s wr, 102 op/s 2024-09-19T06:49:48.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:48 smithi151 ceph-mon[35236]: pgmap v249: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 712 KiB/s rd, 938 KiB/s wr, 102 op/s 2024-09-19T06:49:49.217 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.28 MB/sec execute 256 sec latency 14622.045 ms 2024-09-19T06:49:50.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.25 MB/sec execute 257 sec latency 15622.168 ms 2024-09-19T06:49:50.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:50 smithi096 ceph-mon[35004]: pgmap v250: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 943 KiB/s wr, 105 op/s 2024-09-19T06:49:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:50 smithi037 ceph-mon[31211]: pgmap v250: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 943 KiB/s wr, 105 op/s 2024-09-19T06:49:50.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:50 smithi151 ceph-mon[35236]: pgmap v250: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 718 KiB/s rd, 943 KiB/s wr, 105 op/s 2024-09-19T06:49:51.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.22 MB/sec execute 258 sec latency 16622.283 ms 2024-09-19T06:49:52.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.19 MB/sec execute 259 sec latency 17622.394 ms 2024-09-19T06:49:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:52 smithi096 ceph-mon[35004]: pgmap v251: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-19T06:49:52.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:52 smithi037 ceph-mon[31211]: pgmap v251: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-19T06:49:52.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:52 smithi151 ceph-mon[35236]: pgmap v251: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-19T06:49:53.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.16 MB/sec execute 260 sec latency 18622.490 ms 2024-09-19T06:49:53.750 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:49:54.025 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 -- 172.21.15.37:0/3142324640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26401035a0 msgr2=0x7f264010ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 --2- 172.21.15.37:0/3142324640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26401035a0 0x7f264010ba80 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f263c009900 tx=0x7f263c02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 -- 172.21.15.37:0/3142324640 shutdown_connections 2024-09-19T06:49:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 --2- 172.21.15.37:0/3142324640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f264010c0d0 0x7f264010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 --2- 172.21.15.37:0/3142324640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f26401035a0 0x7f264010ba80 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 --2- 172.21.15.37:0/3142324640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2640102c80 0x7f2640103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 -- 172.21.15.37:0/3142324640 >> 172.21.15.37:0/3142324640 conn(0x7f2640077990 msgr2=0x7f26400fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 -- 172.21.15.37:0/3142324640 shutdown_connections 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.051+0000 7f2647482640 1 -- 172.21.15.37:0/3142324640 wait complete. 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 Processor -- start 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 -- start start 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2640102c80 0x7f2640198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26401035a0 0x7f2640198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f264010c0d0 0x7f264019cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2640111eb0 con 0x7f2640102c80 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2640111d30 con 0x7f26401035a0 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2647482640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2640112030 con 0x7f264010c0d0 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f264521e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2640102c80 0x7f2640198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f264521e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2640102c80 0x7f2640198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55718/0 (socket says 172.21.15.37:55718) 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f264521e640 1 -- 172.21.15.37:0/2528268585 learned_addr learned my addr 172.21.15.37:0/2528268585 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.052+0000 7f2645a1f640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f264010c0d0 0x7f264019cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f2644a1d640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26401035a0 0x7f2640198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f264521e640 1 -- 172.21.15.37:0/2528268585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f264010c0d0 msgr2=0x7f264019cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f264521e640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f264010c0d0 0x7f264019cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f264521e640 1 -- 172.21.15.37:0/2528268585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26401035a0 msgr2=0x7f2640198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f264521e640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26401035a0 0x7f2640198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f264521e640 1 -- 172.21.15.37:0/2528268585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264019d330 con 0x7f2640102c80 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f264521e640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2640102c80 0x7f2640198340 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f263400d950 tx=0x7f263400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f26327fc640 1 -- 172.21.15.37:0/2528268585 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2634017030 con 0x7f2640102c80 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f26327fc640 1 -- 172.21.15.37:0/2528268585 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2634004020 con 0x7f2640102c80 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f264019d620 con 0x7f2640102c80 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f26327fc640 1 -- 172.21.15.37:0/2528268585 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26340047a0 con 0x7f2640102c80 2024-09-19T06:49:54.057 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.053+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26401a4eb0 con 0x7f2640102c80 2024-09-19T06:49:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 Processor -- start 2024-09-19T06:49:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 -- start start 2024-09-19T06:49:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e9576ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9581770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9581cb0 0x5634e95840a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5634e94db2f0 con 0x5634e9577410 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5634e9483270 con 0x5634e9576af0 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.056+0000 7f264a220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5634e94d72b0 con 0x5634e9581cb0 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e9576ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e9576ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:45550/0 (socket says 172.21.15.37:45550) 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 -- 172.21.15.37:0/4237856131 learned_addr learned my addr 172.21.15.37:0/4237856131 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2644a1d640 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9581cb0 0x5634e95840a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 -- 172.21.15.37:0/4237856131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9581cb0 msgr2=0x5634e95840a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9581cb0 0x5634e95840a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 -- 172.21.15.37:0/4237856131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 msgr2=0x5634e9581770 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9581770 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 -- 172.21.15.37:0/4237856131 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e9482340 con 0x5634e9576af0 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f26327fc640 1 -- 172.21.15.37:0/2528268585 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f26340041d0 con 0x7f2640102c80 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2644a1d640 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9581cb0 0x5634e95840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f2645a1f640 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e9576ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2638009f70 tx=0x7f2638010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc7781afa1eb5b63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f262b7fe640 1 -- 172.21.15.37:0/4237856131 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638016070 con 0x5634e9576af0 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f262b7fe640 1 -- 172.21.15.37:0/4237856131 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638004d20 con 0x5634e9576af0 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.057+0000 7f262b7fe640 1 -- 172.21.15.37:0/4237856131 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638012910 con 0x5634e9576af0 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f26327fc640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f262c0a34e0 0x7f262c0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264521e640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f262c0a34e0 0x7f262c0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f26327fc640 1 -- 172.21.15.37:0/2528268585 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f26340cf600 con 0x7f2640102c80 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264521e640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f262c0a34e0 0x7f262c0a59a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5634e9573ec0 tx=0x7f26340662a0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 -- 172.21.15.37:0/4237856131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 msgr2=0x5634e9576ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e9576ed0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2638009f70 tx=0x7f2638010800 comp rx=0 tx=0).stop 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 -- 172.21.15.37:0/4237856131 shutdown_connections 2024-09-19T06:49:54.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9581cb0 0x5634e95840a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9581770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 --2- 172.21.15.37:0/4237856131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e9576ed0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 -- 172.21.15.37:0/4237856131 >> 172.21.15.37:0/4237856131 conn(0x5634e9573060 msgr2=0x5634e9573440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 -- 172.21.15.37:0/4237856131 shutdown_connections 2024-09-19T06:49:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.058+0000 7f264a220740 1 -- 172.21.15.37:0/4237856131 wait complete. 2024-09-19T06:49:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 -- start start 2024-09-19T06:49:54.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 Processor -- start 2024-09-19T06:49:54.066 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e957eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9577410 0x5634e957f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9581cb0 0x5634e957f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5634e9584880 con 0x5634e9581cb0 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5634e9464290 con 0x5634e9576af0 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5634e9584a00 con 0x5634e9577410 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.059+0000 7f264a220740 10 client.0 _ll_register_callbacks cb 0x5634e9495140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9581cb0 0x5634e957f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9581cb0 0x5634e957f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55730/0 (socket says 172.21.15.37:55730) 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 -- 172.21.15.37:0/409668561 learned_addr learned my addr 172.21.15.37:0/409668561 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e957eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2644a1d640 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9577410 0x5634e957f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.067 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 -- 172.21.15.37:0/409668561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9577410 msgr2=0x5634e957f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9577410 0x5634e957f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 -- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 msgr2=0x5634e957eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e957eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 -- 172.21.15.37:0/409668561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e91ee210 con 0x5634e9581cb0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2645a1f640 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9581cb0 0x5634e957f5e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f26380125f0 tx=0x7f26380124f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f959b19afd7388 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2628ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2628ff9640 1 -- 172.21.15.37:0/409668561 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2638016030 con 0x5634e9581cb0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2628ff9640 1 -- 172.21.15.37:0/409668561 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f2638012c90 con 0x5634e9581cb0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2638004d20 con 0x5634e9581cb0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f2628ff9640 1 -- 172.21.15.37:0/409668561 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f2638023040 con 0x5634e9581cb0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f264a220740 10 client.17697 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.060+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5634e92ba620 con 0x5634e9581cb0 2024-09-19T06:49:54.068 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f2628ff9640 1 -- 172.21.15.37:0/409668561 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2638004d20 con 0x5634e9581cb0 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f2628ff9640 1 -- 172.21.15.37:0/409668561 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f263800f040 con 0x5634e9581cb0 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f264a220740 10 client.17697 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f264a220740 10 client.17697 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f264a220740 20 client.17697 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f264a220740 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e9591b30 0x5634e9593f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f264a220740 4 client.17697 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634e94e8a20 con 0x5634e9591b30 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.061+0000 7f2644a1d640 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e9591b30 0x5634e9593f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.062+0000 7f2644a1d640 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e9591b30 0x5634e9593f20 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c04300e35cd5c051 server_cookie=9a1587820e931b2c in_seq=0 out_seq=0 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.062+0000 7f2628ff9640 10 client.17697 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.062+0000 7f2628ff9640 1 -- 172.21.15.37:0/409668561 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5634e94e8a20 con 0x5634e9591b30 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.062+0000 7f2628ff9640 10 client.17697 handle_command_reply: tid=0 2024-09-19T06:49:54.069 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 client.17697 shutdown 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9581cb0 msgr2=0x5634e957f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9581cb0 0x5634e957f5e0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f26380125f0 tx=0x7f26380124f0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 shutdown_connections 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e9591b30 0x5634e9593f20 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9581cb0 0x5634e957f5e0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9577410 0x5634e957f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f2628ff9640 0 client.17697 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 --2- 172.21.15.37:0/409668561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9576af0 0x5634e957eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 >> 172.21.15.37:0/409668561 conn(0x5634e9573060 msgr2=0x5634e9578dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 shutdown_connections 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 1 -- 172.21.15.37:0/409668561 wait complete. 2024-09-19T06:49:54.070 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.063+0000 7f264a220740 20 client.17697 trim_cache size 0 max 16384 2024-09-19T06:49:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.077+0000 7f264a220740 1 Processor -- start 2024-09-19T06:49:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f264a220740 1 -- start start 2024-09-19T06:49:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f264a220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e957f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f264a220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9599610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f264a220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9599b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f264a220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26401b0280 con 0x5634e9577410 2024-09-19T06:49:54.085 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f264a220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26401adaf0 con 0x5634e9581cb0 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f264a220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ab960 con 0x5634e9576af0 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9599610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9599610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55734/0 (socket says 172.21.15.37:55734) 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 -- 172.21.15.37:0/3684914615 learned_addr learned my addr 172.21.15.37:0/3684914615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2645a1f640 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9599b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2645a1f640 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e957f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 -- 172.21.15.37:0/3684914615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 msgr2=0x5634e957f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e957f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.086 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 -- 172.21.15.37:0/3684914615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 msgr2=0x5634e9599b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9599b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 -- 172.21.15.37:0/3684914615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634e9572e90 con 0x5634e9577410 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f2644a1d640 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9599610 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f263c009230 tx=0x7f263c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f9019a9dccbd85b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.078+0000 7f262affd640 1 -- 172.21.15.37:0/3684914615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c035040 con 0x5634e9577410 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f262affd640 1 -- 172.21.15.37:0/3684914615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263c0319d0 con 0x5634e9577410 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f262affd640 1 -- 172.21.15.37:0/3684914615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c0047a0 con 0x5634e9577410 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 -- 172.21.15.37:0/3684914615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 msgr2=0x5634e9599610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9599610 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f263c009230 tx=0x7f263c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 -- 172.21.15.37:0/3684914615 shutdown_connections 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9599b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9599610 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.087 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 --2- 172.21.15.37:0/3684914615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e957f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 -- 172.21.15.37:0/3684914615 >> 172.21.15.37:0/3684914615 conn(0x5634e9573060 msgr2=0x5634e9580be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 -- 172.21.15.37:0/3684914615 shutdown_connections 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 -- 172.21.15.37:0/3684914615 wait complete. 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.079+0000 7f264a220740 1 -- start start 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 1 Processor -- start 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e9585a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9585f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9586480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f26401abfb0 con 0x5634e9577410 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ad420 con 0x5634e9581cb0 2024-09-19T06:49:54.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f26401ac680 con 0x5634e9576af0 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f264a220740 10 client.0 _ll_register_callbacks cb 0x7f2638009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9585f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9585f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55748/0 (socket says 172.21.15.37:55748) 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 -- 172.21.15.37:0/2946795529 learned_addr learned my addr 172.21.15.37:0/2946795529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2645a1f640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9586480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2645a1f640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e9585a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 -- 172.21.15.37:0/2946795529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 msgr2=0x5634e9585a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e9585a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 -- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 msgr2=0x5634e9586480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9586480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2644a1d640 1 -- 172.21.15.37:0/2946795529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634e9579940 con 0x5634e9577410 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.080+0000 7f2645a1f640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e9585a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2644a1d640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9585f40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f263c02a9a0 tx=0x7f263c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9572dfa17ba6ae06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2628ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2628ff9640 1 -- 172.21.15.37:0/2946795529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c035070 con 0x5634e9577410 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2628ff9640 1 -- 172.21.15.37:0/2946795529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f263c043040 con 0x5634e9577410 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f263c0319d0 con 0x5634e9577410 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2628ff9640 1 -- 172.21.15.37:0/2946795529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f263c005020 con 0x5634e9577410 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 10 client.17703 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5634e9573cf0 con 0x5634e9577410 2024-09-19T06:49:54.090 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2628ff9640 1 -- 172.21.15.37:0/2946795529 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f263c0319d0 con 0x5634e9577410 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2628ff9640 1 -- 172.21.15.37:0/2946795529 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f263c004af0 con 0x5634e9577410 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 10 client.17703 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 10 client.17703 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 20 client.17703 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e95fcc60 0x5634e95ff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 4 client.17703 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5634e94e2d70 con 0x5634e95fcc60 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.081+0000 7f2645a1f640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e95fcc60 0x5634e95ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.082+0000 7f2628ff9640 10 client.17703 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.082+0000 7f2645a1f640 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e95fcc60 0x5634e95ff080 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f47eb1d381c2be5 server_cookie=b82952a2e13e1636 in_seq=0 out_seq=0 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f2628ff9640 1 -- 172.21.15.37:0/2946795529 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5634e94e2d70 con 0x5634e95fcc60 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f2628ff9640 10 client.17703 handle_command_reply: tid=0 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 client.17703 shutdown 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 msgr2=0x5634e9585f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9585f40 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f263c02a9a0 tx=0x7f263c02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 shutdown_connections 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5634e95fcc60 0x5634e95ff080 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634e9581cb0 0x5634e9586480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5634e9577410 0x5634e9585f40 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f2628ff9640 0 client.17703 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 --2- 172.21.15.37:0/2946795529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5634e9576af0 0x5634e9585a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.103+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 >> 172.21.15.37:0/2946795529 conn(0x5634e9573060 msgr2=0x5634e95719a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 shutdown_connections 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f264a220740 1 -- 172.21.15.37:0/2946795529 wait complete. 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f264a220740 20 client.17703 trim_cache size 0 max 16384 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:49:54.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:49:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:49:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f262c0a34e0 msgr2=0x7f262c0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f262c0a34e0 0x7f262c0a59a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x5634e9573ec0 tx=0x7f26340662a0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2640102c80 msgr2=0x7f2640198340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2640102c80 0x7f2640198340 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f263400d950 tx=0x7f263400de10 comp rx=0 tx=0).stop 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 shutdown_connections 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f264010c0d0 0x7f264019cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26401035a0 0x7f2640198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f262c0a34e0 0x7f262c0a59a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 --2- 172.21.15.37:0/2528268585 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2640102c80 0x7f2640198340 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 >> 172.21.15.37:0/2528268585 conn(0x7f2640077990 msgr2=0x7f26400fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 shutdown_connections 2024-09-19T06:49:54.109 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:49:54.104+0000 7f2647482640 1 -- 172.21.15.37:0/2528268585 wait complete. 2024-09-19T06:49:54.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.14 MB/sec execute 261 sec latency 19622.613 ms 2024-09-19T06:49:54.649 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:54 smithi037 ceph-mon[31211]: pgmap v252: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-19T06:49:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 -- 172.21.15.37:0/606523419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0481008d0 msgr2=0x7fa048100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 --2- 172.21.15.37:0/606523419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0481008d0 0x7fa048100cb0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fa038009900 tx=0x7fa03802a3f0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 -- 172.21.15.37:0/606523419 shutdown_connections 2024-09-19T06:49:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 --2- 172.21.15.37:0/606523419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa04810e480 0x7fa048110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 --2- 172.21.15.37:0/606523419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa048101280 0x7fa04810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 --2- 172.21.15.37:0/606523419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa0481008d0 0x7fa048100cb0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 -- 172.21.15.37:0/606523419 >> 172.21.15.37:0/606523419 conn(0x7fa0480fc7a0 msgr2=0x7fa0480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 -- 172.21.15.37:0/606523419 shutdown_connections 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 -- 172.21.15.37:0/606523419 wait complete. 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 Processor -- start 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.639+0000 7fa0500e5640 1 -- start start 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa0500e5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0481008d0 0x7fa04819a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa0500e5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa048101280 0x7fa04819ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa0500e5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa04810e480 0x7fa04819eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa0500e5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481141e0 con 0x7fa048101280 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa0500e5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa048114060 con 0x7fa04810e480 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa0500e5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa048114360 con 0x7fa0481008d0 2024-09-19T06:49:54.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa048101280 0x7fa04819ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa048101280 0x7fa04819ab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55762/0 (socket says 172.21.15.37:55762) 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 -- 172.21.15.37:0/1308803134 learned_addr learned my addr 172.21.15.37:0/1308803134 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04de81640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0481008d0 0x7fa04819a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04e682640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa04810e480 0x7fa04819eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 -- 172.21.15.37:0/1308803134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0481008d0 msgr2=0x7fa04819a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0481008d0 0x7fa04819a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 -- 172.21.15.37:0/1308803134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa04810e480 msgr2=0x7fa04819eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa04810e480 0x7fa04819eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 -- 172.21.15.37:0/1308803134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa04819f5d0 con 0x7fa048101280 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04de81640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0481008d0 0x7fa04819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04e682640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa04810e480 0x7fa04819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa04d680640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa048101280 0x7fa04819ab60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00c920 tx=0x7fa03c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa046ffd640 1 -- 172.21.15.37:0/1308803134 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c007c70 con 0x7fa048101280 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa046ffd640 1 -- 172.21.15.37:0/1308803134 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa03c007e10 con 0x7fa048101280 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.640+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa04819f8c0 con 0x7fa048101280 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.641+0000 7fa046ffd640 1 -- 172.21.15.37:0/1308803134 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03c018890 con 0x7fa048101280 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.641+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0481a71a0 con 0x7fa048101280 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.643+0000 7fa046ffd640 1 -- 172.21.15.37:0/1308803134 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa03c018a70 con 0x7fa048101280 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.643+0000 7fa0524be740 1 Processor -- start 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.643+0000 7fa0524be740 1 -- start start 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa0524be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4ae880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa0524be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 0x5619ee4b9140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa046ffd640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0300a3430 0x7fa0300a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa0524be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4b9680 0x5619ee4bba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa0524be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5619ee4142e0 con 0x5619ee4b9680 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa0524be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5619ee3bbce0 con 0x5619ee4ae4a0 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa0524be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5619ee4102a0 con 0x5619ee4aedc0 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4b9680 0x5619ee4bba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04d680640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0300a3430 0x7fa0300a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4b9680 0x5619ee4bba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55770/0 (socket says 172.21.15.37:55770) 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 -- 172.21.15.37:0/595458277 learned_addr learned my addr 172.21.15.37:0/595458277 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa046ffd640 1 -- 172.21.15.37:0/1308803134 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fa03c0cff80 con 0x7fa048101280 2024-09-19T06:49:54.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04de81640 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 0x5619ee4b9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 -- 172.21.15.37:0/595458277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 msgr2=0x5619ee4b9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 0x5619ee4b9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 -- 172.21.15.37:0/595458277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 msgr2=0x5619ee4ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 -- 172.21.15.37:0/595458277 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ee3a0440 con 0x5619ee4b9680 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04d680640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0300a3430 0x7fa0300a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0a4910 tx=0x7fa03c0bc040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.644+0000 7fa04e682640 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4b9680 0x5619ee4bba70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa040012a60 tx=0x7fa04000a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd42a46fcf6ba7b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa02ffff640 1 -- 172.21.15.37:0/595458277 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040004d20 con 0x5619ee4b9680 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa02ffff640 1 -- 172.21.15.37:0/595458277 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040005020 con 0x5619ee4b9680 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa02ffff640 1 -- 172.21.15.37:0/595458277 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040010610 con 0x5619ee4b9680 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa0524be740 1 -- 172.21.15.37:0/595458277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4b9680 msgr2=0x5619ee4bba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa0524be740 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4b9680 0x5619ee4bba70 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa040012a60 tx=0x7fa04000a600 comp rx=0 tx=0).stop 2024-09-19T06:49:54.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa0524be740 1 -- 172.21.15.37:0/595458277 shutdown_connections 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa0524be740 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4b9680 0x5619ee4bba70 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa0524be740 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 0x5619ee4b9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa0524be740 1 --2- 172.21.15.37:0/595458277 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.645+0000 7fa0524be740 1 -- 172.21.15.37:0/595458277 >> 172.21.15.37:0/595458277 conn(0x5619ee4aaa30 msgr2=0x5619ee4aae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 -- 172.21.15.37:0/595458277 shutdown_connections 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 -- 172.21.15.37:0/595458277 wait complete. 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 -- start start 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 Processor -- start 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4b9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4b9680 0x5619ee4b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5619ee4bc250 con 0x5619ee4aedc0 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.646+0000 7fa0524be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5619ee3cf050 con 0x5619ee4ae4a0 2024-09-19T06:49:54.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa0524be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5619ee4bc3d0 con 0x5619ee4b9680 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa0524be740 10 client.0 _ll_register_callbacks cb 0x5619ee40f160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4b62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55778/0 (socket says 172.21.15.37:55778) 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 -- 172.21.15.37:0/3677893047 learned_addr learned my addr 172.21.15.37:0/3677893047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04e682640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4b9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04e682640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4b9680 0x5619ee4b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 -- 172.21.15.37:0/3677893047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4b9680 msgr2=0x5619ee4b6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4b9680 0x5619ee4b6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 -- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 msgr2=0x5619ee4b9000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4b9000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04de81640 1 -- 172.21.15.37:0/3677893047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ee1853c0 con 0x5619ee4aedc0 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.647+0000 7fa04e682640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:54.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa04de81640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4b62b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa038009230 tx=0x7fa038003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8628ed5139476192 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa02d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa02d7fa640 1 -- 172.21.15.37:0/3677893047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa038035070 con 0x5619ee4aedc0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa02d7fa640 1 -- 172.21.15.37:0/3677893047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fa038003c70 con 0x5619ee4aedc0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa040005020 con 0x5619ee4aedc0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa02d7fa640 1 -- 172.21.15.37:0/3677893047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa038005d50 con 0x5619ee4aedc0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa0524be740 10 client.17715 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5619ee4c7c40 con 0x5619ee4aedc0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.648+0000 7fa02d7fa640 1 -- 172.21.15.37:0/3677893047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040005020 con 0x5619ee4aedc0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa02d7fa640 1 -- 172.21.15.37:0/3677893047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fa03803b760 con 0x5619ee4aedc0 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa0524be740 10 client.17715 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa0524be740 10 client.17715 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa0524be740 20 client.17715 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa0524be740 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee4c9150 0x5619ee4cb5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa0524be740 4 client.17715 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619ee421850 con 0x5619ee4c9150 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa04e682640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee4c9150 0x5619ee4cb5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa04e682640 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee4c9150 0x5619ee4cb5b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e8e682db49057c server_cookie=9f17854c06051c59 in_seq=0 out_seq=0 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.649+0000 7fa02d7fa640 10 client.17715 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa02d7fa640 1 -- 172.21.15.37:0/3677893047 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619ee421850 con 0x5619ee4c9150 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa02d7fa640 10 client.17715 handle_command_reply: tid=0 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 client.17715 shutdown 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 msgr2=0x5619ee4b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4b62b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa038009230 tx=0x7fa038003880 comp rx=0 tx=0).stop 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 shutdown_connections 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee4c9150 0x5619ee4cb5b0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4b9680 0x5619ee4b6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.684 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4b62b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 --2- 172.21.15.37:0/3677893047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4ae4a0 0x5619ee4b9000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa02d7fa640 0 client.17715 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.650+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 >> 172.21.15.37:0/3677893047 conn(0x5619ee4aaa30 msgr2=0x5619ee4b0710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.651+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 shutdown_connections 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.651+0000 7fa0524be740 1 -- 172.21.15.37:0/3677893047 wait complete. 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.651+0000 7fa0524be740 20 client.17715 trim_cache size 0 max 16384 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 Processor -- start 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 -- start start 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 0x5619ee4a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481b2410 con 0x5619ee4ae4a0 2024-09-19T06:49:54.685 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481afc80 con 0x5619ee4b9680 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa0524be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481adaf0 con 0x5619ee4aedc0 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55784/0 (socket says 172.21.15.37:55784) 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 -- 172.21.15.37:0/1096272199 learned_addr learned my addr 172.21.15.37:0/1096272199 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 -- 172.21.15.37:0/1096272199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 msgr2=0x5619ee4a9a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 --2- 172.21.15.37:0/1096272199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 0x5619ee4a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 -- 172.21.15.37:0/1096272199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 msgr2=0x5619ee4d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 --2- 172.21.15.37:0/1096272199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 -- 172.21.15.37:0/1096272199 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619ee4b4800 con 0x5619ee4ae4a0 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.668+0000 7fa04e682640 1 --2- 172.21.15.37:0/1096272199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c54b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa040012a60 tx=0x7fa04000ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfc370e29145ee93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa02f7fe640 1 -- 172.21.15.37:0/1096272199 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040031070 con 0x5619ee4ae4a0 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa02f7fe640 1 -- 172.21.15.37:0/1096272199 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040004a00 con 0x5619ee4ae4a0 2024-09-19T06:49:54.686 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa02f7fe640 1 -- 172.21.15.37:0/1096272199 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040027040 con 0x5619ee4ae4a0 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 -- 172.21.15.37:0/1096272199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4ae4a0 msgr2=0x5619ee4c54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 --2- 172.21.15.37:0/1096272199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c54b0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa040012a60 tx=0x7fa04000ad00 comp rx=0 tx=0).stop 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 -- 172.21.15.37:0/1096272199 shutdown_connections 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 --2- 172.21.15.37:0/1096272199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4d0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 --2- 172.21.15.37:0/1096272199 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4aedc0 0x5619ee4a9a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 --2- 172.21.15.37:0/1096272199 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c54b0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 -- 172.21.15.37:0/1096272199 >> 172.21.15.37:0/1096272199 conn(0x5619ee4aaa30 msgr2=0x5619ee4a9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 -- 172.21.15.37:0/1096272199 shutdown_connections 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.669+0000 7fa0524be740 1 -- 172.21.15.37:0/1096272199 wait complete. 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 -- start start 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 Processor -- start 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4bcf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4bd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481ae140 con 0x5619ee4aedc0 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481af5b0 con 0x5619ee4b9680 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa0481ae810 con 0x5619ee4ae4a0 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa0524be740 10 client.0 _ll_register_callbacks cb 0x7fa038009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa04de81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4bcf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa04de81640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4bcf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55786/0 (socket says 172.21.15.37:55786) 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa04de81640 1 -- 172.21.15.37:0/2119608753 learned_addr learned my addr 172.21.15.37:0/2119608753 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.670+0000 7fa04e682640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04de81640 1 -- 172.21.15.37:0/2119608753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4ae4a0 msgr2=0x5619ee4c1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04de81640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04de81640 1 -- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 msgr2=0x5619ee4bd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04e682640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4bd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04de81640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4bd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04de81640 1 -- 172.21.15.37:0/2119608753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619ee4bdb00 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04e682640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4bd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa04de81640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4bcf20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa038006010 tx=0x7fa03803ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=156e82283e8cb0d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa02d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa02d7fa640 1 -- 172.21.15.37:0/2119608753 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa03804f070 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa02d7fa640 1 -- 172.21.15.37:0/2119608753 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fa038003c10 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa040004a00 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa02d7fa640 1 -- 172.21.15.37:0/2119608753 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa038035040 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa0524be740 10 client.17721 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.671+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5619ee533cd0 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa02d7fa640 1 -- 172.21.15.37:0/2119608753 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa040004a00 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa02d7fa640 1 -- 172.21.15.37:0/2119608753 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fa038031640 con 0x5619ee4aedc0 2024-09-19T06:49:54.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa0524be740 10 client.17721 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa0524be740 10 client.17721 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa0524be740 20 client.17721 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa0524be740 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee534470 0x5619ee536890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa0524be740 4 client.17721 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5619ee41bd60 con 0x5619ee534470 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.672+0000 7fa04e682640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee534470 0x5619ee536890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa02d7fa640 10 client.17721 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa04e682640 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee534470 0x5619ee536890 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4eff74e20ea8c6f5 server_cookie=ace7af3d89a7ded3 in_seq=0 out_seq=0 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa02d7fa640 1 -- 172.21.15.37:0/2119608753 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5619ee41bd60 con 0x5619ee534470 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa02d7fa640 10 client.17721 handle_command_reply: tid=0 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 client.17721 shutdown 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 msgr2=0x5619ee4bcf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4bcf20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa038006010 tx=0x7fa03803ba40 comp rx=0 tx=0).stop 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 shutdown_connections 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5619ee534470 0x5619ee536890 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5619ee4b9680 0x5619ee4bd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5619ee4aedc0 0x5619ee4bcf20 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 --2- 172.21.15.37:0/2119608753 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5619ee4ae4a0 0x5619ee4c1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa02d7fa640 0 client.17721 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 >> 172.21.15.37:0/2119608753 conn(0x5619ee4aaa30 msgr2=0x5619ee4b8330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 shutdown_connections 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 1 -- 172.21.15.37:0/2119608753 wait complete. 2024-09-19T06:49:54.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.673+0000 7fa0524be740 20 client.17721 trim_cache size 0 max 16384 2024-09-19T06:49:54.692 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:49:54.692 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:49:54.692 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:49:54.692 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:49:54.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0300a3430 msgr2=0x7fa0300a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0300a3430 0x7fa0300a58f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa03c0a4910 tx=0x7fa03c0bc040 comp rx=0 tx=0).stop 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa048101280 msgr2=0x7fa04819ab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa048101280 0x7fa04819ab60 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00c920 tx=0x7fa03c00cde0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 shutdown_connections 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa04810e480 0x7fa04819eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa0300a3430 0x7fa0300a58f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa048101280 0x7fa04819ab60 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 --2- 172.21.15.37:0/1308803134 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa0481008d0 0x7fa04819a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 >> 172.21.15.37:0/1308803134 conn(0x7fa0480fc7a0 msgr2=0x7fa0480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 shutdown_connections 2024-09-19T06:49:54.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:54.674+0000 7fa0500e5640 1 -- 172.21.15.37:0/1308803134 wait complete. 2024-09-19T06:49:54.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:54 smithi096 ceph-mon[35004]: pgmap v252: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-19T06:49:54.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:54 smithi151 ceph-mon[35236]: pgmap v252: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 175 KiB/s wr, 52 op/s 2024-09-19T06:49:55.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.11 MB/sec execute 262 sec latency 20622.749 ms 2024-09-19T06:49:55.454 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:49:55.454 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:55.709 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 -- 172.21.15.37:0/2355510444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 msgr2=0x7f8b6410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 --2- 172.21.15.37:0/2355510444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6410d950 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c009cd0 tx=0x7f8b5c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 -- 172.21.15.37:0/2355510444 shutdown_connections 2024-09-19T06:49:56.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 --2- 172.21.15.37:0/2355510444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6410d950 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 --2- 172.21.15.37:0/2355510444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b64100a80 0x7f8b64100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 --2- 172.21.15.37:0/2355510444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b64106a80 0x7f8b64106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 -- 172.21.15.37:0/2355510444 >> 172.21.15.37:0/2355510444 conn(0x7f8b640fc7a0 msgr2=0x7f8b640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 -- 172.21.15.37:0/2355510444 shutdown_connections 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.177+0000 7f8b6a306640 1 -- 172.21.15.37:0/2355510444 wait complete. 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 Processor -- start 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 -- start start 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b64100a80 0x7f8b6419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6419aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b64106a80 0x7f8b6419ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64110e80 con 0x7f8b64101420 2024-09-19T06:49:56.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64110d00 con 0x7f8b64106a80 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b6a306640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8b64111000 con 0x7f8b64100a80 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6419aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6419aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55804/0 (socket says 172.21.15.37:55804) 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b63fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b64100a80 0x7f8b6419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b637fe640 1 -- 172.21.15.37:0/1418900923 learned_addr learned my addr 172.21.15.37:0/1418900923 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.178+0000 7f8b688a3640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b64106a80 0x7f8b6419ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b637fe640 1 -- 172.21.15.37:0/1418900923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b64100a80 msgr2=0x7f8b6419a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b637fe640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b64100a80 0x7f8b6419a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b637fe640 1 -- 172.21.15.37:0/1418900923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b64106a80 msgr2=0x7f8b6419ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b637fe640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b64106a80 0x7f8b6419ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b637fe640 1 -- 172.21.15.37:0/1418900923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b6419f550 con 0x7f8b64101420 2024-09-19T06:49:56.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b688a3640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b64106a80 0x7f8b6419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b637fe640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6419aae0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c920 tx=0x7f8b5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b617fa640 1 -- 172.21.15.37:0/1418900923 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58007c70 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b6419f840 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b617fa640 1 -- 172.21.15.37:0/1418900923 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8b58007e10 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b617fa640 1 -- 172.21.15.37:0/1418900923 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b58018890 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.179+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b641a7120 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.180+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b64100ee0 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.181+0000 7f8b617fa640 1 -- 172.21.15.37:0/1418900923 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8b58018a50 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.182+0000 7f8b617fa640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.182+0000 7f8b63fff640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.182+0000 7f8b617fa640 1 -- 172.21.15.37:0/1418900923 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f8b580d0170 con 0x7f8b64101420 2024-09-19T06:49:56.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.183+0000 7f8b63fff640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009b50 tx=0x7f8b54005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:56.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.185+0000 7f8b617fa640 1 -- 172.21.15.37:0/1418900923 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8b58014030 con 0x7f8b64101420 2024-09-19T06:49:56.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.08 MB/sec execute 263 sec latency 21622.793 ms 2024-09-19T06:49:56.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.397+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8b6419b900 con 0x7f8b64101420 2024-09-19T06:49:56.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.398+0000 7f8b617fa640 1 -- 172.21.15.37:0/1418900923 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11907 (secure 0 0 0) 0x7f8b6419b900 con 0x7f8b64101420 2024-09-19T06:49:56.400 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:49:56.400 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"btime":"2024-09-19T06:48:37:250513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:48:36.625010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:49:56.401 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-09-19T06:49:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.401+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8b4c0a33b0 msgr2=0x7f8b4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.401+0000 7f8b6a306640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f8b54009b50 tx=0x7f8b54005f90 comp rx=0 tx=0).stop 2024-09-19T06:49:56.403 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.401+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 msgr2=0x7f8b6419aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.401+0000 7f8b6a306640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6419aae0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c920 tx=0x7f8b5800cde0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 shutdown_connections 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8b64106a80 0x7f8b6419ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8b4c0a33b0 0x7f8b4c0a5870 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8b64101420 0x7f8b6419aae0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 --2- 172.21.15.37:0/1418900923 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8b64100a80 0x7f8b6419a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 >> 172.21.15.37:0/1418900923 conn(0x7f8b640fc7a0 msgr2=0x7f8b6406a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 shutdown_connections 2024-09-19T06:49:56.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:56.402+0000 7f8b6a306640 1 -- 172.21.15.37:0/1418900923 wait complete. 2024-09-19T06:49:56.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:56 smithi037 ceph-mon[31211]: pgmap v253: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-19T06:49:56.710 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:56 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1418900923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:56.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:56 smithi096 ceph-mon[35004]: pgmap v253: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-19T06:49:56.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:56 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1418900923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:56.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:56 smithi151 ceph-mon[35236]: pgmap v253: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-19T06:49:56.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:56 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1418900923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:49:57.106 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:49:57.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.06 MB/sec execute 264 sec latency 22622.842 ms 2024-09-19T06:49:57.368 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:57.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:49:57.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 -- 172.21.15.37:0/2564313091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0541036e0 msgr2=0x7fb054107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 --2- 172.21.15.37:0/2564313091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0541036e0 0x7fb054107b90 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7fb048009930 tx=0x7fb04802a420 comp rx=0 tx=0).stop 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 -- 172.21.15.37:0/2564313091 shutdown_connections 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 --2- 172.21.15.37:0/2564313091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0541082a0 0x7fb05410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 --2- 172.21.15.37:0/2564313091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb0541036e0 0x7fb054107b90 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 --2- 172.21.15.37:0/2564313091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb054102d30 0x7fb054103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 -- 172.21.15.37:0/2564313091 >> 172.21.15.37:0/2564313091 conn(0x7fb0540fc7d0 msgr2=0x7fb0540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 -- 172.21.15.37:0/2564313091 shutdown_connections 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 -- 172.21.15.37:0/2564313091 wait complete. 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.845+0000 7fb059073640 1 Processor -- start 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb059073640 1 -- start start 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb059073640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb054102d30 0x7fb054198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb059073640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0541036e0 0x7fb054198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb059073640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0541082a0 0x7fb05419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb059073640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541120a0 con 0x7fb054102d30 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb059073640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb054111f20 con 0x7fb0541082a0 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb059073640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb054112220 con 0x7fb0541036e0 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb054102d30 0x7fb054198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb054102d30 0x7fb054198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55826/0 (socket says 172.21.15.37:55826) 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 -- 172.21.15.37:0/3818104245 learned_addr learned my addr 172.21.15.37:0/3818104245 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb05259c640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0541036e0 0x7fb054198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb05359e640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0541082a0 0x7fb05419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 -- 172.21.15.37:0/3818104245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0541036e0 msgr2=0x7fb054198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0541036e0 0x7fb054198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 -- 172.21.15.37:0/3818104245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0541082a0 msgr2=0x7fb05419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0541082a0 0x7fb05419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.846+0000 7fb052d9d640 1 -- 172.21.15.37:0/3818104245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb05419d5f0 con 0x7fb054102d30 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.847+0000 7fb052d9d640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb054102d30 0x7fb054198640 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb04400d950 tx=0x7fb04400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.847+0000 7fb03bfff640 1 -- 172.21.15.37:0/3818104245 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb044017030 con 0x7fb054102d30 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.847+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb05419d8e0 con 0x7fb054102d30 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.847+0000 7fb03bfff640 1 -- 172.21.15.37:0/3818104245 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb044004020 con 0x7fb054102d30 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.847+0000 7fb03bfff640 1 -- 172.21.15.37:0/3818104245 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0440047a0 con 0x7fb054102d30 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.847+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0541a51c0 con 0x7fb054102d30 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.849+0000 7fb05b44c740 1 Processor -- start 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.849+0000 7fb05b44c740 1 -- start start 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.849+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8bb40 0x562da4a8a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.849+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 0x562da4a8aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 0x562da4a97760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05b44c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562da49f02f0 con 0x562da4a8bb40 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05b44c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562da4997cf0 con 0x562da4a8a5d0 2024-09-19T06:49:57.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05b44c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562da49ec2b0 con 0x562da4a95370 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb03bfff640 1 -- 172.21.15.37:0/3818104245 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb0440041d0 con 0x7fb054102d30 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8bb40 0x562da4a8a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8bb40 0x562da4a8a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55832/0 (socket says 172.21.15.37:55832) 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 -- 172.21.15.37:0/2107466252 learned_addr learned my addr 172.21.15.37:0/2107466252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05259c640 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 0x562da4a97760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 -- 172.21.15.37:0/2107466252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 msgr2=0x562da4a97760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 0x562da4a97760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 -- 172.21.15.37:0/2107466252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 msgr2=0x562da4a8aa10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 0x562da4a8aa10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 -- 172.21.15.37:0/2107466252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da497c3a0 con 0x562da4a8bb40 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb05359e640 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8bb40 0x562da4a8a090 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009e60 tx=0x7fb04c010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6237078c901fc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb0397fa640 1 -- 172.21.15.37:0/2107466252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c016070 con 0x562da4a8bb40 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb0397fa640 1 -- 172.21.15.37:0/2107466252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c004d20 con 0x562da4a8bb40 2024-09-19T06:49:57.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.850+0000 7fb0397fa640 1 -- 172.21.15.37:0/2107466252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c012830 con 0x562da4a8bb40 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb03bfff640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb03c0a33b0 0x7fb03c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb052d9d640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb03c0a33b0 0x7fb03c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb03bfff640 1 -- 172.21.15.37:0/3818104245 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb0440cf750 con 0x7fb054102d30 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 -- 172.21.15.37:0/2107466252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8bb40 msgr2=0x562da4a8a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8bb40 0x562da4a8a090 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009e60 tx=0x7fb04c010740 comp rx=0 tx=0).stop 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb052d9d640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb03c0a33b0 0x7fb03c0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x562da4a87ac0 tx=0x7fb044086040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 -- 172.21.15.37:0/2107466252 shutdown_connections 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 0x562da4a97760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 0x562da4a8aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 --2- 172.21.15.37:0/2107466252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8bb40 0x562da4a8a090 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 -- 172.21.15.37:0/2107466252 >> 172.21.15.37:0/2107466252 conn(0x562da4a53310 msgr2=0x562da4a86c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 -- 172.21.15.37:0/2107466252 shutdown_connections 2024-09-19T06:49:57.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.851+0000 7fb05b44c740 1 -- 172.21.15.37:0/2107466252 wait complete. 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 -- start start 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 Processor -- start 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a94db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a92060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a95370 0x562da4a927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562da4a98070 con 0x562da4a8a5d0 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562da49ab060 con 0x562da4a95370 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562da4a981f0 con 0x562da4a8bb40 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05b44c740 10 client.0 _ll_register_callbacks cb 0x562da46fe8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a94db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a94db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55842/0 (socket says 172.21.15.37:55842) 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 -- 172.21.15.37:0/3868833252 learned_addr learned my addr 172.21.15.37:0/3868833252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05259c640 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a92060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 -- 172.21.15.37:0/3868833252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 msgr2=0x562da4a92060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a92060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 -- 172.21.15.37:0/3868833252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a95370 msgr2=0x562da4a927f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a95370 0x562da4a927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05359e640 1 -- 172.21.15.37:0/3868833252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da47613c0 con 0x562da4a8a5d0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.852+0000 7fb05259c640 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a92060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05359e640 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a94db0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb04c011010 tx=0x7fb04c004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6f070a08b285b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb036ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb036ff5640 1 -- 172.21.15.37:0/3868833252 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c028060 con 0x562da4a8a5d0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04c004d20 con 0x562da4a8a5d0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb036ff5640 1 -- 172.21.15.37:0/3868833252 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb04c005320 con 0x562da4a8a5d0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb036ff5640 1 -- 172.21.15.37:0/3868833252 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb04c016070 con 0x562da4a8a5d0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 10 client.17739 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562da4aa3bb0 con 0x562da4a8a5d0 2024-09-19T06:49:57.865 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb036ff5640 1 -- 172.21.15.37:0/3868833252 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c004d20 con 0x562da4a8a5d0 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb036ff5640 1 -- 172.21.15.37:0/3868833252 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb04c00f040 con 0x562da4a8a5d0 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 10 client.17739 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 10 client.17739 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 20 client.17739 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4aa7350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 4 client.17739 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.853+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562da49fd860 con 0x562da4aa4f60 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.854+0000 7fb05259c640 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4aa7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.854+0000 7fb05259c640 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4aa7350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ab90d70ab607bcb server_cookie=c447187f9e19ad2d in_seq=0 out_seq=0 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.854+0000 7fb036ff5640 10 client.17739 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb036ff5640 1 -- 172.21.15.37:0/3868833252 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562da49fd860 con 0x562da4aa4f60 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb036ff5640 10 client.17739 handle_command_reply: tid=0 2024-09-19T06:49:57.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 client.17739 shutdown 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 msgr2=0x562da4a94db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a94db0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fb04c011010 tx=0x7fb04c004c70 comp rx=0 tx=0).stop 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 shutdown_connections 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4aa7350 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb036ff5640 0 client.17739 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a95370 0x562da4a927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a92060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 --2- 172.21.15.37:0/3868833252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a94db0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 >> 172.21.15.37:0/3868833252 conn(0x562da4a53310 msgr2=0x562da4a8c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 shutdown_connections 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 1 -- 172.21.15.37:0/3868833252 wait complete. 2024-09-19T06:49:57.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.855+0000 7fb05b44c740 20 client.17739 trim_cache size 0 max 16384 2024-09-19T06:49:57.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 Processor -- start 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 -- start start 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a91040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8bb40 0x562da4aac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 0x562da4aace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541b03b0 con 0x562da4a8a5d0 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541adc20 con 0x562da4a8bb40 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05b44c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541aba90 con 0x562da4a95370 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a91040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a91040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55844/0 (socket says 172.21.15.37:55844) 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05359e640 1 -- 172.21.15.37:0/1470747460 learned_addr learned my addr 172.21.15.37:0/1470747460 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:57.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.869+0000 7fb05259c640 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8bb40 0x562da4aac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb05359e640 1 -- 172.21.15.37:0/1470747460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 msgr2=0x562da4aace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb05359e640 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 0x562da4aace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb05359e640 1 -- 172.21.15.37:0/1470747460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8bb40 msgr2=0x562da4aac8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb05359e640 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8bb40 0x562da4aac8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb05359e640 1 -- 172.21.15.37:0/1470747460 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562da4a8b750 con 0x562da4a8a5d0 2024-09-19T06:49:57.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb05259c640 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8bb40 0x562da4aac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:49:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb05359e640 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a91040 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009680 tx=0x7fb04c0096e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884eefe5cb962033 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb038ff9640 1 -- 172.21.15.37:0/1470747460 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c028070 con 0x562da4a8a5d0 2024-09-19T06:49:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb038ff9640 1 -- 172.21.15.37:0/1470747460 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c004ce0 con 0x562da4a8a5d0 2024-09-19T06:49:57.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.870+0000 7fb038ff9640 1 -- 172.21.15.37:0/1470747460 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb04c012560 con 0x562da4a8a5d0 2024-09-19T06:49:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 -- 172.21.15.37:0/1470747460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 msgr2=0x562da4a91040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a91040 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fb04c009680 tx=0x7fb04c0096e0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 -- 172.21.15.37:0/1470747460 shutdown_connections 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a95370 0x562da4aace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8bb40 0x562da4aac8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 --2- 172.21.15.37:0/1470747460 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a8a5d0 0x562da4a91040 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 -- 172.21.15.37:0/1470747460 >> 172.21.15.37:0/1470747460 conn(0x562da4a53310 msgr2=0x562da4a90930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 -- 172.21.15.37:0/1470747460 shutdown_connections 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 -- 172.21.15.37:0/1470747460 wait complete. 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 -- start start 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.871+0000 7fb05b44c740 1 Processor -- start 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 0x562da4a99180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05b44c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a95370 0x562da4a99c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05b44c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541ac0e0 con 0x562da4a95370 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05b44c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541ad550 con 0x562da4a8a5d0 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05b44c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb0541ac7b0 con 0x562da4a8bb40 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05b44c740 10 client.0 _ll_register_callbacks cb 0x7fb04c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a95370 0x562da4a99c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a95370 0x562da4a99c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55860/0 (socket says 172.21.15.37:55860) 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05359e640 1 -- 172.21.15.37:0/437348569 learned_addr learned my addr 172.21.15.37:0/437348569 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05259c640 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05359e640 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 0x562da4a99180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05259c640 1 -- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 msgr2=0x562da4a99180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05259c640 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 0x562da4a99180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05259c640 1 -- 172.21.15.37:0/437348569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a95370 msgr2=0x562da4a99c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05259c640 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a95370 0x562da4a99c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.872+0000 7fb05259c640 1 -- 172.21.15.37:0/437348569 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562da4b0e8c0 con 0x562da4a8bb40 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb05259c640 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a996c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb04802a940 tx=0x7fb048005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e949115fd306a588 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb036ff5640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb036ff5640 1 -- 172.21.15.37:0/437348569 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb048035040 con 0x562da4a8bb40 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb04c004ce0 con 0x562da4a8bb40 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb036ff5640 1 -- 172.21.15.37:0/437348569 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fb048005030 con 0x562da4a8bb40 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb036ff5640 1 -- 172.21.15.37:0/437348569 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb048052050 con 0x562da4a8bb40 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb05b44c740 10 client.26875 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562da4b0fb40 con 0x562da4a8bb40 2024-09-19T06:49:57.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb036ff5640 1 -- 172.21.15.37:0/437348569 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb04c004ce0 con 0x562da4a8bb40 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.873+0000 7fb036ff5640 1 -- 172.21.15.37:0/437348569 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fb0480035f0 con 0x562da4a8bb40 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05b44c740 10 client.26875 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05b44c740 10 client.26875 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05b44c740 20 client.26875 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05b44c740 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4b12330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05b44c740 4 client.26875 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562da49f7d70 con 0x562da4aa4f60 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05359e640 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4b12330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb036ff5640 10 client.26875 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb05359e640 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4b12330 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6af311314c10c11c server_cookie=2fc347653bfc067c in_seq=0 out_seq=0 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb036ff5640 1 -- 172.21.15.37:0/437348569 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562da49f7d70 con 0x562da4aa4f60 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.874+0000 7fb036ff5640 10 client.26875 handle_command_reply: tid=0 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 client.26875 shutdown 2024-09-19T06:49:57.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 msgr2=0x562da4a996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a996c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb04802a940 tx=0x7fb048005c00 comp rx=0 tx=0).stop 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 shutdown_connections 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x562da4aa4f60 0x562da4b12330 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562da4a95370 0x562da4a99c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562da4a8bb40 0x562da4a996c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb036ff5640 0 client.26875 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 --2- 172.21.15.37:0/437348569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562da4a8a5d0 0x562da4a99180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 >> 172.21.15.37:0/437348569 conn(0x562da4a53310 msgr2=0x562da4a854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 shutdown_connections 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 1 -- 172.21.15.37:0/437348569 wait complete. 2024-09-19T06:49:57.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.875+0000 7fb05b44c740 20 client.26875 trim_cache size 0 max 16384 2024-09-19T06:49:57.884 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:49:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb03c0a33b0 msgr2=0x7fb03c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb03c0a33b0 0x7fb03c0a5870 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x562da4a87ac0 tx=0x7fb044086040 comp rx=0 tx=0).stop 2024-09-19T06:49:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb054102d30 msgr2=0x7fb054198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:57.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb054102d30 0x7fb054198640 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb04400d950 tx=0x7fb04400de10 comp rx=0 tx=0).stop 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 shutdown_connections 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb0541082a0 0x7fb05419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb0541036e0 0x7fb054198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb03c0a33b0 0x7fb03c0a5870 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 --2- 172.21.15.37:0/3818104245 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb054102d30 0x7fb054198640 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 >> 172.21.15.37:0/3818104245 conn(0x7fb0540fc7d0 msgr2=0x7fb054106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 shutdown_connections 2024-09-19T06:49:57.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:57.876+0000 7fb059073640 1 -- 172.21.15.37:0/3818104245 wait complete. 2024-09-19T06:49:58.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.03 MB/sec execute 265 sec latency 23622.880 ms 2024-09-19T06:49:58.619 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:49:58.644 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:58.766 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:49:58 smithi037 ceph-mon[31211]: pgmap v254: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-19T06:49:58.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:49:58 smithi096 ceph-mon[35004]: pgmap v254: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-19T06:49:58.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:49:58 smithi151 ceph-mon[35236]: pgmap v254: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.0 KiB/s wr, 5 op/s 2024-09-19T06:49:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 -- 172.21.15.37:0/4209097076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e8102d40 msgr2=0x7f61e8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4209097076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e8102d40 0x7f61e8103120 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009ca0 tx=0x7f61d802a950 comp rx=0 tx=0).stop 2024-09-19T06:49:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 -- 172.21.15.37:0/4209097076 shutdown_connections 2024-09-19T06:49:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4209097076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e810c170 0x7f61e810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4209097076 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e8103660 0x7f61e810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4209097076 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e8102d40 0x7f61e8103120 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 -- 172.21.15.37:0/4209097076 >> 172.21.15.37:0/4209097076 conn(0x7f61e80fc7a0 msgr2=0x7f61e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 -- 172.21.15.37:0/4209097076 shutdown_connections 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.199+0000 7f61f03ea640 1 -- 172.21.15.37:0/4209097076 wait complete. 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 Processor -- start 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 -- start start 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e8102d40 0x7f61e8198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e8103660 0x7f61e8198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e810c170 0x7f61e819ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8111f70 con 0x7f61e810c170 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f61e8111df0 con 0x7f61e8102d40 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.200+0000 7f61f03ea640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81120f0 con 0x7f61e8103660 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e810c170 0x7f61e819ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e810c170 0x7f61e819ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55884/0 (socket says 172.21.15.37:55884) 2024-09-19T06:49:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 -- 172.21.15.37:0/4234655238 learned_addr learned my addr 172.21.15.37:0/4234655238 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee186640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e8102d40 0x7f61e8198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ed985640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e8103660 0x7f61e8198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 -- 172.21.15.37:0/4234655238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e8103660 msgr2=0x7f61e8198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e8103660 0x7f61e8198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 -- 172.21.15.37:0/4234655238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e8102d40 msgr2=0x7f61e8198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e8102d40 0x7f61e8198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 -- 172.21.15.37:0/4234655238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e819d3d0 con 0x7f61e810c170 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61ee987640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e810c170 0x7f61e819ccf0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d750 tx=0x7f61e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61e77fe640 1 -- 172.21.15.37:0/4234655238 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0017070 con 0x7f61e810c170 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61e77fe640 1 -- 172.21.15.37:0/4234655238 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f61e0005500 con 0x7f61e810c170 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e819d6c0 con 0x7f61e810c170 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61e77fe640 1 -- 172.21.15.37:0/4234655238 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61e0004d90 con 0x7f61e810c170 2024-09-19T06:49:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.201+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e81a4fa0 con 0x7f61e810c170 2024-09-19T06:49:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.202+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e8103120 con 0x7f61e810c170 2024-09-19T06:49:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.203+0000 7f61e77fe640 1 -- 172.21.15.37:0/4234655238 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f61e0004f30 con 0x7f61e810c170 2024-09-19T06:49:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.204+0000 7f61e77fe640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61d00a3430 0x7f61d00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.204+0000 7f61e77fe640 1 -- 172.21.15.37:0/4234655238 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f61e00cfab0 con 0x7f61e810c170 2024-09-19T06:49:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.204+0000 7f61ee186640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61d00a3430 0x7f61d00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.205+0000 7f61ee186640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61d00a3430 0x7f61d00a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f61d8005fc0 tx=0x7f61d8009360 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.207+0000 7f61e77fe640 1 -- 172.21.15.37:0/4234655238 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f61e009c4e0 con 0x7f61e810c170 2024-09-19T06:49:59.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 7.00 MB/sec execute 266 sec latency 24622.922 ms 2024-09-19T06:49:59.248 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:49:59.440 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.438+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e8199780 con 0x7f61e810c170 2024-09-19T06:49:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.439+0000 7f61e77fe640 1 -- 172.21.15.37:0/4234655238 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11907 (secure 0 0 0) 0x7f61e8199780 con 0x7f61e810c170 2024-09-19T06:49:59.441 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:49:59.441 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":36,"btime":"2024-09-19T06:48:37:250513+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:48:36.625010+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:49:59.441 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 36 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61d00a3430 msgr2=0x7f61d00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61d00a3430 0x7f61d00a58f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f61d8005fc0 tx=0x7f61d8009360 comp rx=0 tx=0).stop 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e810c170 msgr2=0x7f61e819ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e810c170 0x7f61e819ccf0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d750 tx=0x7f61e000dc10 comp rx=0 tx=0).stop 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 shutdown_connections 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f61d00a3430 0x7f61d00a58f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f61e810c170 0x7f61e819ccf0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f61e8103660 0x7f61e8198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 --2- 172.21.15.37:0/4234655238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f61e8102d40 0x7f61e8198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 >> 172.21.15.37:0/4234655238 conn(0x7f61e80fc7a0 msgr2=0x7f61e80fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.443 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 shutdown_connections 2024-09-19T06:49:59.444 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.441+0000 7f61f03ea640 1 -- 172.21.15.37:0/4234655238 wait complete. 2024-09-19T06:49:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.966+0000 7fe68978c640 1 -- 172.21.15.37:0/2190482872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684103660 msgr2=0x7fe68410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.966+0000 7fe68978c640 1 --2- 172.21.15.37:0/2190482872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684103660 0x7fe68410bb50 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fe66c009900 tx=0x7fe66c02a480 comp rx=0 tx=0).stop 2024-09-19T06:49:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- 172.21.15.37:0/2190482872 shutdown_connections 2024-09-19T06:49:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 --2- 172.21.15.37:0/2190482872 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe68410c170 0x7fe68410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 --2- 172.21.15.37:0/2190482872 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684103660 0x7fe68410bb50 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 --2- 172.21.15.37:0/2190482872 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe684102d40 0x7fe684103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- 172.21.15.37:0/2190482872 >> 172.21.15.37:0/2190482872 conn(0x7fe6840fc7a0 msgr2=0x7fe6840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- 172.21.15.37:0/2190482872 shutdown_connections 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- 172.21.15.37:0/2190482872 wait complete. 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 Processor -- start 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- start start 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684102d40 0x7fe684198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe684103660 0x7fe684198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68410c170 0x7fe68419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe684111f70 con 0x7fe684102d40 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe684111df0 con 0x7fe684103660 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.967+0000 7fe68978c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841120f0 con 0x7fe68410c170 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684102d40 0x7fe684198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684102d40 0x7fe684198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41732/0 (socket says 172.21.15.37:41732) 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 -- 172.21.15.37:0/2133408541 learned_addr learned my addr 172.21.15.37:0/2133408541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe682ffd640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe684103660 0x7fe684198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe6837fe640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68410c170 0x7fe68419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 -- 172.21.15.37:0/2133408541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68410c170 msgr2=0x7fe68419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68410c170 0x7fe68419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 -- 172.21.15.37:0/2133408541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe684103660 msgr2=0x7fe684198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe684103660 0x7fe684198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 -- 172.21.15.37:0/2133408541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe68419d3e0 con 0x7fe684102d40 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe67bfff640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684102d40 0x7fe684198430 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe67400ea60 tx=0x7fe67400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe680ff9640 1 -- 172.21.15.37:0/2133408541 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67400cde0 con 0x7fe684102d40 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe680ff9640 1 -- 172.21.15.37:0/2133408541 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6740035f0 con 0x7fe684102d40 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe68419d6d0 con 0x7fe684102d40 2024-09-19T06:49:59.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe680ff9640 1 -- 172.21.15.37:0/2133408541 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674010670 con 0x7fe684102d40 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.968+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6841a4fb0 con 0x7fe684102d40 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe680ff9640 1 -- 172.21.15.37:0/2133408541 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe6740037a0 con 0x7fe684102d40 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 Processor -- start 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 -- start start 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036dcd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dcda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dda760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564036d332f0 con 0x564036dceb40 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564036cdacf0 con 0x564036dd8370 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe68bb65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564036d2f2b0 con 0x564036dcd5d0 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe6837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036dcd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe6837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036dcd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41746/0 (socket says 172.21.15.37:41746) 2024-09-19T06:49:59.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe6837fe640 1 -- 172.21.15.37:0/2561363401 learned_addr learned my addr 172.21.15.37:0/2561363401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.971+0000 7fe67bfff640 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dcda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe680ff9640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6640a33b0 0x7fe6640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe682ffd640 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dda760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe6837fe640 1 -- 172.21.15.37:0/2561363401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 msgr2=0x564036dcda10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe6837fe640 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dcda10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe6837fe640 1 -- 172.21.15.37:0/2561363401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 msgr2=0x564036dda760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe6837fe640 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dda760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe6837fe640 1 -- 172.21.15.37:0/2561363401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564036cbf3a0 con 0x564036dceb40 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe682ffd640 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dda760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe680ff9640 1 -- 172.21.15.37:0/2133408541 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fe6740d0000 con 0x7fe684102d40 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe6837fe640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6640a33b0 0x7fe6640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.972+0000 7fe6837fe640 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036dcd090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe67c011590 tx=0x7fe67c011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25971e13b21355fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.973+0000 7fe6797fa640 1 -- 172.21.15.37:0/2561363401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c017050 con 0x564036dceb40 2024-09-19T06:49:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.973+0000 7fe6797fa640 1 -- 172.21.15.37:0/2561363401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c01bb90 con 0x564036dceb40 2024-09-19T06:49:59.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.973+0000 7fe6797fa640 1 -- 172.21.15.37:0/2561363401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c00e3b0 con 0x564036dceb40 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 -- 172.21.15.37:0/2561363401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 msgr2=0x564036dcd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe6837fe640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6640a33b0 0x7fe6640a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe67c022000 tx=0x7fe67c023000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036dcd090 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fe67c011590 tx=0x7fe67c011a50 comp rx=0 tx=0).stop 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 -- 172.21.15.37:0/2561363401 shutdown_connections 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dda760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dcda10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 --2- 172.21.15.37:0/2561363401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036dcd090 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 -- 172.21.15.37:0/2561363401 >> 172.21.15.37:0/2561363401 conn(0x564036d96310 msgr2=0x564036dc9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 -- 172.21.15.37:0/2561363401 shutdown_connections 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 -- 172.21.15.37:0/2561363401 wait complete. 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 -- start start 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.974+0000 7fe68bb65740 1 Processor -- start 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dd7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dceb40 0x564036dd5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dd8370 0x564036dd5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe68bb65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x564036ddb070 con 0x564036dd8370 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe68bb65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564036cee060 con 0x564036dceb40 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe68bb65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x564036ddb1f0 con 0x564036dcd5d0 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe68bb65740 10 client.0 _ll_register_callbacks cb 0x564036a418c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dd8370 0x564036dd5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dd8370 0x564036dd5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41754/0 (socket says 172.21.15.37:41754) 2024-09-19T06:49:59.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 -- 172.21.15.37:0/1249693401 learned_addr learned my addr 172.21.15.37:0/1249693401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe682ffd640 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dceb40 0x564036dd5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe67bfff640 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dd7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 -- 172.21.15.37:0/1249693401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 msgr2=0x564036dd7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dd7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 -- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dceb40 msgr2=0x564036dd5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dceb40 0x564036dd5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 -- 172.21.15.37:0/1249693401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564036aa43c0 con 0x564036dd8370 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6837fe640 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dd8370 0x564036dd5610 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005f50 tx=0x7fe67c0257c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886306e9992bd370 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.975+0000 7fe6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe6727fc640 1 -- 172.21.15.37:0/1249693401 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67c02a040 con 0x564036dd8370 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe6727fc640 1 -- 172.21.15.37:0/1249693401 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fe67c025c80 con 0x564036dd8370 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe67c01bb90 con 0x564036dd8370 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe6727fc640 1 -- 172.21.15.37:0/1249693401 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe67c03c050 con 0x564036dd8370 2024-09-19T06:49:59.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 10 client.17763 fetch_fsmap learned FSMap version 36 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564036de6a80 con 0x564036dd8370 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe6727fc640 1 -- 172.21.15.37:0/1249693401 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67c01bb90 con 0x564036dd8370 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe6727fc640 1 -- 172.21.15.37:0/1249693401 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fe67c02d030 con 0x564036dd8370 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 10 client.17763 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 10 client.17763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 20 client.17763 populate_metadata read hostname 'smithi037' 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036de7e90 0x564036dea280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 4 client.17763 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.976+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564036d40860 con 0x564036de7e90 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.977+0000 7fe67bfff640 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036de7e90 0x564036dea280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.977+0000 7fe67bfff640 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036de7e90 0x564036dea280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5041108cfe4848ad server_cookie=498ed8825c16af46 in_seq=0 out_seq=0 2024-09-19T06:49:59.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.977+0000 7fe6727fc640 10 client.17763 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe6727fc640 1 -- 172.21.15.37:0/1249693401 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564036d40860 con 0x564036de7e90 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe6727fc640 10 client.17763 handle_command_reply: tid=0 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 client.17763 shutdown 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dd8370 msgr2=0x564036dd5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dd8370 0x564036dd5610 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fe67c005f50 tx=0x7fe67c0257c0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 shutdown_connections 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036de7e90 0x564036dea280 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dd8370 0x564036dd5610 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe6727fc640 0 client.17763 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dceb40 0x564036dd5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 --2- 172.21.15.37:0/1249693401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dcd5d0 0x564036dd7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 >> 172.21.15.37:0/1249693401 conn(0x564036d96310 msgr2=0x564036dcf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 shutdown_connections 2024-09-19T06:49:59.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 1 -- 172.21.15.37:0/1249693401 wait complete. 2024-09-19T06:49:59.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.978+0000 7fe68bb65740 20 client.17763 trim_cache size 0 max 16384 2024-09-19T06:49:59.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.991+0000 7fe68bb65740 1 Processor -- start 2024-09-19T06:49:59.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.991+0000 7fe68bb65740 1 -- start start 2024-09-19T06:49:59.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dcd5d0 0x564036de4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dceb40 0x564036dd0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dd0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:49:59.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe68bb65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841b0330 con 0x564036dcd5d0 2024-09-19T06:49:59.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe68bb65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841adba0 con 0x564036dd8370 2024-09-19T06:49:59.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe68bb65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841aba10 con 0x564036dceb40 2024-09-19T06:49:59.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dcd5d0 0x564036de4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dcd5d0 0x564036de4310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41766/0 (socket says 172.21.15.37:41766) 2024-09-19T06:49:59.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 -- 172.21.15.37:0/3523582350 learned_addr learned my addr 172.21.15.37:0/3523582350 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:49:59.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe682ffd640 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dceb40 0x564036dd0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe6837fe640 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dd0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 -- 172.21.15.37:0/3523582350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dceb40 msgr2=0x564036dd0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dceb40 0x564036dd0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 -- 172.21.15.37:0/3523582350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 msgr2=0x564036dd0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dd0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 -- 172.21.15.37:0/3523582350 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564036dd3570 con 0x564036dcd5d0 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe6837fe640 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dd0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.992+0000 7fe67bfff640 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dcd5d0 0x564036de4310 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe67409c240 tx=0x7fe6740d2510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73db3d8d2de0b0d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe678ff9640 1 -- 172.21.15.37:0/3523582350 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe67409bc10 con 0x564036dcd5d0 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe678ff9640 1 -- 172.21.15.37:0/3523582350 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67409c9b0 con 0x564036dcd5d0 2024-09-19T06:49:59.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe678ff9640 1 -- 172.21.15.37:0/3523582350 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe674003b30 con 0x564036dcd5d0 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 -- 172.21.15.37:0/3523582350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dcd5d0 msgr2=0x564036de4310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dcd5d0 0x564036de4310 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe67409c240 tx=0x7fe6740d2510 comp rx=0 tx=0).stop 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 -- 172.21.15.37:0/3523582350 shutdown_connections 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dd8370 0x564036dd0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dceb40 0x564036dd0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3523582350 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dcd5d0 0x564036de4310 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 -- 172.21.15.37:0/3523582350 >> 172.21.15.37:0/3523582350 conn(0x564036d96310 msgr2=0x564036dd4110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 -- 172.21.15.37:0/3523582350 shutdown_connections 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.993+0000 7fe68bb65740 1 -- 172.21.15.37:0/3523582350 wait complete. 2024-09-19T06:50:00.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 -- start start 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 Processor -- start 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dcd5d0 0x564036de0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036ddbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dd8370 0x564036ddc100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841ac060 con 0x564036dceb40 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841ad4d0 con 0x564036dcd5d0 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6841ac730 con 0x564036dd8370 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe68bb65740 10 client.0 _ll_register_callbacks cb 0x7fe67c011f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe682ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036ddbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe682ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036ddbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41776/0 (socket says 172.21.15.37:41776) 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe682ffd640 1 -- 172.21.15.37:0/3405192603 learned_addr learned my addr 172.21.15.37:0/3405192603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:00.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe6837fe640 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dd8370 0x564036ddc100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.994+0000 7fe67bfff640 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dcd5d0 0x564036de0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe682ffd640 1 -- 172.21.15.37:0/3405192603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dd8370 msgr2=0x564036ddc100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe682ffd640 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dd8370 0x564036ddc100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe682ffd640 1 -- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dcd5d0 msgr2=0x564036de0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe682ffd640 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dcd5d0 0x564036de0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe682ffd640 1 -- 172.21.15.37:0/3405192603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564036ddc640 con 0x564036dceb40 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe682ffd640 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036ddbbc0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe66c02a9a0 tx=0x7fe66c031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aabd769a4e7dfa7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe6727fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe6727fc640 1 -- 172.21.15.37:0/3405192603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66c035040 con 0x564036dceb40 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe6727fc640 1 -- 172.21.15.37:0/3405192603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fe66c004450 con 0x564036dceb40 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe67409c9b0 con 0x564036dceb40 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe6727fc640 1 -- 172.21.15.37:0/3405192603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe66c005020 con 0x564036dceb40 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe68bb65740 10 client.17769 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:00.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x564036e52520 con 0x564036dceb40 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.995+0000 7fe6727fc640 1 -- 172.21.15.37:0/3405192603 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe67409c9b0 con 0x564036dceb40 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe6727fc640 1 -- 172.21.15.37:0/3405192603 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fe66c044070 con 0x564036dceb40 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe68bb65740 10 client.17769 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe68bb65740 10 client.17769 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe68bb65740 20 client.17769 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036e52d20 0x564036e55140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe68bb65740 4 client.17769 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564036d3ad70 con 0x564036e52d20 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe67bfff640 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036e52d20 0x564036e55140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe67bfff640 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036e52d20 0x564036e55140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3287d0191abfa95b server_cookie=c1dabe3bf89708ae in_seq=0 out_seq=0 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.996+0000 7fe6727fc640 10 client.17769 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe6727fc640 1 -- 172.21.15.37:0/3405192603 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564036d3ad70 con 0x564036e52d20 2024-09-19T06:50:00.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe6727fc640 10 client.17769 handle_command_reply: tid=0 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 client.17769 shutdown 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 msgr2=0x564036ddbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036ddbbc0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fe66c02a9a0 tx=0x7fe66c031850 comp rx=0 tx=0).stop 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 shutdown_connections 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x564036e52d20 0x564036e55140 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x564036dd8370 0x564036ddc100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x564036dceb40 0x564036ddbbc0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe6727fc640 0 client.17769 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 --2- 172.21.15.37:0/3405192603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564036dcd5d0 0x564036de0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 >> 172.21.15.37:0/3405192603 conn(0x564036d96310 msgr2=0x564036dd6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 shutdown_connections 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 1 -- 172.21.15.37:0/3405192603 wait complete. 2024-09-19T06:50:00.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.997+0000 7fe68bb65740 20 client.17769 trim_cache size 0 max 16384 2024-09-19T06:50:00.005 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:50:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6640a33b0 msgr2=0x7fe6640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:00.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6640a33b0 0x7fe6640a5870 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe67c022000 tx=0x7fe67c023000 comp rx=0 tx=0).stop 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684102d40 msgr2=0x7fe684198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684102d40 0x7fe684198430 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fe67400ea60 tx=0x7fe67400ef20 comp rx=0 tx=0).stop 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 shutdown_connections 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe68410c170 0x7fe68419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe684103660 0x7fe684198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6640a33b0 0x7fe6640a5870 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 --2- 172.21.15.37:0/2133408541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe684102d40 0x7fe684198430 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 >> 172.21.15.37:0/2133408541 conn(0x7fe6840fc7a0 msgr2=0x7fe6840fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 shutdown_connections 2024-09-19T06:50:00.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:49:59.998+0000 7fe68978c640 1 -- 172.21.15.37:0/2133408541 wait complete. 2024-09-19T06:50:00.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 6.98 MB/sec execute 267 sec latency 25622.949 ms 2024-09-19T06:50:00.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:00 smithi096 ceph-mon[35004]: pgmap v255: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 666 KiB/s wr, 35 op/s 2024-09-19T06:50:00.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:00 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4234655238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:00.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:00 smithi096 ceph-mon[35004]: overall HEALTH_OK 2024-09-19T06:50:00.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:00 smithi037 ceph-mon[31211]: pgmap v255: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 666 KiB/s wr, 35 op/s 2024-09-19T06:50:00.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:00 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4234655238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:00.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:00 smithi037 ceph-mon[31211]: overall HEALTH_OK 2024-09-19T06:50:00.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:50:00.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:00 smithi151 ceph-mon[35236]: pgmap v255: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 666 KiB/s wr, 35 op/s 2024-09-19T06:50:00.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:00 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4234655238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:00 smithi151 ceph-mon[35236]: overall HEALTH_OK 2024-09-19T06:50:01.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 6.95 MB/sec execute 268 sec latency 26623.026 ms 2024-09-19T06:50:01.380 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 506b7504 --release --await' 2024-09-19T06:50:01.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 506b7504 --release --await 2024-09-19T06:50:01.449 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:01.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 -- 172.21.15.37:0/2162089172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60106f10 msgr2=0x7fce6010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 --2- 172.21.15.37:0/2162089172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60106f10 0x7fce6010ea90 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fce58009900 tx=0x7fce5802a410 comp rx=0 tx=0).stop 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 -- 172.21.15.37:0/2162089172 shutdown_connections 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 --2- 172.21.15.37:0/2162089172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60106f10 0x7fce6010ea90 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 --2- 172.21.15.37:0/2162089172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102790 0x7fce601067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 --2- 172.21.15.37:0/2162089172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce60101de0 0x7fce601021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 -- 172.21.15.37:0/2162089172 >> 172.21.15.37:0/2162089172 conn(0x7fce600fd8c0 msgr2=0x7fce600ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 -- 172.21.15.37:0/2162089172 shutdown_connections 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.900+0000 7fce65917640 1 -- 172.21.15.37:0/2162089172 wait complete. 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 Processor -- start 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 -- start start 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60101de0 0x7fce601983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102790 0x7fce60198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce60106f10 0x7fce6019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce60111f70 con 0x7fce60101de0 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce60111df0 con 0x7fce60102790 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.901+0000 7fce65917640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce601120f0 con 0x7fce60106f10 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60101de0 0x7fce601983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60101de0 0x7fce601983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41804/0 (socket says 172.21.15.37:41804) 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 -- 172.21.15.37:0/240075945 learned_addr learned my addr 172.21.15.37:0/240075945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102790 0x7fce60198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce60106f10 0x7fce6019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 -- 172.21.15.37:0/240075945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce60106f10 msgr2=0x7fce6019ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce60106f10 0x7fce6019ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 -- 172.21.15.37:0/240075945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102790 msgr2=0x7fce60198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102790 0x7fce60198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 -- 172.21.15.37:0/240075945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce6019d3a0 con 0x7fce60101de0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102790 0x7fce60198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce5effd640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60101de0 0x7fce601983f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fce5000ebc0 tx=0x7fce5000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce64915640 1 -- 172.21.15.37:0/240075945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50010630 con 0x7fce60101de0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce6019d690 con 0x7fce60101de0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce64915640 1 -- 172.21.15.37:0/240075945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce50010cb0 con 0x7fce60101de0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.902+0000 7fce64915640 1 -- 172.21.15.37:0/240075945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50018bd0 con 0x7fce60101de0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.903+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce601a4f70 con 0x7fce60101de0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 Processor -- start 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 -- start start 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce64915640 1 -- 172.21.15.37:0/240075945 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fce500107d0 con 0x7fce60101de0 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f58090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f585d0 0x5570e2f58a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f67780 0x5570e2f69b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.960 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570e2ebe2f0 con 0x5570e2f585d0 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570e2e65cf0 con 0x5570e2f59b40 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce67cf0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570e2eba2b0 con 0x5570e2f67780 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce5effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f585d0 0x5570e2f58a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce5effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f585d0 0x5570e2f58a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41814/0 (socket says 172.21.15.37:41814) 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce5e7fc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f67780 0x5570e2f69b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce5effd640 1 -- 172.21.15.37:0/904433802 learned_addr learned my addr 172.21.15.37:0/904433802 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f58090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce5e7fc640 1 -- 172.21.15.37:0/904433802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 msgr2=0x5570e2f58090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.906+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/904433802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f58090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce5e7fc640 1 -- 172.21.15.37:0/904433802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f585d0 msgr2=0x5570e2f58a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/904433802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f585d0 0x5570e2f58a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.961 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce5e7fc640 1 -- 172.21.15.37:0/904433802 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570e2e4a3a0 con 0x5570e2f67780 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/904433802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f67780 0x5570e2f69b70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce48009f20 tx=0x7fce480118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f3fd379b1acfe29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce55ffb640 1 -- 172.21.15.37:0/904433802 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce48004020 con 0x5570e2f67780 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce55ffb640 1 -- 172.21.15.37:0/904433802 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce480041c0 con 0x5570e2f67780 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce55ffb640 1 -- 172.21.15.37:0/904433802 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce4800f8b0 con 0x5570e2f67780 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce64915640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce4c0a33b0 0x7fce4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.907+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce4c0a33b0 0x7fce4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce64915640 1 -- 172.21.15.37:0/240075945 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fce500d0370 con 0x7fce60101de0 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 -- 172.21.15.37:0/904433802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f67780 msgr2=0x5570e2f69b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 --2- 172.21.15.37:0/904433802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f67780 0x5570e2f69b70 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fce48009f20 tx=0x7fce480118d0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce4c0a33b0 0x7fce4c0a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce58005c40 tx=0x7fce58005b50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 -- 172.21.15.37:0/904433802 shutdown_connections 2024-09-19T06:50:01.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 --2- 172.21.15.37:0/904433802 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f67780 0x5570e2f69b70 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 --2- 172.21.15.37:0/904433802 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f585d0 0x5570e2f58a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 --2- 172.21.15.37:0/904433802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f58090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 -- 172.21.15.37:0/904433802 >> 172.21.15.37:0/904433802 conn(0x5570e2f21310 msgr2=0x5570e2f54c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 -- 172.21.15.37:0/904433802 shutdown_connections 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 -- 172.21.15.37:0/904433802 wait complete. 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.908+0000 7fce67cf0740 1 -- start start 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 1 Processor -- start 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f670a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f59b40 0x5570e2f64350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f67780 0x5570e2f64890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5570e2f6a480 con 0x5570e2f59b40 2024-09-19T06:50:01.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5570e2e79060 con 0x5570e2f67780 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5570e2f6a600 con 0x5570e2f585d0 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce67cf0740 10 client.0 _ll_register_callbacks cb 0x5570e2bcc8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f59b40 0x5570e2f64350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce5effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f670a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce5e7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f59b40 0x5570e2f64350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41818/0 (socket says 172.21.15.37:41818) 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce5e7fc640 1 -- 172.21.15.37:0/2212357108 learned_addr learned my addr 172.21.15.37:0/2212357108 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.909+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f67780 0x5570e2f64890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce5effd640 1 -- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f67780 msgr2=0x5570e2f64890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce5effd640 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f67780 0x5570e2f64890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce5effd640 1 -- 172.21.15.37:0/2212357108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f59b40 msgr2=0x5570e2f64350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce5effd640 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f59b40 0x5570e2f64350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce5effd640 1 -- 172.21.15.37:0/2212357108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570e2c2f3c0 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce5effd640 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f670a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce500ce090 tx=0x7fce500099b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=743fe5a5ccfe1379 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce3effd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce3effd640 1 -- 172.21.15.37:0/2212357108 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50010e60 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce3effd640 1 -- 172.21.15.37:0/2212357108 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fce50098d40 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce480041c0 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce3effd640 1 -- 172.21.15.37:0/2212357108 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fce5009c4a0 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce67cf0740 10 client.26908 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5570e2f75ed0 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce3effd640 1 -- 172.21.15.37:0/2212357108 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce480041c0 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.910+0000 7fce3effd640 1 -- 172.21.15.37:0/2212357108 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fce500b0b40 con 0x5570e2f585d0 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce67cf0740 10 client.26908 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce67cf0740 10 client.26908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:01.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce67cf0740 20 client.26908 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce67cf0740 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2f77380 0x5570e2f797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce67cf0740 4 client.26908 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5570e2ecb860 con 0x5570e2f77380 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2f77380 0x5570e2f797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce3effd640 10 client.26908 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.911+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2f77380 0x5570e2f797e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95c5967a67209415 server_cookie=39968f70b0ed0809 in_seq=0 out_seq=0 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce3effd640 1 -- 172.21.15.37:0/2212357108 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5570e2ecb860 con 0x5570e2f77380 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce3effd640 10 client.26908 handle_command_reply: tid=0 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 client.26908 shutdown 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 msgr2=0x5570e2f670a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f670a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce500ce090 tx=0x7fce500099b0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 shutdown_connections 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2f77380 0x5570e2f797e0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f67780 0x5570e2f64890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f59b40 0x5570e2f64350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 --2- 172.21.15.37:0/2212357108 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f670a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce3effd640 0 client.26908 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 >> 172.21.15.37:0/2212357108 conn(0x5570e2f21310 msgr2=0x5570e2f5a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 shutdown_connections 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.912+0000 7fce67cf0740 1 -- 172.21.15.37:0/2212357108 wait complete. 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.913+0000 7fce67cf0740 20 client.26908 trim_cache size 0 max 16384 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 Processor -- start 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 -- start start 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f54e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f73750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f7f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce601ae120 con 0x5570e2f67780 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce601ab990 con 0x5570e2f59b40 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce67cf0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce601a9800 con 0x5570e2f585d0 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce5effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f54e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f7f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce5e7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f73750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce5effd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f54e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:44054/0 (socket says 172.21.15.37:44054) 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce5effd640 1 -- 172.21.15.37:0/191455431 learned_addr learned my addr 172.21.15.37:0/191455431 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.926+0000 7fce5effd640 1 -- 172.21.15.37:0/191455431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 msgr2=0x5570e2f73750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce5effd640 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f73750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce5effd640 1 -- 172.21.15.37:0/191455431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 msgr2=0x5570e2f7f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce5effd640 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f7f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce5effd640 1 -- 172.21.15.37:0/191455431 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5570e2f5b120 con 0x5570e2f585d0 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f73750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f7f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce5effd640 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f54e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce5009c660 tx=0x7fce50000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56e912f564147b36 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce557fa640 1 -- 172.21.15.37:0/191455431 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce50010e60 con 0x5570e2f585d0 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce557fa640 1 -- 172.21.15.37:0/191455431 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5009c6d0 con 0x5570e2f585d0 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.927+0000 7fce557fa640 1 -- 172.21.15.37:0/191455431 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce5009c030 con 0x5570e2f585d0 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 -- 172.21.15.37:0/191455431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 msgr2=0x5570e2f54e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f54e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce5009c660 tx=0x7fce50000f40 comp rx=0 tx=0).stop 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 -- 172.21.15.37:0/191455431 shutdown_connections 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f7f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f73750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 --2- 172.21.15.37:0/191455431 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f54e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 -- 172.21.15.37:0/191455431 >> 172.21.15.37:0/191455431 conn(0x5570e2f21310 msgr2=0x5570e2f54700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 -- 172.21.15.37:0/191455431 shutdown_connections 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.928+0000 7fce67cf0740 1 -- 172.21.15.37:0/191455431 wait complete. 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 -- start start 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 Processor -- start 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f6fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f70140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce601a9e50 con 0x5570e2f67780 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce601ab2c0 con 0x5570e2f59b40 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce601aa520 con 0x5570e2f585d0 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce67cf0740 10 client.0 _ll_register_callbacks cb 0x7fce5000eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f6b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce5f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f6b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41828/0 (socket says 172.21.15.37:41828) 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce5f7fe640 1 -- 172.21.15.37:0/3921940912 learned_addr learned my addr 172.21.15.37:0/3921940912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce5effd640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f6fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.929+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f70140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5f7fe640 1 -- 172.21.15.37:0/3921940912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 msgr2=0x5570e2f6fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f6fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5f7fe640 1 -- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 msgr2=0x5570e2f70140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f70140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5f7fe640 1 -- 172.21.15.37:0/3921940912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5570e2f6b830 con 0x5570e2f67780 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5effd640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f6fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5e7fc640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f70140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce5f7fe640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f6b000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fce58009900 tx=0x7fce58009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1be2149ac283b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce3effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:01.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce3effd640 1 -- 172.21.15.37:0/3921940912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce58035040 con 0x5570e2f67780 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce3effd640 1 -- 172.21.15.37:0/3921940912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fce58003800 con 0x5570e2f67780 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce5009c6d0 con 0x5570e2f67780 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce3effd640 1 -- 172.21.15.37:0/3921940912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fce58003a00 con 0x5570e2f67780 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce67cf0740 10 client.17787 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.930+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5570e2fe1cc0 con 0x5570e2f67780 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce3effd640 1 -- 172.21.15.37:0/3921940912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce5009c6d0 con 0x5570e2f67780 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce3effd640 1 -- 172.21.15.37:0/3921940912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fce5805d430 con 0x5570e2f67780 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce67cf0740 10 client.17787 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce67cf0740 10 client.17787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce67cf0740 20 client.17787 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce67cf0740 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2fe24f0 0x5570e2fe4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce67cf0740 4 client.17787 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:01.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5570e2ec5d70 con 0x5570e2fe24f0 2024-09-19T06:50:01.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce5effd640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2fe24f0 0x5570e2fe4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:01.973 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:50:01.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce5effd640 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2fe24f0 0x5570e2fe4910 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4451219243a2b6e0 server_cookie=af01be377ada233f in_seq=0 out_seq=0 2024-09-19T06:50:01.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.931+0000 7fce3effd640 10 client.17787 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:01.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce3effd640 1 -- 172.21.15.37:0/3921940912 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5570e2ec5d70 con 0x5570e2fe24f0 2024-09-19T06:50:01.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce3effd640 10 client.17787 handle_command_reply: tid=0 2024-09-19T06:50:01.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 client.17787 shutdown 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 msgr2=0x5570e2f6b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f6b000 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fce58009900 tx=0x7fce58009260 comp rx=0 tx=0).stop 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 shutdown_connections 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5570e2fe24f0 0x5570e2fe4910 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5570e2f67780 0x5570e2f6b000 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5570e2f59b40 0x5570e2f70140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 --2- 172.21.15.37:0/3921940912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5570e2f585d0 0x5570e2f6fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce3effd640 0 client.17787 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 >> 172.21.15.37:0/3921940912 conn(0x5570e2f21310 msgr2=0x5570e2f5a930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 shutdown_connections 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 1 -- 172.21.15.37:0/3921940912 wait complete. 2024-09-19T06:50:01.974 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.932+0000 7fce67cf0740 20 client.17787 trim_cache size 0 max 16384 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce4c0a33b0 msgr2=0x7fce4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce4c0a33b0 0x7fce4c0a5870 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fce58005c40 tx=0x7fce58005b50 comp rx=0 tx=0).stop 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60101de0 msgr2=0x7fce601983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60101de0 0x7fce601983f0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fce5000ebc0 tx=0x7fce5000c6a0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 shutdown_connections 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce60106f10 0x7fce6019ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce60102790 0x7fce60198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce4c0a33b0 0x7fce4c0a5870 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 --2- 172.21.15.37:0/240075945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce60101de0 0x7fce601983f0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 >> 172.21.15.37:0/240075945 conn(0x7fce600fd8c0 msgr2=0x7fce600feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 shutdown_connections 2024-09-19T06:50:01.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:01.933+0000 7fce65917640 1 -- 172.21.15.37:0/240075945 wait complete. 2024-09-19T06:50:02.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567670 6.92 MB/sec execute 269 sec latency 27623.110 ms 2024-09-19T06:50:02.263 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:02.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:02 smithi037 ceph-mon[31211]: pgmap v256: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 661 KiB/s wr, 31 op/s 2024-09-19T06:50:02.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:02 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:50:02.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:02 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:50:02.674 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:02 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:50:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:02 smithi096 ceph-mon[35004]: pgmap v256: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 661 KiB/s wr, 31 op/s 2024-09-19T06:50:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:02 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:50:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:02 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:50:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:02 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:50:02.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:02 smithi151 ceph-mon[35236]: pgmap v256: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 661 KiB/s wr, 31 op/s 2024-09-19T06:50:02.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:02 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:50:02.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:02 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:50:02.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:02 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:50:03.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.172+0000 7fe0cb26d640 1 -- 172.21.15.37:0/3303042373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4106a80 msgr2=0x7fe0c4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.172+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/3303042373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4106a80 0x7fe0c4106e60 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009930 tx=0x7fe0c002a420 comp rx=0 tx=0).stop 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.172+0000 7fe0cb26d640 1 -- 172.21.15.37:0/3303042373 shutdown_connections 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.172+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/3303042373 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c4101420 0x7fe0c410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.172+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/3303042373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4100a80 0x7fe0c4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.172+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/3303042373 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4106a80 0x7fe0c4106e60 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.173+0000 7fe0cb26d640 1 -- 172.21.15.37:0/3303042373 >> 172.21.15.37:0/3303042373 conn(0x7fe0c40fc7a0 msgr2=0x7fe0c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.173+0000 7fe0cb26d640 1 -- 172.21.15.37:0/3303042373 shutdown_connections 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.173+0000 7fe0cb26d640 1 -- 172.21.15.37:0/3303042373 wait complete. 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.173+0000 7fe0cb26d640 1 Processor -- start 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.173+0000 7fe0cb26d640 1 -- start start 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.173+0000 7fe0cb26d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4100a80 0x7fe0c41984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0cb26d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4101420 0x7fe0c4198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0cb26d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c4106a80 0x7fe0c419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0cb26d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4110ea0 con 0x7fe0c4100a80 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0cb26d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4110d20 con 0x7fe0c4101420 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0cb26d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c4111020 con 0x7fe0c4106a80 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4100a80 0x7fe0c41984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4100a80 0x7fe0c41984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41842/0 (socket says 172.21.15.37:41842) 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 -- 172.21.15.37:0/4119859126 learned_addr learned my addr 172.21.15.37:0/4119859126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c8808640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4101420 0x7fe0c4198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 -- 172.21.15.37:0/4119859126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c4106a80 msgr2=0x7fe0c419cd90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c4106a80 0x7fe0c419cd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 -- 172.21.15.37:0/4119859126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4101420 msgr2=0x7fe0c4198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4101420 0x7fe0c4198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 -- 172.21.15.37:0/4119859126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0c419d470 con 0x7fe0c4100a80 2024-09-19T06:50:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c8808640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4101420 0x7fe0c4198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0c9009640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4100a80 0x7fe0c41984c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009230 tx=0x7fe0c0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0ba7fc640 1 -- 172.21.15.37:0/4119859126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c003f030 con 0x7fe0c4100a80 2024-09-19T06:50:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0ba7fc640 1 -- 172.21.15.37:0/4119859126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0c0003ad0 con 0x7fe0c4100a80 2024-09-19T06:50:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0c419d700 con 0x7fe0c4100a80 2024-09-19T06:50:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.174+0000 7fe0ba7fc640 1 -- 172.21.15.37:0/4119859126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c0035070 con 0x7fe0c4100a80 2024-09-19T06:50:03.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.175+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0c41a5040 con 0x7fe0c4100a80 2024-09-19T06:50:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.177+0000 7fe0ba7fc640 1 -- 172.21.15.37:0/4119859126 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe0c0042030 con 0x7fe0c4100a80 2024-09-19T06:50:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.177+0000 7fe0cd646740 1 Processor -- start 2024-09-19T06:50:03.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0cd646740 1 -- start start 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3060 0x55eae1dd3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3980 0x55eae1dddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1de0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0cd646740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eae1d38620 con 0x55eae1dd3060 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0cd646740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eae1cdfe50 con 0x55eae1dd3980 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0cd646740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eae1d345e0 con 0x55eae1dde240 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0ba7fc640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe0b00a33b0 0x7fe0b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3060 0x55eae1dd3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3060 0x55eae1dd3440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41846/0 (socket says 172.21.15.37:41846) 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 -- 172.21.15.37:0/4244911235 learned_addr learned my addr 172.21.15.37:0/4244911235 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:03.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c8808640 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1de0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c9009640 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3980 0x55eae1dddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0ba7fc640 1 -- 172.21.15.37:0/4119859126 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fe0c00f0380 con 0x7fe0c4100a80 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 -- 172.21.15.37:0/4244911235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 msgr2=0x55eae1de0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1de0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 -- 172.21.15.37:0/4244911235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3980 msgr2=0x55eae1dddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3980 0x55eae1dddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c980a640 1 -- 172.21.15.37:0/4244911235 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae1cd0d40 con 0x55eae1dd3060 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.178+0000 7fe0c8808640 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1de0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.179+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe0b00a33b0 0x7fe0b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.179+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3060 0x55eae1dd3440 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009f20 tx=0x7fe0bc00d500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e577a9be930dee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:03.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.179+0000 7fe0af7fe640 1 -- 172.21.15.37:0/4244911235 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc020640 con 0x55eae1dd3060 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.179+0000 7fe0af7fe640 1 -- 172.21.15.37:0/4244911235 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0bc0148c0 con 0x55eae1dd3060 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.179+0000 7fe0af7fe640 1 -- 172.21.15.37:0/4244911235 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc013650 con 0x55eae1dd3060 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.179+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe0b00a33b0 0x7fe0b00a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc01d000 tx=0x7fe0bc01e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 -- 172.21.15.37:0/4244911235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3060 msgr2=0x55eae1dd3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3060 0x55eae1dd3440 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc009f20 tx=0x7fe0bc00d500 comp rx=0 tx=0).stop 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 -- 172.21.15.37:0/4244911235 shutdown_connections 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1de0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3980 0x55eae1dddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4244911235 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3060 0x55eae1dd3440 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 -- 172.21.15.37:0/4244911235 >> 172.21.15.37:0/4244911235 conn(0x55eae1dcf5f0 msgr2=0x55eae1dcf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 -- 172.21.15.37:0/4244911235 shutdown_connections 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 -- 172.21.15.37:0/4244911235 wait complete. 2024-09-19T06:50:03.187 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 -- start start 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.180+0000 7fe0cd646740 1 Processor -- start 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1ddb160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1ddb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1ddbbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0cd646740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eae1de0e10 con 0x55eae1dd3980 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0cd646740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eae1cc0f10 con 0x55eae1dd3060 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0cd646740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eae1de0f90 con 0x55eae1dde240 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0cd646740 10 client.0 _ll_register_callbacks cb 0x55eae1ce4db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1ddb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1ddb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41850/0 (socket says 172.21.15.37:41850) 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 -- 172.21.15.37:0/310688148 learned_addr learned my addr 172.21.15.37:0/310688148 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:03.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c9009640 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1ddb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 -- 172.21.15.37:0/310688148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 msgr2=0x55eae1ddbbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1ddbbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 -- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 msgr2=0x55eae1ddb160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1ddb160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 -- 172.21.15.37:0/310688148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae1a47580 con 0x55eae1dd3980 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0c8808640 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1ddb6a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4010490 tx=0x7fe0b4010950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c58b7f417ea478 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.181+0000 7fe0acff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0acff9640 1 -- 172.21.15.37:0/310688148 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4005220 con 0x55eae1dd3980 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0acff9640 1 -- 172.21.15.37:0/310688148 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fe0b40053c0 con 0x55eae1dd3980 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0bc0148c0 con 0x55eae1dd3980 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0acff9640 1 -- 172.21.15.37:0/310688148 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe0b40055c0 con 0x55eae1dd3980 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 10 client.17799 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:03.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55eae1b11960 con 0x55eae1dd3980 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0acff9640 1 -- 172.21.15.37:0/310688148 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0bc0148c0 con 0x55eae1dd3980 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0acff9640 1 -- 172.21.15.37:0/310688148 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fe0b400f040 con 0x55eae1dd3980 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 10 client.17799 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 10 client.17799 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 20 client.17799 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1df0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 4 client.17799 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.182+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eae1d45b90 con 0x55eae1dedf60 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.183+0000 7fe0c9009640 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1df0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.183+0000 7fe0acff9640 10 client.17799 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:03.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.183+0000 7fe0c9009640 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1df0400 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99832a71fea503f6 server_cookie=727f8954dc25ef43 in_seq=0 out_seq=0 2024-09-19T06:50:03.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.183+0000 7fe0acff9640 1 -- 172.21.15.37:0/310688148 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eae1d45b90 con 0x55eae1dedf60 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.183+0000 7fe0acff9640 10 client.17799 handle_command_reply: tid=0 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 client.17799 shutdown 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 msgr2=0x55eae1ddb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1ddb6a0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4010490 tx=0x7fe0b4010950 comp rx=0 tx=0).stop 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 shutdown_connections 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1df0400 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1ddbbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0acff9640 0 client.17799 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1ddb6a0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 --2- 172.21.15.37:0/310688148 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1ddb160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 >> 172.21.15.37:0/310688148 conn(0x55eae1dcf5f0 msgr2=0x55eae1dd5320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:03.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 shutdown_connections 2024-09-19T06:50:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 1 -- 172.21.15.37:0/310688148 wait complete. 2024-09-19T06:50:03.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.184+0000 7fe0cd646740 20 client.17799 trim_cache size 0 max 16384 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 Processor -- start 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 -- start start 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1dd6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1dce640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1df5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41b02d0 con 0x55eae1dd3980 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41adb40 con 0x55eae1dd3060 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0cd646740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41ab9b0 con 0x55eae1dde240 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0c8808640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1dce640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0c8808640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1dce640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41866/0 (socket says 172.21.15.37:41866) 2024-09-19T06:50:03.207 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0c8808640 1 -- 172.21.15.37:0/845446183 learned_addr learned my addr 172.21.15.37:0/845446183 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.201+0000 7fe0c9009640 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1dd6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0c980a640 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1df5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0c8808640 1 -- 172.21.15.37:0/845446183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 msgr2=0x55eae1df5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0c8808640 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1df5d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0c8808640 1 -- 172.21.15.37:0/845446183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 msgr2=0x55eae1dd6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0c8808640 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1dd6480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0c8808640 1 -- 172.21.15.37:0/845446183 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eae1ddd0a0 con 0x55eae1dd3980 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0c8808640 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1dce640 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d00 tx=0x7fe0b400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=237b7f5f73e157cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0aeffd640 1 -- 172.21.15.37:0/845446183 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4031020 con 0x55eae1dd3980 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0aeffd640 1 -- 172.21.15.37:0/845446183 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b4029040 con 0x55eae1dd3980 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.202+0000 7fe0aeffd640 1 -- 172.21.15.37:0/845446183 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4028070 con 0x55eae1dd3980 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 -- 172.21.15.37:0/845446183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 msgr2=0x55eae1dce640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1dce640 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4005d00 tx=0x7fe0b400ec20 comp rx=0 tx=0).stop 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 -- 172.21.15.37:0/845446183 shutdown_connections 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dde240 0x55eae1df5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dd3980 0x55eae1dce640 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 --2- 172.21.15.37:0/845446183 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1dd6480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 -- 172.21.15.37:0/845446183 >> 172.21.15.37:0/845446183 conn(0x55eae1dcf5f0 msgr2=0x55eae1dcdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 -- 172.21.15.37:0/845446183 shutdown_connections 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 -- 172.21.15.37:0/845446183 wait complete. 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 -- start start 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.203+0000 7fe0cd646740 1 Processor -- start 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1de61e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dd3980 0x55eae1de6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0cd646740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dde240 0x55eae1de31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0cd646740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41ac000 con 0x55eae1dde240 2024-09-19T06:50:03.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0cd646740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41ad470 con 0x55eae1dd3060 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0cd646740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe0c41ac6d0 con 0x55eae1dd3980 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dde240 0x55eae1de31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0cd646740 10 client.0 _ll_register_callbacks cb 0x7fe0b400ecb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dde240 0x55eae1de31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41874/0 (socket says 172.21.15.37:41874) 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 -- 172.21.15.37:0/4280075321 learned_addr learned my addr 172.21.15.37:0/4280075321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c9009640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1de61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c8808640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dd3980 0x55eae1de6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 -- 172.21.15.37:0/4280075321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dd3980 msgr2=0x55eae1de6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dd3980 0x55eae1de6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 -- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 msgr2=0x55eae1de61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1de61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 -- 172.21.15.37:0/4280075321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eae1de37f0 con 0x55eae1dde240 2024-09-19T06:50:03.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c8808640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dd3980 0x55eae1de6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0c980a640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dde240 0x55eae1de31b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc015f50 tx=0x7fe0bc013760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d267761b435b9ec3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0acff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0acff9640 1 -- 172.21.15.37:0/4280075321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc02c070 con 0x55eae1dde240 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0b4029040 con 0x55eae1dde240 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.204+0000 7fe0acff9640 1 -- 172.21.15.37:0/4280075321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fe0bc013c40 con 0x55eae1dde240 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0acff9640 1 -- 172.21.15.37:0/4280075321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fe0bc013e40 con 0x55eae1dde240 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 10 client.17805 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55eae1e59670 con 0x55eae1dde240 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0acff9640 1 -- 172.21.15.37:0/4280075321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe0b4029040 con 0x55eae1dde240 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0acff9640 1 -- 172.21.15.37:0/4280075321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fe0bc028400 con 0x55eae1dde240 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 10 client.17805 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 10 client.17805 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 20 client.17805 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:03.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1e5bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 4 client.17805 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "506b7504", "release": true, "await": true}] 2024-09-19T06:50:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "506b7504", "release": true, "await": true}) -- 0x55eae1d400a0 con 0x55eae1dedf60 2024-09-19T06:50:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.205+0000 7fe0c9009640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1e5bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.206+0000 7fe0c9009640 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1e5bd10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30eff777f100a62f server_cookie=1272a92101b11b1d in_seq=0 out_seq=0 2024-09-19T06:50:03.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.206+0000 7fe0acff9640 10 client.17805 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0acff9640 1 -- 172.21.15.37:0/4280075321 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55eae1d400a0 con 0x55eae1dedf60 2024-09-19T06:50:03.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0acff9640 10 client.17805 handle_command_reply: tid=0 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 client.17805 shutdown 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dde240 msgr2=0x55eae1de31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dde240 0x55eae1de31b0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc015f50 tx=0x7fe0bc013760 comp rx=0 tx=0).stop 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 shutdown_connections 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eae1dedf60 0x55eae1e5bd10 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eae1dde240 0x55eae1de31b0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eae1dd3980 0x55eae1de6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0acff9640 0 client.17805 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 --2- 172.21.15.37:0/4280075321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eae1dd3060 0x55eae1de61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.215+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 >> 172.21.15.37:0/4280075321 conn(0x55eae1dcf5f0 msgr2=0x55eae1dce750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 shutdown_connections 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cd646740 1 -- 172.21.15.37:0/4280075321 wait complete. 2024-09-19T06:50:03.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cd646740 20 client.17805 trim_cache size 0 max 16384 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 36, 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 32, 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "506b7504": { 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 32, 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:50:03.219 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 145.8, 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:03.220 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe0b00a33b0 msgr2=0x7fe0b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe0b00a33b0 0x7fe0b00a5870 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc01d000 tx=0x7fe0bc01e000 comp rx=0 tx=0).stop 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4100a80 msgr2=0x7fe0c41984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4100a80 0x7fe0c41984c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fe0c0009230 tx=0x7fe0c0004270 comp rx=0 tx=0).stop 2024-09-19T06:50:03.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 shutdown_connections 2024-09-19T06:50:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe0c4106a80 0x7fe0c419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe0c4101420 0x7fe0c4198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe0b00a33b0 0x7fe0b00a5870 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 --2- 172.21.15.37:0/4119859126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe0c4100a80 0x7fe0c41984c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.216+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 >> 172.21.15.37:0/4119859126 conn(0x7fe0c40fc7a0 msgr2=0x7fe0c4104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.217+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 shutdown_connections 2024-09-19T06:50:03.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:03.217+0000 7fe0cb26d640 1 -- 172.21.15.37:0/4119859126 wait complete. 2024-09-19T06:50:03.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 567673 6.90 MB/sec execute 270 sec latency 28622.913 ms 2024-09-19T06:50:03.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:03 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:50:03.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:03 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:50:03.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:03 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:50:04.003 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:50:04.003 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:50:04.148 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '506b7504', total seconds elapsed: 32.2 2024-09-19T06:50:04.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 569816 6.92 MB/sec execute 271 sec latency 13.325 ms 2024-09-19T06:50:04.267 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:04.695 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:04 smithi037 ceph-mon[31211]: pgmap v257: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 712 KiB/s wr, 52 op/s 2024-09-19T06:50:04.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:04 smithi096 ceph-mon[35004]: pgmap v257: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 712 KiB/s wr, 52 op/s 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.780+0000 7f63cbb12640 1 -- 172.21.15.37:0/3771915659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 msgr2=0x7f63c410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.780+0000 7f63cbb12640 1 --2- 172.21.15.37:0/3771915659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c410da40 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f63bc009ca0 tx=0x7f63bc02a950 comp rx=0 tx=0).stop 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.780+0000 7f63cbb12640 1 -- 172.21.15.37:0/3771915659 shutdown_connections 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.780+0000 7f63cbb12640 1 --2- 172.21.15.37:0/3771915659 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c410da40 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.780+0000 7f63cbb12640 1 --2- 172.21.15.37:0/3771915659 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c4102570 0x7f63c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.781+0000 7f63cbb12640 1 --2- 172.21.15.37:0/3771915659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63c4100e20 0x7f63c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.781+0000 7f63cbb12640 1 -- 172.21.15.37:0/3771915659 >> 172.21.15.37:0/3771915659 conn(0x7f63c40fc7d0 msgr2=0x7f63c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.781+0000 7f63cbb12640 1 -- 172.21.15.37:0/3771915659 shutdown_connections 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.781+0000 7f63cbb12640 1 -- 172.21.15.37:0/3771915659 wait complete. 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.781+0000 7f63cbb12640 1 Processor -- start 2024-09-19T06:50:04.783 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.781+0000 7f63cbb12640 1 -- start start 2024-09-19T06:50:04.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63cbb12640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c4100e20 0x7f63c4197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63cbb12640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63c4102570 0x7f63c4197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63cbb12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c419bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63cbb12640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4110f80 con 0x7f63c4102ef0 2024-09-19T06:50:04.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63cbb12640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4110e00 con 0x7f63c4102570 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63cbb12640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63c4111100 con 0x7f63c4100e20 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c419bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c419bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41896/0 (socket says 172.21.15.37:41896) 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c90ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63c4102570 0x7f63c4197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63ca0af640 1 -- 172.21.15.37:0/1120876571 learned_addr learned my addr 172.21.15.37:0/1120876571 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c98ae640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c4100e20 0x7f63c4197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c90ad640 1 -- 172.21.15.37:0/1120876571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c4100e20 msgr2=0x7f63c4197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c90ad640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c4100e20 0x7f63c4197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c90ad640 1 -- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 msgr2=0x7f63c419bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c90ad640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c419bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c90ad640 1 -- 172.21.15.37:0/1120876571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63c419c3b0 con 0x7f63c4102570 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63ca0af640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c98ae640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c4100e20 0x7f63c4197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:04.786 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.782+0000 7f63c90ad640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63c4102570 0x7f63c4197940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63b400e9f0 tx=0x7f63b400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.783+0000 7f63baffd640 1 -- 172.21.15.37:0/1120876571 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b400ccc0 con 0x7f63c4102570 2024-09-19T06:50:04.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.783+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63c419c6a0 con 0x7f63c4102570 2024-09-19T06:50:04.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.783+0000 7f63baffd640 1 -- 172.21.15.37:0/1120876571 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63b4005020 con 0x7f63c4102570 2024-09-19T06:50:04.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.783+0000 7f63baffd640 1 -- 172.21.15.37:0/1120876571 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b40049e0 con 0x7f63c4102570 2024-09-19T06:50:04.787 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.783+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63c4069f70 con 0x7f63c4102570 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.785+0000 7f63baffd640 1 -- 172.21.15.37:0/1120876571 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f63b4005560 con 0x7f63c4102570 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.785+0000 7f63cdeeb740 1 Processor -- start 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63cdeeb740 1 -- start start 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfbb60 0x55e75cbfbf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc06800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc09130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e75cb61280 con 0x55e75cbfbb60 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e75cb08b10 con 0x55e75cc06d40 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e75cb5d240 con 0x55e75cbfc480 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfbb60 0x55e75cbfbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfbb60 0x55e75cbfbf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41900/0 (socket says 172.21.15.37:41900) 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63c98ae640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc06800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 -- 172.21.15.37:0/907789517 learned_addr learned my addr 172.21.15.37:0/907789517 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63c90ad640 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc09130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63baffd640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f63b00a33b0 0x7f63b00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 -- 172.21.15.37:0/907789517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 msgr2=0x55e75cc06800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc06800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 -- 172.21.15.37:0/907789517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 msgr2=0x55e75cc09130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc09130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63ca0af640 1 -- 172.21.15.37:0/907789517 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75cb151f0 con 0x55e75cbfbb60 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.786+0000 7f63c98ae640 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc06800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.787+0000 7f63baffd640 1 -- 172.21.15.37:0/1120876571 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f63b40cfca0 con 0x7f63c4102570 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.787+0000 7f63ca0af640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f63b00a33b0 0x7f63b00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.787+0000 7f63ca0af640 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfbb60 0x55e75cbfbf40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f63bc005f70 tx=0x7f63bc02fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84fef19767546e9c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.787+0000 7f63affff640 1 -- 172.21.15.37:0/907789517 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc034040 con 0x55e75cbfbb60 2024-09-19T06:50:04.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.787+0000 7f63affff640 1 -- 172.21.15.37:0/907789517 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63bc003e40 con 0x55e75cbfbb60 2024-09-19T06:50:04.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.787+0000 7f63affff640 1 -- 172.21.15.37:0/907789517 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc039990 con 0x55e75cbfbb60 2024-09-19T06:50:04.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.787+0000 7f63ca0af640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f63b00a33b0 0x7f63b00a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63bc04c000 tx=0x7f63bc049000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 -- 172.21.15.37:0/907789517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfbb60 msgr2=0x55e75cbfbf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfbb60 0x55e75cbfbf40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f63bc005f70 tx=0x7f63bc02fe00 comp rx=0 tx=0).stop 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 -- 172.21.15.37:0/907789517 shutdown_connections 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc09130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc06800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/907789517 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfbb60 0x55e75cbfbf40 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 -- 172.21.15.37:0/907789517 >> 172.21.15.37:0/907789517 conn(0x55e75cbf80f0 msgr2=0x55e75cbf84d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 -- 172.21.15.37:0/907789517 shutdown_connections 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 -- 172.21.15.37:0/907789517 wait complete. 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 -- start start 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.788+0000 7f63cdeeb740 1 Processor -- start 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc03be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfc480 0x55e75cc04120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc04660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e75cc0dd20 con 0x55e75cbfc480 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e75cb1bf10 con 0x55e75cc06d40 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e75cc0dea0 con 0x55e75cbfbb60 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63cdeeb740 10 client.0 _ll_register_callbacks cb 0x55e75cae4a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfc480 0x55e75cc04120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfc480 0x55e75cc04120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41906/0 (socket says 172.21.15.37:41906) 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc04660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 -- 172.21.15.37:0/3276020061 learned_addr learned my addr 172.21.15.37:0/3276020061 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c98ae640 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc03be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 -- 172.21.15.37:0/3276020061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 msgr2=0x55e75cc03be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc03be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 -- 172.21.15.37:0/3276020061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 msgr2=0x55e75cc04660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc04660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c90ad640 1 -- 172.21.15.37:0/3276020061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e75c8d26a0 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.789+0000 7f63c98ae640 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc03be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63c90ad640 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfc480 0x55e75cc04120 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f63b4098350 tx=0x7f63b4005440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e1675af30b940ef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63ad7fa640 1 -- 172.21.15.37:0/3276020061 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63b401c040 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e75cb6e7f0 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63ad7fa640 1 -- 172.21.15.37:0/3276020061 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f63b40a48d0 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63ad7fa640 1 -- 172.21.15.37:0/3276020061 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f63b40cd7f0 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 10 client.17817 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e75cc19950 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63ad7fa640 1 -- 172.21.15.37:0/3276020061 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55e75cb6e7f0 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63ad7fa640 1 -- 172.21.15.37:0/3276020061 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f63b40d5030 con 0x55e75cbfc480 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 10 client.17817 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:04.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 10 client.17817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 20 client.17817 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc1ae10 0x55e75cc1d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 4 client.17817 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.790+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e75cb68d00 con 0x55e75cc1ae10 2024-09-19T06:50:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.791+0000 7f63c98ae640 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc1ae10 0x55e75cc1d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.791+0000 7f63c98ae640 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc1ae10 0x55e75cc1d220 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abeaba10cf12f7cd server_cookie=71b6d7126eea5fbc in_seq=0 out_seq=0 2024-09-19T06:50:04.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.791+0000 7f63ad7fa640 10 client.17817 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63ad7fa640 1 -- 172.21.15.37:0/3276020061 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e75cb68d00 con 0x55e75cc1ae10 2024-09-19T06:50:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63ad7fa640 10 client.17817 handle_command_reply: tid=0 2024-09-19T06:50:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 client.17817 shutdown 2024-09-19T06:50:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfc480 msgr2=0x55e75cc04120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfc480 0x55e75cc04120 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f63b4098350 tx=0x7f63b4005440 comp rx=0 tx=0).stop 2024-09-19T06:50:04.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 shutdown_connections 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc1ae10 0x55e75cc1d220 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cc06d40 0x55e75cc04660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cbfc480 0x55e75cc04120 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3276020061 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc03be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 >> 172.21.15.37:0/3276020061 conn(0x55e75cbf80f0 msgr2=0x55e75cbf7270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63ad7fa640 0 client.17817 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 shutdown_connections 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3276020061 wait complete. 2024-09-19T06:50:04.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.793+0000 7f63cdeeb740 20 client.17817 trim_cache size 0 max 16384 2024-09-19T06:50:04.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 Processor -- start 2024-09-19T06:50:04.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 -- start start 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfbb60 0x55e75cc22ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc23020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cbf6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41ae090 con 0x55e75cc06d40 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41ab900 con 0x55e75cbfbb60 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41a9770 con 0x55e75cbfc480 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cbf6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63c90ad640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc23020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cbf6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41920/0 (socket says 172.21.15.37:41920) 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63ca0af640 1 -- 172.21.15.37:0/3001406605 learned_addr learned my addr 172.21.15.37:0/3001406605 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.812+0000 7f63c98ae640 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfbb60 0x55e75cc22ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63ca0af640 1 -- 172.21.15.37:0/3001406605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 msgr2=0x55e75cc23020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63ca0af640 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc23020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63ca0af640 1 -- 172.21.15.37:0/3001406605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfbb60 msgr2=0x55e75cc22ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63ca0af640 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfbb60 0x55e75cc22ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63ca0af640 1 -- 172.21.15.37:0/3001406605 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e75cc02150 con 0x55e75cc06d40 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63c90ad640 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc23020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63c98ae640 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfbb60 0x55e75cc22ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63ca0af640 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cbf6a50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f63bc039640 tx=0x7f63bc02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3414105c522cc8f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63af7fe640 1 -- 172.21.15.37:0/3001406605 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc060070 con 0x55e75cc06d40 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63af7fe640 1 -- 172.21.15.37:0/3001406605 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63bc039d30 con 0x55e75cc06d40 2024-09-19T06:50:04.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.813+0000 7f63af7fe640 1 -- 172.21.15.37:0/3001406605 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc05b480 con 0x55e75cc06d40 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3001406605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 msgr2=0x55e75cbf6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cbf6a50 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f63bc039640 tx=0x7f63bc02a950 comp rx=0 tx=0).stop 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3001406605 shutdown_connections 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cbf6a50 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfc480 0x55e75cc23020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/3001406605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfbb60 0x55e75cc22ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3001406605 >> 172.21.15.37:0/3001406605 conn(0x55e75cbf80f0 msgr2=0x55e75cc02cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3001406605 shutdown_connections 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 -- 172.21.15.37:0/3001406605 wait complete. 2024-09-19T06:50:04.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 -- start start 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.814+0000 7f63cdeeb740 1 Processor -- start 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc13730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfc480 0x55e75cc00830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63cdeeb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cc00d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41a9dc0 con 0x55e75cc06d40 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41ab230 con 0x55e75cbfc480 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63cdeeb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f63c41aa490 con 0x55e75cbfbb60 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63cdeeb740 10 client.0 _ll_register_callbacks cb 0x7f63b400e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cc00d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cc00d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41930/0 (socket says 172.21.15.37:41930) 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 -- 172.21.15.37:0/1491888904 learned_addr learned my addr 172.21.15.37:0/1491888904 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63c98ae640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc13730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63c90ad640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfc480 0x55e75cc00830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 -- 172.21.15.37:0/1491888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 msgr2=0x55e75cc13730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc13730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 -- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfc480 msgr2=0x55e75cc00830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfc480 0x55e75cc00830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 -- 172.21.15.37:0/1491888904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e75cc01460 con 0x55e75cc06d40 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63c98ae640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc13730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63c90ad640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfc480 0x55e75cc00830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ca0af640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cc00d70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f63bc05e040 tx=0x7f63bc02f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad34958f79be68d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.815+0000 7f63ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.816+0000 7f63ad7fa640 1 -- 172.21.15.37:0/1491888904 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63bc069060 con 0x55e75cc06d40 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.816+0000 7f63ad7fa640 1 -- 172.21.15.37:0/1491888904 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7f63bc039670 con 0x55e75cc06d40 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.816+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63bc039d30 con 0x55e75cc06d40 2024-09-19T06:50:04.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63ad7fa640 1 -- 172.21.15.37:0/1491888904 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f63bc060040 con 0x55e75cc06d40 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 10 client.17823 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e75cc859f0 con 0x55e75cc06d40 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63ad7fa640 1 -- 172.21.15.37:0/1491888904 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63bc039d30 con 0x55e75cc06d40 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63ad7fa640 1 -- 172.21.15.37:0/1491888904 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7f63bc04f070 con 0x55e75cc06d40 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 10 client.17823 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 10 client.17823 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 20 client.17823 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc85cc0 0x55e75cc880b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 4 client.17823 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e75cae5110 con 0x55e75cc85cc0 2024-09-19T06:50:04.822 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.817+0000 7f63c98ae640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc85cc0 0x55e75cc880b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.818+0000 7f63c98ae640 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc85cc0 0x55e75cc880b0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fd64af73c506f7b server_cookie=b26dbb0dba730043 in_seq=0 out_seq=0 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.818+0000 7f63ad7fa640 10 client.17823 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63ad7fa640 1 -- 172.21.15.37:0/1491888904 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e75cae5110 con 0x55e75cc85cc0 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63ad7fa640 10 client.17823 handle_command_reply: tid=0 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 client.17823 shutdown 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 msgr2=0x55e75cc00d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cc00d70 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f63bc05e040 tx=0x7f63bc02f580 comp rx=0 tx=0).stop 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 shutdown_connections 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e75cc85cc0 0x55e75cc880b0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e75cc06d40 0x55e75cc00d70 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e75cbfc480 0x55e75cc00830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63ad7fa640 0 client.17823 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 --2- 172.21.15.37:0/1491888904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e75cbfbb60 0x55e75cc13730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 >> 172.21.15.37:0/1491888904 conn(0x55e75cbf80f0 msgr2=0x55e75cc03ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 shutdown_connections 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 1 -- 172.21.15.37:0/1491888904 wait complete. 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.819+0000 7f63cdeeb740 20 client.17823 trim_cache size 0 max 16384 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "7e4cbc0a-e475-4f00-9d80-513c2d4fec1b", 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:50:04.825 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:50:04.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f63b00a33b0 msgr2=0x7f63b00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f63b00a33b0 0x7f63b00a5870 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f63bc04c000 tx=0x7f63bc049000 comp rx=0 tx=0).stop 2024-09-19T06:50:04.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63c4102570 msgr2=0x7f63c4197940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:04.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63c4102570 0x7f63c4197940 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f63b400e9f0 tx=0x7f63b400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 shutdown_connections 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f63b00a33b0 0x7f63b00a5870 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f63c4102ef0 0x7f63c419bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f63c4102570 0x7f63c4197940 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 --2- 172.21.15.37:0/1120876571 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63c4100e20 0x7f63c4197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 >> 172.21.15.37:0/1120876571 conn(0x7f63c40fc7d0 msgr2=0x7f63c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 shutdown_connections 2024-09-19T06:50:04.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:04.820+0000 7f63cbb12640 1 -- 172.21.15.37:0/1120876571 wait complete. 2024-09-19T06:50:04.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:04 smithi151 ceph-mon[35236]: pgmap v257: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 712 KiB/s wr, 52 op/s 2024-09-19T06:50:05.218 INFO:tasks.workunit.client.0.smithi037.stdout: 1 572350 6.95 MB/sec execute 272 sec latency 41.595 ms 2024-09-19T06:50:05.482 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:50:05.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:05 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:50:05.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:05 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:05 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:50:05.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:05 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:05.784 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:05.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:05 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:50:05.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:05 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:06.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 574859 6.97 MB/sec execute 273 sec latency 41.295 ms 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4246048994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 msgr2=0x7ff5cc10e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4246048994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 0x7ff5cc10e600 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4009ca0 tx=0x7ff5c402a950 comp rx=0 tx=0).stop 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4246048994 shutdown_connections 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4246048994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 0x7ff5cc10e600 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4246048994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5cc103650 0x7ff5cc10bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4246048994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5cc102d30 0x7ff5cc103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4246048994 >> 172.21.15.37:0/4246048994 conn(0x7ff5cc0fc7d0 msgr2=0x7ff5cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4246048994 shutdown_connections 2024-09-19T06:50:06.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.305+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4246048994 wait complete. 2024-09-19T06:50:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.306+0000 7ff5d1aa9640 1 Processor -- start 2024-09-19T06:50:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.306+0000 7ff5d1aa9640 1 -- start start 2024-09-19T06:50:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5d1aa9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5cc102d30 0x7ff5cc1a46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5d1aa9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5cc103650 0x7ff5cc1a4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.309 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5d1aa9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 0x7ff5cc10acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5cc102d30 0x7ff5cc1a46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5d1aa9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111f90 con 0x7ff5cc10c160 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5d1aa9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc111e10 con 0x7ff5cc102d30 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5cc102d30 0x7ff5cc1a46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53940/0 (socket says 172.21.15.37:53940) 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/4081186901 learned_addr learned my addr 172.21.15.37:0/4081186901 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc112110 con 0x7ff5cc103650 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5caffd640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5cc103650 0x7ff5cc1a4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.307+0000 7ff5cbfff640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 0x7ff5cc10acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5caffd640 1 -- 172.21.15.37:0/4081186901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5cc102d30 msgr2=0x7ff5cc1a46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5caffd640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5cc102d30 0x7ff5cc1a46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5caffd640 1 -- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 msgr2=0x7ff5cc10acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5caffd640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 0x7ff5cc10acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5caffd640 1 -- 172.21.15.37:0/4081186901 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5cc10b3e0 con 0x7ff5cc103650 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5cbfff640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 0x7ff5cc10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5caffd640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5cc103650 0x7ff5cc1a4c20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e7c0 tx=0x7ff5c000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5c8ff9640 1 -- 172.21.15.37:0/4081186901 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0009530 con 0x7ff5cc103650 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5c8ff9640 1 -- 172.21.15.37:0/4081186901 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c000ccc0 con 0x7ff5cc103650 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.308+0000 7ff5c8ff9640 1 -- 172.21.15.37:0/4081186901 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c0004a00 con 0x7ff5cc103650 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.309+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5cc10b6d0 con 0x7ff5cc103650 2024-09-19T06:50:06.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.309+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5cc1ab5b0 con 0x7ff5cc103650 2024-09-19T06:50:06.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.312+0000 7ff5c8ff9640 1 -- 172.21.15.37:0/4081186901 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff5c0004ba0 con 0x7ff5cc103650 2024-09-19T06:50:06.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.313+0000 7ff5c8ff9640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff5b40a3430 0x7ff5b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.313+0000 7ff5c8ff9640 1 -- 172.21.15.37:0/4081186901 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7ff5c00cfb90 con 0x7ff5cc103650 2024-09-19T06:50:06.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.313+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff5b40a3430 0x7ff5b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 Processor -- start 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 -- start start 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff5b40a3430 0x7ff5b40a58f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc10bdf0 tx=0x7ff5bc005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b548f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b549a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5562b53f52e0 con 0x5562b548f4a0 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562b539cce0 con 0x5562b548fdc0 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5562b53f12a0 con 0x5562b549a680 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b548f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b549a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5cbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b549a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53956/0 (socket says 172.21.15.37:53956) 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.314+0000 7ff5caffd640 1 -- 172.21.15.37:0/3114427660 learned_addr learned my addr 172.21.15.37:0/3114427660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5caffd640 1 -- 172.21.15.37:0/3114427660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 msgr2=0x5562b549ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5caffd640 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5caffd640 1 -- 172.21.15.37:0/3114427660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 msgr2=0x5562b548f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5caffd640 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b548f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5caffd640 1 -- 172.21.15.37:0/3114427660 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562b5381440 con 0x5562b548fdc0 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b548f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5caffd640 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b549a140 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00982a0 tx=0x7ff5c009b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f3579a982b79b38 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5b1ffb640 1 -- 172.21.15.37:0/3114427660 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c009e070 con 0x5562b548fdc0 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5b1ffb640 1 -- 172.21.15.37:0/3114427660 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c00a01f0 con 0x5562b548fdc0 2024-09-19T06:50:06.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.315+0000 7ff5b1ffb640 1 -- 172.21.15.37:0/3114427660 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c001c040 con 0x5562b548fdc0 2024-09-19T06:50:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 -- 172.21.15.37:0/3114427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 msgr2=0x5562b549a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b549a140 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff5c00982a0 tx=0x7ff5c009b7a0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 -- 172.21.15.37:0/3114427660 shutdown_connections 2024-09-19T06:50:06.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b549a140 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/3114427660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b548f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 -- 172.21.15.37:0/3114427660 >> 172.21.15.37:0/3114427660 conn(0x5562b548ba30 msgr2=0x5562b548be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 -- 172.21.15.37:0/3114427660 shutdown_connections 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.316+0000 7ff5d3e82740 1 -- 172.21.15.37:0/3114427660 wait complete. 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.317+0000 7ff5d3e82740 1 -- start start 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.317+0000 7ff5d3e82740 1 Processor -- start 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.317+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b548f4a0 0x5562b549a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.317+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54972c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b549a680 0x5562b5497840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5562b549d250 con 0x5562b548fdc0 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562b53b0050 con 0x5562b549a680 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41986/0 (socket says 172.21.15.37:41986) 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5562b549d3d0 con 0x5562b548f4a0 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5caffd640 1 -- 172.21.15.37:0/2476786455 learned_addr learned my addr 172.21.15.37:0/2476786455 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b548f4a0 0x5562b549a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5d3e82740 10 client.0 _ll_register_callbacks cb 0x5562b53f0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cbfff640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b549a680 0x5562b5497840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b549a680 msgr2=0x5562b5497840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b549a680 0x5562b5497840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/2476786455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 msgr2=0x5562b54972c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54972c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/2476786455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562b51663c0 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5caffd640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b548f4a0 0x5562b549a010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc01dbd0 tx=0x7ff5bc004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f30ecf96d686121a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.318+0000 7ff5aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2476786455 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc01cbd0 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2476786455 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7ff5bc01cd70 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c00a01f0 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2476786455 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc005020 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 10 client.26953 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5562b54a8c20 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2476786455 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c00a01f0 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2476786455 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7ff5bc026410 con 0x5562b548f4a0 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 10 client.26953 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:06.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 10 client.26953 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:06.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 20 client.26953 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:06.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b54ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 4 client.26953 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:06.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.319+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562b5402850 con 0x5562b54aa130 2024-09-19T06:50:06.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.320+0000 7ff5caffd640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b54ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.320+0000 7ff5aaffd640 10 client.26953 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:06.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.320+0000 7ff5caffd640 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b54ac540 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5b777fe6972b2a7 server_cookie=f964005c431b4230 in_seq=0 out_seq=0 2024-09-19T06:50:06.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.321+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2476786455 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5562b5402850 con 0x5562b54aa130 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.321+0000 7ff5aaffd640 10 client.26953 handle_command_reply: tid=0 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.321+0000 7ff5d3e82740 1 client.26953 shutdown 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.321+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b548f4a0 msgr2=0x5562b549a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.321+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b548f4a0 0x5562b549a010 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc01dbd0 tx=0x7ff5bc004730 comp rx=0 tx=0).stop 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 shutdown_connections 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b54ac540 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5aaffd640 0 client.26953 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b549a680 0x5562b5497840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54972c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2476786455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b548f4a0 0x5562b549a010 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 >> 172.21.15.37:0/2476786455 conn(0x5562b548ba30 msgr2=0x5562b548ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 shutdown_connections 2024-09-19T06:50:06.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2476786455 wait complete. 2024-09-19T06:50:06.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.322+0000 7ff5d3e82740 20 client.26953 trim_cache size 0 max 16384 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 Processor -- start 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 -- start start 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b5496b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b548aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b54b1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b69b0 con 0x5562b548f4a0 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b4220 con 0x5562b548fdc0 2024-09-19T06:50:06.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b2090 con 0x5562b549a680 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b5496b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5caffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b548aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b5496b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42000/0 (socket says 172.21.15.37:42000) 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/922967793 learned_addr learned my addr 172.21.15.37:0/922967793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.342+0000 7ff5cbfff640 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b54b1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5caffd640 1 -- 172.21.15.37:0/922967793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 msgr2=0x5562b54b1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5caffd640 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b54b1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5caffd640 1 -- 172.21.15.37:0/922967793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 msgr2=0x5562b5496b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5caffd640 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b5496b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5caffd640 1 -- 172.21.15.37:0/922967793 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562b53301e0 con 0x5562b548fdc0 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5caffd640 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b548aa80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009230 tx=0x7ff5c000cf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2bda1608c651683c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5b17fa640 1 -- 172.21.15.37:0/922967793 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c001c040 con 0x5562b548fdc0 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5b17fa640 1 -- 172.21.15.37:0/922967793 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c00984b0 con 0x5562b548fdc0 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.343+0000 7ff5b17fa640 1 -- 172.21.15.37:0/922967793 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5c009e040 con 0x5562b548fdc0 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 -- 172.21.15.37:0/922967793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 msgr2=0x5562b548aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b548aa80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009230 tx=0x7ff5c000cf20 comp rx=0 tx=0).stop 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 -- 172.21.15.37:0/922967793 shutdown_connections 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b54b1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548fdc0 0x5562b548aa80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/922967793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548f4a0 0x5562b5496b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 -- 172.21.15.37:0/922967793 >> 172.21.15.37:0/922967793 conn(0x5562b548ba30 msgr2=0x5562b548a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 -- 172.21.15.37:0/922967793 shutdown_connections 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 -- 172.21.15.37:0/922967793 wait complete. 2024-09-19T06:50:06.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 -- start start 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.344+0000 7ff5d3e82740 1 Processor -- start 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548f4a0 0x5562b54a29e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54a2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5d3e82740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b26e0 con 0x5562b548fdc0 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b3b50 con 0x5562b548f4a0 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5d3e82740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff5cc1b2db0 con 0x5562b549a680 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5d3e82740 10 client.0 _ll_register_callbacks cb 0x7ff5bc0099b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548f4a0 0x5562b54a29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548f4a0 0x5562b54a29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53980/0 (socket says 172.21.15.37:53980) 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/2253463079 learned_addr learned my addr 172.21.15.37:0/2253463079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5caffd640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54a2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cbfff640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/2253463079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 msgr2=0x5562b549ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/2253463079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 msgr2=0x5562b54a2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54a2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 -- 172.21.15.37:0/2253463079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562b54a0780 con 0x5562b548f4a0 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cbfff640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5caffd640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5cb7fe640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548f4a0 0x5562b54a29e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc016040 tx=0x7ff5bc026690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ea74ccbf8ab0540 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.345+0000 7ff5aaffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2253463079 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc01d7b0 con 0x5562b548f4a0 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2253463079 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7ff5bc009040 con 0x5562b548f4a0 2024-09-19T06:50:06.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff5c00984b0 con 0x5562b548f4a0 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2253463079 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff5bc011d20 con 0x5562b548f4a0 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 10 client.27182 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5562b5514a80 con 0x5562b548f4a0 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2253463079 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff5c00984b0 con 0x5562b548f4a0 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2253463079 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7ff5bc037070 con 0x5562b548f4a0 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 10 client.27182 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 10 client.27182 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 20 client.27182 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b5517260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 4 client.27182 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562b53fcd60 con 0x5562b54aa130 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.346+0000 7ff5caffd640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b5517260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:06.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.347+0000 7ff5aaffd640 10 client.27182 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.347+0000 7ff5caffd640 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b5517260 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=617b7d7e5eeb6ca8 server_cookie=18850919aa1d4a52 in_seq=0 out_seq=0 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.347+0000 7ff5aaffd640 1 -- 172.21.15.37:0/2253463079 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5562b53fcd60 con 0x5562b54aa130 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.347+0000 7ff5aaffd640 10 client.27182 handle_command_reply: tid=0 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.347+0000 7ff5d3e82740 1 client.27182 shutdown 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.347+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548f4a0 msgr2=0x5562b54a29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.347+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548f4a0 0x5562b54a29e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc016040 tx=0x7ff5bc026690 comp rx=0 tx=0).stop 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 shutdown_connections 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5562b54aa130 0x5562b5517260 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5562b549a680 0x5562b549ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5562b548fdc0 0x5562b54a2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 --2- 172.21.15.37:0/2253463079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562b548f4a0 0x5562b54a29e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 >> 172.21.15.37:0/2253463079 conn(0x5562b548ba30 msgr2=0x5562b548ab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5aaffd640 0 client.27182 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 shutdown_connections 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 1 -- 172.21.15.37:0/2253463079 wait complete. 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.348+0000 7ff5d3e82740 20 client.27182 trim_cache size 0 max 16384 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "7e4cbc0a-e475-4f00-9d80-513c2d4fec1b": { 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:50:06.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "7e4cbc0a-e475-4f00-9d80-513c2d4fec1b", 2024-09-19T06:50:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:50:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:06.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:06.359 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff5b40a3430 msgr2=0x7ff5b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff5b40a3430 0x7ff5b40a58f0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff5cc10bdf0 tx=0x7ff5bc005f90 comp rx=0 tx=0).stop 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5cc103650 msgr2=0x7ff5cc1a4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5cc103650 0x7ff5cc1a4c20 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff5c000e7c0 tx=0x7ff5c000ec80 comp rx=0 tx=0).stop 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 shutdown_connections 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff5b40a3430 0x7ff5b40a58f0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff5cc10c160 0x7ff5cc10acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff5cc103650 0x7ff5cc1a4c20 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 --2- 172.21.15.37:0/4081186901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5cc102d30 0x7ff5cc1a46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:06.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 >> 172.21.15.37:0/4081186901 conn(0x7ff5cc0fc7d0 msgr2=0x7ff5cc0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:06.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 shutdown_connections 2024-09-19T06:50:06.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:06.349+0000 7ff5d1aa9640 1 -- 172.21.15.37:0/4081186901 wait complete. 2024-09-19T06:50:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:06 smithi096 ceph-mon[35004]: pgmap v258: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 712 KiB/s wr, 50 op/s 2024-09-19T06:50:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:06 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:50:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:06 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:06 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:50:06.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:06 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:06.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:06 smithi037 ceph-mon[31211]: pgmap v258: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 712 KiB/s wr, 50 op/s 2024-09-19T06:50:06.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:06 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:50:06.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:06 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:06.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:06 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:50:06.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:06 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:06.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:06 smithi151 ceph-mon[35236]: pgmap v258: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 712 KiB/s wr, 50 op/s 2024-09-19T06:50:06.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:06 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:50:06.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:06 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:06.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:06 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:50:06.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:06 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:07.148 INFO:tasks.cephfs.filesystem:scrub status for tag:7e4cbc0a-e475-4f00-9d80-513c2d4fec1b - {'path': '/', 'tag': '7e4cbc0a-e475-4f00-9d80-513c2d4fec1b', 'options': 'recursive,force'} 2024-09-19T06:50:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.202+0000 7fd4a02e6640 1 -- 172.21.15.37:0/4214219573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd49810c110 msgr2=0x7fd49810e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.204 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.202+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/4214219573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd49810c110 0x7fd49810e590 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fd490009ca0 tx=0x7fd49002a950 comp rx=0 tx=0).stop 2024-09-19T06:50:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.202+0000 7fd4a02e6640 1 -- 172.21.15.37:0/4214219573 shutdown_connections 2024-09-19T06:50:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.202+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/4214219573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd49810c110 0x7fd49810e590 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.202+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/4214219573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd4981035e0 0x7fd49810bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.202+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/4214219573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 0x7fd4981030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.202+0000 7fd4a02e6640 1 -- 172.21.15.37:0/4214219573 >> 172.21.15.37:0/4214219573 conn(0x7fd4980fc6b0 msgr2=0x7fd4980fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.203+0000 7fd4a02e6640 1 -- 172.21.15.37:0/4214219573 shutdown_connections 2024-09-19T06:50:07.205 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.203+0000 7fd4a02e6640 1 -- 172.21.15.37:0/4214219573 wait complete. 2024-09-19T06:50:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.203+0000 7fd4a02e6640 1 Processor -- start 2024-09-19T06:50:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.203+0000 7fd4a02e6640 1 -- start start 2024-09-19T06:50:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd4a02e6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 0x7fd4981982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd4a02e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4981035e0 0x7fd498198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd4a02e6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49810c110 0x7fd49819cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd4a02e6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd498111eb0 con 0x7fd4981035e0 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd4a02e6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd498111d30 con 0x7fd498102cc0 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd4a02e6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd498112030 con 0x7fd49810c110 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 0x7fd4981982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49d881640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4981035e0 0x7fd498198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49d881640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4981035e0 0x7fd498198820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42022/0 (socket says 172.21.15.37:42022) 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 0x7fd4981982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53994/0 (socket says 172.21.15.37:53994) 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49d881640 1 -- 172.21.15.37:0/408057291 learned_addr learned my addr 172.21.15.37:0/408057291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e883640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49810c110 0x7fd49819cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 -- 172.21.15.37:0/408057291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49810c110 msgr2=0x7fd49819cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49810c110 0x7fd49819cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 -- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4981035e0 msgr2=0x7fd498198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4981035e0 0x7fd498198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 -- 172.21.15.37:0/408057291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd49819d2d0 con 0x7fd498102cc0 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49d881640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4981035e0 0x7fd498198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd49e082640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 0x7fd4981982e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd49400e9f0 tx=0x7fd49400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.204+0000 7fd48f7fe640 1 -- 172.21.15.37:0/408057291 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49400ccc0 con 0x7fd498102cc0 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.205+0000 7fd48f7fe640 1 -- 172.21.15.37:0/408057291 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd494010910 con 0x7fd498102cc0 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.205+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd49819d5c0 con 0x7fd498102cc0 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.205+0000 7fd48f7fe640 1 -- 172.21.15.37:0/408057291 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494018960 con 0x7fd498102cc0 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.205+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981a4e50 con 0x7fd498102cc0 2024-09-19T06:50:07.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.207+0000 7fd48f7fe640 1 -- 172.21.15.37:0/408057291 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd494010ac0 con 0x7fd498102cc0 2024-09-19T06:50:07.212 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 Processor -- start 2024-09-19T06:50:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 -- start start 2024-09-19T06:50:07.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd48f7fe640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4840a3460 0x7fd4840a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626270ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x55962627b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x55962627e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5596261d52f0 con 0x55962627bcb0 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55962617d270 con 0x559626270af0 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd4a3077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5596261d12b0 con 0x559626271410 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.208+0000 7fd49e883640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4840a3460 0x7fd4840a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626270ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626270ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54010/0 (socket says 172.21.15.37:54010) 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 -- 172.21.15.37:0/1118885304 learned_addr learned my addr 172.21.15.37:0/1118885304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e883640 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x55962627e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49d881640 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x55962627b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd48f7fe640 1 -- 172.21.15.37:0/408057291 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fd4940d0120 con 0x7fd498102cc0 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 -- 172.21.15.37:0/1118885304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 msgr2=0x55962627b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x55962627b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 -- 172.21.15.37:0/1118885304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 msgr2=0x55962627e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x55962627e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 -- 172.21.15.37:0/1118885304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962617c340 con 0x559626270af0 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e082640 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626270ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd4940cd840 tx=0x7fd49400cf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbad739fc251be13 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd49e883640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4840a3460 0x7fd4840a5920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd490005fe0 tx=0x7fd49002a950 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd48cbfe640 1 -- 172.21.15.37:0/1118885304 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49407fec0 con 0x559626270af0 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd48cbfe640 1 -- 172.21.15.37:0/1118885304 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49409c420 con 0x559626270af0 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.209+0000 7fd48cbfe640 1 -- 172.21.15.37:0/1118885304 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd494013190 con 0x559626270af0 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 -- 172.21.15.37:0/1118885304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 msgr2=0x559626270ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626270ed0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd4940cd840 tx=0x7fd49400cf60 comp rx=0 tx=0).stop 2024-09-19T06:50:07.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 -- 172.21.15.37:0/1118885304 shutdown_connections 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x55962627e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x55962627b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 --2- 172.21.15.37:0/1118885304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626270ed0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 -- 172.21.15.37:0/1118885304 >> 172.21.15.37:0/1118885304 conn(0x55962626d060 msgr2=0x55962626d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 -- 172.21.15.37:0/1118885304 shutdown_connections 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 -- 172.21.15.37:0/1118885304 wait complete. 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.210+0000 7fd4a3077740 1 -- start start 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 1 Processor -- start 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626270af0 0x559626278b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626271410 0x5596262790c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55962627bcb0 0x559626279600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55962627e880 con 0x559626271410 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55962615e290 con 0x55962627bcb0 2024-09-19T06:50:07.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55962627ea00 con 0x559626270af0 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd4a3077740 10 client.0 _ll_register_callbacks cb 0x55962618f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626271410 0x5596262790c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626271410 0x5596262790c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42048/0 (socket says 172.21.15.37:42048) 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 -- 172.21.15.37:0/2302434222 learned_addr learned my addr 172.21.15.37:0/2302434222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49e883640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55962627bcb0 0x559626279600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49e082640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626270af0 0x559626278b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 -- 172.21.15.37:0/2302434222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626270af0 msgr2=0x559626278b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626270af0 0x559626278b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 -- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55962627bcb0 msgr2=0x559626279600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55962627bcb0 0x559626279600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 -- 172.21.15.37:0/2302434222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559625ee8210 con 0x559626271410 2024-09-19T06:50:07.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49e082640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626270af0 0x559626278b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd49d881640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626271410 0x5596262790c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd4880044f0 tx=0x7fd4880039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ef0d0be4c6e8604 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.211+0000 7fd481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd481ffb640 1 -- 172.21.15.37:0/2302434222 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd488011830 con 0x559626271410 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd481ffb640 1 -- 172.21.15.37:0/2302434222 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fd4880119d0 con 0x559626271410 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd49409c420 con 0x559626271410 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd481ffb640 1 -- 172.21.15.37:0/2302434222 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd488011bd0 con 0x559626271410 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 10 client.17850 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x559625fb4620 con 0x559626271410 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd481ffb640 1 -- 172.21.15.37:0/2302434222 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd49409c420 con 0x559626271410 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd481ffb640 1 -- 172.21.15.37:0/2302434222 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd488022070 con 0x559626271410 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 10 client.17850 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 10 client.17850 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:07.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 20 client.17850 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55962628bcc0 0x55962628e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 4 client.17850 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596261e2a20 con 0x55962628bcc0 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.212+0000 7fd49e082640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55962628bcc0 0x55962628e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.213+0000 7fd49e082640 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55962628bcc0 0x55962628e110 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b0dc48f9d914180 server_cookie=2f5afb98bf0c7774 in_seq=0 out_seq=0 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.213+0000 7fd481ffb640 10 client.17850 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd481ffb640 1 -- 172.21.15.37:0/2302434222 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596261e2a20 con 0x55962628bcc0 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd481ffb640 10 client.17850 handle_command_reply: tid=0 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 client.17850 shutdown 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626271410 msgr2=0x5596262790c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626271410 0x5596262790c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fd4880044f0 tx=0x7fd4880039c0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 shutdown_connections 2024-09-19T06:50:07.223 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55962628bcc0 0x55962628e110 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55962627bcb0 0x559626279600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd481ffb640 0 client.17850 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626271410 0x5596262790c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 --2- 172.21.15.37:0/2302434222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626270af0 0x559626278b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 >> 172.21.15.37:0/2302434222 conn(0x55962626d060 msgr2=0x559626272dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 shutdown_connections 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 1 -- 172.21.15.37:0/2302434222 wait complete. 2024-09-19T06:50:07.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.214+0000 7fd4a3077740 20 client.17850 trim_cache size 0 max 16384 2024-09-19T06:50:07.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 576940 6.99 MB/sec execute 274 sec latency 21.627 ms 2024-09-19T06:50:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.230+0000 7fd4a3077740 1 Processor -- start 2024-09-19T06:50:07.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd4a3077740 1 -- start start 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626279430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x559626293a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x559626293fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd4a3077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981b02c0 con 0x55962627bcb0 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd4a3077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981adb30 con 0x559626270af0 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd4a3077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981ab9a0 con 0x559626271410 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x559626293fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e082640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626279430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x559626293fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42062/0 (socket says 172.21.15.37:42062) 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 -- 172.21.15.37:0/3368770807 learned_addr learned my addr 172.21.15.37:0/3368770807 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49d881640 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x559626293a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 -- 172.21.15.37:0/3368770807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 msgr2=0x559626293a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x559626293a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 -- 172.21.15.37:0/3368770807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 msgr2=0x559626279430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626279430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 -- 172.21.15.37:0/3368770807 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55962626ce90 con 0x55962627bcb0 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e082640 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626279430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.231+0000 7fd49e883640 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x559626293fb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd49002fa40 tx=0x7fd49003b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33384477194d673c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd483fff640 1 -- 172.21.15.37:0/3368770807 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490044040 con 0x55962627bcb0 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd483fff640 1 -- 172.21.15.37:0/3368770807 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd490003cf0 con 0x55962627bcb0 2024-09-19T06:50:07.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd483fff640 1 -- 172.21.15.37:0/3368770807 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd490034040 con 0x55962627bcb0 2024-09-19T06:50:07.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 -- 172.21.15.37:0/3368770807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 msgr2=0x559626293fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x559626293fb0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fd49002fa40 tx=0x7fd49003b000 comp rx=0 tx=0).stop 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 -- 172.21.15.37:0/3368770807 shutdown_connections 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55962627bcb0 0x559626293fb0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559626271410 0x559626293a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3368770807 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626270af0 0x559626279430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 -- 172.21.15.37:0/3368770807 >> 172.21.15.37:0/3368770807 conn(0x55962626d060 msgr2=0x55962627ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 -- 172.21.15.37:0/3368770807 shutdown_connections 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.232+0000 7fd4a3077740 1 -- 172.21.15.37:0/3368770807 wait complete. 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 -- start start 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 Processor -- start 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626270af0 0x559626284310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626271410 0x55962627f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55962627bcb0 0x55962627fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981abff0 con 0x559626270af0 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981ad460 con 0x559626271410 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4981ac6c0 con 0x55962627bcb0 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd4a3077740 10 client.0 _ll_register_callbacks cb 0x7fd48800bdf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626270af0 0x559626284310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626270af0 0x559626284310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42064/0 (socket says 172.21.15.37:42064) 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 -- 172.21.15.37:0/3774505594 learned_addr learned my addr 172.21.15.37:0/3774505594 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49d881640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626271410 0x55962627f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e883640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55962627bcb0 0x55962627fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 -- 172.21.15.37:0/3774505594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55962627bcb0 msgr2=0x55962627fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55962627bcb0 0x55962627fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 -- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626271410 msgr2=0x55962627f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:07.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626271410 0x55962627f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e082640 1 -- 172.21.15.37:0/3774505594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559626280440 con 0x559626270af0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.233+0000 7fd49e883640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55962627bcb0 0x55962627fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd49e082640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626270af0 0x559626284310 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd494098df0 tx=0x7fd494098040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f7182f44cd3ac24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd481ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd481ffb640 1 -- 172.21.15.37:0/3774505594 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd49409c420 con 0x559626270af0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd481ffb640 1 -- 172.21.15.37:0/3774505594 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8571+0+0 (secure 0 0 0) 0x7fd49409b870 con 0x559626270af0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd490003cf0 con 0x559626270af0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd481ffb640 1 -- 172.21.15.37:0/3774505594 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fd4940cdc60 con 0x559626270af0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 10 client.17856 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5596262f6990 con 0x559626270af0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd481ffb640 1 -- 172.21.15.37:0/3774505594 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd490003cf0 con 0x559626270af0 2024-09-19T06:50:07.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd481ffb640 1 -- 172.21.15.37:0/3774505594 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fd494018b00 con 0x559626270af0 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 10 client.17856 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 10 client.17856 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 20 client.17856 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5596262f6cf0 0x5596262f90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 4 client.17856 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5596261dcd70 con 0x5596262f6cf0 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.234+0000 7fd49d881640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5596262f6cf0 0x5596262f90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.235+0000 7fd49d881640 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5596262f6cf0 0x5596262f90e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e31ef6a27f5d6b server_cookie=93ad17d2433bfd5f in_seq=0 out_seq=0 2024-09-19T06:50:07.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.235+0000 7fd481ffb640 10 client.17856 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd481ffb640 1 -- 172.21.15.37:0/3774505594 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5596261dcd70 con 0x5596262f6cf0 2024-09-19T06:50:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd481ffb640 10 client.17856 handle_command_reply: tid=0 2024-09-19T06:50:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 client.17856 shutdown 2024-09-19T06:50:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626270af0 msgr2=0x559626284310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626270af0 0x559626284310 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fd494098df0 tx=0x7fd494098040 comp rx=0 tx=0).stop 2024-09-19T06:50:07.312 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 shutdown_connections 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5596262f6cf0 0x5596262f90e0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55962627bcb0 0x55962627fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559626271410 0x55962627f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd481ffb640 0 client.17856 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 --2- 172.21.15.37:0/3774505594 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559626270af0 0x559626284310 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 >> 172.21.15.37:0/3774505594 conn(0x55962626d060 msgr2=0x55962626b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 shutdown_connections 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 1 -- 172.21.15.37:0/3774505594 wait complete. 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.310+0000 7fd4a3077740 20 client.17856 trim_cache size 0 max 16384 2024-09-19T06:50:07.313 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:50:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:50:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:50:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:50:07.314 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4840a3460 msgr2=0x7fd4840a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4840a3460 0x7fd4840a5920 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fd490005fe0 tx=0x7fd49002a950 comp rx=0 tx=0).stop 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 msgr2=0x7fd4981982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 0x7fd4981982e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd49400e9f0 tx=0x7fd49400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 shutdown_connections 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd49810c110 0x7fd49819cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4840a3460 0x7fd4840a5920 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4981035e0 0x7fd498198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 --2- 172.21.15.37:0/408057291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd498102cc0 0x7fd4981982e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 >> 172.21.15.37:0/408057291 conn(0x7fd4980fc6b0 msgr2=0x7fd4980fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 shutdown_connections 2024-09-19T06:50:07.315 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:07.311+0000 7fd4a02e6640 1 -- 172.21.15.37:0/408057291 wait complete. 2024-09-19T06:50:08.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 578355 6.99 MB/sec execute 275 sec latency 43.949 ms 2024-09-19T06:50:08.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:08 smithi096 ceph-mon[35004]: pgmap v259: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 712 KiB/s wr, 50 op/s 2024-09-19T06:50:08.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:08 smithi096 ceph-mon[35004]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:50:08.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:08 smithi037 ceph-mon[31211]: pgmap v259: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 712 KiB/s wr, 50 op/s 2024-09-19T06:50:08.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:08 smithi037 ceph-mon[31211]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:50:08.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:08 smithi151 ceph-mon[35236]: pgmap v259: 97 pgs: 97 active+clean; 61 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 712 KiB/s wr, 50 op/s 2024-09-19T06:50:08.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:08 smithi151 ceph-mon[35236]: osdmap e91: 12 total, 12 up, 12 in 2024-09-19T06:50:09.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 579699 6.99 MB/sec execute 276 sec latency 26.140 ms 2024-09-19T06:50:10.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 582630 7.02 MB/sec execute 277 sec latency 14.177 ms 2024-09-19T06:50:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:10 smithi096 ceph-mon[35004]: pgmap v261: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 63 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-09-19T06:50:10.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:10 smithi037 ceph-mon[31211]: pgmap v261: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 63 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-09-19T06:50:10.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:10 smithi151 ceph-mon[35236]: pgmap v261: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 63 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-09-19T06:50:11.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 584827 7.03 MB/sec execute 278 sec latency 16.602 ms 2024-09-19T06:50:12.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 587021 7.05 MB/sec execute 279 sec latency 12.457 ms 2024-09-19T06:50:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:12 smithi096 ceph-mon[35004]: pgmap v262: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 63 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-09-19T06:50:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:12 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:50:12.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:12 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:50:12.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:12 smithi037 ceph-mon[31211]: pgmap v262: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 63 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-09-19T06:50:12.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:12 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:50:12.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:12 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:50:12.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:12 smithi151 ceph-mon[35236]: pgmap v262: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 63 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.2 MiB/s wr, 175 op/s 2024-09-19T06:50:12.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:12 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:50:12.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:12 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:50:13.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 589905 7.09 MB/sec execute 280 sec latency 13.147 ms 2024-09-19T06:50:14.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 592197 7.09 MB/sec execute 281 sec latency 48.974 ms 2024-09-19T06:50:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:14 smithi096 ceph-mon[35004]: pgmap v263: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-19T06:50:14.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:14 smithi037 ceph-mon[31211]: pgmap v263: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-19T06:50:14.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:14 smithi151 ceph-mon[35236]: pgmap v263: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-19T06:50:15.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 594652 7.13 MB/sec execute 282 sec latency 13.878 ms 2024-09-19T06:50:16.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:15 smithi151 ceph-mon[35236]: pgmap v264: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-19T06:50:16.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:15 smithi151 ceph-mon[35236]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T06:50:16.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 597286 7.15 MB/sec execute 283 sec latency 17.670 ms 2024-09-19T06:50:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:15 smithi096 ceph-mon[35004]: pgmap v264: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-19T06:50:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:15 smithi096 ceph-mon[35004]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T06:50:16.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:15 smithi037 ceph-mon[31211]: pgmap v264: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 2.1 MiB/s wr, 247 op/s 2024-09-19T06:50:16.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:15 smithi037 ceph-mon[31211]: osdmap e92: 12 total, 12 up, 12 in 2024-09-19T06:50:17.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 599734 7.17 MB/sec execute 284 sec latency 85.579 ms 2024-09-19T06:50:18.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:17 smithi151 ceph-mon[35236]: pgmap v266: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-09-19T06:50:18.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 601895 7.19 MB/sec execute 285 sec latency 12.585 ms 2024-09-19T06:50:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:17 smithi096 ceph-mon[35004]: pgmap v266: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-09-19T06:50:18.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:17 smithi037 ceph-mon[31211]: pgmap v266: 97 pgs: 97 active+clean; 71 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.3 MiB/s wr, 267 op/s 2024-09-19T06:50:19.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 604470 7.21 MB/sec execute 286 sec latency 11.999 ms 2024-09-19T06:50:20.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:19 smithi151 ceph-mon[35236]: pgmap v267: 97 pgs: 97 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-19T06:50:20.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 607352 7.24 MB/sec execute 287 sec latency 12.720 ms 2024-09-19T06:50:20.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:19 smithi096 ceph-mon[35004]: pgmap v267: 97 pgs: 97 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-19T06:50:20.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:19 smithi037 ceph-mon[31211]: pgmap v267: 97 pgs: 97 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-19T06:50:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.406+0000 7fa53f378640 1 -- 172.21.15.37:0/1835245242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 msgr2=0x7fa53810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.406+0000 7fa53f378640 1 --2- 172.21.15.37:0/1835245242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa53810d880 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fa530009cd0 tx=0x7fa53002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 -- 172.21.15.37:0/1835245242 shutdown_connections 2024-09-19T06:50:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 --2- 172.21.15.37:0/1835245242 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa53810d880 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 --2- 172.21.15.37:0/1835245242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5381009b0 0x7fa538100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 --2- 172.21.15.37:0/1835245242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5381069c0 0x7fa538106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 -- 172.21.15.37:0/1835245242 >> 172.21.15.37:0/1835245242 conn(0x7fa538077990 msgr2=0x7fa5380fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 -- 172.21.15.37:0/1835245242 shutdown_connections 2024-09-19T06:50:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 -- 172.21.15.37:0/1835245242 wait complete. 2024-09-19T06:50:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 Processor -- start 2024-09-19T06:50:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.407+0000 7fa53f378640 1 -- start start 2024-09-19T06:50:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5381009b0 0x7fa538198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa538198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5381069c0 0x7fa53819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa538110e00 con 0x7fa538101350 2024-09-19T06:50:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa538110c80 con 0x7fa5381009b0 2024-09-19T06:50:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa538110f80 con 0x7fa5381069c0 2024-09-19T06:50:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa538198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa538198840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41868/0 (socket says 172.21.15.37:41868) 2024-09-19T06:50:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 -- 172.21.15.37:0/212760812 learned_addr learned my addr 172.21.15.37:0/212760812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53d114640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5381009b0 0x7fa538198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53d915640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5381069c0 0x7fa53819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 -- 172.21.15.37:0/212760812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5381069c0 msgr2=0x7fa53819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5381069c0 0x7fa53819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 -- 172.21.15.37:0/212760812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5381009b0 msgr2=0x7fa538198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5381009b0 0x7fa538198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 -- 172.21.15.37:0/212760812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa53819d2f0 con 0x7fa538101350 2024-09-19T06:50:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53d114640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5381009b0 0x7fa538198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53c913640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa538198840 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9f0 tx=0x7fa52800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa52e7fc640 1 -- 172.21.15.37:0/212760812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa52800cd80 con 0x7fa538101350 2024-09-19T06:50:20.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa52e7fc640 1 -- 172.21.15.37:0/212760812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa528010910 con 0x7fa538101350 2024-09-19T06:50:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa53819d5e0 con 0x7fa538101350 2024-09-19T06:50:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa52e7fc640 1 -- 172.21.15.37:0/212760812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa528018990 con 0x7fa538101350 2024-09-19T06:50:20.417 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.408+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5381a4e70 con 0x7fa538101350 2024-09-19T06:50:20.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.411+0000 7fa5421f1740 1 Processor -- start 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.411+0000 7fa52e7fc640 1 -- 172.21.15.37:0/212760812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa528010ac0 con 0x7fa538101350 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.411+0000 7fa5421f1740 1 -- start start 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa5421f1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5603d5d2f2f0 con 0x5603d5dcaaf0 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa5421f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603d5cd7270 con 0x5603d5dcb410 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa5421f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603d5d2b2b0 con 0x5603d5dd5cb0 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d114640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcaed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41876/0 (socket says 172.21.15.37:41876) 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d114640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:55010/0 (socket says 172.21.15.37:55010) 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 -- 172.21.15.37:0/2661651241 learned_addr learned my addr 172.21.15.37:0/2661651241 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53c913640 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 -- 172.21.15.37:0/2661651241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dd5cb0 msgr2=0x5603d5dd80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 -- 172.21.15.37:0/2661651241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 msgr2=0x5603d5dd5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 -- 172.21.15.37:0/2661651241 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d5cd6340 con 0x5603d5dcaaf0 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d114640 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa52e7fc640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5240a3460 0x7fa5240a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.430 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d114640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5240a3460 0x7fa5240a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa52e7fc640 1 -- 172.21.15.37:0/212760812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fa5280d0040 con 0x7fa538101350 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.412+0000 7fa53d915640 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcaed0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa530009cd0 tx=0x7fa530009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae0081880e70c8fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5237fe640 1 -- 172.21.15.37:0/2661651241 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530048070 con 0x5603d5dcaaf0 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5237fe640 1 -- 172.21.15.37:0/2661651241 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530036070 con 0x5603d5dcaaf0 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5237fe640 1 -- 172.21.15.37:0/2661651241 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530004460 con 0x5603d5dcaaf0 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 -- 172.21.15.37:0/2661651241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 msgr2=0x5603d5dcaed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcaed0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa530009cd0 tx=0x7fa530009a20 comp rx=0 tx=0).stop 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa53d114640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5240a3460 0x7fa5240a5920 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa53400bfd0 tx=0x7fa53400b410 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 -- 172.21.15.37:0/2661651241 shutdown_connections 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.431 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2661651241 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcaed0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 -- 172.21.15.37:0/2661651241 >> 172.21.15.37:0/2661651241 conn(0x5603d5dc7060 msgr2=0x5603d5dc7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.413+0000 7fa5421f1740 1 -- 172.21.15.37:0/2661651241 shutdown_connections 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 -- 172.21.15.37:0/2661651241 wait complete. 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 -- start start 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 Processor -- start 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcaaf0 0x5603d5dd2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5603d5dd8880 con 0x5603d5dd5cb0 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5603d5cb8290 con 0x5603d5dcb410 2024-09-19T06:50:20.432 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5603d5dd8a00 con 0x5603d5dcaaf0 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa5421f1740 10 client.0 _ll_register_callbacks cb 0x5603d5ce9140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa53d915640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa53c913640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa53d915640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd35e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41888/0 (socket says 172.21.15.37:41888) 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.414+0000 7fa53d915640 1 -- 172.21.15.37:0/2132414979 learned_addr learned my addr 172.21.15.37:0/2132414979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53d114640 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcaaf0 0x5603d5dd2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53d915640 1 -- 172.21.15.37:0/2132414979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcaaf0 msgr2=0x5603d5dd2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53d915640 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcaaf0 0x5603d5dd2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53d915640 1 -- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 msgr2=0x5603d5dd30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.433 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53d915640 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53d915640 1 -- 172.21.15.37:0/2132414979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d5a42210 con 0x5603d5dd5cb0 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53c913640 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa53d915640 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd35e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa530009bf0 tx=0x7fa530003fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e17db22e809a83b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa520ff9640 1 -- 172.21.15.37:0/2132414979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa530048040 con 0x5603d5dd5cb0 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa530036070 con 0x5603d5dd5cb0 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa520ff9640 1 -- 172.21.15.37:0/2132414979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fa530031ac0 con 0x5603d5dd5cb0 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa520ff9640 1 -- 172.21.15.37:0/2132414979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa53005d050 con 0x5603d5dd5cb0 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa5421f1740 10 client.17868 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:20.434 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5603d5b0e620 con 0x5603d5dd5cb0 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.415+0000 7fa520ff9640 1 -- 172.21.15.37:0/2132414979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa530036070 con 0x5603d5dd5cb0 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa520ff9640 1 -- 172.21.15.37:0/2132414979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fa530042030 con 0x5603d5dd5cb0 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa5421f1740 10 client.17868 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa5421f1740 10 client.17868 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa5421f1740 20 client.17868 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5de5a50 0x5603d5de7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa5421f1740 4 client.17868 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603d5d3ca20 con 0x5603d5de5a50 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa53d114640 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5de5a50 0x5603d5de7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa53d114640 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5de5a50 0x5603d5de7e40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24ba9d441ca9f99 server_cookie=b34cb72515e7a4cd in_seq=0 out_seq=0 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.416+0000 7fa520ff9640 10 client.17868 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:20.435 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa520ff9640 1 -- 172.21.15.37:0/2132414979 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5603d5d3ca20 con 0x5603d5de5a50 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa520ff9640 10 client.17868 handle_command_reply: tid=0 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 client.17868 shutdown 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dd5cb0 msgr2=0x5603d5dd35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd35e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa530009bf0 tx=0x7fa530003fb0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 shutdown_connections 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5de5a50 0x5603d5de7e40 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd35e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dcb410 0x5603d5dd30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 --2- 172.21.15.37:0/2132414979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcaaf0 0x5603d5dd2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa520ff9640 0 client.17868 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 >> 172.21.15.37:0/2132414979 conn(0x5603d5dc7060 msgr2=0x5603d5dccdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 shutdown_connections 2024-09-19T06:50:20.436 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.417+0000 7fa5421f1740 1 -- 172.21.15.37:0/2132414979 wait complete. 2024-09-19T06:50:20.437 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.418+0000 7fa5421f1740 20 client.17868 trim_cache size 0 max 16384 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 Processor -- start 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 -- start start 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5dd33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5ded470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381b00f0 con 0x5603d5dcaaf0 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ad960 con 0x5603d5dd5cb0 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa5421f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ab7d0 con 0x5603d5dcb410 2024-09-19T06:50:20.442 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41896/0 (socket says 172.21.15.37:41896) 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 -- 172.21.15.37:0/1513453694 learned_addr learned my addr 172.21.15.37:0/1513453694 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d915640 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5ded470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53c913640 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5dd33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 -- 172.21.15.37:0/1513453694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 msgr2=0x5603d5dd33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5dd33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 -- 172.21.15.37:0/1513453694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 msgr2=0x5603d5ded470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5ded470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.432+0000 7fa53d114640 1 -- 172.21.15.37:0/1513453694 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603d5dc8110 con 0x5603d5dcaaf0 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa53d114640 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcd560 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa534005d00 tx=0x7fa534019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78a4f8b94b2ce67 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa522ffd640 1 -- 172.21.15.37:0/1513453694 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534010dc0 con 0x5603d5dcaaf0 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa522ffd640 1 -- 172.21.15.37:0/1513453694 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53401edd0 con 0x5603d5dcaaf0 2024-09-19T06:50:20.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa522ffd640 1 -- 172.21.15.37:0/1513453694 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53401c430 con 0x5603d5dcaaf0 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa5421f1740 1 -- 172.21.15.37:0/1513453694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 msgr2=0x5603d5dcd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa5421f1740 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcd560 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa534005d00 tx=0x7fa534019000 comp rx=0 tx=0).stop 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa5421f1740 1 -- 172.21.15.37:0/1513453694 shutdown_connections 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa5421f1740 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5ded470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa5421f1740 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5dd33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa5421f1740 1 --2- 172.21.15.37:0/1513453694 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5dcd560 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.433+0000 7fa5421f1740 1 -- 172.21.15.37:0/1513453694 >> 172.21.15.37:0/1513453694 conn(0x5603d5dc7060 msgr2=0x5603d5dd4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 -- 172.21.15.37:0/1513453694 shutdown_connections 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 -- 172.21.15.37:0/1513453694 wait complete. 2024-09-19T06:50:20.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 -- start start 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 Processor -- start 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5e48740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5e48c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381abe20 con 0x5603d5dcaaf0 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ad290 con 0x5603d5dd5cb0 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa5381ac4f0 con 0x5603d5dcb410 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa5421f1740 10 client.0 _ll_register_callbacks cb 0x7fa530046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.434+0000 7fa53d114640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5e48740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5e48740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41912/0 (socket says 172.21.15.37:41912) 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 -- 172.21.15.37:0/3277231868 learned_addr learned my addr 172.21.15.37:0/3277231868 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d915640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53c913640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5e48c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 -- 172.21.15.37:0/3277231868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 msgr2=0x5603d5e48c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5e48c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 -- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 msgr2=0x5603d5dd9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 -- 172.21.15.37:0/3277231868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603d5dca920 con 0x5603d5dcaaf0 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d915640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa53d114640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5e48740 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa534010a30 tx=0x7fa534010a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8664022b86073878 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa520ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa520ff9640 1 -- 172.21.15.37:0/3277231868 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53402a620 con 0x5603d5dcaaf0 2024-09-19T06:50:20.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa520ff9640 1 -- 172.21.15.37:0/3277231868 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fa53402a980 con 0x5603d5dcaaf0 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa53401edd0 con 0x5603d5dcaaf0 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa520ff9640 1 -- 172.21.15.37:0/3277231868 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fa53402ab80 con 0x5603d5dcaaf0 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa5421f1740 10 client.17874 fetch_fsmap learned FSMap version 36 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.435+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5603d5e4fde0 con 0x5603d5dcaaf0 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa520ff9640 1 -- 172.21.15.37:0/3277231868 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa53401edd0 con 0x5603d5dcaaf0 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa520ff9640 1 -- 172.21.15.37:0/3277231868 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 36) ==== 8358+0+0 (secure 0 0 0) 0x7fa534044070 con 0x5603d5dcaaf0 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa5421f1740 10 client.17874 fetch_fsmap finished waiting for FSMap version 36 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa5421f1740 10 client.17874 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa5421f1740 20 client.17874 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa5421f1740 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5e504a0 0x5603d5e528c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa5421f1740 4 client.17874 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5603d5d36d70 con 0x5603d5e504a0 2024-09-19T06:50:20.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa53c913640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5e504a0 0x5603d5e528c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa53c913640 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5e504a0 0x5603d5e528c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4031ba8e010e3d0 server_cookie=dce0a0e6cb482ad4 in_seq=0 out_seq=0 2024-09-19T06:50:20.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.436+0000 7fa520ff9640 10 client.17874 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.601+0000 7fa520ff9640 1 -- 172.21.15.37:0/3277231868 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5603d5d36d70 con 0x5603d5e504a0 2024-09-19T06:50:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.601+0000 7fa520ff9640 10 client.17874 handle_command_reply: tid=0 2024-09-19T06:50:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.601+0000 7fa5421f1740 1 client.17874 shutdown 2024-09-19T06:50:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.601+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 msgr2=0x5603d5e48740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.601+0000 7fa5421f1740 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5e48740 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fa534010a30 tx=0x7fa534010a60 comp rx=0 tx=0).stop 2024-09-19T06:50:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.601+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 shutdown_connections 2024-09-19T06:50:20.604 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.601+0000 7fa5421f1740 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5603d5e504a0 0x5603d5e528c0 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa520ff9640 0 client.17874 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa5421f1740 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5603d5dd5cb0 0x5603d5dd9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa5421f1740 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5603d5dcb410 0x5603d5e48c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa5421f1740 1 --2- 172.21.15.37:0/3277231868 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5603d5dcaaf0 0x5603d5e48740 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 >> 172.21.15.37:0/3277231868 conn(0x5603d5dc7060 msgr2=0x5603d5dd43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 shutdown_connections 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa5421f1740 1 -- 172.21.15.37:0/3277231868 wait complete. 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.602+0000 7fa5421f1740 20 client.17874 trim_cache size 0 max 16384 2024-09-19T06:50:20.605 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:50:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:50:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:50:20.606 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:50:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5240a3460 msgr2=0x7fa5240a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5240a3460 0x7fa5240a5920 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fa53400bfd0 tx=0x7fa53400b410 comp rx=0 tx=0).stop 2024-09-19T06:50:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 msgr2=0x7fa538198840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa538198840 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa52800e9f0 tx=0x7fa52800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 shutdown_connections 2024-09-19T06:50:20.607 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa5381069c0 0x7fa53819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5240a3460 0x7fa5240a5920 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa538101350 0x7fa538198840 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.603+0000 7fa53f378640 1 --2- 172.21.15.37:0/212760812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa5381009b0 0x7fa538198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.604+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 >> 172.21.15.37:0/212760812 conn(0x7fa538077990 msgr2=0x7fa5380ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.604+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 shutdown_connections 2024-09-19T06:50:20.608 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:20.604+0000 7fa53f378640 1 -- 172.21.15.37:0/212760812 wait complete. 2024-09-19T06:50:21.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 608993 7.24 MB/sec execute 288 sec latency 90.795 ms 2024-09-19T06:50:22.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:21 smithi151 ceph-mon[35236]: pgmap v268: 97 pgs: 97 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-19T06:50:22.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 611514 7.27 MB/sec execute 289 sec latency 11.188 ms 2024-09-19T06:50:22.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:21 smithi096 ceph-mon[35004]: pgmap v268: 97 pgs: 97 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-19T06:50:22.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:21 smithi037 ceph-mon[31211]: pgmap v268: 97 pgs: 97 active+clean; 81 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 2.2 MiB/s wr, 203 op/s 2024-09-19T06:50:23.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 614443 7.30 MB/sec execute 290 sec latency 14.335 ms 2024-09-19T06:50:24.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:23 smithi151 ceph-mon[35236]: pgmap v269: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-09-19T06:50:24.219 INFO:tasks.workunit.client.0.smithi037.stdout: 1 616776 7.31 MB/sec execute 291 sec latency 12.586 ms 2024-09-19T06:50:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:23 smithi096 ceph-mon[35004]: pgmap v269: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-09-19T06:50:24.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:23 smithi037 ceph-mon[31211]: pgmap v269: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-09-19T06:50:25.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 619056 7.33 MB/sec execute 292 sec latency 13.417 ms 2024-09-19T06:50:26.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:25 smithi151 ceph-mon[35236]: pgmap v270: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-09-19T06:50:26.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 621607 7.36 MB/sec execute 293 sec latency 219.859 ms 2024-09-19T06:50:26.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:25 smithi096 ceph-mon[35004]: pgmap v270: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-09-19T06:50:26.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:25 smithi037 ceph-mon[31211]: pgmap v270: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.6 MiB/s wr, 324 op/s 2024-09-19T06:50:27.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 623506 7.36 MB/sec execute 294 sec latency 19.962 ms 2024-09-19T06:50:28.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:27 smithi151 ceph-mon[35236]: pgmap v271: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-09-19T06:50:28.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:27 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:28.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 625989 7.38 MB/sec execute 295 sec latency 18.745 ms 2024-09-19T06:50:28.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:27 smithi096 ceph-mon[35004]: pgmap v271: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-09-19T06:50:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:27 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:28.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:27 smithi037 ceph-mon[31211]: pgmap v271: 97 pgs: 97 active+clean; 68 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.1 MiB/s wr, 272 op/s 2024-09-19T06:50:28.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:27 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:29.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 628880 7.42 MB/sec execute 296 sec latency 13.423 ms 2024-09-19T06:50:30.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:29 smithi151 ceph-mon[35236]: pgmap v272: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 354 op/s 2024-09-19T06:50:30.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 631199 7.42 MB/sec execute 297 sec latency 20.362 ms 2024-09-19T06:50:30.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:29 smithi096 ceph-mon[35004]: pgmap v272: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 354 op/s 2024-09-19T06:50:30.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:29 smithi037 ceph-mon[31211]: pgmap v272: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 MiB/s rd, 3.1 MiB/s wr, 354 op/s 2024-09-19T06:50:31.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 633425 7.44 MB/sec execute 298 sec latency 13.628 ms 2024-09-19T06:50:32.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:31 smithi151 ceph-mon[35236]: pgmap v273: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-09-19T06:50:32.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 636029 7.47 MB/sec execute 299 sec latency 27.012 ms 2024-09-19T06:50:32.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:31 smithi096 ceph-mon[35004]: pgmap v273: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-09-19T06:50:32.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:31 smithi037 ceph-mon[31211]: pgmap v273: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 2.1 MiB/s wr, 265 op/s 2024-09-19T06:50:33.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 638477 7.48 MB/sec execute 300 sec latency 22.519 ms 2024-09-19T06:50:33.705 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/3289265987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 msgr2=0x7f2bc410d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/3289265987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 0x7f2bc410d8a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009ca0 tx=0x7f2bbc02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/3289265987 shutdown_connections 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/3289265987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 0x7f2bc410d8a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/3289265987 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 0x7f2bc4100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/3289265987 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bc41069e0 0x7f2bc4106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/3289265987 >> 172.21.15.37:0/3289265987 conn(0x7f2bc40fc6b0 msgr2=0x7f2bc40fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/3289265987 shutdown_connections 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.703+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/3289265987 wait complete. 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.704+0000 7f2bc8cf1640 1 Processor -- start 2024-09-19T06:50:33.706 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.704+0000 7f2bc8cf1640 1 -- start start 2024-09-19T06:50:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.704+0000 7f2bc8cf1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 0x7f2bc4199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.704+0000 7f2bc8cf1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 0x7f2bc4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.704+0000 7f2bc8cf1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bc41069e0 0x7f2bc41a41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.708 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc8cf1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4110e20 con 0x7f2bc41009d0 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc8cf1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4110ca0 con 0x7f2bc4101370 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc8cf1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4110fa0 con 0x7f2bc41069e0 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 0x7f2bc4199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 0x7f2bc4199120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51130/0 (socket says 172.21.15.37:51130) 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 0x7f2bc4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc259c640 1 -- 172.21.15.37:0/2965864614 learned_addr learned my addr 172.21.15.37:0/2965864614 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bc41069e0 0x7f2bc41a41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc1d9b640 1 -- 172.21.15.37:0/2965864614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bc41069e0 msgr2=0x7f2bc41a41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bc41069e0 0x7f2bc41a41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc1d9b640 1 -- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 msgr2=0x7f2bc4199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 0x7f2bc4199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc1d9b640 1 -- 172.21.15.37:0/2965864614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc41a46e0 con 0x7f2bc4101370 2024-09-19T06:50:33.709 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc259c640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 0x7f2bc4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 0x7f2bc4199660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e9f0 tx=0x7f2bb800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bb37fe640 1 -- 172.21.15.37:0/2965864614 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb800cd80 con 0x7f2bc4101370 2024-09-19T06:50:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bb37fe640 1 -- 172.21.15.37:0/2965864614 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb80035f0 con 0x7f2bc4101370 2024-09-19T06:50:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.705+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc41a48b0 con 0x7f2bc4101370 2024-09-19T06:50:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.706+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc41a4e20 con 0x7f2bc4101370 2024-09-19T06:50:33.710 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.706+0000 7f2bb37fe640 1 -- 172.21.15.37:0/2965864614 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb8010690 con 0x7f2bc4101370 2024-09-19T06:50:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.708+0000 7f2bb37fe640 1 -- 172.21.15.37:0/2965864614 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2bb8003b30 con 0x7f2bc4101370 2024-09-19T06:50:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.708+0000 7f2bcba8f740 1 Processor -- start 2024-09-19T06:50:33.714 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.708+0000 7f2bcba8f740 1 -- start start 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bb37fe640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2bac0a34e0 0x7f2bac0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deedaf0 0x55675deeded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deee410 0x55675def8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675defb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55675de522f0 con 0x55675deedaf0 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55675ddfa270 con 0x55675def8cb0 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bb37fe640 1 -- 172.21.15.37:0/2965864614 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f2bb80cffd0 con 0x7f2bc4101370 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55675de4e2b0 con 0x55675deee410 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2bac0a34e0 0x7f2bac0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deedaf0 0x55675deeded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deedaf0 0x55675deeded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51136/0 (socket says 172.21.15.37:51136) 2024-09-19T06:50:33.715 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 -- 172.21.15.37:0/2971389071 learned_addr learned my addr 172.21.15.37:0/2971389071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deee410 0x55675def8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675defb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 -- 172.21.15.37:0/2971389071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deee410 msgr2=0x55675def8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deee410 0x55675def8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 -- 172.21.15.37:0/2971389071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 msgr2=0x55675defb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675defb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 -- 172.21.15.37:0/2971389071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675ddf9340 con 0x55675deedaf0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2bac0a34e0 0x7f2bac0a59a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009430 tx=0x7f2bbc009a20 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.709+0000 7f2bc259c640 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deedaf0 0x55675deeded0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40115c0 tx=0x7f2bb4011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73cbd50046ec92ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bb0ff9640 1 -- 172.21.15.37:0/2971389071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb400f660 con 0x55675deedaf0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bb0ff9640 1 -- 172.21.15.37:0/2971389071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb400fce0 con 0x55675deedaf0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bb0ff9640 1 -- 172.21.15.37:0/2971389071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb4019da0 con 0x55675deedaf0 2024-09-19T06:50:33.716 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 -- 172.21.15.37:0/2971389071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deedaf0 msgr2=0x55675deeded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deedaf0 0x55675deeded0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f2bb40115c0 tx=0x7f2bb4011a80 comp rx=0 tx=0).stop 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 -- 172.21.15.37:0/2971389071 shutdown_connections 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675defb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deee410 0x55675def8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/2971389071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deedaf0 0x55675deeded0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 -- 172.21.15.37:0/2971389071 >> 172.21.15.37:0/2971389071 conn(0x55675deea060 msgr2=0x55675deea440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 -- 172.21.15.37:0/2971389071 shutdown_connections 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.710+0000 7f2bcba8f740 1 -- 172.21.15.37:0/2971389071 wait complete. 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 -- start start 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 Processor -- start 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.717 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deee410 0x55675def60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675def65f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55675defb880 con 0x55675deee410 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55675dddb290 con 0x55675def8cb0 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55675defba00 con 0x55675deedaf0 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bc1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deee410 0x55675def60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bc1d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deee410 0x55675def60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51140/0 (socket says 172.21.15.37:51140) 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bc1d9b640 1 -- 172.21.15.37:0/199586052 learned_addr learned my addr 172.21.15.37:0/199586052 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.711+0000 7f2bcba8f740 10 client.0 _ll_register_callbacks cb 0x55675de0c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675def65f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc1d9b640 1 -- 172.21.15.37:0/199586052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 msgr2=0x55675def5b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def5b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc1d9b640 1 -- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 msgr2=0x55675def65f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:33.718 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675def65f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc1d9b640 1 -- 172.21.15.37:0/199586052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675db65210 con 0x55675deee410 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675def65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deee410 0x55675def60b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c6a0 tx=0x7f2bb809ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fffd036477fce62 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/199586052 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bb80b4070 con 0x55675deee410 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/199586052 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f2bb80988d0 con 0x55675deee410 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.712+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bb400fce0 con 0x55675deee410 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/199586052 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2bb80b5260 con 0x55675deee410 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 10 client.17886 fetch_fsmap learned FSMap version 37 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55675dc31620 con 0x55675deee410 2024-09-19T06:50:33.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/199586052 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bb400fce0 con 0x55675deee410 2024-09-19T06:50:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/199586052 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2bb80bc0c0 con 0x55675deee410 2024-09-19T06:50:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 10 client.17886 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T06:50:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 10 client.17886 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 20 client.17886 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:33.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df08c20 0x55675df0b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 4 client.17886 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55675de5fa20 con 0x55675df08c20 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.713+0000 7f2bc259c640 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df08c20 0x55675df0b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.714+0000 7f2ba9ffb640 10 client.17886 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.714+0000 7f2bc259c640 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df08c20 0x55675df0b070 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17899d15b92a0428 server_cookie=eff38bf5fd091f60 in_seq=0 out_seq=0 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.715+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/199586052 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55675de5fa20 con 0x55675df08c20 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.715+0000 7f2ba9ffb640 10 client.17886 handle_command_reply: tid=0 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 client.17886 shutdown 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deee410 msgr2=0x55675def60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deee410 0x55675def60b0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800c6a0 tx=0x7f2bb809ca20 comp rx=0 tx=0).stop 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 shutdown_connections 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df08c20 0x55675df0b070 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675def8cb0 0x55675def65f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2ba9ffb640 0 client.17886 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675deee410 0x55675def60b0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/199586052 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 >> 172.21.15.37:0/199586052 conn(0x55675deea060 msgr2=0x55675deefd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 shutdown_connections 2024-09-19T06:50:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 1 -- 172.21.15.37:0/199586052 wait complete. 2024-09-19T06:50:33.723 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.716+0000 7f2bcba8f740 20 client.17886 trim_cache size 0 max 16384 2024-09-19T06:50:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.731+0000 7f2bcba8f740 1 Processor -- start 2024-09-19T06:50:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bcba8f740 1 -- start start 2024-09-19T06:50:33.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675df10980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675df10ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc41adfd0 con 0x55675def8cb0 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc41ab870 con 0x55675deee410 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc41a9700 con 0x55675deedaf0 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675df10ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675df10ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51146/0 (socket says 172.21.15.37:51146) 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/3421935142 learned_addr learned my addr 172.21.15.37:0/3421935142 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:41226/0 (socket says 172.21.15.37:41226) 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675df10980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/3421935142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 msgr2=0x55675def6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/3421935142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 msgr2=0x55675df10980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675df10980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/3421935142 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55675df0b6f0 con 0x55675def8cb0 2024-09-19T06:50:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.732+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675df10ec0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc035000 tx=0x7f2bbc04d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9121bc29b70c00e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2babfff640 1 -- 172.21.15.37:0/3421935142 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc054070 con 0x55675def8cb0 2024-09-19T06:50:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2babfff640 1 -- 172.21.15.37:0/3421935142 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc040820 con 0x55675def8cb0 2024-09-19T06:50:33.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2babfff640 1 -- 172.21.15.37:0/3421935142 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc049040 con 0x55675def8cb0 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2bcba8f740 1 -- 172.21.15.37:0/3421935142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 msgr2=0x55675df10ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675df10ec0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc035000 tx=0x7f2bbc04d000 comp rx=0 tx=0).stop 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2bcba8f740 1 -- 172.21.15.37:0/3421935142 shutdown_connections 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675df10ec0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.733+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675df10980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/3421935142 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675def6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 -- 172.21.15.37:0/3421935142 >> 172.21.15.37:0/3421935142 conn(0x55675deea060 msgr2=0x55675def7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:33.739 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 -- 172.21.15.37:0/3421935142 shutdown_connections 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 -- 172.21.15.37:0/3421935142 wait complete. 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 -- start start 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 Processor -- start 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675df01560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675def2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675def31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc41a9d50 con 0x55675def8cb0 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc41ab1a0 con 0x55675deee410 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc41aa400 con 0x55675deedaf0 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bcba8f740 10 client.0 _ll_register_callbacks cb 0x7f2bb800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675def31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bc2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675def31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51160/0 (socket says 172.21.15.37:51160) 2024-09-19T06:50:33.740 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/503801415 learned_addr learned my addr 172.21.15.37:0/503801415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bc1d9b640 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675def2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.734+0000 7f2bc259c640 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675df01560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/503801415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 msgr2=0x55675df01560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675df01560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 msgr2=0x55675def2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675def2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bc2d9d640 1 -- 172.21.15.37:0/503801415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55675df72950 con 0x55675def8cb0 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bc2d9d640 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675def31b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc02ade0 tx=0x7f2bbc040d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3dda5bd89e53aad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2ba9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:33.741 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/503801415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc049030 con 0x55675def8cb0 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/503801415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f2bbc031650 con 0x55675def8cb0 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2bbc040820 con 0x55675def8cb0 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/503801415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) ==== 24+0+0 (secure 0 0 0) 0x7f2bbc067050 con 0x55675def8cb0 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bcba8f740 10 client.17892 fetch_fsmap learned FSMap version 37 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) -- 0x55675df73b40 con 0x55675def8cb0 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.735+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/503801415 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2bbc040820 con 0x55675def8cb0 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/503801415 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 37) ==== 8354+0+0 (secure 0 0 0) 0x7f2bbc062030 con 0x55675def8cb0 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bcba8f740 10 client.17892 fetch_fsmap finished waiting for FSMap version 37 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bcba8f740 10 client.17892 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bcba8f740 20 client.17892 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df74260 0x55675df76680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:33.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bcba8f740 4 client.17892 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55675de59d70 con 0x55675df74260 2024-09-19T06:50:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bc259c640 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df74260 0x55675df76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2bc259c640 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df74260 0x55675df76680 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6833ebdcb0b5b65e server_cookie=dd961d5ede6f8e3e in_seq=0 out_seq=0 2024-09-19T06:50:33.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.736+0000 7f2ba9ffb640 10 client.17892 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:33.887 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2ba9ffb640 1 -- 172.21.15.37:0/503801415 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55675de59d70 con 0x55675df74260 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2ba9ffb640 10 client.17892 handle_command_reply: tid=0 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2bcba8f740 1 client.17892 shutdown 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 msgr2=0x55675def31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675def31b0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc02ade0 tx=0x7f2bbc040d60 comp rx=0 tx=0).stop 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 shutdown_connections 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55675df74260 0x55675df76680 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.885+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55675def8cb0 0x55675def31b0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.886+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55675deee410 0x55675def2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.886+0000 7f2bcba8f740 1 --2- 172.21.15.37:0/503801415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55675deedaf0 0x55675df01560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.888 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.886+0000 7f2ba9ffb640 0 client.17892 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.886+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 >> 172.21.15.37:0/503801415 conn(0x55675deea060 msgr2=0x55675def7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.886+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 shutdown_connections 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.886+0000 7f2bcba8f740 1 -- 172.21.15.37:0/503801415 wait complete. 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.886+0000 7f2bcba8f740 20 client.17892 trim_cache size 0 max 16384 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:50:33.889 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:50:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2bac0a34e0 msgr2=0x7f2bac0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2bac0a34e0 0x7f2bac0a59a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009430 tx=0x7f2bbc009a20 comp rx=0 tx=0).stop 2024-09-19T06:50:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 msgr2=0x7f2bc4199660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 0x7f2bc4199660 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2bb800e9f0 tx=0x7f2bb800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 shutdown_connections 2024-09-19T06:50:33.890 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2bc41069e0 0x7f2bc41a41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2bc4101370 0x7f2bc4199660 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2bac0a34e0 0x7f2bac0a59a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 --2- 172.21.15.37:0/2965864614 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2bc41009d0 0x7f2bc4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 >> 172.21.15.37:0/2965864614 conn(0x7f2bc40fc6b0 msgr2=0x7f2bc40fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 shutdown_connections 2024-09-19T06:50:33.891 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:33.887+0000 7f2bc8cf1640 1 -- 172.21.15.37:0/2965864614 wait complete. 2024-09-19T06:50:34.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:33 smithi151 ceph-mon[35236]: pgmap v274: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-09-19T06:50:34.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 640235 7.49 MB/sec execute 301 sec latency 88.801 ms 2024-09-19T06:50:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:33 smithi096 ceph-mon[35004]: pgmap v274: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-09-19T06:50:34.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:33 smithi037 ceph-mon[31211]: pgmap v274: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.0 MiB/s wr, 352 op/s 2024-09-19T06:50:35.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 643051 7.52 MB/sec execute 302 sec latency 20.129 ms 2024-09-19T06:50:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:34 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:35.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:34 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:35.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:34 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:36.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 645566 7.54 MB/sec execute 303 sec latency 15.611 ms 2024-09-19T06:50:36.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:35 smithi096 ceph-mon[35004]: pgmap v275: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-19T06:50:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:35 smithi037 ceph-mon[31211]: pgmap v275: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-19T06:50:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:35 smithi151 ceph-mon[35236]: pgmap v275: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-19T06:50:37.150 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:50:37.220 INFO:tasks.workunit.client.0.smithi037.stdout: 1 647834 7.55 MB/sec execute 304 sec latency 12.895 ms 2024-09-19T06:50:37.419 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:38.117 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:37 smithi037 ceph-mon[31211]: pgmap v276: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-19T06:50:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/2123693222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc102d30 msgr2=0x7f2ccc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/2123693222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc102d30 0x7f2ccc103110 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009930 tx=0x7f2cc002a420 comp rx=0 tx=0).stop 2024-09-19T06:50:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/2123693222 shutdown_connections 2024-09-19T06:50:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/2123693222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1082a0 0x7f2ccc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/2123693222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc1036e0 0x7f2ccc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/2123693222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc102d30 0x7f2ccc103110 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/2123693222 >> 172.21.15.37:0/2123693222 conn(0x7f2ccc0fc7d0 msgr2=0x7f2ccc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/2123693222 shutdown_connections 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.107+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/2123693222 wait complete. 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 Processor -- start 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 -- start start 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc102d30 0x7f2ccc1a0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1036e0 0x7f2ccc1a0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1082a0 0x7f2ccc19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1120e0 con 0x7f2ccc1082a0 2024-09-19T06:50:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc111f60 con 0x7f2ccc1036e0 2024-09-19T06:50:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd2ce5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc112260 con 0x7f2ccc102d30 2024-09-19T06:50:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd1282640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1082a0 0x7f2ccc19a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1036e0 0x7f2ccc1a0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd1282640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1082a0 0x7f2ccc19a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51186/0 (socket says 172.21.15.37:51186) 2024-09-19T06:50:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1036e0 0x7f2ccc1a0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:53182/0 (socket says 172.21.15.37:53182) 2024-09-19T06:50:38.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd1282640 1 -- 172.21.15.37:0/4120244583 learned_addr learned my addr 172.21.15.37:0/4120244583 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.108+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc102d30 0x7f2ccc1a0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd1282640 1 -- 172.21.15.37:0/4120244583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc102d30 msgr2=0x7f2ccc1a0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd1282640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc102d30 0x7f2ccc1a0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd1282640 1 -- 172.21.15.37:0/4120244583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1036e0 msgr2=0x7f2ccc1a0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd1282640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1036e0 0x7f2ccc1a0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd1282640 1 -- 172.21.15.37:0/4120244583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ccc19ad20 con 0x7f2ccc1082a0 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1036e0 0x7f2ccc1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:38.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd1282640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1082a0 0x7f2ccc19a5f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400efd0 tx=0x7f2cc400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cc9ffb640 1 -- 172.21.15.37:0/4120244583 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc4003b70 con 0x7f2ccc1082a0 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ccc19b010 con 0x7f2ccc1082a0 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cc9ffb640 1 -- 172.21.15.37:0/4120244583 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc4003d10 con 0x7f2ccc1082a0 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cc9ffb640 1 -- 172.21.15.37:0/4120244583 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40109f0 con 0x7f2ccc1082a0 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.109+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ccc1a7300 con 0x7f2ccc1082a0 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.112+0000 7f2cd50be740 1 Processor -- start 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.112+0000 7f2cc9ffb640 1 -- 172.21.15.37:0/4120244583 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f2cc4004020 con 0x7f2ccc1082a0 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.112+0000 7f2cd50be740 1 -- start start 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.112+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce04f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.112+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce05a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.112+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce05a680 0x5652ce05ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd50be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5652cdfb52e0 con 0x5652ce04f4a0 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd50be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5652cdf5cce0 con 0x5652ce05a680 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd50be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5652cdfb12a0 con 0x5652ce04fdc0 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce04f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce05a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce05a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:41268/0 (socket says 172.21.15.37:41268) 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 -- 172.21.15.37:0/913234387 learned_addr learned my addr 172.21.15.37:0/913234387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce04f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51190/0 (socket says 172.21.15.37:51190) 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce05a680 0x5652ce05ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 -- 172.21.15.37:0/913234387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce05a680 msgr2=0x5652ce05ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce05a680 0x5652ce05ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 -- 172.21.15.37:0/913234387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 msgr2=0x5652ce04f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce04f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 -- 172.21.15.37:0/913234387 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652cdf41440 con 0x5652ce04fdc0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cc9ffb640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cbc0a33b0 0x7f2cbc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce04f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cbc0a33b0 0x7f2cbc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce05a140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009930 tx=0x7f2cc0005de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91efb0b0d6b60767 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.113+0000 7f2cc9ffb640 1 -- 172.21.15.37:0/4120244583 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f2cc40d00f0 con 0x7f2ccc1082a0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cb6ffd640 1 -- 172.21.15.37:0/913234387 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0035040 con 0x5652ce04fdc0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cb6ffd640 1 -- 172.21.15.37:0/913234387 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0005520 con 0x5652ce04fdc0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cb6ffd640 1 -- 172.21.15.37:0/913234387 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc00035f0 con 0x5652ce04fdc0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cd50be740 1 -- 172.21.15.37:0/913234387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 msgr2=0x5652ce05a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cbc0a33b0 0x7f2cbc0a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5652ce04cdb0 tx=0x7f2cb8015040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cd50be740 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce05a140 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0009930 tx=0x7f2cc0005de0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cd50be740 1 -- 172.21.15.37:0/913234387 shutdown_connections 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cd50be740 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce05a680 0x5652ce05ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cd50be740 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce05a140 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cd50be740 1 --2- 172.21.15.37:0/913234387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce04f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.114+0000 7f2cd50be740 1 -- 172.21.15.37:0/913234387 >> 172.21.15.37:0/913234387 conn(0x5652ce04ba30 msgr2=0x5652ce04be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 -- 172.21.15.37:0/913234387 shutdown_connections 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 -- 172.21.15.37:0/913234387 wait complete. 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 -- start start 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 Processor -- start 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce059f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce0571e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce05a680 0x5652ce057720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5652ce05d250 con 0x5652ce05a680 2024-09-19T06:50:38.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5652cdf70050 con 0x5652ce04f4a0 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5652ce05d3d0 con 0x5652ce04fdc0 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.115+0000 7f2cd50be740 10 client.0 _ll_register_callbacks cb 0x5652cdfb0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce0571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2cd0a81640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce059f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2cd1282640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce05a680 0x5652ce057720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce0571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:41282/0 (socket says 172.21.15.37:41282) 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 -- 172.21.15.37:0/2065670326 learned_addr learned my addr 172.21.15.37:0/2065670326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 -- 172.21.15.37:0/2065670326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 msgr2=0x5652ce059f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce059f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 -- 172.21.15.37:0/2065670326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce05a680 msgr2=0x5652ce057720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce05a680 0x5652ce057720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 -- 172.21.15.37:0/2065670326 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652cdd263c0 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce0571e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0697d0 tx=0x7f2cb8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47909e9c30f8fd19 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2cabfff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2cabfff640 1 -- 172.21.15.37:0/2065670326 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb8021e30 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc0005520 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.116+0000 7f2cabfff640 1 -- 172.21.15.37:0/2065670326 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f2cb802d480 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cabfff640 1 -- 172.21.15.37:0/2065670326 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2cb802d680 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 10 client.27022 fetch_fsmap learned FSMap version 38 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5652ce068c50 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cabfff640 1 -- 172.21.15.37:0/2065670326 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc0005520 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cabfff640 1 -- 172.21.15.37:0/2065670326 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f2cb8034070 con 0x5652ce04fdc0 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 10 client.27022 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:50:38.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 10 client.27022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 20 client.27022 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce06a060 0x5652ce06c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 4 client.27022 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.117+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5652cdfc2850 con 0x5652ce06a060 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.118+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce06a060 0x5652ce06c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.118+0000 7f2cabfff640 10 client.27022 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.118+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce06a060 0x5652ce06c480 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21eb01570f9e8e3d server_cookie=19e8a7a70b0b4780 in_seq=0 out_seq=0 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cabfff640 1 -- 172.21.15.37:0/2065670326 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5652cdfc2850 con 0x5652ce06a060 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cabfff640 10 client.27022 handle_command_reply: tid=0 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 client.27022 shutdown 2024-09-19T06:50:38.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 msgr2=0x5652ce0571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce0571e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc0697d0 tx=0x7f2cb8005f90 comp rx=0 tx=0).stop 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 shutdown_connections 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce06a060 0x5652ce06c480 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce05a680 0x5652ce057720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cabfff640 0 client.27022 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04fdc0 0x5652ce0571e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 --2- 172.21.15.37:0/2065670326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce059f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 >> 172.21.15.37:0/2065670326 conn(0x5652ce04ba30 msgr2=0x5652ce051760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 shutdown_connections 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.119+0000 7f2cd50be740 1 -- 172.21.15.37:0/2065670326 wait complete. 2024-09-19T06:50:38.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.120+0000 7f2cd50be740 20 client.27022 trim_cache size 0 max 16384 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.137+0000 7f2cd50be740 1 Processor -- start 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.137+0000 7f2cd50be740 1 -- start start 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce0664e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce071d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04fdc0 0x5652ce072290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd50be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1b24f0 con 0x5652ce04f4a0 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd50be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1afd90 con 0x5652ce04fdc0 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd50be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1adc00 con 0x5652ce04ba30 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce071d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2ccbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce071d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51204/0 (socket says 172.21.15.37:51204) 2024-09-19T06:50:38.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd1282640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04fdc0 0x5652ce072290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2ccbfff640 1 -- 172.21.15.37:0/1390441575 learned_addr learned my addr 172.21.15.37:0/1390441575 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce0664e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd1282640 1 -- 172.21.15.37:0/1390441575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 msgr2=0x5652ce0664e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd1282640 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce0664e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd1282640 1 -- 172.21.15.37:0/1390441575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 msgr2=0x5652ce071d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd1282640 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce071d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd1282640 1 -- 172.21.15.37:0/1390441575 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652ce04b860 con 0x5652ce04fdc0 2024-09-19T06:50:38.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.138+0000 7f2cd1282640 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04fdc0 0x5652ce072290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40cd810 tx=0x7f2cc4004920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af2fb6438e11fae6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cb67fc640 1 -- 172.21.15.37:0/1390441575 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40b4070 con 0x5652ce04fdc0 2024-09-19T06:50:38.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cb67fc640 1 -- 172.21.15.37:0/1390441575 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc409c3a0 con 0x5652ce04fdc0 2024-09-19T06:50:38.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cb67fc640 1 -- 172.21.15.37:0/1390441575 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc40b5380 con 0x5652ce04fdc0 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cd50be740 1 -- 172.21.15.37:0/1390441575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04fdc0 msgr2=0x5652ce072290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cd50be740 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04fdc0 0x5652ce072290 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f2cc40cd810 tx=0x7f2cc4004920 comp rx=0 tx=0).stop 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cd50be740 1 -- 172.21.15.37:0/1390441575 shutdown_connections 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cd50be740 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04fdc0 0x5652ce072290 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cd50be740 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04f4a0 0x5652ce071d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cd50be740 1 --2- 172.21.15.37:0/1390441575 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce0664e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.139+0000 7f2cd50be740 1 -- 172.21.15.37:0/1390441575 >> 172.21.15.37:0/1390441575 conn(0x5652ce057880 msgr2=0x5652ce055ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 -- 172.21.15.37:0/1390441575 shutdown_connections 2024-09-19T06:50:38.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 -- 172.21.15.37:0/1390441575 wait complete. 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 -- start start 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 Processor -- start 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce05e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce05eb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04fdc0 0x5652ce05f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ae250 con 0x5652ce04fdc0 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1af6c0 con 0x5652ce04f4a0 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f2ccc1ae920 con 0x5652ce04ba30 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd50be740 10 client.0 _ll_register_callbacks cb 0x7f2cb8013670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:38.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.140+0000 7f2cd0a81640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce05e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd1282640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04fdc0 0x5652ce05f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce05e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:41304/0 (socket says 172.21.15.37:41304) 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd1282640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04fdc0 0x5652ce05f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:51212/0 (socket says 172.21.15.37:51212) 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 -- 172.21.15.37:0/3731494484 learned_addr learned my addr 172.21.15.37:0/3731494484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce05eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 -- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 msgr2=0x5652ce05eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce05eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 -- 172.21.15.37:0/3731494484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04fdc0 msgr2=0x5652ce05f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04fdc0 0x5652ce05f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 -- 172.21.15.37:0/3731494484 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652ce0d3f70 con 0x5652ce04ba30 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd1282640 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04fdc0 0x5652ce05f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:38.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd0a81640 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce05e620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005bc0 tx=0x7f2cc0005b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3352d7983a461387 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cabfff640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cabfff640 1 -- 172.21.15.37:0/3731494484 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cc0035070 con 0x5652ce04ba30 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cabfff640 1 -- 172.21.15.37:0/3731494484 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f2cc00315b0 con 0x5652ce04ba30 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.141+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2cc409c3a0 con 0x5652ce04ba30 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cabfff640 1 -- 172.21.15.37:0/3731494484 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f2cc00317b0 con 0x5652ce04ba30 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 10 client.27028 fetch_fsmap learned FSMap version 38 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5652ce0d52e0 con 0x5652ce04ba30 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cabfff640 1 -- 172.21.15.37:0/3731494484 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2cc409c3a0 con 0x5652ce04ba30 2024-09-19T06:50:38.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cabfff640 1 -- 172.21.15.37:0/3731494484 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7f2cc0003b80 con 0x5652ce04ba30 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 10 client.27028 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 10 client.27028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 20 client.27028 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce05a680 0x5652ce0d7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 4 client.27028 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5652cdfbcd60 con 0x5652ce05a680 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.142+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce05a680 0x5652ce0d7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2ccbfff640 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce05a680 0x5652ce0d7580 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9711fcafdf838161 server_cookie=dadb08ce58d5a82c in_seq=0 out_seq=0 2024-09-19T06:50:38.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cabfff640 10 client.27028 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cabfff640 1 -- 172.21.15.37:0/3731494484 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5652cdfbcd60 con 0x5652ce05a680 2024-09-19T06:50:38.152 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cabfff640 10 client.27028 handle_command_reply: tid=0 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cd50be740 1 client.27028 shutdown 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 msgr2=0x5652ce05e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cd50be740 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce05e620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0005bc0 tx=0x7f2cc0005b50 comp rx=0 tx=0).stop 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 shutdown_connections 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.143+0000 7f2cd50be740 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5652ce05a680 0x5652ce0d7580 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cd50be740 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5652ce04fdc0 0x5652ce05f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cabfff640 0 client.27028 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:38.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cd50be740 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5652ce04f4a0 0x5652ce05eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cd50be740 1 --2- 172.21.15.37:0/3731494484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5652ce04ba30 0x5652ce05e620 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 >> 172.21.15.37:0/3731494484 conn(0x5652ce057880 msgr2=0x5652ce058a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 shutdown_connections 2024-09-19T06:50:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cd50be740 1 -- 172.21.15.37:0/3731494484 wait complete. 2024-09-19T06:50:38.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.144+0000 7f2cd50be740 20 client.27028 trim_cache size 0 max 16384 2024-09-19T06:50:38.154 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:50:38.154 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:50:38.155 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:50:38.155 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:50:38.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cbc0a33b0 msgr2=0x7f2cbc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cbc0a33b0 0x7f2cbc0a5870 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5652ce04cdb0 tx=0x7f2cb8015040 comp rx=0 tx=0).stop 2024-09-19T06:50:38.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1082a0 msgr2=0x7f2ccc19a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:38.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1082a0 0x7f2ccc19a5f0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f2cc400efd0 tx=0x7f2cc400c6a0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 shutdown_connections 2024-09-19T06:50:38.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f2cbc0a33b0 0x7f2cbc0a5870 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2ccc1082a0 0x7f2ccc19a5f0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2ccc1036e0 0x7f2ccc1a0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 --2- 172.21.15.37:0/4120244583 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ccc102d30 0x7f2ccc1a0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 >> 172.21.15.37:0/4120244583 conn(0x7f2ccc0fc7d0 msgr2=0x7f2ccc101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 shutdown_connections 2024-09-19T06:50:38.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:38.145+0000 7f2cd2ce5640 1 -- 172.21.15.37:0/4120244583 wait complete. 2024-09-19T06:50:38.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:37 smithi151 ceph-mon[35236]: pgmap v276: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-19T06:50:38.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 650274 7.57 MB/sec execute 305 sec latency 30.662 ms 2024-09-19T06:50:38.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:37 smithi096 ceph-mon[35004]: pgmap v276: 97 pgs: 97 active+clean; 89 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 1.9 MiB/s wr, 171 op/s 2024-09-19T06:50:38.889 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:50:38.889 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:50:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:39.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 652807 7.59 MB/sec execute 306 sec latency 13.939 ms 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.679+0000 7f1e5a73b640 1 -- 172.21.15.37:0/2894681136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54103660 msgr2=0x7f1e5410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.679+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/2894681136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54103660 0x7f1e5410bb50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009900 tx=0x7f1e4802a480 comp rx=0 tx=0).stop 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 -- 172.21.15.37:0/2894681136 shutdown_connections 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/2894681136 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e5410c170 0x7f1e5410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/2894681136 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54103660 0x7f1e5410bb50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/2894681136 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e54102d40 0x7f1e54103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 -- 172.21.15.37:0/2894681136 >> 172.21.15.37:0/2894681136 conn(0x7f1e540fc7a0 msgr2=0x7f1e540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.687 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 -- 172.21.15.37:0/2894681136 shutdown_connections 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 -- 172.21.15.37:0/2894681136 wait complete. 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 Processor -- start 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.680+0000 7f1e5a73b640 1 -- start start 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e5a73b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54102d40 0x7f1e541009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e5a73b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e54103660 0x7f1e54100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e5a73b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5410c170 0x7f1e54101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e5a73b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54111f70 con 0x7f1e54102d40 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e5a73b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1e54111df0 con 0x7f1e5410c170 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e5a73b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1e541120f0 con 0x7f1e54103660 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54102d40 0x7f1e541009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54102d40 0x7f1e541009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33540/0 (socket says 172.21.15.37:33540) 2024-09-19T06:50:39.688 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 -- 172.21.15.37:0/1949816698 learned_addr learned my addr 172.21.15.37:0/1949816698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e58cd8640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5410c170 0x7f1e54101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e537fe640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e54103660 0x7f1e54100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 -- 172.21.15.37:0/1949816698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e54103660 msgr2=0x7f1e54100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e54103660 0x7f1e54100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 -- 172.21.15.37:0/1949816698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5410c170 msgr2=0x7f1e54101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5410c170 0x7f1e54101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 -- 172.21.15.37:0/1949816698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e541a6910 con 0x7f1e54102d40 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e53fff640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54102d40 0x7f1e541009a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d950 tx=0x7f1e4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e517fa640 1 -- 172.21.15.37:0/1949816698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44017030 con 0x7f1e54102d40 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e517fa640 1 -- 172.21.15.37:0/1949816698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1e44004020 con 0x7f1e54102d40 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.681+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e541a6c00 con 0x7f1e54102d40 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.682+0000 7f1e517fa640 1 -- 172.21.15.37:0/1949816698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440047a0 con 0x7f1e54102d40 2024-09-19T06:50:39.689 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.682+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e541a7110 con 0x7f1e54102d40 2024-09-19T06:50:39.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.683+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e54103120 con 0x7f1e54102d40 2024-09-19T06:50:39.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.684+0000 7f1e517fa640 1 -- 172.21.15.37:0/1949816698 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1e440041d0 con 0x7f1e54102d40 2024-09-19T06:50:39.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.685+0000 7f1e517fa640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:39.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.685+0000 7f1e517fa640 1 -- 172.21.15.37:0/1949816698 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f1e440cf660 con 0x7f1e54102d40 2024-09-19T06:50:39.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.685+0000 7f1e537fe640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:39.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.685+0000 7f1e537fe640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009230 tx=0x7f1e480058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:39.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.689+0000 7f1e517fa640 1 -- 172.21.15.37:0/1949816698 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1e4409c010 con 0x7f1e54102d40 2024-09-19T06:50:39.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.909+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e54106c50 con 0x7f1e54102d40 2024-09-19T06:50:39.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.910+0000 7f1e517fa640 1 -- 172.21.15.37:0/1949816698 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11904 (secure 0 0 0) 0x7f1e4409b910 con 0x7f1e54102d40 2024-09-19T06:50:39.911 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:50:39.911 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":38,"btime":"2024-09-19T06:50:33:849103+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:33.041948+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24449,24470,24497]},"id":1}]} 2024-09-19T06:50:39.911 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 38 2024-09-19T06:50:39.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.912+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e3c0a33b0 msgr2=0x7f1e3c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.912+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f1e48009230 tx=0x7f1e480058d0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.912+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54102d40 msgr2=0x7f1e541009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:39.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.912+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54102d40 0x7f1e541009a0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d950 tx=0x7f1e4400de10 comp rx=0 tx=0).stop 2024-09-19T06:50:39.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 shutdown_connections 2024-09-19T06:50:39.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1e5410c170 0x7f1e54101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1e54103660 0x7f1e54100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1e3c0a33b0 0x7f1e3c0a5870 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 --2- 172.21.15.37:0/1949816698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1e54102d40 0x7f1e541009a0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:39.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 >> 172.21.15.37:0/1949816698 conn(0x7f1e540fc7a0 msgr2=0x7f1e540fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:39.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 shutdown_connections 2024-09-19T06:50:39.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:39.913+0000 7f1e5a73b640 1 -- 172.21.15.37:0/1949816698 wait complete. 2024-09-19T06:50:40.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 654443 7.59 MB/sec execute 307 sec latency 41.536 ms 2024-09-19T06:50:40.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:39 smithi096 ceph-mon[35004]: pgmap v277: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.2 MiB/s wr, 368 op/s 2024-09-19T06:50:40.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:39 smithi037 ceph-mon[31211]: pgmap v277: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.2 MiB/s wr, 368 op/s 2024-09-19T06:50:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:39 smithi151 ceph-mon[35236]: pgmap v277: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 3.2 MiB/s wr, 368 op/s 2024-09-19T06:50:41.057 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:50:41.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 657238 7.62 MB/sec execute 308 sec latency 62.524 ms 2024-09-19T06:50:41.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:40 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1949816698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:41.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:40 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1949816698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:41.319 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:41.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:40 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1949816698' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:50:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.825+0000 7fd2c9eca640 1 -- 172.21.15.37:0/444112187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4100e20 msgr2=0x7fd2c4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.825+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/444112187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4100e20 0x7fd2c4101200 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009890 tx=0x7fd2b402a410 comp rx=0 tx=0).stop 2024-09-19T06:50:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 -- 172.21.15.37:0/444112187 shutdown_connections 2024-09-19T06:50:41.839 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/444112187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102ef0 0x7fd2c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/444112187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4102570 0x7fd2c41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/444112187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4100e20 0x7fd2c4101200 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 -- 172.21.15.37:0/444112187 >> 172.21.15.37:0/444112187 conn(0x7fd2c40fc7d0 msgr2=0x7fd2c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 -- 172.21.15.37:0/444112187 shutdown_connections 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 -- 172.21.15.37:0/444112187 wait complete. 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 Processor -- start 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.826+0000 7fd2c9eca640 1 -- start start 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c9eca640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4100e20 0x7fd2c4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c9eca640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102570 0x7fd2c41976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c9eca640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4102ef0 0x7fd2c4197be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c9eca640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4110f80 con 0x7fd2c4102570 2024-09-19T06:50:41.840 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c9eca640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4110e00 con 0x7fd2c4102ef0 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c9eca640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c4111100 con 0x7fd2c4100e20 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102570 0x7fd2c41976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102570 0x7fd2c41976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33548/0 (socket says 172.21.15.37:33548) 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2bbfff640 1 -- 172.21.15.37:0/3372411191 learned_addr learned my addr 172.21.15.37:0/3372411191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4102ef0 0x7fd2c4197be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4100e20 0x7fd2c4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c3fff640 1 -- 172.21.15.37:0/3372411191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4100e20 msgr2=0x7fd2c4197160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4100e20 0x7fd2c4197160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c3fff640 1 -- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102570 msgr2=0x7fd2c41976a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102570 0x7fd2c41976a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c3fff640 1 -- 172.21.15.37:0/3372411191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c41a69d0 con 0x7fd2c4102ef0 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102570 0x7fd2c41976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:41.841 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4102ef0 0x7fd2c4197be0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00ef00 tx=0x7fd2bc00c740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c17fa640 1 -- 172.21.15.37:0/3372411191 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc004b20 con 0x7fd2c4102ef0 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c17fa640 1 -- 172.21.15.37:0/3372411191 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2bc003e50 con 0x7fd2c4102ef0 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c41a6cc0 con 0x7fd2c4102ef0 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.827+0000 7fd2c17fa640 1 -- 172.21.15.37:0/3372411191 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00b570 con 0x7fd2c4102ef0 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.828+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c41a7200 con 0x7fd2c4102ef0 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2c17fa640 1 -- 172.21.15.37:0/3372411191 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd2bc01d020 con 0x7fd2c4102ef0 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 Processor -- start 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 -- start start 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f865090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f8655d0 0x55df0f865a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f872760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df0f7cb2f0 con 0x55df0f870370 2024-09-19T06:50:41.842 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55df0f772cf0 con 0x55df0f8655d0 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df0f7c72b0 con 0x55df0f866b40 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f872760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f872760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33556/0 (socket says 172.21.15.37:33556) 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2bbfff640 1 -- 172.21.15.37:0/3013655078 learned_addr learned my addr 172.21.15.37:0/3013655078 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f8655d0 0x55df0f865a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.830+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f865090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2bbfff640 1 -- 172.21.15.37:0/3013655078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 msgr2=0x55df0f865090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f865090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2bbfff640 1 -- 172.21.15.37:0/3013655078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f8655d0 msgr2=0x55df0f865a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f8655d0 0x55df0f865a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2bbfff640 1 -- 172.21.15.37:0/3013655078 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0f7573a0 con 0x55df0f870370 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2c17fa640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2b00a3430 0x7fd2b00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f872760 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac011ca0 tx=0x7fd2ac009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20acfd65f7e4fdb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.843 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2b00a3430 0x7fd2b00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2c17fa640 1 -- 172.21.15.37:0/3372411191 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fd2bc0cfb50 con 0x7fd2c4102ef0 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2b9ffb640 1 -- 172.21.15.37:0/3013655078 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac00fc50 con 0x55df0f870370 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2b9ffb640 1 -- 172.21.15.37:0/3013655078 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac00fdf0 con 0x55df0f870370 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.831+0000 7fd2b9ffb640 1 -- 172.21.15.37:0/3013655078 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac01d5a0 con 0x55df0f870370 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/3013655078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 msgr2=0x55df0f872760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f872760 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac011ca0 tx=0x7fd2ac009c60 comp rx=0 tx=0).stop 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2b00a3430 0x7fd2b00a58f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40095f0 tx=0x7fd2b4005dc0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/3013655078 shutdown_connections 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f872760 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f8655d0 0x55df0f865a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/3013655078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f865090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/3013655078 >> 172.21.15.37:0/3013655078 conn(0x55df0f82e310 msgr2=0x55df0f861c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/3013655078 shutdown_connections 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/3013655078 wait complete. 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.832+0000 7fd2cc2a3740 1 -- start start 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 1 Processor -- start 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f8655d0 0x55df0f86fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f86d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f86d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55df0f873070 con 0x55df0f8655d0 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55df0f786060 con 0x55df0f870370 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55df0f8731f0 con 0x55df0f866b40 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2cc2a3740 10 client.0 _ll_register_callbacks cb 0x55df0f4d98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f8655d0 0x55df0f86fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c3fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f86d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f86d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f8655d0 0x55df0f86fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33564/0 (socket says 172.21.15.37:33564) 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c37fe640 1 -- 172.21.15.37:0/2425849342 learned_addr learned my addr 172.21.15.37:0/2425849342 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c3fff640 1 -- 172.21.15.37:0/2425849342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 msgr2=0x55df0f86d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f86d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c3fff640 1 -- 172.21.15.37:0/2425849342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f8655d0 msgr2=0x55df0f86fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f8655d0 0x55df0f86fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.833+0000 7fd2c3fff640 1 -- 172.21.15.37:0/2425849342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0f53c3c0 con 0x55df0f870370 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f8655d0 0x55df0f86fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f86d640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0cd270 tx=0x7fd2bc09c5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b876098db9053773 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/2425849342 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc01a070 con 0x55df0f870370 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/2425849342 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fd2bc098470 con 0x55df0f870370 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2ac00fdf0 con 0x55df0f870370 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/2425849342 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc0a4a70 con 0x55df0f870370 2024-09-19T06:50:41.846 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 10 client.27269 fetch_fsmap learned FSMap version 38 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55df0f87ec20 con 0x55df0f870370 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/2425849342 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac00fdf0 con 0x55df0f870370 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/2425849342 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fd2bc00b040 con 0x55df0f870370 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 10 client.27269 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 10 client.27269 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 20 client.27269 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f880090 0x55df0f882480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 4 client.27269 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.834+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df0f7d8860 con 0x55df0f880090 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.835+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f880090 0x55df0f882480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.835+0000 7fd2a2ffd640 10 client.27269 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.835+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f880090 0x55df0f882480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6de17f56278042c server_cookie=1e810cb15acce0c0 in_seq=0 out_seq=0 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/2425849342 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55df0f7d8860 con 0x55df0f880090 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2a2ffd640 10 client.27269 handle_command_reply: tid=0 2024-09-19T06:50:41.847 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 client.27269 shutdown 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 msgr2=0x55df0f86d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f86d640 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc0cd270 tx=0x7fd2bc09c5c0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 shutdown_connections 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f880090 0x55df0f882480 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f86d640 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f866b40 0x55df0f86d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2425849342 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f8655d0 0x55df0f86fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 >> 172.21.15.37:0/2425849342 conn(0x55df0f82e310 msgr2=0x55df0f867960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2a2ffd640 0 client.27269 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 shutdown_connections 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2425849342 wait complete. 2024-09-19T06:50:41.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.836+0000 7fd2cc2a3740 20 client.27269 trim_cache size 0 max 16384 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.850+0000 7fd2cc2a3740 1 Processor -- start 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.850+0000 7fd2cc2a3740 1 -- start start 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f87c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f866b40 0x55df0f861700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f861c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41b23e0 con 0x55df0f870370 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41afc80 con 0x55df0f866b40 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41adaf0 con 0x55df0f8655d0 2024-09-19T06:50:41.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f866b40 0x55df0f861700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f861c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f866b40 0x55df0f861700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48682/0 (socket says 172.21.15.37:48682) 2024-09-19T06:50:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 -- 172.21.15.37:0/2402744426 learned_addr learned my addr 172.21.15.37:0/2402744426 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f87c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 -- 172.21.15.37:0/2402744426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 msgr2=0x55df0f87c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f87c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 -- 172.21.15.37:0/2402744426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 msgr2=0x55df0f861c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f861c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 -- 172.21.15.37:0/2402744426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df0f862180 con 0x55df0f866b40 2024-09-19T06:50:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.851+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f866b40 0x55df0f861700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097f0 tx=0x7fd2ac00ee00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8e4ea91b637a11d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2b97fa640 1 -- 172.21.15.37:0/2402744426 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac028070 con 0x55df0f866b40 2024-09-19T06:50:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2b97fa640 1 -- 172.21.15.37:0/2402744426 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac005550 con 0x55df0f866b40 2024-09-19T06:50:41.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2b97fa640 1 -- 172.21.15.37:0/2402744426 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2ac003c10 con 0x55df0f866b40 2024-09-19T06:50:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2402744426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f866b40 msgr2=0x55df0f861700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f866b40 0x55df0f861700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac0097f0 tx=0x7fd2ac00ee00 comp rx=0 tx=0).stop 2024-09-19T06:50:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2402744426 shutdown_connections 2024-09-19T06:50:41.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f870370 0x55df0f861c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f866b40 0x55df0f861700 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/2402744426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f87c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2402744426 >> 172.21.15.37:0/2402744426 conn(0x55df0f82e310 msgr2=0x55df0f8682c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2402744426 shutdown_connections 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.852+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/2402744426 wait complete. 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 -- start start 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 Processor -- start 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f8786e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f866b40 0x55df0f878c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f869880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41ae140 con 0x55df0f866b40 2024-09-19T06:50:41.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41af5b0 con 0x55df0f870370 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c41ae810 con 0x55df0f8655d0 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2cc2a3740 10 client.0 _ll_register_callbacks cb 0x7fd2bc004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f866b40 0x55df0f878c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2bbfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f866b40 0x55df0f878c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33588/0 (socket says 172.21.15.37:33588) 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2bbfff640 1 -- 172.21.15.37:0/603820304 learned_addr learned my addr 172.21.15.37:0/603820304 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f869880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.853+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f8786e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2c3fff640 1 -- 172.21.15.37:0/603820304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 msgr2=0x55df0f8786e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f8786e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2c3fff640 1 -- 172.21.15.37:0/603820304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f866b40 msgr2=0x55df0f878c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f866b40 0x55df0f878c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2c3fff640 1 -- 172.21.15.37:0/603820304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df0f86a100 con 0x55df0f870370 2024-09-19T06:50:41.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2bbfff640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f866b40 0x55df0f878c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2c3fff640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f869880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc098260 tx=0x7fd2bc009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=118a11d6b90a3f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/603820304 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc01a040 con 0x55df0f870370 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2ac005550 con 0x55df0f870370 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.854+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/603820304 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fd2bc098470 con 0x55df0f870370 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/603820304 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc0980a0 con 0x55df0f870370 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 10 client.27275 fetch_fsmap learned FSMap version 38 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55df0f8ea760 con 0x55df0f870370 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/603820304 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd2ac005550 con 0x55df0f870370 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/603820304 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8350+0+0 (secure 0 0 0) 0x7fd2bc0a45e0 con 0x55df0f870370 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 10 client.27275 fetch_fsmap finished waiting for FSMap version 38 2024-09-19T06:50:41.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 10 client.27275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 20 client.27275 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f8eacf0 0x55df0f8ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 4 client.27275 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55df0f7d2d70 con 0x55df0f8eacf0 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f8eacf0 0x55df0f8ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.855+0000 7fd2a2ffd640 10 client.27275 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2c37fe640 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f8eacf0 0x55df0f8ed110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cb3c3c7f1261daab server_cookie=c3ee183e793e5708 in_seq=0 out_seq=0 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2a2ffd640 1 -- 172.21.15.37:0/603820304 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55df0f7d2d70 con 0x55df0f8eacf0 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2a2ffd640 10 client.27275 handle_command_reply: tid=0 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 client.27275 shutdown 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 msgr2=0x55df0f869880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f869880 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc098260 tx=0x7fd2bc009b90 comp rx=0 tx=0).stop 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 shutdown_connections 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55df0f8eacf0 0x55df0f8ed110 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55df0f870370 0x55df0f869880 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2a2ffd640 0 client.27275 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55df0f866b40 0x55df0f878c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 --2- 172.21.15.37:0/603820304 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55df0f8655d0 0x55df0f8786e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 >> 172.21.15.37:0/603820304 conn(0x55df0f82e310 msgr2=0x55df0f86b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 shutdown_connections 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 1 -- 172.21.15.37:0/603820304 wait complete. 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.856+0000 7fd2cc2a3740 20 client.27275 trim_cache size 0 max 16384 2024-09-19T06:50:41.862 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:50:41.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2b00a3430 msgr2=0x7fd2b00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2b00a3430 0x7fd2b00a58f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40095f0 tx=0x7fd2b4005dc0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4102ef0 msgr2=0x7fd2c4197be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4102ef0 0x7fd2c4197be0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc00ef00 tx=0x7fd2bc00c740 comp rx=0 tx=0).stop 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 shutdown_connections 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd2c4102ef0 0x7fd2c4197be0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2b00a3430 0x7fd2b00a58f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd2c4102570 0x7fd2c41976a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 --2- 172.21.15.37:0/3372411191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd2c4100e20 0x7fd2c4197160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 >> 172.21.15.37:0/3372411191 conn(0x7fd2c40fc7d0 msgr2=0x7fd2c40fd160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 shutdown_connections 2024-09-19T06:50:41.864 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:41.857+0000 7fd2c9eca640 1 -- 172.21.15.37:0/3372411191 wait complete. 2024-09-19T06:50:42.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:41 smithi037 ceph-mon[31211]: pgmap v278: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-09-19T06:50:42.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 659391 7.62 MB/sec execute 309 sec latency 52.180 ms 2024-09-19T06:50:42.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:41 smithi096 ceph-mon[35004]: pgmap v278: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-09-19T06:50:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:41 smithi151 ceph-mon[35236]: pgmap v278: 97 pgs: 97 active+clean; 78 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 2.2 MiB/s wr, 284 op/s 2024-09-19T06:50:42.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:50:43.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:42 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:43.043 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:43.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 661602 7.64 MB/sec execute 310 sec latency 13.149 ms 2024-09-19T06:50:43.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:42 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:43.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:42 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:50:43.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/1989367143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 msgr2=0x7fe6e8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/1989367143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8101200 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8009900 tx=0x7fe6d802a480 comp rx=0 tx=0).stop 2024-09-19T06:50:43.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/1989367143 shutdown_connections 2024-09-19T06:50:43.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/1989367143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e8102ef0 0x7fe6e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/1989367143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e8102570 0x7fe6e81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/1989367143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8101200 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/1989367143 >> 172.21.15.37:0/1989367143 conn(0x7fe6e80fc7d0 msgr2=0x7fe6e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.636+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/1989367143 shutdown_connections 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.637+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/1989367143 wait complete. 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.637+0000 7fe6ee5cc640 1 Processor -- start 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.637+0000 7fe6ee5cc640 1 -- start start 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.637+0000 7fe6ee5cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ee5cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e8102570 0x7fe6e8198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ee5cc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e8102ef0 0x7fe6e819ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ee5cc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8110f80 con 0x7fe6e8100e20 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ee5cc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8110e00 con 0x7fe6e8102570 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ee5cc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e8111100 con 0x7fe6e8102ef0 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8198540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33598/0 (socket says 172.21.15.37:33598) 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 -- 172.21.15.37:0/3257938448 learned_addr learned my addr 172.21.15.37:0/3257938448 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e77fe640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e8102570 0x7fe6e8198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 -- 172.21.15.37:0/3257938448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e8102ef0 msgr2=0x7fe6e819ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e8102ef0 0x7fe6e819ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e8102ef0 0x7fe6e819ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 -- 172.21.15.37:0/3257938448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e8102570 msgr2=0x7fe6e8198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e8102570 0x7fe6e8198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 -- 172.21.15.37:0/3257938448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6e819d4f0 con 0x7fe6e8100e20 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8198540 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8006010 tx=0x7fe6d8003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e57fa640 1 -- 172.21.15.37:0/3257938448 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d803f070 con 0x7fe6e8100e20 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e57fa640 1 -- 172.21.15.37:0/3257938448 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d8031c30 con 0x7fe6e8100e20 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6e819d780 con 0x7fe6e8100e20 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6e57fa640 1 -- 172.21.15.37:0/3257938448 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8035030 con 0x7fe6e8100e20 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.638+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6e81a50c0 con 0x7fe6e8100e20 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.641+0000 7fe6e57fa640 1 -- 172.21.15.37:0/3257938448 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe6d8042070 con 0x7fe6e8100e20 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.641+0000 7fe6f09a5740 1 Processor -- start 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.641+0000 7fe6f09a5740 1 -- start start 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.641+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee158e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee158ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee159b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eee14f42f0 con 0x55eee1599370 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eee149bcf0 con 0x55eee158fb40 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eee14f02b0 con 0x55eee158e5d0 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e57fa640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6d00a33b0 0x7fe6d00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee159b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee159b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33604/0 (socket says 172.21.15.37:33604) 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e77fe640 1 -- 172.21.15.37:0/925417212 learned_addr learned my addr 172.21.15.37:0/925417212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee158ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6d00a33b0 0x7fe6d00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee158e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e57fa640 1 -- 172.21.15.37:0/3257938448 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fe6d80f0450 con 0x7fe6e8100e20 2024-09-19T06:50:43.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e7fff640 1 -- 172.21.15.37:0/925417212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 msgr2=0x55eee158e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee158e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e7fff640 1 -- 172.21.15.37:0/925417212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 msgr2=0x55eee159b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee159b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e7fff640 1 -- 172.21.15.37:0/925417212 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee14803a0 con 0x55eee158e5d0 2024-09-19T06:50:43.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e77fe640 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee159b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6d00a33b0 0x7fe6d00a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0005c40 tx=0x7fe6e0009c10 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.642+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee158ea10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80edb70 tx=0x7fe6d80b1b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fcc7391703e977b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.643+0000 7fe6ce7fc640 1 -- 172.21.15.37:0/925417212 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8049030 con 0x55eee158e5d0 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.643+0000 7fe6ce7fc640 1 -- 172.21.15.37:0/925417212 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d80bce00 con 0x55eee158e5d0 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.643+0000 7fe6ce7fc640 1 -- 172.21.15.37:0/925417212 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d803b460 con 0x55eee158e5d0 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 -- 172.21.15.37:0/925417212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 msgr2=0x55eee158ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee158ea10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80edb70 tx=0x7fe6d80b1b90 comp rx=0 tx=0).stop 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 -- 172.21.15.37:0/925417212 shutdown_connections 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee159b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee158ea10 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/925417212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee158e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 -- 172.21.15.37:0/925417212 >> 172.21.15.37:0/925417212 conn(0x55eee1557310 msgr2=0x55eee158ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:43.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 -- 172.21.15.37:0/925417212 shutdown_connections 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 -- 172.21.15.37:0/925417212 wait complete. 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 -- start start 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.644+0000 7fe6f09a5740 1 Processor -- start 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1598e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee15960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1596870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eee159be30 con 0x55eee1599370 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eee14af060 con 0x55eee158fb40 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eee159bfb0 con 0x55eee158e5d0 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6f09a5740 10 client.0 _ll_register_callbacks cb 0x55eee12028c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1596870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee15960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1596870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33606/0 (socket says 172.21.15.37:33606) 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee15960e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48734/0 (socket says 172.21.15.37:48734) 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 -- 172.21.15.37:0/2371804291 learned_addr learned my addr 172.21.15.37:0/2371804291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1598e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 -- 172.21.15.37:0/2371804291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 msgr2=0x55eee1598e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1598e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 -- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 msgr2=0x55eee15960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee15960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 -- 172.21.15.37:0/2371804291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee12653c0 con 0x55eee1599370 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6e77fe640 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee15960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1596870 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009fc0 tx=0x7fe6e001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=602f2c6a715ebf8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.645+0000 7fe6c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:43.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6c77fe640 1 -- 172.21.15.37:0/2371804291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6e0024030 con 0x55eee1599370 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6c77fe640 1 -- 172.21.15.37:0/2371804291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fe6e0012e00 con 0x55eee1599370 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d80bce00 con 0x55eee1599370 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6c77fe640 1 -- 172.21.15.37:0/2371804291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6e0020480 con 0x55eee1599370 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 10 client.17946 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eee15a78e0 con 0x55eee1599370 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6c77fe640 1 -- 172.21.15.37:0/2371804291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d80bce00 con 0x55eee1599370 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6c77fe640 1 -- 172.21.15.37:0/2371804291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe6e001d050 con 0x55eee1599370 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 10 client.17946 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 10 client.17946 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 20 client.17946 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee15a8d40 0x55eee15ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 4 client.17946 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:43.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.646+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee1501860 con 0x55eee15a8d40 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.647+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee15a8d40 0x55eee15ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.647+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee15a8d40 0x55eee15ab1a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9755871682dbf6c7 server_cookie=3d302c4bd6987408 in_seq=0 out_seq=0 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.647+0000 7fe6c77fe640 10 client.17946 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6c77fe640 1 -- 172.21.15.37:0/2371804291 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eee1501860 con 0x55eee15a8d40 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6c77fe640 10 client.17946 handle_command_reply: tid=0 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 client.17946 shutdown 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 msgr2=0x55eee1596870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1596870 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0009fc0 tx=0x7fe6e001b000 comp rx=0 tx=0).stop 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 shutdown_connections 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee15a8d40 0x55eee15ab1a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1596870 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee15960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/2371804291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1598e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 >> 172.21.15.37:0/2371804291 conn(0x55eee1557310 msgr2=0x55eee1590960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:43.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6c77fe640 0 client.17946 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:50:43.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 shutdown_connections 2024-09-19T06:50:43.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 1 -- 172.21.15.37:0/2371804291 wait complete. 2024-09-19T06:50:43.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.648+0000 7fe6f09a5740 20 client.17946 trim_cache size 0 max 16384 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 Processor -- start 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 -- start start 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee15959a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee158fb40 0x55eee1598050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee1599370 0x55eee1591120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81b0230 con 0x55eee158fb40 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81adb00 con 0x55eee1599370 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81ab970 con 0x55eee158e5d0 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee15959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6e77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee158fb40 0x55eee1598050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee15959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:58518/0 (socket says 172.21.15.37:58518) 2024-09-19T06:50:43.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6ecb69640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee1599370 0x55eee1591120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.663+0000 7fe6e7fff640 1 -- 172.21.15.37:0/944409141 learned_addr learned my addr 172.21.15.37:0/944409141 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:43.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6e7fff640 1 -- 172.21.15.37:0/944409141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee1599370 msgr2=0x55eee1591120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee1599370 0x55eee1591120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6e7fff640 1 -- 172.21.15.37:0/944409141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee158fb40 msgr2=0x55eee1598050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee158fb40 0x55eee1598050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6e7fff640 1 -- 172.21.15.37:0/944409141 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee158b960 con 0x55eee158e5d0 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee1599370 0x55eee1591120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee15959a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80ee170 tx=0x7fe6d80b89a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e59a375f48ed5c1c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6cdffb640 1 -- 172.21.15.37:0/944409141 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d8049040 con 0x55eee158e5d0 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6cdffb640 1 -- 172.21.15.37:0/944409141 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d80bce00 con 0x55eee158e5d0 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.664+0000 7fe6cdffb640 1 -- 172.21.15.37:0/944409141 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d803b580 con 0x55eee158e5d0 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 -- 172.21.15.37:0/944409141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 msgr2=0x55eee15959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee15959a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80ee170 tx=0x7fe6d80b89a0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 -- 172.21.15.37:0/944409141 shutdown_connections 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee1599370 0x55eee1591120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee158fb40 0x55eee1598050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/944409141 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee15959a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 -- 172.21.15.37:0/944409141 >> 172.21.15.37:0/944409141 conn(0x55eee1557310 msgr2=0x55eee1597940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 -- 172.21.15.37:0/944409141 shutdown_connections 2024-09-19T06:50:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 -- 172.21.15.37:0/944409141 wait complete. 2024-09-19T06:50:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.665+0000 7fe6f09a5740 1 -- start start 2024-09-19T06:50:43.668 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 1 Processor -- start 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1593030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee1593570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1593ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81abfc0 con 0x55eee1599370 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81ad430 con 0x55eee158fb40 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe6e81ac690 con 0x55eee158e5d0 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6f09a5740 10 client.0 _ll_register_callbacks cb 0x7fe6e0011d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1593030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee1593570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6ecb69640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1593ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6e7fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1593030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:58520/0 (socket says 172.21.15.37:58520) 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6e7fff640 1 -- 172.21.15.37:0/4155723890 learned_addr learned my addr 172.21.15.37:0/4155723890 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.666+0000 7fe6e77fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee1593570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48754/0 (socket says 172.21.15.37:48754) 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6e7fff640 1 -- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 msgr2=0x55eee1593570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee1593570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6e7fff640 1 -- 172.21.15.37:0/4155723890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 msgr2=0x55eee1593ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1593ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6e7fff640 1 -- 172.21.15.37:0/4155723890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee158bdd0 con 0x55eee158e5d0 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6e77fe640 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee1593570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:43.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6ecb69640 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1593ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6e7fff640 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1593030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80c4e40 tx=0x7fe6d8005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b047e3493e18d01c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6c77fe640 1 -- 172.21.15.37:0/4155723890 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d80d4070 con 0x55eee158e5d0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6c77fe640 1 -- 172.21.15.37:0/4155723890 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fe6d80eddf0 con 0x55eee158e5d0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6d80bce00 con 0x55eee158e5d0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6c77fe640 1 -- 172.21.15.37:0/4155723890 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe6d80bc060 con 0x55eee158e5d0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6f09a5740 10 client.27067 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eee1617ec0 con 0x55eee158e5d0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.667+0000 7fe6c77fe640 1 -- 172.21.15.37:0/4155723890 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6d80bce00 con 0x55eee158e5d0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6c77fe640 1 -- 172.21.15.37:0/4155723890 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fe6d80bbb50 con 0x55eee158e5d0 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6f09a5740 10 client.27067 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6f09a5740 10 client.27067 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:50:43.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6f09a5740 20 client.27067 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee16180c0 0x55eee161a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6f09a5740 4 client.27067 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eee14fbd70 con 0x55eee16180c0 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6e77fe640 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee16180c0 0x55eee161a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6e77fe640 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee16180c0 0x55eee161a4e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d5d375fc04837aaa server_cookie=b4e037f98577a158 in_seq=0 out_seq=0 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6c77fe640 10 client.27067 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6c77fe640 1 -- 172.21.15.37:0/4155723890 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eee14fbd70 con 0x55eee16180c0 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6c77fe640 10 client.27067 handle_command_reply: tid=0 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.668+0000 7fe6f09a5740 1 client.27067 shutdown 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 msgr2=0x55eee1593030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1593030 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe6d80c4e40 tx=0x7fe6d8005c90 comp rx=0 tx=0).stop 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 shutdown_connections 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55eee16180c0 0x55eee161a4e0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eee1599370 0x55eee1593ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eee158fb40 0x55eee1593570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 --2- 172.21.15.37:0/4155723890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eee158e5d0 0x55eee1593030 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 >> 172.21.15.37:0/4155723890 conn(0x55eee1557310 msgr2=0x55eee1598160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6c77fe640 0 client.27067 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 shutdown_connections 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 1 -- 172.21.15.37:0/4155723890 wait complete. 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.669+0000 7fe6f09a5740 20 client.27067 trim_cache size 0 max 16384 2024-09-19T06:50:43.675 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:50:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6d00a33b0 msgr2=0x7fe6d00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6d00a33b0 0x7fe6d00a5870 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe6e0005c40 tx=0x7fe6e0009c10 comp rx=0 tx=0).stop 2024-09-19T06:50:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 msgr2=0x7fe6e8198540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8198540 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe6d8006010 tx=0x7fe6d8003c10 comp rx=0 tx=0).stop 2024-09-19T06:50:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 shutdown_connections 2024-09-19T06:50:43.676 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe6e8102ef0 0x7fe6e819ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6e8102570 0x7fe6e8198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe6d00a33b0 0x7fe6d00a5870 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 --2- 172.21.15.37:0/3257938448 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe6e8100e20 0x7fe6e8198540 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 >> 172.21.15.37:0/3257938448 conn(0x7fe6e80fc7d0 msgr2=0x7fe6e8105f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 shutdown_connections 2024-09-19T06:50:43.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:43.670+0000 7fe6ee5cc640 1 -- 172.21.15.37:0/3257938448 wait complete. 2024-09-19T06:50:44.200 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:43 smithi037 ceph-mon[31211]: pgmap v279: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-09-19T06:50:44.369 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:43 smithi151 ceph-mon[35236]: pgmap v279: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-09-19T06:50:44.369 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:43 smithi096 ceph-mon[35004]: pgmap v279: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 3.8 MiB/s wr, 390 op/s 2024-09-19T06:50:44.369 INFO:tasks.workunit.client.0.smithi037.stdout: 1 664318 7.67 MB/sec execute 311 sec latency 41.417 ms 2024-09-19T06:50:44.652 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:50:44.929 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:45.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 665779 7.67 MB/sec execute 312 sec latency 51.531 ms 2024-09-19T06:50:45.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.426+0000 7f4abb95e640 1 -- 172.21.15.37:0/3417759028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 msgr2=0x7f4ab4101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.426+0000 7f4abb95e640 1 --2- 172.21.15.37:0/3417759028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab4101200 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f4ab0009860 tx=0x7f4ab002a3d0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 -- 172.21.15.37:0/3417759028 shutdown_connections 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 --2- 172.21.15.37:0/3417759028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ab4102ef0 0x7f4ab410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 --2- 172.21.15.37:0/3417759028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab4102570 0x7f4ab41029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 --2- 172.21.15.37:0/3417759028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab4101200 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 -- 172.21.15.37:0/3417759028 >> 172.21.15.37:0/3417759028 conn(0x7f4ab40fc7d0 msgr2=0x7f4ab40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 -- 172.21.15.37:0/3417759028 shutdown_connections 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 -- 172.21.15.37:0/3417759028 wait complete. 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 Processor -- start 2024-09-19T06:50:45.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.427+0000 7f4abb95e640 1 -- start start 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4abb95e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab41984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4abb95e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab4102570 0x7f4ab4198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4abb95e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ab4102ef0 0x7f4ab419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4abb95e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41120a0 con 0x7f4ab4100e20 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4abb95e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4111f20 con 0x7f4ab4102ef0 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4abb95e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab4112220 con 0x7f4ab4102570 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab41984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab41984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33640/0 (socket says 172.21.15.37:33640) 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 -- 172.21.15.37:0/2679386005 learned_addr learned my addr 172.21.15.37:0/2679386005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab9efb640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ab4102ef0 0x7f4ab419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab4102570 0x7f4ab4198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 -- 172.21.15.37:0/2679386005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab4102570 msgr2=0x7f4ab4198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab4102570 0x7f4ab4198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 -- 172.21.15.37:0/2679386005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ab4102ef0 msgr2=0x7f4ab419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ab4102ef0 0x7f4ab419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 -- 172.21.15.37:0/2679386005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ab419d480 con 0x7f4ab4100e20 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab4102570 0x7f4ab4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab9efb640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ab4102ef0 0x7f4ab419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.428+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab41984d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4ab002a8f0 tx=0x7f4ab00306c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.429+0000 7f4aaa7fc640 1 -- 172.21.15.37:0/2679386005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab0040020 con 0x7f4ab4100e20 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.429+0000 7f4aaa7fc640 1 -- 172.21.15.37:0/2679386005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4ab0004460 con 0x7f4ab4100e20 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.429+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ab419d710 con 0x7f4ab4100e20 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.429+0000 7f4aaa7fc640 1 -- 172.21.15.37:0/2679386005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00046c0 con 0x7f4ab4100e20 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.429+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ab41a5050 con 0x7f4ab4100e20 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.431+0000 7f4aaa7fc640 1 -- 172.21.15.37:0/2679386005 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4ab0043020 con 0x7f4ab4100e20 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 Processor -- start 2024-09-19T06:50:45.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 -- start start 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c549c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c549ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4aaa7fc640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4aa00a3430 0x7f4aa00a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54adb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f5c54022f0 con 0x55f5c549db40 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5c53a9cf0 con 0x55f5c549c5d0 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4abdd37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f5c53fe2b0 con 0x55f5c54ab780 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4ab9efb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c549c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4ab9efb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c549c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33644/0 (socket says 172.21.15.37:33644) 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4ab9efb640 1 -- 172.21.15.37:0/1031089059 learned_addr learned my addr 172.21.15.37:0/1031089059 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4aa00a3430 0x7f4aa00a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c549ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4aaa7fc640 1 -- 172.21.15.37:0/2679386005 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f4ab00f0c90 con 0x7f4ab4100e20 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.432+0000 7f4ab9efb640 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4ab9efb640 1 -- 172.21.15.37:0/1031089059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 msgr2=0x55f5c54adb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4ab9efb640 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54adb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4ab9efb640 1 -- 172.21.15.37:0/1031089059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 msgr2=0x55f5c549ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4ab9efb640 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c549ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4ab9efb640 1 -- 172.21.15.37:0/1031089059 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c538e3a0 con 0x55f5c549db40 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4ab9efb640 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c549c090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009850 tx=0x7f4aac009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38b5e483788415fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4aa00a3430 0x7f4aa00a58f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40ffee0 tx=0x7f4aa4005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4a9f7fe640 1 -- 172.21.15.37:0/1031089059 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac017040 con 0x55f5c549db40 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4a9f7fe640 1 -- 172.21.15.37:0/1031089059 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac003e50 con 0x55f5c549db40 2024-09-19T06:50:45.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.433+0000 7f4a9f7fe640 1 -- 172.21.15.37:0/1031089059 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aac0045e0 con 0x55f5c549db40 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 -- 172.21.15.37:0/1031089059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 msgr2=0x55f5c549c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c549c090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f4aac009850 tx=0x7f4aac009ca0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 -- 172.21.15.37:0/1031089059 shutdown_connections 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54adb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c549ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1031089059 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c549c090 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 -- 172.21.15.37:0/1031089059 >> 172.21.15.37:0/1031089059 conn(0x55f5c5465310 msgr2=0x55f5c5498c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 -- 172.21.15.37:0/1031089059 shutdown_connections 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 -- 172.21.15.37:0/1031089059 wait complete. 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 -- start start 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.434+0000 7f4abdd37740 1 Processor -- start 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c549c5d0 0x55f5c54a4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549db40 0x55f5c54a4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c54ab780 0x55f5c54a4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4abdd37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f5c549f430 con 0x55f5c54ab780 2024-09-19T06:50:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4abdd37740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f5c53bd060 con 0x55f5c549db40 2024-09-19T06:50:45.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4abdd37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f5c549f5b0 con 0x55f5c549c5d0 2024-09-19T06:50:45.494 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4abdd37740 10 client.0 _ll_register_callbacks cb 0x55f5c51108c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549db40 0x55f5c54a4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab9efb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c54ab780 0x55f5c54a4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549db40 0x55f5c54a4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48784/0 (socket says 172.21.15.37:48784) 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/2943454181 learned_addr learned my addr 172.21.15.37:0/2943454181 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c549c5d0 0x55f5c54a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/2943454181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c549c5d0 msgr2=0x55f5c54a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c549c5d0 0x55f5c54a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/2943454181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c54ab780 msgr2=0x55f5c54a4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c54ab780 0x55f5c54a4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/2943454181 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c51733c0 con 0x55f5c549db40 2024-09-19T06:50:45.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c549c5d0 0x55f5c54a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549db40 0x55f5c54a4ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40045c0 tx=0x7f4aa4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb4e04ba30fbeccb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.435+0000 7f4a9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4a9cff9640 1 -- 172.21.15.37:0/2943454181 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4024020 con 0x55f5c549db40 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aac003e50 con 0x55f5c549db40 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4a9cff9640 1 -- 172.21.15.37:0/2943454181 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f4aa4012920 con 0x55f5c549db40 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4a9cff9640 1 -- 172.21.15.37:0/2943454181 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4aa402f050 con 0x55f5c549db40 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 10 client.27305 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f5c54b5b60 con 0x55f5c549db40 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4a9cff9640 1 -- 172.21.15.37:0/2943454181 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aac003e50 con 0x55f5c549db40 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4a9cff9640 1 -- 172.21.15.37:0/2943454181 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4aa401f390 con 0x55f5c549db40 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 10 client.27305 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:45.496 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 10 client.27305 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 20 client.27305 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c54b7020 0x55f5c54b9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 4 client.27305 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.436+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f5c540f860 con 0x55f5c54b7020 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.437+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c54b7020 0x55f5c54b9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.437+0000 7f4a9cff9640 10 client.27305 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.437+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c54b7020 0x55f5c54b9470 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3d30356d5ba039 server_cookie=b227689229e9b422 in_seq=0 out_seq=0 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4a9cff9640 1 -- 172.21.15.37:0/2943454181 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f5c540f860 con 0x55f5c54b7020 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4a9cff9640 10 client.27305 handle_command_reply: tid=0 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 client.27305 shutdown 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549db40 msgr2=0x55f5c54a4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549db40 0x55f5c54a4ab0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f4aa40045c0 tx=0x7f4aa4004730 comp rx=0 tx=0).stop 2024-09-19T06:50:45.497 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 shutdown_connections 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c54b7020 0x55f5c54b9470 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c54ab780 0x55f5c54a4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549db40 0x55f5c54a4ab0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 --2- 172.21.15.37:0/2943454181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c549c5d0 0x55f5c54a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 >> 172.21.15.37:0/2943454181 conn(0x55f5c5465310 msgr2=0x55f5c549e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4a9cff9640 0 client.27305 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 shutdown_connections 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 1 -- 172.21.15.37:0/2943454181 wait complete. 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.438+0000 7f4abdd37740 20 client.27305 trim_cache size 0 max 16384 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 Processor -- start 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 -- start start 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54a34b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.498 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54bed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41b6820 con 0x55f5c549db40 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41b40f0 con 0x55f5c549c5d0 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4abdd37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41b1f60 con 0x55f5c54ab780 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab96fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54a34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab9efb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54bed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33654/0 (socket says 172.21.15.37:33654) 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/1575305494 learned_addr learned my addr 172.21.15.37:0/1575305494 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/1575305494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 msgr2=0x55f5c54bed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54bed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/1575305494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 msgr2=0x55f5c54a34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.499 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54a34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/1575305494 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5c54ab580 con 0x55f5c549db40 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.452+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.453+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54a4e40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4aa401f7b0 tx=0x7f4aa4011cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71fc4f658f91a101 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.453+0000 7f4a9effd640 1 -- 172.21.15.37:0/1575305494 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4037070 con 0x55f5c549db40 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.453+0000 7f4a9effd640 1 -- 172.21.15.37:0/1575305494 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4016070 con 0x55f5c549db40 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.453+0000 7f4a9effd640 1 -- 172.21.15.37:0/1575305494 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4024040 con 0x55f5c549db40 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- 172.21.15.37:0/1575305494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 msgr2=0x55f5c54a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54a4e40 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f4aa401f7b0 tx=0x7f4aa4011cf0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- 172.21.15.37:0/1575305494 shutdown_connections 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54bed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54a4e40 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 --2- 172.21.15.37:0/1575305494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54a34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- 172.21.15.37:0/1575305494 >> 172.21.15.37:0/1575305494 conn(0x55f5c5465310 msgr2=0x55f5c54a8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- 172.21.15.37:0/1575305494 shutdown_connections 2024-09-19T06:50:45.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- 172.21.15.37:0/1575305494 wait complete. 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- start start 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 Processor -- start 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54b3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54af050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41b25b0 con 0x55f5c549db40 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41b3a20 con 0x55f5c549c5d0 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.454+0000 7f4abdd37740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4ab41b2c80 con 0x55f5c54ab780 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4abdd37740 10 client.0 _ll_register_callbacks cb 0x7f4aa4011c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33668/0 (socket says 172.21.15.37:33668) 2024-09-19T06:50:45.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/113941288 learned_addr learned my addr 172.21.15.37:0/113941288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab96fa640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54b3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab9efb640 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/113941288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 msgr2=0x55f5c54af5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54af5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 msgr2=0x55f5c54b3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54b3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 -- 172.21.15.37:0/113941288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5c54afe00 con 0x55f5c549db40 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4ab8ef9640 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54af050 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4039000 tx=0x7f4aa4005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e21e542ce1f57fe server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4a9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4a9cff9640 1 -- 172.21.15.37:0/113941288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4aa4037070 con 0x55f5c549db40 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4aa4016070 con 0x55f5c549db40 2024-09-19T06:50:45.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.455+0000 7f4a9cff9640 1 -- 172.21.15.37:0/113941288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f4aa401f9d0 con 0x55f5c549db40 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4a9cff9640 1 -- 172.21.15.37:0/113941288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4aa4024040 con 0x55f5c549db40 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 10 client.17970 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f5c5521df0 con 0x55f5c549db40 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4a9cff9640 1 -- 172.21.15.37:0/113941288 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4aa4016070 con 0x55f5c549db40 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4a9cff9640 1 -- 172.21.15.37:0/113941288 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f4aa404b070 con 0x55f5c549db40 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 10 client.17970 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 10 client.17970 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 20 client.17970 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c5522460 0x55f5c5524880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 4 client.17970 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f5c5409d70 con 0x55f5c5522460 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.456+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c5522460 0x55f5c5524880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:45.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4ab96fa640 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c5522460 0x55f5c5524880 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90ea0021a1b9185 server_cookie=783a2818835874b in_seq=0 out_seq=0 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4a9cff9640 10 client.17970 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4a9cff9640 1 -- 172.21.15.37:0/113941288 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f5c5409d70 con 0x55f5c5522460 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4a9cff9640 10 client.17970 handle_command_reply: tid=0 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 client.17970 shutdown 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 msgr2=0x55f5c54af050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54af050 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4aa4039000 tx=0x7f4aa4005640 comp rx=0 tx=0).stop 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 shutdown_connections 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f5c5522460 0x55f5c5524880 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f5c54ab780 0x55f5c54af5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4a9cff9640 0 client.17970 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f5c549db40 0x55f5c54af050 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 --2- 172.21.15.37:0/113941288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f5c549c5d0 0x55f5c54b3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.457+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 >> 172.21.15.37:0/113941288 conn(0x55f5c5465310 msgr2=0x55f5c54974e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 shutdown_connections 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abdd37740 1 -- 172.21.15.37:0/113941288 wait complete. 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abdd37740 20 client.17970 trim_cache size 0 max 16384 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4aa00a3430 msgr2=0x7f4aa00a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4aa00a3430 0x7f4aa00a58f0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4ab40ffee0 tx=0x7f4aa4005f90 comp rx=0 tx=0).stop 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 msgr2=0x7f4ab41984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab41984d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f4ab002a8f0 tx=0x7f4ab00306c0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 shutdown_connections 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4ab4102ef0 0x7f4ab419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4ab4102570 0x7f4ab4198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4aa00a3430 0x7f4aa00a58f0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 --2- 172.21.15.37:0/2679386005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4ab4100e20 0x7f4ab41984d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 >> 172.21.15.37:0/2679386005 conn(0x7f4ab40fc7d0 msgr2=0x7f4ab40fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:45.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 shutdown_connections 2024-09-19T06:50:45.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:45.458+0000 7f4abb95e640 1 -- 172.21.15.37:0/2679386005 wait complete. 2024-09-19T06:50:46.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 668101 7.69 MB/sec execute 313 sec latency 28.088 ms 2024-09-19T06:50:46.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:45 smithi096 ceph-mon[35004]: pgmap v280: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-09-19T06:50:46.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:45 smithi037 ceph-mon[31211]: pgmap v280: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-09-19T06:50:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:45 smithi151 ceph-mon[35236]: pgmap v280: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-09-19T06:50:46.985 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 -- 172.21.15.37:0/762186466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 msgr2=0x7f2428107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 --2- 172.21.15.37:0/762186466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 0x7f2428107f50 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f2418009cd0 tx=0x7f241802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 -- 172.21.15.37:0/762186466 shutdown_connections 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 --2- 172.21.15.37:0/762186466 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428102470 0x7f242810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 --2- 172.21.15.37:0/762186466 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2428101ad0 0x7f2428101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 --2- 172.21.15.37:0/762186466 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 0x7f2428107f50 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 -- 172.21.15.37:0/762186466 >> 172.21.15.37:0/762186466 conn(0x7f24280fd7d0 msgr2=0x7f24280ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 -- 172.21.15.37:0/762186466 shutdown_connections 2024-09-19T06:50:46.986 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.985+0000 7f242f3f6640 1 -- 172.21.15.37:0/762186466 wait complete. 2024-09-19T06:50:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 Processor -- start 2024-09-19T06:50:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 -- start start 2024-09-19T06:50:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428101ad0 0x7f24281983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.988 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2428102470 0x7f24281988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 0x7f242819cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2428111f20 con 0x7f2428107b70 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2428111da0 con 0x7f2428102470 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242f3f6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24281120a0 con 0x7f2428101ad0 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242d993640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 0x7f242819cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242c991640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2428102470 0x7f24281988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242c991640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2428102470 0x7f24281988f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48814/0 (socket says 172.21.15.37:48814) 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242c991640 1 -- 172.21.15.37:0/4134735994 learned_addr learned my addr 172.21.15.37:0/4134735994 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.986+0000 7f242d192640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428101ad0 0x7f24281983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242d993640 1 -- 172.21.15.37:0/4134735994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428101ad0 msgr2=0x7f24281983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242d993640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428101ad0 0x7f24281983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242d993640 1 -- 172.21.15.37:0/4134735994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2428102470 msgr2=0x7f24281988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:46.989 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242d993640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2428102470 0x7f24281988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242d993640 1 -- 172.21.15.37:0/4134735994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f242819d3a0 con 0x7f2428107b70 2024-09-19T06:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242d993640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 0x7f242819cc70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f242000e970 tx=0x7f242000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f24267fc640 1 -- 172.21.15.37:0/4134735994 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f242000ccc0 con 0x7f2428107b70 2024-09-19T06:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f242819d690 con 0x7f2428107b70 2024-09-19T06:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f24267fc640 1 -- 172.21.15.37:0/4134735994 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2420005020 con 0x7f2428107b70 2024-09-19T06:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f24267fc640 1 -- 172.21.15.37:0/4134735994 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2420004a00 con 0x7f2428107b70 2024-09-19T06:50:46.990 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.987+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24281a4f20 con 0x7f2428107b70 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.989+0000 7f24267fc640 1 -- 172.21.15.37:0/4134735994 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f24200051d0 con 0x7f2428107b70 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 Processor -- start 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 -- start start 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e20ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e2b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e2e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563158d852f0 con 0x563158e2bcb0 2024-09-19T06:50:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563158d2d270 con 0x563158e21410 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.990+0000 7f243226b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563158d812b0 con 0x563158e20af0 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e2b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242c991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e2e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e2b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48824/0 (socket says 172.21.15.37:48824) 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 -- 172.21.15.37:0/2704923205 learned_addr learned my addr 172.21.15.37:0/2704923205 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242c991640 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e20ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 -- 172.21.15.37:0/2704923205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 msgr2=0x563158e20ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e20ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 -- 172.21.15.37:0/2704923205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 msgr2=0x563158e2e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f24267fc640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24100a3460 0x7f24100a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e2e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 -- 172.21.15.37:0/2704923205 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563158d2c340 con 0x563158e21410 2024-09-19T06:50:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242c991640 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e20ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:50:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d993640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24100a3460 0x7f24100a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f24267fc640 1 -- 172.21.15.37:0/4134735994 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f24200cfcf0 con 0x7f2428107b70 2024-09-19T06:50:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.991+0000 7f242d192640 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e2b770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2418000f80 tx=0x7f2418009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a8818b137f9fe82c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f240f7fe640 1 -- 172.21.15.37:0/2704923205 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180318c0 con 0x563158e21410 2024-09-19T06:50:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f240f7fe640 1 -- 172.21.15.37:0/2704923205 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2418036040 con 0x563158e21410 2024-09-19T06:50:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f240f7fe640 1 -- 172.21.15.37:0/2704923205 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418004500 con 0x563158e21410 2024-09-19T06:50:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f242d993640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24100a3460 0x7f24100a5920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f242000e940 tx=0x7f2420015f00 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 -- 172.21.15.37:0/2704923205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 msgr2=0x563158e2b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:47.000 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e2b770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2418000f80 tx=0x7f2418009a20 comp rx=0 tx=0).stop 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 -- 172.21.15.37:0/2704923205 shutdown_connections 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e2e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e2b770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 --2- 172.21.15.37:0/2704923205 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e20ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 -- 172.21.15.37:0/2704923205 >> 172.21.15.37:0/2704923205 conn(0x563158e1d060 msgr2=0x563158e1d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.992+0000 7f243226b740 1 -- 172.21.15.37:0/2704923205 shutdown_connections 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.993+0000 7f243226b740 1 -- 172.21.15.37:0/2704923205 wait complete. 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.993+0000 7f243226b740 1 -- start start 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.993+0000 7f243226b740 1 Processor -- start 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.993+0000 7f243226b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e28c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.993+0000 7f243226b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e29190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.993+0000 7f243226b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e296d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.001 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f243226b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563158e32c90 con 0x563158e20af0 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f243226b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563158d0e290 con 0x563158e2bcb0 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f243226b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563158e32e10 con 0x563158e21410 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f243226b740 10 client.0 _ll_register_callbacks cb 0x563158d3f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e28c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e28c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33698/0 (socket says 172.21.15.37:33698) 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 -- 172.21.15.37:0/307771463 learned_addr learned my addr 172.21.15.37:0/307771463 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242d192640 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e29190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e296d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 -- 172.21.15.37:0/307771463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 msgr2=0x563158e29190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e29190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.002 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 -- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 msgr2=0x563158e296d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e296d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 -- 172.21.15.37:0/307771463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563158a98210 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f242c991640 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e28c50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f241c010d70 tx=0x7f241c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd0aff8e861b666 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f240cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f240cff9640 1 -- 172.21.15.37:0/307771463 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c00fe60 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f240cff9640 1 -- 172.21.15.37:0/307771463 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f241c004030 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2418036040 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f240cff9640 1 -- 172.21.15.37:0/307771463 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f241c025050 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f243226b740 10 client.17982 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.994+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563158b64620 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f240cff9640 1 -- 172.21.15.37:0/307771463 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2418036040 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f240cff9640 1 -- 172.21.15.37:0/307771463 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f241c020390 con 0x563158e20af0 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f243226b740 10 client.17982 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:47.003 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f243226b740 10 client.17982 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f243226b740 20 client.17982 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f243226b740 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158e400c0 0x563158e424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f243226b740 4 client.17982 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563158d92a20 con 0x563158e400c0 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f242d192640 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158e400c0 0x563158e424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f240cff9640 10 client.17982 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.995+0000 7f242d192640 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158e400c0 0x563158e424b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6225fbedd05434 server_cookie=805f9a6707a7a5ff in_seq=0 out_seq=0 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f240cff9640 1 -- 172.21.15.37:0/307771463 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563158d92a20 con 0x563158e400c0 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f240cff9640 10 client.17982 handle_command_reply: tid=0 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 client.17982 shutdown 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 msgr2=0x563158e28c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:47.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e28c50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f241c010d70 tx=0x7f241c004730 comp rx=0 tx=0).stop 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 shutdown_connections 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158e400c0 0x563158e424b0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e296d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e29190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f240cff9640 0 client.17982 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 --2- 172.21.15.37:0/307771463 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e28c50 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 >> 172.21.15.37:0/307771463 conn(0x563158e1d060 msgr2=0x563158e22d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 shutdown_connections 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.997+0000 7f243226b740 1 -- 172.21.15.37:0/307771463 wait complete. 2024-09-19T06:50:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:46.998+0000 7f243226b740 20 client.17982 trim_cache size 0 max 16384 2024-09-19T06:50:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.013+0000 7f243226b740 1 Processor -- start 2024-09-19T06:50:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f243226b740 1 -- start start 2024-09-19T06:50:47.015 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f243226b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e22c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f243226b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e47d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f243226b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e482b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f243226b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24281adff0 con 0x563158e2bcb0 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f243226b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24281ab8c0 con 0x563158e21410 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f243226b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24281a9730 con 0x563158e20af0 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242c991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e47d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e47d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48838/0 (socket says 172.21.15.37:48838) 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242c991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33702/0 (socket says 172.21.15.37:33702) 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 -- 172.21.15.37:0/1373707841 learned_addr learned my addr 172.21.15.37:0/1373707841 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242c991640 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e22c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 -- 172.21.15.37:0/1373707841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 msgr2=0x563158e22c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:47.016 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e22c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 -- 172.21.15.37:0/1373707841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 msgr2=0x563158e482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242d192640 1 -- 172.21.15.37:0/1373707841 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563158e42820 con 0x563158e21410 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.014+0000 7f242c991640 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f242d192640 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e47d70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2418000f80 tx=0x7f241803d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1366540af89890a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f240effd640 1 -- 172.21.15.37:0/1373707841 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24180318c0 con 0x563158e21410 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f240effd640 1 -- 172.21.15.37:0/1373707841 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2418036050 con 0x563158e21410 2024-09-19T06:50:47.017 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f240effd640 1 -- 172.21.15.37:0/1373707841 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2418004330 con 0x563158e21410 2024-09-19T06:50:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f243226b740 1 -- 172.21.15.37:0/1373707841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 msgr2=0x563158e47d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f243226b740 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e47d70 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2418000f80 tx=0x7f241803d040 comp rx=0 tx=0).stop 2024-09-19T06:50:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f243226b740 1 -- 172.21.15.37:0/1373707841 shutdown_connections 2024-09-19T06:50:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f243226b740 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e2bcb0 0x563158e482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.020 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f243226b740 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e21410 0x563158e47d70 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f243226b740 1 --2- 172.21.15.37:0/1373707841 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e20af0 0x563158e22c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.015+0000 7f243226b740 1 -- 172.21.15.37:0/1373707841 >> 172.21.15.37:0/1373707841 conn(0x563158e1d060 msgr2=0x563158e22550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 -- 172.21.15.37:0/1373707841 shutdown_connections 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 -- 172.21.15.37:0/1373707841 wait complete. 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 -- start start 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 Processor -- start 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e25e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e26370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e268b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f24281a9d80 con 0x563158e20af0 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24281ab1f0 con 0x563158e2bcb0 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f24281aa450 con 0x563158e21410 2024-09-19T06:50:47.021 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f243226b740 10 client.0 _ll_register_callbacks cb 0x7f241c0095a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.016+0000 7f242c991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e25e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e25e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33710/0 (socket says 172.21.15.37:33710) 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 -- 172.21.15.37:0/573413672 learned_addr learned my addr 172.21.15.37:0/573413672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242d192640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e26370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e268b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 -- 172.21.15.37:0/573413672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 msgr2=0x563158e26370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e26370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 -- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 msgr2=0x563158e268b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e268b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 -- 172.21.15.37:0/573413672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563158ea9f90 con 0x563158e20af0 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242d192640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e26370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f242c991640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e25e30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f241c023040 tx=0x7f241c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78d0641f0f7a9221 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f240cff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:47.022 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f240cff9640 1 -- 172.21.15.37:0/573413672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f241c015040 con 0x563158e20af0 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f240cff9640 1 -- 172.21.15.37:0/573413672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f241c009860 con 0x563158e20af0 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2418036050 con 0x563158e20af0 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f240cff9640 1 -- 172.21.15.37:0/573413672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f241c03e050 con 0x563158e20af0 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f243226b740 10 client.17988 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.017+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563158eab240 con 0x563158e20af0 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f240cff9640 1 -- 172.21.15.37:0/573413672 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2418036050 con 0x563158e20af0 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f240cff9640 1 -- 172.21.15.37:0/573413672 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f241c036020 con 0x563158e20af0 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f243226b740 10 client.17988 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f243226b740 10 client.17988 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f243226b740 20 client.17988 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f243226b740 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158eab910 0x563158eadd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f243226b740 4 client.17988 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563158d8cd70 con 0x563158eab910 2024-09-19T06:50:47.023 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f242d192640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158eab910 0x563158eadd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f242d192640 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158eab910 0x563158eadd00 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80826b8962bc7ff6 server_cookie=7feadc1c28880132 in_seq=0 out_seq=0 2024-09-19T06:50:47.024 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.018+0000 7f240cff9640 10 client.17988 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.165+0000 7f240cff9640 1 -- 172.21.15.37:0/573413672 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563158d8cd70 con 0x563158eab910 2024-09-19T06:50:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.165+0000 7f240cff9640 10 client.17988 handle_command_reply: tid=0 2024-09-19T06:50:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 client.17988 shutdown 2024-09-19T06:50:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 msgr2=0x563158e25e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e25e30 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f241c023040 tx=0x7f241c009f60 comp rx=0 tx=0).stop 2024-09-19T06:50:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 shutdown_connections 2024-09-19T06:50:47.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563158eab910 0x563158eadd00 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563158e2bcb0 0x563158e268b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563158e21410 0x563158e26370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f240cff9640 0 client.17988 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 --2- 172.21.15.37:0/573413672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563158e20af0 0x563158e25e30 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 >> 172.21.15.37:0/573413672 conn(0x563158e1d060 msgr2=0x563158e22d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 shutdown_connections 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 1 -- 172.21.15.37:0/573413672 wait complete. 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.166+0000 7f243226b740 20 client.17988 trim_cache size 0 max 16384 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:50:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:50:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:50:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:50:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24100a3460 msgr2=0x7f24100a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24100a3460 0x7f24100a5920 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f242000e940 tx=0x7f2420015f00 comp rx=0 tx=0).stop 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 msgr2=0x7f242819cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 0x7f242819cc70 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f242000e970 tx=0x7f242000ee30 comp rx=0 tx=0).stop 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 shutdown_connections 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24100a3460 0x7f24100a5920 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2428107b70 0x7f242819cc70 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2428102470 0x7f24281988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 --2- 172.21.15.37:0/4134735994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2428101ad0 0x7f24281983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 >> 172.21.15.37:0/4134735994 conn(0x7f24280fd7d0 msgr2=0x7f24280fdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 shutdown_connections 2024-09-19T06:50:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:50:47.167+0000 7f242f3f6640 1 -- 172.21.15.37:0/4134735994 wait complete. 2024-09-19T06:50:47.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 670388 7.70 MB/sec execute 314 sec latency 86.452 ms 2024-09-19T06:50:47.765 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:50:47.766 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:50:48.036 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:48.159 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:47 smithi037 ceph-mon[31211]: pgmap v281: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-09-19T06:50:48.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 672657 7.71 MB/sec execute 315 sec latency 12.195 ms 2024-09-19T06:50:48.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:47 smithi096 ceph-mon[35004]: pgmap v281: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-09-19T06:50:48.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:47 smithi151 ceph-mon[35236]: pgmap v281: 97 pgs: 97 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 2.9 MiB/s wr, 303 op/s 2024-09-19T06:50:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.516+0000 7f7428d17640 1 -- 172.21.15.37:0/275486767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742410c170 msgr2=0x7f742410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.524 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.516+0000 7f7428d17640 1 --2- 172.21.15.37:0/275486767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742410c170 0x7f742410e610 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f741c009ca0 tx=0x7f741c02a950 comp rx=0 tx=0).stop 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.516+0000 7f7428d17640 1 -- 172.21.15.37:0/275486767 shutdown_connections 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.516+0000 7f7428d17640 1 --2- 172.21.15.37:0/275486767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f742410c170 0x7f742410e610 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.516+0000 7f7428d17640 1 --2- 172.21.15.37:0/275486767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7424103660 0x7f742410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.516+0000 7f7428d17640 1 --2- 172.21.15.37:0/275486767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7424102d40 0x7f7424103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.516+0000 7f7428d17640 1 -- 172.21.15.37:0/275486767 >> 172.21.15.37:0/275486767 conn(0x7f74240fc7a0 msgr2=0x7f74240febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.517+0000 7f7428d17640 1 -- 172.21.15.37:0/275486767 shutdown_connections 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.517+0000 7f7428d17640 1 -- 172.21.15.37:0/275486767 wait complete. 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 Processor -- start 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 -- start start 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102d40 0x7f7424103b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7424103660 0x7f74241040d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f742410c170 0x7f7424104610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7424111f30 con 0x7f7424102d40 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7424111db0 con 0x7f742410c170 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7428d17640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f74241120b0 con 0x7f7424103660 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7422d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f742410c170 0x7f7424104610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7422d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f742410c170 0x7f7424104610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48880/0 (socket says 172.21.15.37:48880) 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7422d9d640 1 -- 172.21.15.37:0/7075560 learned_addr learned my addr 172.21.15.37:0/7075560 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.520+0000 7f7421d9b640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7424103660 0x7f74241040d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f742259c640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102d40 0x7f7424103b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f7422d9d640 1 -- 172.21.15.37:0/7075560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7424103660 msgr2=0x7f74241040d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f7422d9d640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7424103660 0x7f74241040d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f7422d9d640 1 -- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102d40 msgr2=0x7f7424103b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f7422d9d640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102d40 0x7f7424103b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f7422d9d640 1 -- 172.21.15.37:0/7075560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74241aac30 con 0x7f742410c170 2024-09-19T06:50:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f742259c640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102d40 0x7f7424103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:50:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.521+0000 7f7422d9d640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f742410c170 0x7f7424104610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f741c006010 tx=0x7f741c02f5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.522+0000 7f74137fe640 1 -- 172.21.15.37:0/7075560 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c034040 con 0x7f742410c170 2024-09-19T06:50:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.522+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74241aaec0 con 0x7f742410c170 2024-09-19T06:50:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.522+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74241ab3c0 con 0x7f742410c170 2024-09-19T06:50:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.522+0000 7f74137fe640 1 -- 172.21.15.37:0/7075560 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f741c030920 con 0x7f742410c170 2024-09-19T06:50:48.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.522+0000 7f74137fe640 1 -- 172.21.15.37:0/7075560 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741c004850 con 0x7f742410c170 2024-09-19T06:50:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.526+0000 7f742b0f0740 1 Processor -- start 2024-09-19T06:50:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.526+0000 7f74137fe640 1 -- 172.21.15.37:0/7075560 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f741c037020 con 0x7f742410c170 2024-09-19T06:50:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.526+0000 7f742b0f0740 1 -- start start 2024-09-19T06:50:48.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359645f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359645f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359645f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:58668/0 (socket says 172.21.15.37:58668) 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x557359650800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f7421d9b640 1 -- 172.21.15.37:0/2327498358 learned_addr learned my addr 172.21.15.37:0/2327498358 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359650d40 0x557359653130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5573595ab280 con 0x557359650d40 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557359552b10 con 0x557359646480 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5573595a7240 con 0x557359645b60 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f74137fe640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f740c0a35d0 0x7f740c0a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f742259c640 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x557359650800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f7421d9b640 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359650d40 0x557359653130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f74137fe640 1 -- 172.21.15.37:0/7075560 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f741c0f0960 con 0x7f742410c170 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.527+0000 7f7421d9b640 1 -- 172.21.15.37:0/2327498358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 msgr2=0x557359650800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7421d9b640 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x557359650800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7421d9b640 1 -- 172.21.15.37:0/2327498358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359650d40 msgr2=0x557359653130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7421d9b640 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359650d40 0x557359653130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7421d9b640 1 -- 172.21.15.37:0/2327498358 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55735955f1f0 con 0x557359645b60 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7421d9b640 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359645f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f741800b570 tx=0x7f741800b750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=951909ffe95d487b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7422d9d640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f740c0a35d0 0x7f740c0a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7410ff9640 1 -- 172.21.15.37:0/2327498358 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418004030 con 0x557359645b60 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7410ff9640 1 -- 172.21.15.37:0/2327498358 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74180041d0 con 0x557359645b60 2024-09-19T06:50:48.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.528+0000 7f7410ff9640 1 -- 172.21.15.37:0/2327498358 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741800f6e0 con 0x557359645b60 2024-09-19T06:50:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.529+0000 7f7422d9d640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f740c0a35d0 0x7f740c0a5a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f741c032000 tx=0x7f741c02ff50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.529+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 msgr2=0x557359645f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.529+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359645f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f741800b570 tx=0x7f741800b750 comp rx=0 tx=0).stop 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.529+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 shutdown_connections 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.529+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359650d40 0x557359653130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.529+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x557359650800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.529+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2327498358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359645f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.530+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 >> 172.21.15.37:0/2327498358 conn(0x5573596420f0 msgr2=0x5573596424d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.530+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 shutdown_connections 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.530+0000 7f742b0f0740 1 -- 172.21.15.37:0/2327498358 wait complete. 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 -- start start 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 Processor -- start 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359645b60 0x55735964db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x55735964e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 0x55735964e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5573596539b0 con 0x557359645b60 2024-09-19T06:50:48.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557359565f10 con 0x557359646480 2024-09-19T06:50:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.531+0000 7f742b0f0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557359653b30 con 0x557359650d40 2024-09-19T06:50:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f742b0f0740 10 client.0 _ll_register_callbacks cb 0x55735952ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f742259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x55735964e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359645b60 0x55735964db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f742259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x55735964e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48896/0 (socket says 172.21.15.37:48896) 2024-09-19T06:50:48.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f742259c640 1 -- 172.21.15.37:0/2752024938 learned_addr learned my addr 172.21.15.37:0/2752024938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f7421d9b640 1 -- 172.21.15.37:0/2752024938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 msgr2=0x55735964e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f7421d9b640 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 0x55735964e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f7421d9b640 1 -- 172.21.15.37:0/2752024938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 msgr2=0x55735964e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f7421d9b640 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x55735964e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.532+0000 7f7421d9b640 1 -- 172.21.15.37:0/2752024938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735931c6a0 con 0x557359645b60 2024-09-19T06:50:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.533+0000 7f7421d9b640 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359645b60 0x55735964db60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7418003e90 tx=0x7f741800b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ca656e806c4626 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.533+0000 7f7409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.533+0000 7f7409ffb640 1 -- 172.21.15.37:0/2752024938 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418004030 con 0x557359645b60 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.533+0000 7f7409ffb640 1 -- 172.21.15.37:0/2752024938 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f74180035f0 con 0x557359645b60 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.533+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74180041d0 con 0x557359645b60 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.533+0000 7f7409ffb640 1 -- 172.21.15.37:0/2752024938 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f74180037f0 con 0x557359645b60 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.534+0000 7f742b0f0740 10 client.18000 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.534+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55735965f590 con 0x557359645b60 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.534+0000 7f7409ffb640 1 -- 172.21.15.37:0/2752024938 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f74180041d0 con 0x557359645b60 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.534+0000 7f7409ffb640 1 -- 172.21.15.37:0/2752024938 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f741801d070 con 0x557359645b60 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.534+0000 7f742b0f0740 10 client.18000 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.534+0000 7f742b0f0740 10 client.18000 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.534+0000 7f742b0f0740 20 client.18000 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.535+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596608e0 0x557359662cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.535+0000 7f742b0f0740 4 client.18000 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:48.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.535+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573595b87f0 con 0x5573596608e0 2024-09-19T06:50:48.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.535+0000 7f742259c640 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596608e0 0x557359662cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.536+0000 7f742259c640 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596608e0 0x557359662cd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98753b540f418a35 server_cookie=94ca1ddd6fb75eb1 in_seq=0 out_seq=0 2024-09-19T06:50:48.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.536+0000 7f7409ffb640 10 client.18000 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f7409ffb640 1 -- 172.21.15.37:0/2752024938 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5573595b87f0 con 0x5573596608e0 2024-09-19T06:50:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f7409ffb640 10 client.18000 handle_command_reply: tid=0 2024-09-19T06:50:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 client.18000 shutdown 2024-09-19T06:50:48.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359645b60 msgr2=0x55735964db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359645b60 0x55735964db60 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f7418003e90 tx=0x7f741800b2e0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 shutdown_connections 2024-09-19T06:50:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596608e0 0x557359662cd0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 0x55735964e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359646480 0x55735964e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f7409ffb640 0 client.18000 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 --2- 172.21.15.37:0/2752024938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359645b60 0x55735964db60 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 >> 172.21.15.37:0/2752024938 conn(0x5573596420f0 msgr2=0x557359647dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 shutdown_connections 2024-09-19T06:50:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 1 -- 172.21.15.37:0/2752024938 wait complete. 2024-09-19T06:50:48.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.537+0000 7f742b0f0740 20 client.18000 trim_cache size 0 max 16384 2024-09-19T06:50:48.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 Processor -- start 2024-09-19T06:50:48.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 -- start start 2024-09-19T06:50:48.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359645b60 0x557359641160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.557 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x5573596684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 0x5573596689e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7404011120 con 0x557359646480 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f740400e9f0 con 0x557359645b60 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742b0f0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f740400c860 con 0x557359650d40 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x5573596684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359645b60 0x557359641160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359645b60 0x557359641160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48898/0 (socket says 172.21.15.37:48898) 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f742259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x5573596684a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33736/0 (socket says 172.21.15.37:33736) 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f7421d9b640 1 -- 172.21.15.37:0/1357588722 learned_addr learned my addr 172.21.15.37:0/1357588722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.556+0000 7f7421d9b640 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 0x5573596689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f7421d9b640 1 -- 172.21.15.37:0/1357588722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 msgr2=0x5573596689e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f7421d9b640 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 0x5573596689e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f7421d9b640 1 -- 172.21.15.37:0/1357588722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 msgr2=0x5573596684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.558 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f7421d9b640 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x5573596684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f7421d9b640 1 -- 172.21.15.37:0/1357588722 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557359641f20 con 0x557359645b60 2024-09-19T06:50:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f742259c640 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x5573596684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:48.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f7421d9b640 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359645b60 0x557359641160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7418003d60 tx=0x7f74180124a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc6bb7f096464d21 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f740bfff640 1 -- 172.21.15.37:0/1357588722 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741803a040 con 0x557359645b60 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f740bfff640 1 -- 172.21.15.37:0/1357588722 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7418003990 con 0x557359645b60 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.557+0000 7f740bfff640 1 -- 172.21.15.37:0/1357588722 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418004e70 con 0x557359645b60 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 -- 172.21.15.37:0/1357588722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359645b60 msgr2=0x557359641160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359645b60 0x557359641160 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7418003d60 tx=0x7f74180124a0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 -- 172.21.15.37:0/1357588722 shutdown_connections 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359650d40 0x5573596689e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x5573596684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 --2- 172.21.15.37:0/1357588722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359645b60 0x557359641160 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 -- 172.21.15.37:0/1357588722 >> 172.21.15.37:0/1357588722 conn(0x5573596420f0 msgr2=0x557359640a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 -- 172.21.15.37:0/1357588722 shutdown_connections 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 -- 172.21.15.37:0/1357588722 wait complete. 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 -- start start 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.558+0000 7f742b0f0740 1 Processor -- start 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359658e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x557359659360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742b0f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359650d40 0x557359653dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742b0f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f740400ceb0 con 0x557359646480 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742b0f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f740400e320 con 0x557359650d40 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742b0f0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f740400d580 con 0x557359645b60 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742b0f0740 10 client.0 _ll_register_callbacks cb 0x7f74180123f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:48.563 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x557359659360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359650d40 0x557359653dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x557359659360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:33750/0 (socket says 172.21.15.37:33750) 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742259c640 1 -- 172.21.15.37:0/4252453894 learned_addr learned my addr 172.21.15.37:0/4252453894 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359658e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 -- 172.21.15.37:0/4252453894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 msgr2=0x557359658e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359658e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 -- 172.21.15.37:0/4252453894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 msgr2=0x557359659360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x557359659360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 -- 172.21.15.37:0/4252453894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55735963d830 con 0x557359650d40 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f742259c640 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x557359659360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7421d9b640 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359650d40 0x557359653dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f741800ba00 tx=0x7f7418023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=586aa919235e4f2a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.559+0000 7f7409ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f7409ffb640 1 -- 172.21.15.37:0/4252453894 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7418005850 con 0x557359650d40 2024-09-19T06:50:48.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f7409ffb640 1 -- 172.21.15.37:0/4252453894 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f7418004150 con 0x557359650d40 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7418003990 con 0x557359650d40 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f7409ffb640 1 -- 172.21.15.37:0/4252453894 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7418004e70 con 0x557359650d40 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 10 client.27347 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5573596cbc30 con 0x557359650d40 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f7409ffb640 1 -- 172.21.15.37:0/4252453894 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7418003990 con 0x557359650d40 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f7409ffb640 1 -- 172.21.15.37:0/4252453894 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f741803f040 con 0x557359650d40 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 10 client.27347 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 10 client.27347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 20 client.27347 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596cc240 0x5573596ce630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 4 client.27347 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.560+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5573595b2d00 con 0x5573596cc240 2024-09-19T06:50:48.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.561+0000 7f742259c640 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596cc240 0x5573596ce630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.561+0000 7f742259c640 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596cc240 0x5573596ce630 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f453fa8b197f59ef server_cookie=15552dd0c11188f3 in_seq=0 out_seq=0 2024-09-19T06:50:48.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.561+0000 7f7409ffb640 10 client.27347 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f7409ffb640 1 -- 172.21.15.37:0/4252453894 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5573595b2d00 con 0x5573596cc240 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f7409ffb640 10 client.27347 handle_command_reply: tid=0 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 client.27347 shutdown 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359650d40 msgr2=0x557359653dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359650d40 0x557359653dd0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f741800ba00 tx=0x7f7418023040 comp rx=0 tx=0).stop 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 shutdown_connections 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5573596cc240 0x5573596ce630 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557359650d40 0x557359653dd0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f7409ffb640 0 client.27347 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557359646480 0x557359659360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 --2- 172.21.15.37:0/4252453894 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557359645b60 0x557359658e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 >> 172.21.15.37:0/4252453894 conn(0x5573596420f0 msgr2=0x557359647dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 shutdown_connections 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 1 -- 172.21.15.37:0/4252453894 wait complete. 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.562+0000 7f742b0f0740 20 client.27347 trim_cache size 0 max 16384 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "9e64056d-43a5-4bb8-904a-ce002e6a32ce", 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:50:48.568 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:50:48.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f740c0a35d0 msgr2=0x7f740c0a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f740c0a35d0 0x7f740c0a5a90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f741c032000 tx=0x7f741c02ff50 comp rx=0 tx=0).stop 2024-09-19T06:50:48.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f742410c170 msgr2=0x7f7424104610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:48.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f742410c170 0x7f7424104610 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f741c006010 tx=0x7f741c02f5b0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 shutdown_connections 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f742410c170 0x7f7424104610 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7424103660 0x7f74241040d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f740c0a35d0 0x7f740c0a5a90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 --2- 172.21.15.37:0/7075560 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7424102d40 0x7f7424103b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 >> 172.21.15.37:0/7075560 conn(0x7f74240fc7a0 msgr2=0x7f742410d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 shutdown_connections 2024-09-19T06:50:48.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:48.563+0000 7f7428d17640 1 -- 172.21.15.37:0/7075560 wait complete. 2024-09-19T06:50:49.190 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:48 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:50:49.190 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:48 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:49.191 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:48 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:50:49.191 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:48 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:49.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 675297 7.74 MB/sec execute 316 sec latency 12.125 ms 2024-09-19T06:50:49.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:48 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:50:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:48 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:48 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:50:49.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:48 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:48 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:50:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:48 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:48 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:50:49.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:48 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:49.379 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:50:49.675 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:50:50.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:49 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:50:50.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:49 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:50.060 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:49 smithi037 ceph-mon[31211]: pgmap v282: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.7 MiB/s wr, 523 op/s 2024-09-19T06:50:50.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 -- 172.21.15.37:0/1024471480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1036e0 msgr2=0x7f113c107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 --2- 172.21.15.37:0/1024471480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1036e0 0x7f113c107b90 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f112c009cd0 tx=0x7f112c02a8f0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 -- 172.21.15.37:0/1024471480 shutdown_connections 2024-09-19T06:50:50.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 --2- 172.21.15.37:0/1024471480 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f113c1082a0 0x7f113c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 --2- 172.21.15.37:0/1024471480 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1036e0 0x7f113c107b90 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 --2- 172.21.15.37:0/1024471480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f113c102d30 0x7f113c103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 -- 172.21.15.37:0/1024471480 >> 172.21.15.37:0/1024471480 conn(0x7f113c0fc7d0 msgr2=0x7f113c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:50.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 -- 172.21.15.37:0/1024471480 shutdown_connections 2024-09-19T06:50:50.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 -- 172.21.15.37:0/1024471480 wait complete. 2024-09-19T06:50:50.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 Processor -- start 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.149+0000 7f1142e47640 1 -- start start 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f1142e47640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f113c102d30 0x7f113c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f1142e47640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f113c1036e0 0x7f113c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f1142e47640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1082a0 0x7f113c19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f1142e47640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1120a0 con 0x7f113c1082a0 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f1142e47640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f113c111f20 con 0x7f113c102d30 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f1142e47640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f113c112220 con 0x7f113c1036e0 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f11413e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1082a0 0x7f113c19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f11413e4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1082a0 0x7f113c19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35242/0 (socket says 172.21.15.37:35242) 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f11413e4640 1 -- 172.21.15.37:0/1985508920 learned_addr learned my addr 172.21.15.37:0/1985508920 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f1140be3640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f113c102d30 0x7f113c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f113bfff640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f113c1036e0 0x7f113c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f11413e4640 1 -- 172.21.15.37:0/1985508920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f113c1036e0 msgr2=0x7f113c198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f11413e4640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f113c1036e0 0x7f113c198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f11413e4640 1 -- 172.21.15.37:0/1985508920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f113c102d30 msgr2=0x7f113c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.150+0000 7f11413e4640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f113c102d30 0x7f113c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f11413e4640 1 -- 172.21.15.37:0/1985508920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113c19d5f0 con 0x7f113c1082a0 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f1140be3640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f113c102d30 0x7f113c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f11413e4640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1082a0 0x7f113c19cf10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f112400e970 tx=0x7f112400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f1139ffb640 1 -- 172.21.15.37:0/1985508920 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112400ccc0 con 0x7f113c1082a0 2024-09-19T06:50:50.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f113c19d8e0 con 0x7f113c1082a0 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f1139ffb640 1 -- 172.21.15.37:0/1985508920 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112400ce60 con 0x7f113c1082a0 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f1139ffb640 1 -- 172.21.15.37:0/1985508920 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1124010450 con 0x7f113c1082a0 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.151+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f113c1008c0 con 0x7f113c1082a0 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1139ffb640 1 -- 172.21.15.37:0/1985508920 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1124005020 con 0x7f113c1082a0 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 Processor -- start 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 -- start start 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7d4a0 0x5648eac7d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac88140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac8aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648eabe32e0 con 0x5648eac88680 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5648eab8ace0 con 0x5648eac7d4a0 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1145220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648eabdf2a0 con 0x5648eac7ddc0 2024-09-19T06:50:50.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f1139ffb640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11280a33b0 0x7f11280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.154+0000 7f113bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7d4a0 0x5648eac7d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7d4a0 0x5648eac7d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:37002/0 (socket says 172.21.15.37:37002) 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 -- 172.21.15.37:0/2097242433 learned_addr learned my addr 172.21.15.37:0/2097242433 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f11413e4640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11280a33b0 0x7f11280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f1139ffb640 1 -- 172.21.15.37:0/1985508920 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f11240d00c0 con 0x7f113c1082a0 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac8aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f1140be3640 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac88140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 -- 172.21.15.37:0/2097242433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 msgr2=0x5648eac88140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac88140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 -- 172.21.15.37:0/2097242433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7d4a0 msgr2=0x5648eac7d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7d4a0 0x5648eac7d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 -- 172.21.15.37:0/2097242433 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648eab6f440 con 0x5648eac88680 2024-09-19T06:50:50.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f11413e4640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11280a33b0 0x7f11280a5870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11240cdde0 tx=0x7f11240077f0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f113bfff640 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac8aa70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f113c0697d0 tx=0x7f112c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99cc63400be0d9cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f1132ffd640 1 -- 172.21.15.37:0/2097242433 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c030740 con 0x5648eac88680 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f1132ffd640 1 -- 172.21.15.37:0/2097242433 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112c036030 con 0x5648eac88680 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.155+0000 7f1132ffd640 1 -- 172.21.15.37:0/2097242433 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c004230 con 0x5648eac88680 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 -- 172.21.15.37:0/2097242433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 msgr2=0x5648eac8aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac8aa70 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f113c0697d0 tx=0x7f112c000f40 comp rx=0 tx=0).stop 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 -- 172.21.15.37:0/2097242433 shutdown_connections 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac8aa70 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac88140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 --2- 172.21.15.37:0/2097242433 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7d4a0 0x5648eac7d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 -- 172.21.15.37:0/2097242433 >> 172.21.15.37:0/2097242433 conn(0x5648eac79a30 msgr2=0x5648eac79e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 -- 172.21.15.37:0/2097242433 shutdown_connections 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 -- 172.21.15.37:0/2097242433 wait complete. 2024-09-19T06:50:50.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.156+0000 7f1145220740 1 -- start start 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 1 Processor -- start 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac88010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7ddc0 0x5648eac852c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac85840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648eac8b250 con 0x5648eac88680 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5648eab9e050 con 0x5648eac7ddc0 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648eac8b3d0 con 0x5648eac7d4a0 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1145220740 10 client.0 _ll_register_callbacks cb 0x5648eabde160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f113bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac85840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1140be3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7ddc0 0x5648eac852c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1140be3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7ddc0 0x5648eac852c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:37008/0 (socket says 172.21.15.37:37008) 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f113bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac85840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35260/0 (socket says 172.21.15.37:35260) 2024-09-19T06:50:50.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f1140be3640 1 -- 172.21.15.37:0/3810556088 learned_addr learned my addr 172.21.15.37:0/3810556088 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.157+0000 7f113bfff640 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac88010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1140be3640 1 -- 172.21.15.37:0/3810556088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 msgr2=0x5648eac88010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1140be3640 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac88010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1140be3640 1 -- 172.21.15.37:0/3810556088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 msgr2=0x5648eac85840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1140be3640 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac85840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1140be3640 1 -- 172.21.15.37:0/3810556088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648ea9543c0 con 0x5648eac7ddc0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1140be3640 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7ddc0 0x5648eac852c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1134009fc0 tx=0x7f113400b750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=975c1937e18ba7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f111bfff640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f111bfff640 1 -- 172.21.15.37:0/3810556088 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113401f020 con 0x5648eac7ddc0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f112c036030 con 0x5648eac7ddc0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f111bfff640 1 -- 172.21.15.37:0/3810556088 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f1134004460 con 0x5648eac7ddc0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f111bfff640 1 -- 172.21.15.37:0/3810556088 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f113402a050 con 0x5648eac7ddc0 2024-09-19T06:50:50.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1145220740 10 client.27359 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5648eac96cf0 con 0x5648eac7ddc0 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.158+0000 7f111bfff640 1 -- 172.21.15.37:0/3810556088 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112c036030 con 0x5648eac7ddc0 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f111bfff640 1 -- 172.21.15.37:0/3810556088 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f113401b850 con 0x5648eac7ddc0 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f1145220740 10 client.27359 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f1145220740 10 client.27359 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f1145220740 20 client.27359 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f1145220740 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648eac981a0 0x5648eac9a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f1145220740 4 client.27359 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648eabf0850 con 0x5648eac981a0 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f113bfff640 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648eac981a0 0x5648eac9a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f113bfff640 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648eac981a0 0x5648eac9a5e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d139ee631fca414 server_cookie=30dd97878498d917 in_seq=0 out_seq=0 2024-09-19T06:50:50.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.159+0000 7f111bfff640 10 client.27359 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f111bfff640 1 -- 172.21.15.37:0/3810556088 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5648eabf0850 con 0x5648eac981a0 2024-09-19T06:50:50.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f111bfff640 10 client.27359 handle_command_reply: tid=0 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 client.27359 shutdown 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7ddc0 msgr2=0x5648eac852c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7ddc0 0x5648eac852c0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1134009fc0 tx=0x7f113400b750 comp rx=0 tx=0).stop 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 shutdown_connections 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648eac981a0 0x5648eac9a5e0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac88680 0x5648eac85840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac7ddc0 0x5648eac852c0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f111bfff640 0 client.27359 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 --2- 172.21.15.37:0/3810556088 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac88010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 >> 172.21.15.37:0/3810556088 conn(0x5648eac79a30 msgr2=0x5648eac7f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 shutdown_connections 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 1 -- 172.21.15.37:0/3810556088 wait complete. 2024-09-19T06:50:50.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.162+0000 7f1145220740 20 client.27359 trim_cache size 0 max 16384 2024-09-19T06:50:50.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.179+0000 7f1145220740 1 Processor -- start 2024-09-19T06:50:50.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.179+0000 7f1145220740 1 -- start start 2024-09-19T06:50:50.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f1145220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7d4a0 0x5648eac80b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f1145220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f1145220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac9feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f1145220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1ae0a0 con 0x5648eac7d4a0 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f1145220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1ab970 con 0x5648eac88680 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f1145220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1a9800 con 0x5648eac7ddc0 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7d4a0 0x5648eac80b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7d4a0 0x5648eac80b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35266/0 (socket says 172.21.15.37:35266) 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 -- 172.21.15.37:0/1251123490 learned_addr learned my addr 172.21.15.37:0/1251123490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f1140be3640 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac9feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 -- 172.21.15.37:0/1251123490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 msgr2=0x5648eac944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 -- 172.21.15.37:0/1251123490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 msgr2=0x5648eac9feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac9feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 -- 172.21.15.37:0/1251123490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648eac85e20 con 0x5648eac7d4a0 2024-09-19T06:50:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f113bfff640 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7d4a0 0x5648eac80b60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f112c005f60 tx=0x7f112c030620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4571d7ecd467bdb1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.180+0000 7f11327fc640 1 -- 172.21.15.37:0/1251123490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c003b70 con 0x5648eac7d4a0 2024-09-19T06:50:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f11327fc640 1 -- 172.21.15.37:0/1251123490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112c036040 con 0x5648eac7d4a0 2024-09-19T06:50:50.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f11327fc640 1 -- 172.21.15.37:0/1251123490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c004620 con 0x5648eac7d4a0 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f1145220740 1 -- 172.21.15.37:0/1251123490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7d4a0 msgr2=0x5648eac80b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f1145220740 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7d4a0 0x5648eac80b60 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f112c005f60 tx=0x7f112c030620 comp rx=0 tx=0).stop 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f1145220740 1 -- 172.21.15.37:0/1251123490 shutdown_connections 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f1145220740 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac9feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f1145220740 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7ddc0 0x5648eac944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f1145220740 1 --2- 172.21.15.37:0/1251123490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7d4a0 0x5648eac80b60 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.181+0000 7f1145220740 1 -- 172.21.15.37:0/1251123490 >> 172.21.15.37:0/1251123490 conn(0x5648eac79a30 msgr2=0x5648eac87340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:50.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 -- 172.21.15.37:0/1251123490 shutdown_connections 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 -- 172.21.15.37:0/1251123490 wait complete. 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 -- start start 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 Processor -- start 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7ddc0 0x5648eac81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1a9e50 con 0x5648eac7ddc0 2024-09-19T06:50:50.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1ab2a0 con 0x5648eac88680 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f113c1aa500 con 0x5648eac7d4a0 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.182+0000 7f1145220740 10 client.0 _ll_register_callbacks cb 0x7f1134009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7ddc0 0x5648eac81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7ddc0 0x5648eac81f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35274/0 (socket says 172.21.15.37:35274) 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 -- 172.21.15.37:0/2649119724 learned_addr learned my addr 172.21.15.37:0/2649119724 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f113bfff640 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f113bfff640 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 -- 172.21.15.37:0/2649119724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 msgr2=0x5648eac90c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac90c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 -- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 msgr2=0x5648eac824c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:50:50.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac824c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 -- 172.21.15.37:0/2649119724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648eac82c00 con 0x5648eac7ddc0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1140be3640 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7ddc0 0x5648eac81f80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f113401bc70 tx=0x7f113400bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d644ecd0351e920c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f111bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f111bfff640 1 -- 172.21.15.37:0/2649119724 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113401f070 con 0x5648eac7ddc0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f111bfff640 1 -- 172.21.15.37:0/2649119724 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7f1134005610 con 0x5648eac7ddc0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.183+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f112c036040 con 0x5648eac7ddc0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f111bfff640 1 -- 172.21.15.37:0/2649119724 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f1134005810 con 0x5648eac7ddc0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 10 client.18024 fetch_fsmap learned FSMap version 39 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5648ead02dc0 con 0x5648eac7ddc0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f111bfff640 1 -- 172.21.15.37:0/2649119724 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f112c036040 con 0x5648eac7ddc0 2024-09-19T06:50:50.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f111bfff640 1 -- 172.21.15.37:0/2649119724 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f1134003ac0 con 0x5648eac7ddc0 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 10 client.18024 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 20 client.18024 populate_metadata read hostname 'smithi037' 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648ead03530 0x5648ead05950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 4 client.18024 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5648eabead60 con 0x5648ead03530 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.184+0000 7f113bfff640 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648ead03530 0x5648ead05950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.185+0000 7f111bfff640 10 client.18024 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.185+0000 7f113bfff640 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648ead03530 0x5648ead05950 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a36ea89b5e46c2a2 server_cookie=f524f91ec82ead8e in_seq=0 out_seq=0 2024-09-19T06:50:50.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f111bfff640 1 -- 172.21.15.37:0/2649119724 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5648eabead60 con 0x5648ead03530 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f111bfff640 10 client.18024 handle_command_reply: tid=0 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 client.18024 shutdown 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7ddc0 msgr2=0x5648eac81f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7ddc0 0x5648eac81f80 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f113401bc70 tx=0x7f113400bfd0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 shutdown_connections 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648ead03530 0x5648ead05950 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648eac88680 0x5648eac824c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648eac7ddc0 0x5648eac81f80 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 --2- 172.21.15.37:0/2649119724 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648eac7d4a0 0x5648eac90c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 >> 172.21.15.37:0/2649119724 conn(0x5648eac79a30 msgr2=0x5648eac83a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.186+0000 7f111bfff640 0 client.18024 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:50:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 shutdown_connections 2024-09-19T06:50:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1145220740 1 -- 172.21.15.37:0/2649119724 wait complete. 2024-09-19T06:50:50.194 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1145220740 20 client.18024 trim_cache size 0 max 16384 2024-09-19T06:50:50.194 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:50:50.194 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:50:50.195 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:50:50.195 INFO:teuthology.orchestra.run.smithi037.stdout: "9e64056d-43a5-4bb8-904a-ce002e6a32ce": { 2024-09-19T06:50:50.195 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:50:50.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "9e64056d-43a5-4bb8-904a-ce002e6a32ce", 2024-09-19T06:50:50.195 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:50:50.195 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:50.196 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:50:50.196 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:50:50.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11280a33b0 msgr2=0x7f11280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11280a33b0 0x7f11280a5870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f11240cdde0 tx=0x7f11240077f0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1082a0 msgr2=0x7f113c19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1082a0 0x7f113c19cf10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f112400e970 tx=0x7f112400ed70 comp rx=0 tx=0).stop 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 shutdown_connections 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11280a33b0 0x7f11280a5870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f113c1082a0 0x7f113c19cf10 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f113c1036e0 0x7f113c198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 --2- 172.21.15.37:0/1985508920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f113c102d30 0x7f113c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.187+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 >> 172.21.15.37:0/1985508920 conn(0x7f113c0fc7d0 msgr2=0x7f113c106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.188+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 shutdown_connections 2024-09-19T06:50:50.197 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:50:50.188+0000 7f1142e47640 1 -- 172.21.15.37:0/1985508920 wait complete. 2024-09-19T06:50:50.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 677695 7.75 MB/sec execute 317 sec latency 58.945 ms 2024-09-19T06:50:50.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:49 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:50:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:49 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:49 smithi096 ceph-mon[35004]: pgmap v282: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.7 MiB/s wr, 523 op/s 2024-09-19T06:50:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:49 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:50:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:49 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:50:50.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:49 smithi151 ceph-mon[35236]: pgmap v282: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.1 MiB/s rd, 4.7 MiB/s wr, 523 op/s 2024-09-19T06:50:50.914 INFO:tasks.cephfs.filesystem:scrub status for tag:9e64056d-43a5-4bb8-904a-ce002e6a32ce - {'path': '/', 'tag': '9e64056d-43a5-4bb8-904a-ce002e6a32ce', 'options': 'recursive,force'} 2024-09-19T06:50:51.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 679958 7.76 MB/sec execute 318 sec latency 20.405 ms 2024-09-19T06:50:52.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 682568 7.78 MB/sec execute 319 sec latency 13.854 ms 2024-09-19T06:50:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:51 smithi096 ceph-mon[35004]: pgmap v283: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-09-19T06:50:52.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:51 smithi037 ceph-mon[31211]: pgmap v283: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-09-19T06:50:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:51 smithi151 ceph-mon[35236]: pgmap v283: 97 pgs: 97 active+clean; 74 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 3.4 MiB/s wr, 326 op/s 2024-09-19T06:50:53.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 685540 7.81 MB/sec execute 320 sec latency 14.045 ms 2024-09-19T06:50:54.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 687824 7.82 MB/sec execute 321 sec latency 28.801 ms 2024-09-19T06:50:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:53 smithi096 ceph-mon[35004]: pgmap v284: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 447 op/s 2024-09-19T06:50:54.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:53 smithi037 ceph-mon[31211]: pgmap v284: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 447 op/s 2024-09-19T06:50:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:53 smithi151 ceph-mon[35236]: pgmap v284: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 5.2 MiB/s wr, 447 op/s 2024-09-19T06:50:55.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 689738 7.83 MB/sec execute 322 sec latency 101.013 ms 2024-09-19T06:50:56.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 692571 7.86 MB/sec execute 323 sec latency 19.439 ms 2024-09-19T06:50:56.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:55 smithi096 ceph-mon[35004]: pgmap v285: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-09-19T06:50:56.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:55 smithi037 ceph-mon[31211]: pgmap v285: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-09-19T06:50:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:55 smithi151 ceph-mon[35236]: pgmap v285: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-09-19T06:50:57.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 694458 7.86 MB/sec execute 324 sec latency 16.698 ms 2024-09-19T06:50:57.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:56 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:50:57.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:56 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:50:57.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:56 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:50:58.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 696841 7.88 MB/sec execute 325 sec latency 13.257 ms 2024-09-19T06:50:58.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:57 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:50:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:57 smithi096 ceph-mon[35004]: pgmap v286: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-09-19T06:50:58.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:57 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:50:58.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:57 smithi037 ceph-mon[31211]: pgmap v286: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-09-19T06:50:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:57 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:50:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:57 smithi151 ceph-mon[35236]: pgmap v286: 97 pgs: 97 active+clean; 83 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 3.6 MiB/s wr, 341 op/s 2024-09-19T06:50:59.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 699690 7.90 MB/sec execute 326 sec latency 13.004 ms 2024-09-19T06:51:00.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 701232 7.90 MB/sec execute 327 sec latency 55.326 ms 2024-09-19T06:51:00.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.263+0000 7fa3b034b640 1 -- 172.21.15.37:0/4183998967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 msgr2=0x7fa3a810d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.263+0000 7fa3b034b640 1 --2- 172.21.15.37:0/4183998967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a810d880 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0009930 tx=0x7fa3a002a420 comp rx=0 tx=0).stop 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.263+0000 7fa3b034b640 1 -- 172.21.15.37:0/4183998967 shutdown_connections 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 --2- 172.21.15.37:0/4183998967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a810d880 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 --2- 172.21.15.37:0/4183998967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3a81009b0 0x7fa3a8100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 --2- 172.21.15.37:0/4183998967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3a81069c0 0x7fa3a8106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 -- 172.21.15.37:0/4183998967 >> 172.21.15.37:0/4183998967 conn(0x7fa3a8077990 msgr2=0x7fa3a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 -- 172.21.15.37:0/4183998967 shutdown_connections 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 -- 172.21.15.37:0/4183998967 wait complete. 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 Processor -- start 2024-09-19T06:51:00.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 -- start start 2024-09-19T06:51:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3a81009b0 0x7fa3a8199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a81996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3a81069c0 0x7fa3a81a41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8110e00 con 0x7fa3a8101350 2024-09-19T06:51:00.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8110c80 con 0x7fa3a81069c0 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.264+0000 7fa3b034b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a8110f80 con 0x7fa3a81009b0 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a81996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ae0e7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3a81009b0 0x7fa3a8199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a81996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60044/0 (socket says 172.21.15.37:60044) 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2131272756 learned_addr learned my addr 172.21.15.37:0/2131272756 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3a81069c0 0x7fa3a81a41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2131272756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3a81009b0 msgr2=0x7fa3a8199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3a81009b0 0x7fa3a8199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2131272756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3a81069c0 msgr2=0x7fa3a81a41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3a81069c0 0x7fa3a81a41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2131272756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3a81a4730 con 0x7fa3a8101350 2024-09-19T06:51:00.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a81996b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9f0 tx=0x7fa39c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3a77fe640 1 -- 172.21.15.37:0/2131272756 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c00cd80 con 0x7fa3a8101350 2024-09-19T06:51:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3a81a4900 con 0x7fa3a8101350 2024-09-19T06:51:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3a77fe640 1 -- 172.21.15.37:0/2131272756 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa39c010910 con 0x7fa3a8101350 2024-09-19T06:51:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3a77fe640 1 -- 172.21.15.37:0/2131272756 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c018990 con 0x7fa3a8101350 2024-09-19T06:51:00.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.265+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3a81a4e70 con 0x7fa3a8101350 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3a77fe640 1 -- 172.21.15.37:0/2131272756 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa39c010ac0 con 0x7fa3a8101350 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 Processor -- start 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 -- start start 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805050ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x55580505b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55580505bcb0 0x55580505e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555804fb52f0 con 0x555805051410 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555804f5d270 con 0x555805050af0 2024-09-19T06:51:00.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x555804fb12b0 con 0x55580505bcb0 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3ae0e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x55580505b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3ae8e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805050ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3ae0e7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x55580505b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60048/0 (socket says 172.21.15.37:60048) 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3ae0e7640 1 -- 172.21.15.37:0/498146660 learned_addr learned my addr 172.21.15.37:0/498146660 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.269+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55580505bcb0 0x55580505e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae0e7640 1 -- 172.21.15.37:0/498146660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55580505bcb0 msgr2=0x55580505e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55580505bcb0 0x55580505e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae0e7640 1 -- 172.21.15.37:0/498146660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 msgr2=0x555805050ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805050ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae0e7640 1 -- 172.21.15.37:0/498146660 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555804f5c340 con 0x555805051410 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805050ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:00.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x55580505b770 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa398010370 tx=0x7fa398010830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf005aede4a4e3e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3a77fe640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3900a34e0 0x7fa3900a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3a4bfe640 1 -- 172.21.15.37:0/498146660 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa398016070 con 0x555805051410 2024-09-19T06:51:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3900a34e0 0x7fa3900a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3a77fe640 1 -- 172.21.15.37:0/2131272756 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fa39c0cffc0 con 0x7fa3a8101350 2024-09-19T06:51:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3a4bfe640 1 -- 172.21.15.37:0/498146660 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398004d20 con 0x555805051410 2024-09-19T06:51:00.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.270+0000 7fa3a4bfe640 1 -- 172.21.15.37:0/498146660 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3980054c0 con 0x555805051410 2024-09-19T06:51:00.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:50:59 smithi096 ceph-mon[35004]: pgmap v287: 97 pgs: 97 active+clean; 90 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 453 op/s 2024-09-19T06:51:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 -- 172.21.15.37:0/498146660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 msgr2=0x55580505b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3900a34e0 0x7fa3900a59a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006010 tx=0x7fa3a0005d70 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x55580505b770 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7fa398010370 tx=0x7fa398010830 comp rx=0 tx=0).stop 2024-09-19T06:51:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 -- 172.21.15.37:0/498146660 shutdown_connections 2024-09-19T06:51:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55580505bcb0 0x55580505e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x55580505b770 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/498146660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805050ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 -- 172.21.15.37:0/498146660 >> 172.21.15.37:0/498146660 conn(0x55580504d060 msgr2=0x55580504d440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 -- 172.21.15.37:0/498146660 shutdown_connections 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 -- 172.21.15.37:0/498146660 wait complete. 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.271+0000 7fa3b31c5740 1 -- start start 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 1 Processor -- start 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805058b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x555805059050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55580505bcb0 0x555805059590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55580505e880 con 0x555805051410 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555804f3e290 con 0x55580505bcb0 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55580505ea00 con 0x555805050af0 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3b31c5740 10 client.0 _ll_register_callbacks cb 0x555804f6f140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:00.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x555805059050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x555805059050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60064/0 (socket says 172.21.15.37:60064) 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2567531104 learned_addr learned my addr 172.21.15.37:0/2567531104 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55580505bcb0 0x555805059590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805058b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2567531104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 msgr2=0x555805058b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805058b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55580505bcb0 msgr2=0x555805059590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55580505bcb0 0x555805059590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 -- 172.21.15.37:0/2567531104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555804cc8210 con 0x555805051410 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x555805059050 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9c0 tx=0x7fa39c0d3700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbf8cc9726ca09f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.272+0000 7fa38dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa38dffb640 1 -- 172.21.15.37:0/2567531104 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa39c09bc80 con 0x555805051410 2024-09-19T06:51:00.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa38dffb640 1 -- 172.21.15.37:0/2567531104 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fa39c0d2110 con 0x555805051410 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa398004d20 con 0x555805051410 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa38dffb640 1 -- 172.21.15.37:0/2567531104 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa39c09c320 con 0x555805051410 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 10 client.18036 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x555804d94620 con 0x555805051410 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa38dffb640 1 -- 172.21.15.37:0/2567531104 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa398004d20 con 0x555805051410 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa38dffb640 1 -- 172.21.15.37:0/2567531104 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa39c0bc170 con 0x555805051410 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 10 client.18036 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 10 client.18036 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 20 client.18036 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55580506bb20 0x55580506df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 4 client.18036 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555804fc2a20 con 0x55580506bb20 2024-09-19T06:51:00.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.273+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55580506bb20 0x55580506df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.274+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55580506bb20 0x55580506df10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7b807c1472d0c13 server_cookie=660eb6af87be4e29 in_seq=0 out_seq=0 2024-09-19T06:51:00.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.274+0000 7fa38dffb640 10 client.18036 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:00.281 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:50:59 smithi037 ceph-mon[31211]: pgmap v287: 97 pgs: 97 active+clean; 90 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 453 op/s 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.274+0000 7fa38dffb640 1 -- 172.21.15.37:0/2567531104 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555804fc2a20 con 0x55580506bb20 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.274+0000 7fa38dffb640 10 client.18036 handle_command_reply: tid=0 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.274+0000 7fa3b31c5740 1 client.18036 shutdown 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.274+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 msgr2=0x555805059050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.274+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x555805059050 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9c0 tx=0x7fa39c0d3700 comp rx=0 tx=0).stop 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 shutdown_connections 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55580506bb20 0x55580506df10 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55580505bcb0 0x555805059590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x555805051410 0x555805059050 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa38dffb640 0 client.18036 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/2567531104 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805058b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.282 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 >> 172.21.15.37:0/2567531104 conn(0x55580504d060 msgr2=0x555805052dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 shutdown_connections 2024-09-19T06:51:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 1 -- 172.21.15.37:0/2567531104 wait complete. 2024-09-19T06:51:00.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.275+0000 7fa3b31c5740 20 client.18036 trim_cache size 0 max 16384 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.290+0000 7fa3b31c5740 1 Processor -- start 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.290+0000 7fa3b31c5740 1 -- start start 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805053560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805051410 0x5558050593c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x5558050737e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81b0210 con 0x55580505bcb0 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81adae0 con 0x555805050af0 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81ab950 con 0x555805051410 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x5558050737e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x5558050737e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60078/0 (socket says 172.21.15.37:60078) 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/131572202 learned_addr learned my addr 172.21.15.37:0/131572202 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805053560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805051410 0x5558050593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/131572202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805051410 msgr2=0x5558050593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805051410 0x5558050593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/131572202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 msgr2=0x555805053560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805053560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/131572202 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55580504ce90 con 0x55580505bcb0 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x5558050737e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0033000 tx=0x7fa3a003f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33093259f56d53aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.291+0000 7fa38ffff640 1 -- 172.21.15.37:0/131572202 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0035040 con 0x55580505bcb0 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa38ffff640 1 -- 172.21.15.37:0/131572202 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a004f070 con 0x55580505bcb0 2024-09-19T06:51:00.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa38ffff640 1 -- 172.21.15.37:0/131572202 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a003de40 con 0x55580505bcb0 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 -- 172.21.15.37:0/131572202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 msgr2=0x5558050737e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x5558050737e0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0033000 tx=0x7fa3a003f000 comp rx=0 tx=0).stop 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 -- 172.21.15.37:0/131572202 shutdown_connections 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x5558050737e0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805051410 0x5558050593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/131572202 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805050af0 0x555805053560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 -- 172.21.15.37:0/131572202 >> 172.21.15.37:0/131572202 conn(0x55580504d060 msgr2=0x55580505ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 -- 172.21.15.37:0/131572202 shutdown_connections 2024-09-19T06:51:00.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.292+0000 7fa3b31c5740 1 -- 172.21.15.37:0/131572202 wait complete. 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 -- start start 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 Processor -- start 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805064560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805051410 0x555805055930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x555805056070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81abfa0 con 0x55580505bcb0 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81ad410 con 0x555805051410 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa3a81ac670 con 0x555805050af0 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3b31c5740 10 client.0 _ll_register_callbacks cb 0x7fa39c0d3800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x555805056070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x555805056070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60090/0 (socket says 172.21.15.37:60090) 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/1134014774 learned_addr learned my addr 172.21.15.37:0/1134014774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:00.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805064560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805051410 0x555805055930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/1134014774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 msgr2=0x555805064560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805064560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805051410 msgr2=0x555805055930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805051410 0x555805055930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.293+0000 7fa3ae8e8640 1 -- 172.21.15.37:0/1134014774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55580504dcf0 con 0x55580505bcb0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa3ad8e6640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805051410 0x555805055930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa3ae8e8640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x555805056070 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0031ba0 tx=0x7fa3a0009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f93be8e3ee9e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa38dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa38dffb640 1 -- 172.21.15.37:0/1134014774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3a0062060 con 0x55580505bcb0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa38dffb640 1 -- 172.21.15.37:0/1134014774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 8571+0+0 (secure 0 0 0) 0x7fa3a003dbc0 con 0x55580505bcb0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3a004f070 con 0x55580505bcb0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa38dffb640 1 -- 172.21.15.37:0/1134014774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa3a003ddc0 con 0x55580505bcb0 2024-09-19T06:51:00.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa3b31c5740 10 client.18042 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5558050d6730 con 0x55580505bcb0 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa38dffb640 1 -- 172.21.15.37:0/1134014774 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa3a004f070 con 0x55580505bcb0 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa38dffb640 1 -- 172.21.15.37:0/1134014774 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa3a0054880 con 0x55580505bcb0 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.294+0000 7fa3b31c5740 10 client.18042 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa3b31c5740 10 client.18042 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa3b31c5740 20 client.18042 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558050d6da0 0x5558050d9190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa3b31c5740 4 client.18042 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555804fbcd70 con 0x5558050d6da0 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558050d6da0 0x5558050d9190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa3ae0e7640 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558050d6da0 0x5558050d9190 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14dc7b9ddafdecb9 server_cookie=4b93f0bf8cf6e9fa in_seq=0 out_seq=0 2024-09-19T06:51:00.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.295+0000 7fa38dffb640 10 client.18042 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:00.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:50:59 smithi151 ceph-mon[35236]: pgmap v287: 97 pgs: 97 active+clean; 90 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 5.4 MiB/s wr, 453 op/s 2024-09-19T06:51:00.454 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.453+0000 7fa38dffb640 1 -- 172.21.15.37:0/1134014774 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555804fbcd70 con 0x5558050d6da0 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.453+0000 7fa38dffb640 10 client.18042 handle_command_reply: tid=0 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.453+0000 7fa3b31c5740 1 client.18042 shutdown 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.453+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 msgr2=0x555805056070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.453+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x555805056070 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0031ba0 tx=0x7fa3a0009290 comp rx=0 tx=0).stop 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 shutdown_connections 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558050d6da0 0x5558050d9190 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55580505bcb0 0x555805056070 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa38dffb640 0 client.18042 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555805051410 0x555805055930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 --2- 172.21.15.37:0/1134014774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x555805050af0 0x555805064560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.455 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 >> 172.21.15.37:0/1134014774 conn(0x55580504d060 msgr2=0x55580504b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 shutdown_connections 2024-09-19T06:51:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 1 -- 172.21.15.37:0/1134014774 wait complete. 2024-09-19T06:51:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.454+0000 7fa3b31c5740 20 client.18042 trim_cache size 0 max 16384 2024-09-19T06:51:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:51:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:51:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:51:00.456 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3900a34e0 msgr2=0x7fa3900a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3900a34e0 0x7fa3900a59a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa3a0006010 tx=0x7fa3a0005d70 comp rx=0 tx=0).stop 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 msgr2=0x7fa3a81996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a81996b0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7fa39c00e9f0 tx=0x7fa39c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 shutdown_connections 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa3a81069c0 0x7fa3a81a41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa3900a34e0 0x7fa3900a59a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa3a8101350 0x7fa3a81996b0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.457 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 --2- 172.21.15.37:0/2131272756 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa3a81009b0 0x7fa3a8199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 >> 172.21.15.37:0/2131272756 conn(0x7fa3a8077990 msgr2=0x7fa3a8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 shutdown_connections 2024-09-19T06:51:00.458 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:00.455+0000 7fa3b034b640 1 -- 172.21.15.37:0/2131272756 wait complete. 2024-09-19T06:51:01.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 702877 7.90 MB/sec execute 328 sec latency 83.650 ms 2024-09-19T06:51:02.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 704540 7.91 MB/sec execute 329 sec latency 24.797 ms 2024-09-19T06:51:02.533 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:01 smithi151 ceph-mon[35236]: pgmap v288: 97 pgs: 97 active+clean; 90 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-19T06:51:02.534 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:01 smithi096 ceph-mon[35004]: pgmap v288: 97 pgs: 97 active+clean; 90 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-19T06:51:02.534 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:01 smithi037 ceph-mon[31211]: pgmap v288: 97 pgs: 97 active+clean; 90 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.6 MiB/s wr, 233 op/s 2024-09-19T06:51:03.221 INFO:tasks.workunit.client.0.smithi037.stdout: 1 705737 7.89 MB/sec execute 330 sec latency 33.983 ms 2024-09-19T06:51:03.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:02 smithi096 ceph-mon[35004]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T06:51:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:02 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:51:03.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:02 smithi037 ceph-mon[31211]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T06:51:03.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:02 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:51:03.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:02 smithi151 ceph-mon[35236]: osdmap e93: 12 total, 12 up, 12 in 2024-09-19T06:51:03.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:02 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:51:04.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 708083 7.92 MB/sec execute 331 sec latency 12.993 ms 2024-09-19T06:51:04.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:03 smithi096 ceph-mon[35004]: pgmap v290: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:03 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:51:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:03 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:51:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:03 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:51:04.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:03 smithi037 ceph-mon[31211]: pgmap v290: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:04.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:03 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:51:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:03 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:51:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:03 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:51:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:03 smithi151 ceph-mon[35236]: pgmap v290: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:03 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:51:04.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:03 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:51:04.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:03 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:51:05.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 710578 7.94 MB/sec execute 332 sec latency 50.328 ms 2024-09-19T06:51:06.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 712825 7.94 MB/sec execute 333 sec latency 23.238 ms 2024-09-19T06:51:06.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:05 smithi096 ceph-mon[35004]: pgmap v291: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:06.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:05 smithi037 ceph-mon[31211]: pgmap v291: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:05 smithi151 ceph-mon[35236]: pgmap v291: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:07.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 714450 7.95 MB/sec execute 334 sec latency 26.168 ms 2024-09-19T06:51:08.224 INFO:tasks.workunit.client.0.smithi037.stdout: 1 717130 7.97 MB/sec execute 335 sec latency 15.457 ms 2024-09-19T06:51:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:07 smithi096 ceph-mon[35004]: pgmap v292: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:08.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:07 smithi037 ceph-mon[31211]: pgmap v292: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:07 smithi151 ceph-mon[35236]: pgmap v292: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 72 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 4.2 MiB/s wr, 420 op/s 2024-09-19T06:51:09.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 718287 7.96 MB/sec execute 336 sec latency 18.961 ms 2024-09-19T06:51:10.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 720867 7.99 MB/sec execute 337 sec latency 108.198 ms 2024-09-19T06:51:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:09 smithi096 ceph-mon[35004]: pgmap v293: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 408 op/s 2024-09-19T06:51:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:09 smithi096 ceph-mon[35004]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T06:51:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:09 smithi037 ceph-mon[31211]: pgmap v293: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 408 op/s 2024-09-19T06:51:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:09 smithi037 ceph-mon[31211]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T06:51:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:09 smithi151 ceph-mon[35236]: pgmap v293: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 408 op/s 2024-09-19T06:51:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:09 smithi151 ceph-mon[35236]: osdmap e94: 12 total, 12 up, 12 in 2024-09-19T06:51:11.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 722365 7.98 MB/sec execute 338 sec latency 18.303 ms 2024-09-19T06:51:12.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 724663 8.00 MB/sec execute 339 sec latency 15.062 ms 2024-09-19T06:51:12.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:11 smithi096 ceph-mon[35004]: pgmap v295: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 463 op/s 2024-09-19T06:51:12.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:11 smithi037 ceph-mon[31211]: pgmap v295: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 463 op/s 2024-09-19T06:51:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:11 smithi151 ceph-mon[35236]: pgmap v295: 97 pgs: 97 active+clean; 80 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.7 MiB/s wr, 463 op/s 2024-09-19T06:51:13.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 727178 8.01 MB/sec execute 340 sec latency 17.425 ms 2024-09-19T06:51:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 -- 172.21.15.37:0/3441209078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8106a00 msgr2=0x7f77b8106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 --2- 172.21.15.37:0/3441209078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8106a00 0x7f77b8106de0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009930 tx=0x7f77a802a420 comp rx=0 tx=0).stop 2024-09-19T06:51:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 -- 172.21.15.37:0/3441209078 shutdown_connections 2024-09-19T06:51:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 --2- 172.21.15.37:0/3441209078 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b8101390 0x7f77b810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.548 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 --2- 172.21.15.37:0/3441209078 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77b81009f0 0x7f77b8100e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 --2- 172.21.15.37:0/3441209078 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8106a00 0x7f77b8106de0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 -- 172.21.15.37:0/3441209078 >> 172.21.15.37:0/3441209078 conn(0x7f77b80fc6b0 msgr2=0x7f77b80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 -- 172.21.15.37:0/3441209078 shutdown_connections 2024-09-19T06:51:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.546+0000 7f77be02e640 1 -- 172.21.15.37:0/3441209078 wait complete. 2024-09-19T06:51:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 Processor -- start 2024-09-19T06:51:13.549 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 -- start start 2024-09-19T06:51:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b81009f0 0x7f77b81972b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8101390 0x7f77b81977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77b8106a00 0x7f77b819bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8110e40 con 0x7f77b8101390 2024-09-19T06:51:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8110cc0 con 0x7f77b8106a00 2024-09-19T06:51:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77be02e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77b8110fc0 con 0x7f77b81009f0 2024-09-19T06:51:13.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8101390 0x7f77b81977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8101390 0x7f77b81977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46098/0 (socket says 172.21.15.37:46098) 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77b6ffd640 1 -- 172.21.15.37:0/3460568082 learned_addr learned my addr 172.21.15.37:0/3460568082 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77b8106a00 0x7f77b819bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77b77fe640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b81009f0 0x7f77b81972b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77b6ffd640 1 -- 172.21.15.37:0/3460568082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b81009f0 msgr2=0x7f77b81972b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.547+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b81009f0 0x7f77b81972b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b6ffd640 1 -- 172.21.15.37:0/3460568082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77b8106a00 msgr2=0x7f77b819bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77b8106a00 0x7f77b819bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b6ffd640 1 -- 172.21.15.37:0/3460568082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77b819c2a0 con 0x7f77b8101390 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77b8106a00 0x7f77b819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8101390 0x7f77b81977f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00eb70 tx=0x7f77ac00b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:13.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b4ff9640 1 -- 172.21.15.37:0/3460568082 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac00be60 con 0x7f77b8101390 2024-09-19T06:51:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77b819c590 con 0x7f77b8101390 2024-09-19T06:51:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b4ff9640 1 -- 172.21.15.37:0/3460568082 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77ac0035f0 con 0x7f77b8101390 2024-09-19T06:51:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77b4ff9640 1 -- 172.21.15.37:0/3460568082 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77ac0146e0 con 0x7f77b8101390 2024-09-19T06:51:13.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.548+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77b806a300 con 0x7f77b8101390 2024-09-19T06:51:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.550+0000 7f77b4ff9640 1 -- 172.21.15.37:0/3460568082 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f77ac0037a0 con 0x7f77b8101390 2024-09-19T06:51:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 Processor -- start 2024-09-19T06:51:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 -- start start 2024-09-19T06:51:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f09af0 0x55e6f2f09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.559 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f0a410 0x55e6f2f14770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f170a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6f2e6e2f0 con 0x55e6f2f0a410 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6f2e16270 con 0x55e6f2f09af0 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e6f2e6a2b0 con 0x55e6f2f14cb0 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f0a410 0x55e6f2f14770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b4ff9640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77a00a3460 0x7f77a00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f0a410 0x55e6f2f14770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46102/0 (socket says 172.21.15.37:46102) 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 -- 172.21.15.37:0/2228831391 learned_addr learned my addr 172.21.15.37:0/2228831391 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b7fff640 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f09af0 0x55e6f2f09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77a00a3460 0x7f77a00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f170a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.560 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b4ff9640 1 -- 172.21.15.37:0/3460568082 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f77ac0d17e0 con 0x7f77b8101390 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 -- 172.21.15.37:0/2228831391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 msgr2=0x55e6f2f170a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f170a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 -- 172.21.15.37:0/2228831391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f09af0 msgr2=0x55e6f2f09ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f09af0 0x55e6f2f09ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b77fe640 1 -- 172.21.15.37:0/2228831391 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f2e15340 con 0x55e6f2f0a410 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b7fff640 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f09af0 0x55e6f2f09ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.551+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.552+0000 7f77b77fe640 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f0a410 0x55e6f2f14770 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009930 tx=0x7f77a8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb09bfcc8df9e3a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.552+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77a00a3460 0x7f77a00a5920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77b000afd0 tx=0x7f77b000a4d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.552+0000 7f779dffb640 1 -- 172.21.15.37:0/2228831391 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8040070 con 0x55e6f2f0a410 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.552+0000 7f779dffb640 1 -- 172.21.15.37:0/2228831391 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a80319f0 con 0x55e6f2f0a410 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.552+0000 7f779dffb640 1 -- 172.21.15.37:0/2228831391 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8035040 con 0x55e6f2f0a410 2024-09-19T06:51:13.561 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2228831391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f0a410 msgr2=0x55e6f2f14770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f0a410 0x55e6f2f14770 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f77a8009930 tx=0x7f77a8005e80 comp rx=0 tx=0).stop 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2228831391 shutdown_connections 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f170a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f0a410 0x55e6f2f14770 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2228831391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f09af0 0x55e6f2f09ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2228831391 >> 172.21.15.37:0/2228831391 conn(0x55e6f2f06060 msgr2=0x55e6f2f06440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2228831391 shutdown_connections 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2228831391 wait complete. 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 -- start start 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.553+0000 7f77c0dc9740 1 Processor -- start 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f11b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f12060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.562 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e6f2f17880 con 0x55e6f2f09af0 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6f2df7290 con 0x55e6f2f0a410 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e6f2f17a00 con 0x55e6f2f14cb0 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 10 client.0 _ll_register_callbacks cb 0x55e6f2e28140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f11b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f11b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46104/0 (socket says 172.21.15.37:46104) 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 -- 172.21.15.37:0/2154571498 learned_addr learned my addr 172.21.15.37:0/2154571498 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f12060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b7fff640 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 -- 172.21.15.37:0/2154571498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 msgr2=0x55e6f2f125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 -- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 msgr2=0x55e6f2f12060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f12060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 -- 172.21.15.37:0/2154571498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f2b81210 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77b77fe640 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f11b20 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f77a802a940 tx=0x7f77a80052d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2cc8df8bbcaee1b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f7796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f7796ffd640 1 -- 172.21.15.37:0/2154571498 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77a8040030 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f7796ffd640 1 -- 172.21.15.37:0/2154571498 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f77a8004260 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.554+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77a80319f0 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f7796ffd640 1 -- 172.21.15.37:0/2154571498 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f77a8056050 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 10 client.18054 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e6f2c4d620 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f7796ffd640 1 -- 172.21.15.37:0/2154571498 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77a80319f0 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f7796ffd640 1 -- 172.21.15.37:0/2154571498 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f77a8003850 con 0x55e6f2f09af0 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 10 client.18054 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 10 client.18054 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 20 client.18054 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:13.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f24980 0x55e6f2f26d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 4 client.18054 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6f2e7ba20 con 0x55e6f2f24980 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.555+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f24980 0x55e6f2f26d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.556+0000 7f7796ffd640 10 client.18054 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.556+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f24980 0x55e6f2f26d70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b754cb19628b647 server_cookie=5996fe2529b409d6 in_seq=0 out_seq=0 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.556+0000 7f7796ffd640 1 -- 172.21.15.37:0/2154571498 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e6f2e7ba20 con 0x55e6f2f24980 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.556+0000 7f7796ffd640 10 client.18054 handle_command_reply: tid=0 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.556+0000 7f77c0dc9740 1 client.18054 shutdown 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.556+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 msgr2=0x55e6f2f11b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.556+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f11b20 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f77a802a940 tx=0x7f77a80052d0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 shutdown_connections 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f24980 0x55e6f2f26d70 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f12060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/2154571498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f11b20 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 >> 172.21.15.37:0/2154571498 conn(0x55e6f2f06060 msgr2=0x55e6f2f0bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f7796ffd640 0 client.18054 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 shutdown_connections 2024-09-19T06:51:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 1 -- 172.21.15.37:0/2154571498 wait complete. 2024-09-19T06:51:13.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.557+0000 7f77c0dc9740 20 client.18054 trim_cache size 0 max 16384 2024-09-19T06:51:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.571+0000 7f77c0dc9740 1 Processor -- start 2024-09-19T06:51:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.571+0000 7f77c0dc9740 1 -- start start 2024-09-19T06:51:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f09af0 0x55e6f2f123d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f0ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f0f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.574 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77b81adfc0 con 0x55e6f2f14cb0 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77b81ab890 con 0x55e6f2f0a410 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77b81a9700 con 0x55e6f2f09af0 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f0f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f0ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f0f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46120/0 (socket says 172.21.15.37:46120) 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f0ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39364/0 (socket says 172.21.15.37:39364) 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 -- 172.21.15.37:0/3717132289 learned_addr learned my addr 172.21.15.37:0/3717132289 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b77fe640 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f09af0 0x55e6f2f123d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 -- 172.21.15.37:0/3717132289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f09af0 msgr2=0x55e6f2f123d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f09af0 0x55e6f2f123d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 -- 172.21.15.37:0/3717132289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 msgr2=0x55e6f2f0ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f0ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.575 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 -- 172.21.15.37:0/3717132289 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6f2f05e90 con 0x55e6f2f14cb0 2024-09-19T06:51:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f0f2a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f77b0013ed0 tx=0x7f77b000ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb41578870c87b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f779d7fa640 1 -- 172.21.15.37:0/3717132289 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b000fe60 con 0x55e6f2f14cb0 2024-09-19T06:51:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.572+0000 7f779d7fa640 1 -- 172.21.15.37:0/3717132289 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b0003970 con 0x55e6f2f14cb0 2024-09-19T06:51:13.576 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f779d7fa640 1 -- 172.21.15.37:0/3717132289 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b00045d0 con 0x55e6f2f14cb0 2024-09-19T06:51:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3717132289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f14cb0 msgr2=0x55e6f2f0f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f0f2a0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f77b0013ed0 tx=0x7f77b000ac80 comp rx=0 tx=0).stop 2024-09-19T06:51:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3717132289 shutdown_connections 2024-09-19T06:51:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f0f2a0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.579 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f0a410 0x55e6f2f0ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3717132289 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f09af0 0x55e6f2f123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3717132289 >> 172.21.15.37:0/3717132289 conn(0x55e6f2f06060 msgr2=0x55e6f2f13ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3717132289 shutdown_connections 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.573+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3717132289 wait complete. 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 -- start start 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 Processor -- start 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f25b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f0a410 0x55e6f2f26050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f26590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f77b81a9d50 con 0x55e6f2f09af0 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77b81ab1c0 con 0x55e6f2f14cb0 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f77b81aa420 con 0x55e6f2f0a410 2024-09-19T06:51:13.580 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77c0dc9740 10 client.0 _ll_register_callbacks cb 0x7f77b000ad30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f25b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.574+0000 7f77b7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f26590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f25b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46132/0 (socket says 172.21.15.37:46132) 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b77fe640 1 -- 172.21.15.37:0/3445831642 learned_addr learned my addr 172.21.15.37:0/3445831642 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b6ffd640 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f0a410 0x55e6f2f26050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b7fff640 1 -- 172.21.15.37:0/3445831642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f0a410 msgr2=0x55e6f2f26050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f0a410 0x55e6f2f26050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b7fff640 1 -- 172.21.15.37:0/3445831642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 msgr2=0x55e6f2f25b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f25b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b7fff640 1 -- 172.21.15.37:0/3445831642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6f2f8e2e0 con 0x55e6f2f14cb0 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77b7fff640 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f26590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77b0013ea0 tx=0x7f77b0011ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6099fd17b907daa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f7796ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:51:13.581 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f7796ffd640 1 -- 172.21.15.37:0/3445831642 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b000fe60 con 0x55e6f2f14cb0 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f7796ffd640 1 -- 172.21.15.37:0/3445831642 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f77b0004c50 con 0x55e6f2f14cb0 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.575+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b0003970 con 0x55e6f2f14cb0 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f7796ffd640 1 -- 172.21.15.37:0/3445831642 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f77b0004270 con 0x55e6f2f14cb0 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 10 client.27401 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e6f2f8f5b0 con 0x55e6f2f14cb0 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f7796ffd640 1 -- 172.21.15.37:0/3445831642 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f77b0003970 con 0x55e6f2f14cb0 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f7796ffd640 1 -- 172.21.15.37:0/3445831642 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f77b001d040 con 0x55e6f2f14cb0 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 10 client.27401 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 10 client.27401 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 20 client.27401 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f8fcd0 0x55e6f2f920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 4 client.27401 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:13.582 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6f2e75d70 con 0x55e6f2f8fcd0 2024-09-19T06:51:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f77b77fe640 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f8fcd0 0x55e6f2f920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.576+0000 7f7796ffd640 10 client.27401 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:13.583 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.577+0000 7f77b77fe640 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f8fcd0 0x55e6f2f920f0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e561ebbf69fc92c server_cookie=25ae46f292af438c in_seq=0 out_seq=0 2024-09-19T06:51:13.733 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f7796ffd640 1 -- 172.21.15.37:0/3445831642 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6f2e75d70 con 0x55e6f2f8fcd0 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f7796ffd640 10 client.27401 handle_command_reply: tid=0 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 client.27401 shutdown 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f14cb0 msgr2=0x55e6f2f26590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f26590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f77b0013ea0 tx=0x7f77b0011ef0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 shutdown_connections 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e6f2f8fcd0 0x55e6f2f920f0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6f2f14cb0 0x55e6f2f26590 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e6f2f0a410 0x55e6f2f26050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 --2- 172.21.15.37:0/3445831642 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e6f2f09af0 0x55e6f2f25b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 >> 172.21.15.37:0/3445831642 conn(0x55e6f2f06060 msgr2=0x55e6f2f049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f7796ffd640 0 client.27401 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:13.734 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 shutdown_connections 2024-09-19T06:51:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.731+0000 7f77c0dc9740 1 -- 172.21.15.37:0/3445831642 wait complete. 2024-09-19T06:51:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.732+0000 7f77c0dc9740 20 client.27401 trim_cache size 0 max 16384 2024-09-19T06:51:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:51:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:51:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:51:13.735 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:51:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.732+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77a00a3460 msgr2=0x7f77a00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.732+0000 7f77be02e640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77a00a3460 0x7f77a00a5920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f77b000afd0 tx=0x7f77b000a4d0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.732+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8101390 msgr2=0x7f77b81977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.732+0000 7f77be02e640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8101390 0x7f77b81977f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f77ac00eb70 tx=0x7f77ac00b690 comp rx=0 tx=0).stop 2024-09-19T06:51:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 shutdown_connections 2024-09-19T06:51:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77b8106a00 0x7f77b819bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.736 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f77a00a3460 0x7f77a00a5920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f77b8101390 0x7f77b81977f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 --2- 172.21.15.37:0/3460568082 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f77b81009f0 0x7f77b81972b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 >> 172.21.15.37:0/3460568082 conn(0x7f77b80fc6b0 msgr2=0x7f77b80fcda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 shutdown_connections 2024-09-19T06:51:13.737 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:13.733+0000 7f77be02e640 1 -- 172.21.15.37:0/3460568082 wait complete. 2024-09-19T06:51:14.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 728540 8.01 MB/sec execute 341 sec latency 58.887 ms 2024-09-19T06:51:14.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:13 smithi096 ceph-mon[35004]: pgmap v296: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:14.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:13 smithi037 ceph-mon[31211]: pgmap v296: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:13 smithi151 ceph-mon[35236]: pgmap v296: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:15.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 731219 8.03 MB/sec execute 342 sec latency 41.792 ms 2024-09-19T06:51:16.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 732474 8.02 MB/sec execute 343 sec latency 31.870 ms 2024-09-19T06:51:16.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:15 smithi096 ceph-mon[35004]: pgmap v297: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:16.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:15 smithi037 ceph-mon[31211]: pgmap v297: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:15 smithi151 ceph-mon[35236]: pgmap v297: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:17.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 735147 8.05 MB/sec execute 344 sec latency 12.695 ms 2024-09-19T06:51:18.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 737177 8.04 MB/sec execute 345 sec latency 17.129 ms 2024-09-19T06:51:18.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:17 smithi096 ceph-mon[35004]: pgmap v298: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:18.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:17 smithi037 ceph-mon[31211]: pgmap v298: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:18.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:17 smithi151 ceph-mon[35236]: pgmap v298: 97 pgs: 97 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.1 MiB/s wr, 240 op/s 2024-09-19T06:51:19.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 739040 8.06 MB/sec execute 346 sec latency 52.953 ms 2024-09-19T06:51:20.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 741933 8.08 MB/sec execute 347 sec latency 18.177 ms 2024-09-19T06:51:20.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:19 smithi096 ceph-mon[35004]: pgmap v299: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 341 op/s 2024-09-19T06:51:20.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:19 smithi037 ceph-mon[31211]: pgmap v299: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 341 op/s 2024-09-19T06:51:20.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:19 smithi151 ceph-mon[35236]: pgmap v299: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.8 MiB/s wr, 341 op/s 2024-09-19T06:51:20.915 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:51:21.190 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:21.222 INFO:tasks.workunit.client.0.smithi037.stdout: 1 742805 8.07 MB/sec execute 348 sec latency 80.272 ms 2024-09-19T06:51:21.690 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 -- 172.21.15.37:0/2425481293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac101de0 msgr2=0x7f87ac1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 --2- 172.21.15.37:0/2425481293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac101de0 0x7f87ac1021c0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009900 tx=0x7f87a802a410 comp rx=0 tx=0).stop 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 -- 172.21.15.37:0/2425481293 shutdown_connections 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 --2- 172.21.15.37:0/2425481293 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87ac106f10 0x7f87ac10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 --2- 172.21.15.37:0/2425481293 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87ac102790 0x7f87ac1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 --2- 172.21.15.37:0/2425481293 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac101de0 0x7f87ac1021c0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 -- 172.21.15.37:0/2425481293 >> 172.21.15.37:0/2425481293 conn(0x7f87ac0fd8c0 msgr2=0x7f87ac0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 -- 172.21.15.37:0/2425481293 shutdown_connections 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 -- 172.21.15.37:0/2425481293 wait complete. 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 Processor -- start 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.679+0000 7f87b4406640 1 -- start start 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b4406640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87ac101de0 0x7f87ac198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b4406640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac102790 0x7f87ac198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.691 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b4406640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87ac106f10 0x7f87ac19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b4406640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac111f70 con 0x7f87ac102790 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b4406640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac111df0 con 0x7f87ac106f10 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b4406640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac1120f0 con 0x7f87ac101de0 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac102790 0x7f87ac198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac102790 0x7f87ac198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40222/0 (socket says 172.21.15.37:40222) 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 -- 172.21.15.37:0/2929339809 learned_addr learned my addr 172.21.15.37:0/2929339809 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b29a3640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87ac106f10 0x7f87ac19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b21a2640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87ac101de0 0x7f87ac198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 -- 172.21.15.37:0/2929339809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87ac101de0 msgr2=0x7f87ac198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87ac101de0 0x7f87ac198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.692 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 -- 172.21.15.37:0/2929339809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87ac106f10 msgr2=0x7f87ac19cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87ac106f10 0x7f87ac19cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 -- 172.21.15.37:0/2929339809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87ac19d3e0 con 0x7f87ac102790 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b21a2640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87ac101de0 0x7f87ac198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b29a3640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87ac106f10 0x7f87ac19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.680+0000 7f87b19a1640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac102790 0x7f87ac198970 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f879c00e9f0 tx=0x7f879c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.681+0000 7f87a37fe640 1 -- 172.21.15.37:0/2929339809 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c00cd80 con 0x7f87ac102790 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.681+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87ac19d6d0 con 0x7f87ac102790 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.681+0000 7f87a37fe640 1 -- 172.21.15.37:0/2929339809 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879c010910 con 0x7f87ac102790 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.681+0000 7f87a37fe640 1 -- 172.21.15.37:0/2929339809 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c018990 con 0x7f87ac102790 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.681+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87ac1a4fb0 con 0x7f87ac102790 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.683+0000 7f87a37fe640 1 -- 172.21.15.37:0/2929339809 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f879c010ac0 con 0x7f87ac102790 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.683+0000 7f87b67df740 1 Processor -- start 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.683+0000 7f87b67df740 1 -- start start 2024-09-19T06:51:21.693 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b67df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7db880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b67df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e6140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b67df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b67df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a46b7412e0 con 0x55a46b7e6680 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b67df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a46b6e8ce0 con 0x55a46b7db4a0 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b67df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a46b73d2a0 con 0x55a46b7dbdc0 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b29a3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7db880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40228/0 (socket says 172.21.15.37:40228) 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 -- 172.21.15.37:0/3222161985 learned_addr learned my addr 172.21.15.37:0/3222161985 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b21a2640 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87a37fe640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f87980a33b0 0x7f87980a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.694 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b29a3640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f87980a33b0 0x7f87980a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87a37fe640 1 -- 172.21.15.37:0/2929339809 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f879c0d0170 con 0x7f87ac102790 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 -- 172.21.15.37:0/3222161985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7dbdc0 msgr2=0x55a46b7e6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 -- 172.21.15.37:0/3222161985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 msgr2=0x55a46b7db880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7db880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b19a1640 1 -- 172.21.15.37:0/3222161985 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46b6cd440 con 0x55a46b7e6680 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b29a3640 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.684+0000 7f87b21a2640 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.685+0000 7f87b19a1640 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e8a70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f879c09c0c0 tx=0x7f879c018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a45e0915a1dbb29 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.685+0000 7f87a0dea640 1 -- 172.21.15.37:0/3222161985 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c09c3c0 con 0x55a46b7e6680 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.685+0000 7f87a0dea640 1 -- 172.21.15.37:0/3222161985 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879c0a0770 con 0x55a46b7e6680 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.685+0000 7f87a0dea640 1 -- 172.21.15.37:0/3222161985 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c0d20d0 con 0x55a46b7e6680 2024-09-19T06:51:21.695 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.685+0000 7f87b29a3640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f87980a33b0 0x7f87980a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f87a4013e00 tx=0x7f87a4004ab0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.685+0000 7f87b67df740 1 -- 172.21.15.37:0/3222161985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 msgr2=0x55a46b7e8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.685+0000 7f87b67df740 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e8a70 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7f879c09c0c0 tx=0x7f879c018420 comp rx=0 tx=0).stop 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- 172.21.15.37:0/3222161985 shutdown_connections 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e8a70 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 --2- 172.21.15.37:0/3222161985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7db880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- 172.21.15.37:0/3222161985 >> 172.21.15.37:0/3222161985 conn(0x55a46b7d7a30 msgr2=0x55a46b7d7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- 172.21.15.37:0/3222161985 shutdown_connections 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- 172.21.15.37:0/3222161985 wait complete. 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- start start 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 Processor -- start 2024-09-19T06:51:21.696 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a46b7ed600 con 0x55a46b7e6680 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a46b6fc050 con 0x55a46b7dbdc0 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a46b7ed780 con 0x55a46b7db4a0 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.686+0000 7f87b67df740 10 client.0 _ll_register_callbacks cb 0x55a46b73c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:49370/0 (socket says 172.21.15.37:49370) 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 -- 172.21.15.37:0/562722988 learned_addr learned my addr 172.21.15.37:0/562722988 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 -- 172.21.15.37:0/562722988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7dbdc0 msgr2=0x55a46b7e3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 -- 172.21.15.37:0/562722988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 msgr2=0x55a46b7e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.697 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 -- 172.21.15.37:0/562722988 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46b4b23c0 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f87b21a2640 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7e5fb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009900 tx=0x7f87a8005a60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65fef910f52962c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.687+0000 7f8795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f8795ffb640 1 -- 172.21.15.37:0/562722988 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8051070 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f8795ffb640 1 -- 172.21.15.37:0/562722988 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f87a8004240 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f879c0a0770 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f8795ffb640 1 -- 172.21.15.37:0/562722988 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87a8035040 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 10 client.27160 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a46b7f9000 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f8795ffb640 1 -- 172.21.15.37:0/562722988 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879c0a0770 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f8795ffb640 1 -- 172.21.15.37:0/562722988 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f87a805c070 con 0x55a46b7db4a0 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 10 client.27160 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:21.698 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 10 client.27160 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 20 client.27160 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7fa510 0x55a46b7fc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 4 client.27160 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.688+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a46b74e850 con 0x55a46b7fa510 2024-09-19T06:51:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.689+0000 7f87b19a1640 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7fa510 0x55a46b7fc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.689+0000 7f87b19a1640 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7fa510 0x55a46b7fc9a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4558f8aa932c7d3 server_cookie=2cef05a84cfd8d99 in_seq=0 out_seq=0 2024-09-19T06:51:21.699 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.689+0000 7f8795ffb640 10 client.27160 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:21.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f8795ffb640 1 -- 172.21.15.37:0/562722988 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a46b74e850 con 0x55a46b7fa510 2024-09-19T06:51:21.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f8795ffb640 10 client.27160 handle_command_reply: tid=0 2024-09-19T06:51:21.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 client.27160 shutdown 2024-09-19T06:51:21.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 msgr2=0x55a46b7e5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7e5fb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009900 tx=0x7f87a8005a60 comp rx=0 tx=0).stop 2024-09-19T06:51:21.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 shutdown_connections 2024-09-19T06:51:21.700 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7fa510 0x55a46b7fc9a0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7e6680 0x55a46b7e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7dbdc0 0x55a46b7e3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 --2- 172.21.15.37:0/562722988 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7e5fb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f8795ffb640 0 client.27160 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 >> 172.21.15.37:0/562722988 conn(0x55a46b7d7a30 msgr2=0x55a46b7dd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 shutdown_connections 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 1 -- 172.21.15.37:0/562722988 wait complete. 2024-09-19T06:51:21.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.690+0000 7f87b67df740 20 client.27160 trim_cache size 0 max 16384 2024-09-19T06:51:21.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.709+0000 7f87b67df740 1 Processor -- start 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.709+0000 7f87b67df740 1 -- start start 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b67df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7f6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b67df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b8022b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b67df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7e6680 0x55a46b8027f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b67df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac1a5880 con 0x55a46b7dbdc0 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b67df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac074600 con 0x55a46b7e6680 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b67df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac072470 con 0x55a46b7db4a0 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b8022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b8022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40252/0 (socket says 172.21.15.37:40252) 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 -- 172.21.15.37:0/2979581700 learned_addr learned my addr 172.21.15.37:0/2979581700 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b29a3640 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7e6680 0x55a46b8027f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b21a2640 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7f6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 -- 172.21.15.37:0/2979581700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 msgr2=0x55a46b7f6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7f6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 -- 172.21.15.37:0/2979581700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7e6680 msgr2=0x55a46b8027f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7e6680 0x55a46b8027f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 -- 172.21.15.37:0/2979581700 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a46b7d8ba0 con 0x55a46b7dbdc0 2024-09-19T06:51:21.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f87b19a1640 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b8022b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f879c09a670 tx=0x7f879c098540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cf37172316d246b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f8797fff640 1 -- 172.21.15.37:0/2979581700 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c098a30 con 0x55a46b7dbdc0 2024-09-19T06:51:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.710+0000 7f8797fff640 1 -- 172.21.15.37:0/2979581700 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879c0cdab0 con 0x55a46b7dbdc0 2024-09-19T06:51:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f8797fff640 1 -- 172.21.15.37:0/2979581700 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f879c018d30 con 0x55a46b7dbdc0 2024-09-19T06:51:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 -- 172.21.15.37:0/2979581700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 msgr2=0x55a46b8022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b8022b0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f879c09a670 tx=0x7f879c098540 comp rx=0 tx=0).stop 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 -- 172.21.15.37:0/2979581700 shutdown_connections 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7e6680 0x55a46b8027f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b8022b0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 --2- 172.21.15.37:0/2979581700 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7db4a0 0x55a46b7f6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 -- 172.21.15.37:0/2979581700 >> 172.21.15.37:0/2979581700 conn(0x55a46b7d7a30 msgr2=0x55a46b7e1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 -- 172.21.15.37:0/2979581700 shutdown_connections 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.711+0000 7f87b67df740 1 -- 172.21.15.37:0/2979581700 wait complete. 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 -- start start 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 Processor -- start 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7ee780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b7eecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7e6680 0x55a46b7ef200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac072ac0 con 0x55a46b7dbdc0 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac073f30 con 0x55a46b7db4a0 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f87ac073190 con 0x55a46b7e6680 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b67df740 10 client.0 _ll_register_callbacks cb 0x7f87a8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b7eecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b21a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b19a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b7eecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40260/0 (socket says 172.21.15.37:40260) 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b19a1640 1 -- 172.21.15.37:0/1983661127 learned_addr learned my addr 172.21.15.37:0/1983661127 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.712+0000 7f87b21a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7ee780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:50576/0 (socket says 172.21.15.37:50576) 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b29a3640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7e6680 0x55a46b7ef200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b21a2640 1 -- 172.21.15.37:0/1983661127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7e6680 msgr2=0x55a46b7ef200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b21a2640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7e6680 0x55a46b7ef200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b21a2640 1 -- 172.21.15.37:0/1983661127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 msgr2=0x55a46b7eecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b21a2640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b7eecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b21a2640 1 -- 172.21.15.37:0/1983661127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a46b864280 con 0x55a46b7db4a0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b19a1640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b7eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b29a3640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7e6680 0x55a46b7ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b21a2640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7ee780 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000c00 tx=0x7f87a8009660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e176efcb093f9d7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f8795ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f8795ffb640 1 -- 172.21.15.37:0/1983661127 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87a8051070 con 0x55a46b7db4a0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.713+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f879c0cdab0 con 0x55a46b7db4a0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f8795ffb640 1 -- 172.21.15.37:0/1983661127 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f87a8031bb0 con 0x55a46b7db4a0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f8795ffb640 1 -- 172.21.15.37:0/1983661127 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f87a8062040 con 0x55a46b7db4a0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 10 client.27416 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a46b7e9180 con 0x55a46b7db4a0 2024-09-19T06:51:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f8795ffb640 1 -- 172.21.15.37:0/1983661127 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f879c0cdab0 con 0x55a46b7db4a0 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f8795ffb640 1 -- 172.21.15.37:0/1983661127 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f87a8065020 con 0x55a46b7db4a0 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 10 client.27416 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 10 client.27416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 20 client.27416 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7e9850 0x55a46b7ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 4 client.27416 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a46b748d60 con 0x55a46b7e9850 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.714+0000 7f87b19a1640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7e9850 0x55a46b7ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.715+0000 7f8795ffb640 10 client.27416 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.715+0000 7f87b19a1640 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7e9850 0x55a46b7ebc40 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb39c0982c4282d2 server_cookie=6c87ed50edbcc3fc in_seq=0 out_seq=0 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f8795ffb640 1 -- 172.21.15.37:0/1983661127 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55a46b748d60 con 0x55a46b7e9850 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f8795ffb640 10 client.27416 handle_command_reply: tid=0 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 client.27416 shutdown 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 msgr2=0x55a46b7ee780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7ee780 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f87a8000c00 tx=0x7f87a8009660 comp rx=0 tx=0).stop 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 shutdown_connections 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a46b7e9850 0x55a46b7ebc40 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a46b7e6680 0x55a46b7ef200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f8795ffb640 0 client.27416 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:21.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a46b7dbdc0 0x55a46b7eecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 --2- 172.21.15.37:0/1983661127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a46b7db4a0 0x55a46b7ee780 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 >> 172.21.15.37:0/1983661127 conn(0x55a46b7d7a30 msgr2=0x55a46b7d6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 shutdown_connections 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 1 -- 172.21.15.37:0/1983661127 wait complete. 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.716+0000 7f87b67df740 20 client.27416 trim_cache size 0 max 16384 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:51:21.723 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:51:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f87980a33b0 msgr2=0x7f87980a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f87980a33b0 0x7f87980a5870 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f87a4013e00 tx=0x7f87a4004ab0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac102790 msgr2=0x7f87ac198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac102790 0x7f87ac198970 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f879c00e9f0 tx=0x7f879c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 shutdown_connections 2024-09-19T06:51:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87ac106f10 0x7f87ac19cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f87980a33b0 0x7f87980a5870 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f87ac102790 0x7f87ac198970 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 --2- 172.21.15.37:0/2929339809 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f87ac101de0 0x7f87ac198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 >> 172.21.15.37:0/2929339809 conn(0x7f87ac0fd8c0 msgr2=0x7f87ac0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 shutdown_connections 2024-09-19T06:51:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:21.717+0000 7f87b4406640 1 -- 172.21.15.37:0/2929339809 wait complete. 2024-09-19T06:51:22.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 745634 8.09 MB/sec execute 349 sec latency 37.556 ms 2024-09-19T06:51:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:21 smithi096 ceph-mon[35004]: pgmap v300: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-09-19T06:51:22.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:21 smithi037 ceph-mon[31211]: pgmap v300: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-09-19T06:51:22.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:21 smithi151 ceph-mon[35236]: pgmap v300: 97 pgs: 97 active+clean; 78 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.5 MiB/s wr, 313 op/s 2024-09-19T06:51:22.547 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:51:22.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:51:22.832 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:23.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 746748 8.08 MB/sec execute 350 sec latency 42.406 ms 2024-09-19T06:51:23.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 -- 172.21.15.37:0/1185963982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 msgr2=0x7f458c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 --2- 172.21.15.37:0/1185963982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c103110 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f4588009930 tx=0x7f458802a420 comp rx=0 tx=0).stop 2024-09-19T06:51:23.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 -- 172.21.15.37:0/1185963982 shutdown_connections 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 --2- 172.21.15.37:0/1185963982 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c1082a0 0x7f458c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 --2- 172.21.15.37:0/1185963982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458c1036e0 0x7f458c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 --2- 172.21.15.37:0/1185963982 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c103110 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 -- 172.21.15.37:0/1185963982 >> 172.21.15.37:0/1185963982 conn(0x7f458c0fc7d0 msgr2=0x7f458c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 -- 172.21.15.37:0/1185963982 shutdown_connections 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.279+0000 7f459443d640 1 -- 172.21.15.37:0/1185963982 wait complete. 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 Processor -- start 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 -- start start 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c1036e0 0x7f458c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458c1082a0 0x7f458c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f458c1120a0 con 0x7f458c102d30 2024-09-19T06:51:23.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f458c111f20 con 0x7f458c1082a0 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f459443d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f458c112220 con 0x7f458c1036e0 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40278/0 (socket says 172.21.15.37:40278) 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 -- 172.21.15.37:0/423446533 learned_addr learned my addr 172.21.15.37:0/423446533 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45919d8640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c1036e0 0x7f458c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45929da640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458c1082a0 0x7f458c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 -- 172.21.15.37:0/423446533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c1036e0 msgr2=0x7f458c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c1036e0 0x7f458c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 -- 172.21.15.37:0/423446533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458c1082a0 msgr2=0x7f458c101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458c1082a0 0x7f458c101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.280+0000 7f45921d9640 1 -- 172.21.15.37:0/423446533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458c1a6a60 con 0x7f458c102d30 2024-09-19T06:51:23.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.281+0000 7f45921d9640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c100990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4588009230 tx=0x7f4588004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.281+0000 7f45837fe640 1 -- 172.21.15.37:0/423446533 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458803f030 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.281+0000 7f45837fe640 1 -- 172.21.15.37:0/423446533 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4588004440 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.281+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458c1a6cf0 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.281+0000 7f45837fe640 1 -- 172.21.15.37:0/423446533 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4588035070 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.281+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f458c1a7180 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.282+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f458c103110 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.283+0000 7f45837fe640 1 -- 172.21.15.37:0/423446533 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4588042030 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.284+0000 7f45837fe640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f45780a33b0 0x7f45780a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.284+0000 7f45837fe640 1 -- 172.21.15.37:0/423446533 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f45880f0030 con 0x7f458c102d30 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.284+0000 7f45919d8640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f45780a33b0 0x7f45780a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.284+0000 7f45919d8640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f45780a33b0 0x7f45780a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f457c0095f0 tx=0x7f457c005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:23.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.288+0000 7f45837fe640 1 -- 172.21.15.37:0/423446533 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f45880bc390 con 0x7f458c102d30 2024-09-19T06:51:23.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.506+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f458c10a2b0 con 0x7f458c102d30 2024-09-19T06:51:23.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.506+0000 7f45837fe640 1 -- 172.21.15.37:0/423446533 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7f45880c0690 con 0x7f458c102d30 2024-09-19T06:51:23.508 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:51:23.508 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:51:23.509 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:51:23.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f45780a33b0 msgr2=0x7f45780a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f45780a33b0 0x7f45780a5870 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f457c0095f0 tx=0x7f457c005f90 comp rx=0 tx=0).stop 2024-09-19T06:51:23.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 msgr2=0x7f458c100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c100990 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f4588009230 tx=0x7f4588004040 comp rx=0 tx=0).stop 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 shutdown_connections 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f458c1082a0 0x7f458c101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f458c1036e0 0x7f458c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f45780a33b0 0x7f45780a5870 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 --2- 172.21.15.37:0/423446533 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f458c102d30 0x7f458c100990 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 >> 172.21.15.37:0/423446533 conn(0x7f458c0fc7d0 msgr2=0x7f458c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 shutdown_connections 2024-09-19T06:51:23.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:23.509+0000 7f459443d640 1 -- 172.21.15.37:0/423446533 wait complete. 2024-09-19T06:51:24.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 749329 8.10 MB/sec execute 351 sec latency 51.267 ms 2024-09-19T06:51:24.254 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:23 smithi037 ceph-mon[31211]: pgmap v301: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 382 op/s 2024-09-19T06:51:24.254 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:23 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/423446533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:24.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:23 smithi096 ceph-mon[35004]: pgmap v301: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 382 op/s 2024-09-19T06:51:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:23 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/423446533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:24.281 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:51:24.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:23 smithi151 ceph-mon[35236]: pgmap v301: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.9 MiB/s wr, 382 op/s 2024-09-19T06:51:24.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:23 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/423446533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:24.543 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:25.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 -- 172.21.15.37:0/2984678456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 msgr2=0x7f313c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 --2- 172.21.15.37:0/2984678456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c103110 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f312c009930 tx=0x7f312c02a420 comp rx=0 tx=0).stop 2024-09-19T06:51:25.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 -- 172.21.15.37:0/2984678456 shutdown_connections 2024-09-19T06:51:25.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 --2- 172.21.15.37:0/2984678456 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f313c1082a0 0x7f313c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 --2- 172.21.15.37:0/2984678456 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c1036e0 0x7f313c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 --2- 172.21.15.37:0/2984678456 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c103110 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 -- 172.21.15.37:0/2984678456 >> 172.21.15.37:0/2984678456 conn(0x7f313c0fc7d0 msgr2=0x7f313c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 -- 172.21.15.37:0/2984678456 shutdown_connections 2024-09-19T06:51:25.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 -- 172.21.15.37:0/2984678456 wait complete. 2024-09-19T06:51:25.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.019+0000 7f3142fd7640 1 Processor -- start 2024-09-19T06:51:25.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3142fd7640 1 -- start start 2024-09-19T06:51:25.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3142fd7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3142fd7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c1036e0 0x7f313c198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3142fd7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f313c1082a0 0x7f313c19ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.023 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3142fd7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1120a0 con 0x7f313c102d30 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3142fd7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f313c111f20 con 0x7f313c1082a0 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3142fd7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f313c112220 con 0x7f313c1036e0 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40284/0 (socket says 172.21.15.37:40284) 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 -- 172.21.15.37:0/654486771 learned_addr learned my addr 172.21.15.37:0/654486771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f313bfff640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c1036e0 0x7f313c198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3141574640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f313c1082a0 0x7f313c19ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 -- 172.21.15.37:0/654486771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c1036e0 msgr2=0x7f313c198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c1036e0 0x7f313c198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 -- 172.21.15.37:0/654486771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f313c1082a0 msgr2=0x7f313c19ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f313c1082a0 0x7f313c19ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.020+0000 7f3140d73640 1 -- 172.21.15.37:0/654486771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c19d590 con 0x7f313c102d30 2024-09-19T06:51:25.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.021+0000 7f3140d73640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c1985e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f312c009230 tx=0x7f312c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.021+0000 7f3139ffb640 1 -- 172.21.15.37:0/654486771 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c03f030 con 0x7f313c102d30 2024-09-19T06:51:25.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.021+0000 7f3139ffb640 1 -- 172.21.15.37:0/654486771 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f312c004440 con 0x7f313c102d30 2024-09-19T06:51:25.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.021+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c19d820 con 0x7f313c102d30 2024-09-19T06:51:25.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.021+0000 7f3139ffb640 1 -- 172.21.15.37:0/654486771 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f312c035070 con 0x7f313c102d30 2024-09-19T06:51:25.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.021+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c1a5160 con 0x7f313c102d30 2024-09-19T06:51:25.026 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.023+0000 7f3139ffb640 1 -- 172.21.15.37:0/654486771 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f312c042030 con 0x7f313c102d30 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 Processor -- start 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 -- start start 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e3a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e3aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e49780 0x55e599e4bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e599da02f0 con 0x55e599e3a5d0 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e599d47cf0 con 0x55e599e3bb40 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f31453b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e599d9c2b0 con 0x55e599e49780 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f3140d73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e3aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f3141574640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e3a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f3140d73640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e3aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40298/0 (socket says 172.21.15.37:40298) 2024-09-19T06:51:25.027 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f3140d73640 1 -- 172.21.15.37:0/3982171986 learned_addr learned my addr 172.21.15.37:0/3982171986 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f313bfff640 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e49780 0x55e599e4bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.024+0000 7f3139ffb640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f31240a33b0 0x7f31240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3141574640 1 -- 172.21.15.37:0/3982171986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e49780 msgr2=0x55e599e4bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3141574640 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e49780 0x55e599e4bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3141574640 1 -- 172.21.15.37:0/3982171986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 msgr2=0x55e599e3aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3141574640 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e3aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3141574640 1 -- 172.21.15.37:0/3982171986 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e599d2c3a0 con 0x55e599e3bb40 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3141574640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f31240a33b0 0x7f31240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3139ffb640 1 -- 172.21.15.37:0/654486771 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f312c0f00b0 con 0x7f313c102d30 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3141574640 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e3a090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3134009fa0 tx=0x7f31340118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ee8194394062f8c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3122ffd640 1 -- 172.21.15.37:0/3982171986 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134017050 con 0x55e599e3bb40 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3122ffd640 1 -- 172.21.15.37:0/3982171986 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3134004750 con 0x55e599e3bb40 2024-09-19T06:51:25.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.025+0000 7f3122ffd640 1 -- 172.21.15.37:0/3982171986 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134025ae0 con 0x55e599e3bb40 2024-09-19T06:51:25.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 -- 172.21.15.37:0/3982171986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 msgr2=0x55e599e3a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e3a090 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3134009fa0 tx=0x7f31340118d0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f3141574640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f31240a33b0 0x7f31240a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3134020000 tx=0x7f3134021000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 -- 172.21.15.37:0/3982171986 shutdown_connections 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e49780 0x55e599e4bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e3aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 --2- 172.21.15.37:0/3982171986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e3a090 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 -- 172.21.15.37:0/3982171986 >> 172.21.15.37:0/3982171986 conn(0x55e599e03310 msgr2=0x55e599e36c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 -- 172.21.15.37:0/3982171986 shutdown_connections 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.026+0000 7f31453b0740 1 -- 172.21.15.37:0/3982171986 wait complete. 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 -- start start 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 Processor -- start 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e3a5d0 0x55e599e49160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3bb40 0x55e599e46410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e49780 0x55e599e469e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e599e4c480 con 0x55e599e3bb40 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e599d5b060 con 0x55e599e49780 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e599e4c600 con 0x55e599e3a5d0 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f31453b0740 10 client.0 _ll_register_callbacks cb 0x55e599aae8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f313bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3bb40 0x55e599e46410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f3140d73640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e3a5d0 0x55e599e49160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f313bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3bb40 0x55e599e46410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40310/0 (socket says 172.21.15.37:40310) 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f313bfff640 1 -- 172.21.15.37:0/3210797423 learned_addr learned my addr 172.21.15.37:0/3210797423 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.027+0000 7f3141574640 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e49780 0x55e599e469e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f313bfff640 1 -- 172.21.15.37:0/3210797423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e3a5d0 msgr2=0x55e599e49160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f313bfff640 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e3a5d0 0x55e599e49160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f313bfff640 1 -- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e49780 msgr2=0x55e599e469e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f313bfff640 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e49780 0x55e599e469e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f313bfff640 1 -- 172.21.15.37:0/3210797423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e599b113c0 con 0x55e599e3bb40 2024-09-19T06:51:25.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f313bfff640 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3bb40 0x55e599e46410 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3130012700 tx=0x7f3130012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4624a782885181c5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f311bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f311bfff640 1 -- 172.21.15.37:0/3210797423 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130004050 con 0x55e599e3bb40 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f311bfff640 1 -- 172.21.15.37:0/3210797423 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f31300041f0 con 0x55e599e3bb40 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3134004750 con 0x55e599e3bb40 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f311bfff640 1 -- 172.21.15.37:0/3210797423 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f31300043f0 con 0x55e599e3bb40 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f31453b0740 10 client.18093 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.028+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e599e57f60 con 0x55e599e3bb40 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f311bfff640 1 -- 172.21.15.37:0/3210797423 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3134004750 con 0x55e599e3bb40 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f311bfff640 1 -- 172.21.15.37:0/3210797423 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3130023070 con 0x55e599e3bb40 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f31453b0740 10 client.18093 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f31453b0740 10 client.18093 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:51:25.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f31453b0740 20 client.18093 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:25.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f31453b0740 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e59470 0x55e599e5b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f31453b0740 4 client.18093 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:25.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e599dad860 con 0x55e599e59470 2024-09-19T06:51:25.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f3140d73640 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e59470 0x55e599e5b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f3140d73640 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e59470 0x55e599e5b8b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96b7947d820f8aeb server_cookie=ad8ea9b12c155f61 in_seq=0 out_seq=0 2024-09-19T06:51:25.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.029+0000 7f311bfff640 10 client.18093 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:51:25.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f311bfff640 1 -- 172.21.15.37:0/3210797423 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e599dad860 con 0x55e599e59470 2024-09-19T06:51:25.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f311bfff640 10 client.18093 handle_command_reply: tid=0 2024-09-19T06:51:25.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 client.18093 shutdown 2024-09-19T06:51:25.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3bb40 msgr2=0x55e599e46410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3bb40 0x55e599e46410 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3130012700 tx=0x7f3130012bc0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 shutdown_connections 2024-09-19T06:51:25.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e59470 0x55e599e5b8b0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e49780 0x55e599e469e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3bb40 0x55e599e46410 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 --2- 172.21.15.37:0/3210797423 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e3a5d0 0x55e599e49160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 >> 172.21.15.37:0/3210797423 conn(0x55e599e03310 msgr2=0x55e599e3c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.030+0000 7f311bfff640 0 client.18093 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.031+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 shutdown_connections 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.031+0000 7f31453b0740 1 -- 172.21.15.37:0/3210797423 wait complete. 2024-09-19T06:51:25.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.031+0000 7f31453b0740 20 client.18093 trim_cache size 0 max 16384 2024-09-19T06:51:25.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 Processor -- start 2024-09-19T06:51:25.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 -- start start 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e55720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e36700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e453f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1ae080 con 0x55e599e3a5d0 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1ab950 con 0x55e599e3bb40 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f31453b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1a97c0 con 0x55e599e03310 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e36700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e36700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40316/0 (socket says 172.21.15.37:40316) 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 -- 172.21.15.37:0/2141554118 learned_addr learned my addr 172.21.15.37:0/2141554118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f3141574640 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f3140d73640 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e55720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 -- 172.21.15.37:0/2141554118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 msgr2=0x55e599e55720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e55720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 -- 172.21.15.37:0/2141554118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 msgr2=0x55e599e453f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e453f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f313bfff640 1 -- 172.21.15.37:0/2141554118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e599e36ed0 con 0x55e599e3a5d0 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.044+0000 7f3141574640 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f313bfff640 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e36700 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f31300126d0 tx=0x7f31300096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee66818a5def1836 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31227fc640 1 -- 172.21.15.37:0/2141554118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130029070 con 0x55e599e3a5d0 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31227fc640 1 -- 172.21.15.37:0/2141554118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31300035f0 con 0x55e599e3a5d0 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31227fc640 1 -- 172.21.15.37:0/2141554118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313000f790 con 0x55e599e3a5d0 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31453b0740 1 -- 172.21.15.37:0/2141554118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 msgr2=0x55e599e36700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31453b0740 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e36700 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f31300126d0 tx=0x7f31300096a0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31453b0740 1 -- 172.21.15.37:0/2141554118 shutdown_connections 2024-09-19T06:51:25.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31453b0740 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e453f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31453b0740 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e36700 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.045+0000 7f31453b0740 1 --2- 172.21.15.37:0/2141554118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e55720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.046+0000 7f31453b0740 1 -- 172.21.15.37:0/2141554118 >> 172.21.15.37:0/2141554118 conn(0x55e599e41e00 msgr2=0x55e599e44ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.046+0000 7f31453b0740 1 -- 172.21.15.37:0/2141554118 shutdown_connections 2024-09-19T06:51:25.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.046+0000 7f31453b0740 1 -- 172.21.15.37:0/2141554118 wait complete. 2024-09-19T06:51:25.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.046+0000 7f31453b0740 1 -- start start 2024-09-19T06:51:25.050 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.046+0000 7f31453b0740 1 Processor -- start 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.046+0000 7f31453b0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e51c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f31453b0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e52160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f31453b0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e4d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f31453b0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1a9e10 con 0x55e599e3a5d0 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f31453b0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1ab280 con 0x55e599e3bb40 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f31453b0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f313c1aa4e0 con 0x55e599e03310 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f31453b0740 10 client.0 _ll_register_callbacks cb 0x7f3130009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:25.053 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e52160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f3141574640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e4d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e52160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40330/0 (socket says 172.21.15.37:40330) 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f3140d73640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e51c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 -- 172.21.15.37:0/40611577 learned_addr learned my addr 172.21.15.37:0/40611577 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 -- 172.21.15.37:0/40611577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 msgr2=0x55e599e51c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e51c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 -- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 msgr2=0x55e599e4d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:25.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e4d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 -- 172.21.15.37:0/40611577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e599e4d9d0 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f3140d73640 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e51c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f3141574640 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e4d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f313bfff640 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e52160 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3130006010 tx=0x7f31300052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fd091ccad08ec5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f311bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.047+0000 7f311bfff640 1 -- 172.21.15.37:0/40611577 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313003a060 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f311bfff640 1 -- 172.21.15.37:0/40611577 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f31300037a0 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31300035f0 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f311bfff640 1 -- 172.21.15.37:0/40611577 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3130029040 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 10 client.18099 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e599ec3db0 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f311bfff640 1 -- 172.21.15.37:0/40611577 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f31300035f0 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f311bfff640 1 -- 172.21.15.37:0/40611577 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f3130023070 con 0x55e599e3a5d0 2024-09-19T06:51:25.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 10 client.18099 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 10 client.18099 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 20 client.18099 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e49780 0x55e599ec6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 4 client.18099 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e599da7d70 con 0x55e599e49780 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.048+0000 7f3140d73640 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e49780 0x55e599ec6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f3140d73640 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e49780 0x55e599ec6430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be8197b36219c6b server_cookie=2934cfecefab0f13 in_seq=0 out_seq=0 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f311bfff640 10 client.18099 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f311bfff640 1 -- 172.21.15.37:0/40611577 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e599da7d70 con 0x55e599e49780 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f311bfff640 10 client.18099 handle_command_reply: tid=0 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 client.18099 shutdown 2024-09-19T06:51:25.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 msgr2=0x55e599e52160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e52160 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3130006010 tx=0x7f31300052f0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 shutdown_connections 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55e599e49780 0x55e599ec6430 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f311bfff640 0 client.18099 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e599e3bb40 0x55e599e4d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e599e3a5d0 0x55e599e52160 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 --2- 172.21.15.37:0/40611577 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e599e03310 0x55e599e51c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.049+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 >> 172.21.15.37:0/40611577 conn(0x55e599e41e00 msgr2=0x55e599e45500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 shutdown_connections 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f31453b0740 1 -- 172.21.15.37:0/40611577 wait complete. 2024-09-19T06:51:25.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f31453b0740 20 client.18099 trim_cache size 0 max 16384 2024-09-19T06:51:25.058 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:51:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f31240a33b0 msgr2=0x7f31240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f31240a33b0 0x7f31240a5870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3134020000 tx=0x7f3134021000 comp rx=0 tx=0).stop 2024-09-19T06:51:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 msgr2=0x7f313c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c1985e0 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f312c009230 tx=0x7f312c004040 comp rx=0 tx=0).stop 2024-09-19T06:51:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 shutdown_connections 2024-09-19T06:51:25.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f313c1082a0 0x7f313c19ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f313c1036e0 0x7f313c198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f31240a33b0 0x7f31240a5870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 --2- 172.21.15.37:0/654486771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f313c102d30 0x7f313c1985e0 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:25.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 >> 172.21.15.37:0/654486771 conn(0x7f313c0fc7d0 msgr2=0x7f313c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:25.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 shutdown_connections 2024-09-19T06:51:25.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:25.050+0000 7f3142fd7640 1 -- 172.21.15.37:0/654486771 wait complete. 2024-09-19T06:51:25.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 750868 8.10 MB/sec execute 352 sec latency 23.509 ms 2024-09-19T06:51:25.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:51:26.121 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:26.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 752885 8.11 MB/sec execute 353 sec latency 49.878 ms 2024-09-19T06:51:26.243 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:25 smithi037 ceph-mon[31211]: pgmap v302: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-09-19T06:51:26.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:25 smithi096 ceph-mon[35004]: pgmap v302: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-09-19T06:51:26.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:25 smithi151 ceph-mon[35236]: pgmap v302: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/319552347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 msgr2=0x7f7f58100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/319552347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 0x7f7f58100ed0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c009930 tx=0x7f7f4c02a420 comp rx=0 tx=0).stop 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/319552347 shutdown_connections 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/319552347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58101410 0x7f7f5810d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/319552347 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 0x7f7f58100ed0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/319552347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f58106a70 0x7f7f58106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/319552347 >> 172.21.15.37:0/319552347 conn(0x7f7f580fc7d0 msgr2=0x7f7f580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/319552347 shutdown_connections 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.609+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/319552347 wait complete. 2024-09-19T06:51:26.612 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 Processor -- start 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 -- start start 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 0x7f7f58197310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f58101410 0x7f7f58197850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58106a70 0x7f7f5819bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58110e70 con 0x7f7f58100a70 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58110cf0 con 0x7f7f58101410 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.610+0000 7f7f5d3c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58110ff0 con 0x7f7f58106a70 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f577fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58106a70 0x7f7f5819bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f577fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58106a70 0x7f7f5819bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:49494/0 (socket says 172.21.15.37:49494) 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f577fe640 1 -- 172.21.15.37:0/671445478 learned_addr learned my addr 172.21.15.37:0/671445478 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f567fc640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f58101410 0x7f7f58197850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 0x7f7f58197310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f567fc640 1 -- 172.21.15.37:0/671445478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58106a70 msgr2=0x7f7f5819bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f567fc640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58106a70 0x7f7f5819bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f567fc640 1 -- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 msgr2=0x7f7f58197310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f567fc640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 0x7f7f58197310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f567fc640 1 -- 172.21.15.37:0/671445478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f5819c2e0 con 0x7f7f58101410 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f577fe640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58106a70 0x7f7f5819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 0x7f7f58197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.611+0000 7f7f567fc640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f58101410 0x7f7f58197850 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c005940 tx=0x7f7f4c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.612+0000 7f7f3ffff640 1 -- 172.21.15.37:0/671445478 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c035050 con 0x7f7f58101410 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.612+0000 7f7f3ffff640 1 -- 172.21.15.37:0/671445478 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4c004440 con 0x7f7f58101410 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.612+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f5819c570 con 0x7f7f58101410 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.612+0000 7f7f3ffff640 1 -- 172.21.15.37:0/671445478 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c038a50 con 0x7f7f58101410 2024-09-19T06:51:26.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.612+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f5806a790 con 0x7f7f58101410 2024-09-19T06:51:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 Processor -- start 2024-09-19T06:51:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 -- start start 2024-09-19T06:51:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee0ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee1cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55edbed712f0 con 0x55edbee0b5d0 2024-09-19T06:51:26.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55edbed18cf0 con 0x55edbee1a780 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.614+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55edbed6d2b0 con 0x55edbee0cb40 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f3ffff640 1 -- 172.21.15.37:0/671445478 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7f4c0035f0 con 0x7f7f58101410 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee0ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee0ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40342/0 (socket says 172.21.15.37:40342) 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2154699026 learned_addr learned my addr 172.21.15.37:0/2154699026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f577fe640 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee1cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2154699026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 msgr2=0x55edbee0b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee0b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2154699026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 msgr2=0x55edbee1cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee1cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2154699026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edbecfd3a0 con 0x55edbee0b5d0 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee1cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:26.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee0ba10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7f480114f0 tx=0x7f7f480119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0229d9f20d0540 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.615+0000 7f7f3d7fa640 1 -- 172.21.15.37:0/2154699026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4801e070 con 0x55edbee0b5d0 2024-09-19T06:51:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f3ffff640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7f400a3430 0x7f7f400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f3d7fa640 1 -- 172.21.15.37:0/2154699026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4800faa0 con 0x55edbee0b5d0 2024-09-19T06:51:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f3d7fa640 1 -- 172.21.15.37:0/2154699026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f48019b50 con 0x55edbee0b5d0 2024-09-19T06:51:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f577fe640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7f400a3430 0x7f7f400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f3ffff640 1 -- 172.21.15.37:0/671445478 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f7f4c0ef940 con 0x7f7f58101410 2024-09-19T06:51:26.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f577fe640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7f400a3430 0x7f7f400a58f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f50010860 tx=0x7f7f500075e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2154699026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 msgr2=0x55edbee0ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee0ba10 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f7f480114f0 tx=0x7f7f480119b0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2154699026 shutdown_connections 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee1cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee0ba10 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2154699026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee0b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.616+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2154699026 >> 172.21.15.37:0/2154699026 conn(0x55edbedd4310 msgr2=0x55edbee07c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2154699026 shutdown_connections 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2154699026 wait complete. 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 -- start start 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 Processor -- start 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee1a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee173f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55edbee1d480 con 0x55edbee0b5d0 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55edbed2c060 con 0x55edbee1a780 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55edbee1d600 con 0x55edbee0cb40 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f5f7a1740 10 client.0 _ll_register_callbacks cb 0x55edbea7f8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee1a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee1a140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40356/0 (socket says 172.21.15.37:40356) 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2663786708 learned_addr learned my addr 172.21.15.37:0/2663786708 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee173f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.617+0000 7f7f577fe640 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2663786708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 msgr2=0x55edbee173f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee173f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 msgr2=0x55edbee179e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee179e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2663786708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edbeae23c0 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee1a140 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7f48000c00 tx=0x7f7f48005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d875bab4d2281f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2663786708 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4801e040 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f4800faa0 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2663786708 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f7f4800f640 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2663786708 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f48030050 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f5f7a1740 10 client.18111 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.618+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55edbee28ee0 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2663786708 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f4800faa0 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2663786708 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7f48021070 con 0x55edbee0b5d0 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f5f7a1740 10 client.18111 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:26.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f5f7a1740 10 client.18111 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:51:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f5f7a1740 20 client.18111 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee2a280 0x55edbee2c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f5f7a1740 4 client.18111 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edbed7e860 con 0x55edbee2a280 2024-09-19T06:51:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee2a280 0x55edbee2c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f3aff5640 10 client.18111 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:51:26.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.619+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee2a280 0x55edbee2c670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56bdad6d22944814 server_cookie=568ffeadb4f407b0 in_seq=0 out_seq=0 2024-09-19T06:51:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.620+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2663786708 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55edbed7e860 con 0x55edbee2a280 2024-09-19T06:51:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.620+0000 7f7f3aff5640 10 client.18111 handle_command_reply: tid=0 2024-09-19T06:51:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.620+0000 7f7f5f7a1740 1 client.18111 shutdown 2024-09-19T06:51:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 msgr2=0x55edbee1a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee1a140 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f7f48000c00 tx=0x7f7f48005750 comp rx=0 tx=0).stop 2024-09-19T06:51:26.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 shutdown_connections 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee2a280 0x55edbee2c670 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee179e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0cb40 0x55edbee173f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2663786708 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0b5d0 0x55edbee1a140 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 >> 172.21.15.37:0/2663786708 conn(0x55edbedd4310 msgr2=0x55edbee0d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f3aff5640 0 client.18111 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 shutdown_connections 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2663786708 wait complete. 2024-09-19T06:51:26.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.621+0000 7f7f5f7a1740 20 client.18111 trim_cache size 0 max 16384 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 Processor -- start 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 -- start start 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0cb40 0x55edbee06bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee31db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f581b68a0 con 0x55edbee0cb40 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f580736e0 con 0x55edbee1a780 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58071550 con 0x55edbee0b5d0 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.635+0000 7f7f567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0cb40 0x55edbee06bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f567fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0cb40 0x55edbee06bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40364/0 (socket says 172.21.15.37:40364) 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f567fc640 1 -- 172.21.15.37:0/2579414934 learned_addr learned my addr 172.21.15.37:0/2579414934 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f577fe640 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee31db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2579414934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 msgr2=0x55edbee31db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee31db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2579414934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0cb40 msgr2=0x55edbee06bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0cb40 0x55edbee06bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2579414934 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edbee12fd0 con 0x55edbee0b5d0 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0cb40 0x55edbee06bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee26720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f48010010 tx=0x7f7f48009b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb381c0b34957f97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f3cff9640 1 -- 172.21.15.37:0/2579414934 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4801e040 con 0x55edbee0b5d0 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f3cff9640 1 -- 172.21.15.37:0/2579414934 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f48027070 con 0x55edbee0b5d0 2024-09-19T06:51:26.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.636+0000 7f7f3cff9640 1 -- 172.21.15.37:0/2579414934 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f480054c0 con 0x55edbee0b5d0 2024-09-19T06:51:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2579414934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 msgr2=0x55edbee26720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee26720 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7f48010010 tx=0x7f7f48009b90 comp rx=0 tx=0).stop 2024-09-19T06:51:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2579414934 shutdown_connections 2024-09-19T06:51:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee1a780 0x55edbee31db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee0cb40 0x55edbee06bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2579414934 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee26720 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2579414934 >> 172.21.15.37:0/2579414934 conn(0x55edbedd4310 msgr2=0x55edbee064e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2579414934 shutdown_connections 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2579414934 wait complete. 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.637+0000 7f7f5f7a1740 1 -- start start 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 1 Processor -- start 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee1dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee0cb40 0x55edbee1e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee1a780 0x55edbee92d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58071ba0 con 0x55edbee1a780 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58073010 con 0x55edbee0cb40 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7f58072270 con 0x55edbee0b5d0 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee1dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f5f7a1740 10 client.0 _ll_register_callbacks cb 0x7f7f4801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f56ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee1dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:49522/0 (socket says 172.21.15.37:49522) 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f56ffd640 1 -- 172.21.15.37:0/2030812009 learned_addr learned my addr 172.21.15.37:0/2030812009 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee0cb40 0x55edbee1e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f577fe640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee1a780 0x55edbee92d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f567fc640 1 -- 172.21.15.37:0/2030812009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 msgr2=0x55edbee1dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee1dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f567fc640 1 -- 172.21.15.37:0/2030812009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee1a780 msgr2=0x55edbee92d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee1a780 0x55edbee92d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f567fc640 1 -- 172.21.15.37:0/2030812009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edbee932b0 con 0x55edbee0cb40 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.638+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee1dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f567fc640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee0cb40 0x55edbee1e150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0ed0c0 tx=0x7f7f4c0efb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6766ce4b7f49f6a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2030812009 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c0be500 con 0x55edbee0cb40 2024-09-19T06:51:26.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2030812009 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f7f4c0bbea0 con 0x55edbee0cb40 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f48027070 con 0x55edbee0cb40 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2030812009 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7f4c0bde00 con 0x55edbee0cb40 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f5f7a1740 10 client.27458 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55edbee944e0 con 0x55edbee0cb40 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.639+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2030812009 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7f48027070 con 0x55edbee0cb40 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2030812009 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7f4c0ed2e0 con 0x55edbee0cb40 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f5f7a1740 10 client.27458 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f5f7a1740 10 client.27458 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f5f7a1740 20 client.27458 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee94c60 0x55edbee97080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f5f7a1740 4 client.27458 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55edbed78d70 con 0x55edbee94c60 2024-09-19T06:51:26.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee94c60 0x55edbee97080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f3aff5640 10 client.27458 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f56ffd640 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee94c60 0x55edbee97080 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=759fd8b0cc18f710 server_cookie=706558f359d20c6c in_seq=0 out_seq=0 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f3aff5640 1 -- 172.21.15.37:0/2030812009 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55edbed78d70 con 0x55edbee94c60 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.640+0000 7f7f3aff5640 10 client.27458 handle_command_reply: tid=0 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 client.27458 shutdown 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee0cb40 msgr2=0x55edbee1e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee0cb40 0x55edbee1e150 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c0ed0c0 tx=0x7f7f4c0efb40 comp rx=0 tx=0).stop 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 shutdown_connections 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55edbee94c60 0x55edbee97080 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55edbee1a780 0x55edbee92d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f3aff5640 0 client.27458 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55edbee0cb40 0x55edbee1e150 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 --2- 172.21.15.37:0/2030812009 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55edbee0b5d0 0x55edbee1dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 >> 172.21.15.37:0/2030812009 conn(0x55edbedd4310 msgr2=0x55edbee06d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 shutdown_connections 2024-09-19T06:51:26.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 1 -- 172.21.15.37:0/2030812009 wait complete. 2024-09-19T06:51:26.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.641+0000 7f7f5f7a1740 20 client.27458 trim_cache size 0 max 16384 2024-09-19T06:51:26.648 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7f400a3430 msgr2=0x7f7f400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7f400a3430 0x7f7f400a58f0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7f50010860 tx=0x7f7f500075e0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f58101410 msgr2=0x7f7f58197850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f58101410 0x7f7f58197850 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c005940 tx=0x7f7f4c004040 comp rx=0 tx=0).stop 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 shutdown_connections 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7f58106a70 0x7f7f5819bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7f58101410 0x7f7f58197850 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7f400a3430 0x7f7f400a58f0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 --2- 172.21.15.37:0/671445478 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7f58100a70 0x7f7f58197310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 >> 172.21.15.37:0/671445478 conn(0x7f7f580fc7d0 msgr2=0x7f7f580fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 shutdown_connections 2024-09-19T06:51:26.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:26.642+0000 7f7f5d3c8640 1 -- 172.21.15.37:0/671445478 wait complete. 2024-09-19T06:51:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 -- 172.21.15.37:0/1262201859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a34101350 msgr2=0x7f6a3410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 --2- 172.21.15.37:0/1262201859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a34101350 0x7f6a3410d880 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009cd0 tx=0x7f6a2c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.837 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 -- 172.21.15.37:0/1262201859 shutdown_connections 2024-09-19T06:51:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 --2- 172.21.15.37:0/1262201859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a34101350 0x7f6a3410d880 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 --2- 172.21.15.37:0/1262201859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a341009b0 0x7f6a34100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 --2- 172.21.15.37:0/1262201859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a341069c0 0x7f6a34106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 -- 172.21.15.37:0/1262201859 >> 172.21.15.37:0/1262201859 conn(0x7f6a34077990 msgr2=0x7f6a340fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 -- 172.21.15.37:0/1262201859 shutdown_connections 2024-09-19T06:51:26.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.835+0000 7f6a39d49640 1 -- 172.21.15.37:0/1262201859 wait complete. 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.836+0000 7f6a39d49640 1 Processor -- start 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.836+0000 7f6a39d49640 1 -- start start 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a39d49640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a341009b0 0x7f6a341982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a39d49640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a34101350 0x7f6a34198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a39d49640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a341069c0 0x7f6a3419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a39d49640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34110e00 con 0x7f6a341009b0 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a39d49640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34110c80 con 0x7f6a341069c0 2024-09-19T06:51:26.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a39d49640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a34110f80 con 0x7f6a34101350 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a341009b0 0x7f6a341982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a341009b0 0x7f6a341982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40398/0 (socket says 172.21.15.37:40398) 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a341069c0 0x7f6a3419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 -- 172.21.15.37:0/3793294490 learned_addr learned my addr 172.21.15.37:0/3793294490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a2affd640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a34101350 0x7f6a34198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 -- 172.21.15.37:0/3793294490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a34101350 msgr2=0x7f6a34198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a34101350 0x7f6a34198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 -- 172.21.15.37:0/3793294490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a341069c0 msgr2=0x7f6a3419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a341069c0 0x7f6a3419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.837+0000 7f6a337fe640 1 -- 172.21.15.37:0/3793294490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a3419d2e0 con 0x7f6a341009b0 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.838+0000 7f6a337fe640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a341009b0 0x7f6a341982f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e9f0 tx=0x7f6a2400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.838+0000 7f6a317fa640 1 -- 172.21.15.37:0/3793294490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2400cd80 con 0x7f6a341009b0 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.838+0000 7f6a317fa640 1 -- 172.21.15.37:0/3793294490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a24005500 con 0x7f6a341009b0 2024-09-19T06:51:26.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.838+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a3419d5d0 con 0x7f6a341009b0 2024-09-19T06:51:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.838+0000 7f6a317fa640 1 -- 172.21.15.37:0/3793294490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a24010690 con 0x7f6a341009b0 2024-09-19T06:51:26.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.838+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a341a4e60 con 0x7f6a341009b0 2024-09-19T06:51:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.840+0000 7f6a317fa640 1 -- 172.21.15.37:0/3793294490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6a240046c0 con 0x7f6a341009b0 2024-09-19T06:51:26.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.840+0000 7f6a3cbbc740 1 Processor -- start 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.840+0000 7f6a3cbbc740 1 -- start start 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044ceed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044cf410 0x5621044d9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044d9cb0 0x5621044dc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5621044332f0 con 0x5621044cf410 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621043db270 con 0x5621044ceaf0 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56210442f2b0 con 0x5621044d9cb0 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a337fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044cf410 0x5621044d9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044ceed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044ceed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:50736/0 (socket says 172.21.15.37:50736) 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 -- 172.21.15.37:0/3439023110 learned_addr learned my addr 172.21.15.37:0/3439023110 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a2affd640 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044d9cb0 0x5621044dc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 -- 172.21.15.37:0/3439023110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044d9cb0 msgr2=0x5621044dc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044d9cb0 0x5621044dc0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 -- 172.21.15.37:0/3439023110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044cf410 msgr2=0x5621044d9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044cf410 0x5621044d9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a33fff640 1 -- 172.21.15.37:0/3439023110 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621043da340 con 0x5621044ceaf0 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.841+0000 7f6a317fa640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a200a34e0 0x7f6a200a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a337fe640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a200a34e0 0x7f6a200a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a317fa640 1 -- 172.21.15.37:0/3793294490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f6a240d0040 con 0x7f6a341009b0 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a33fff640 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044ceed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009cd0 tx=0x7f6a2c009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=674066a5cd260e7e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a29ffb640 1 -- 172.21.15.37:0/3439023110 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c048070 con 0x5621044ceaf0 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a29ffb640 1 -- 172.21.15.37:0/3439023110 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c036070 con 0x5621044ceaf0 2024-09-19T06:51:26.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a29ffb640 1 -- 172.21.15.37:0/3439023110 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c004440 con 0x5621044ceaf0 2024-09-19T06:51:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/3439023110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 msgr2=0x5621044ceed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044ceed0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009cd0 tx=0x7f6a2c009a20 comp rx=0 tx=0).stop 2024-09-19T06:51:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.842+0000 7f6a337fe640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a200a34e0 0x7f6a200a59a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a240cd810 tx=0x7f6a240d3b90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/3439023110 shutdown_connections 2024-09-19T06:51:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044d9cb0 0x5621044dc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044cf410 0x5621044d9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/3439023110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044ceed0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/3439023110 >> 172.21.15.37:0/3439023110 conn(0x5621044cb060 msgr2=0x5621044cb440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/3439023110 shutdown_connections 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/3439023110 wait complete. 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- start start 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 Processor -- start 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044d7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5621044e0c90 con 0x5621044d9cb0 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621043bc290 con 0x5621044ceaf0 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.843+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5621044e0e10 con 0x5621044cf410 2024-09-19T06:51:26.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a3cbbc740 10 client.0 _ll_register_callbacks cb 0x5621043ed140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a2affd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044d7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40410/0 (socket says 172.21.15.37:40410) 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a33fff640 1 -- 172.21.15.37:0/2100303528 learned_addr learned my addr 172.21.15.37:0/2100303528 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a33fff640 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a2affd640 1 -- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 msgr2=0x5621044d6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a2affd640 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a2affd640 1 -- 172.21.15.37:0/2100303528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 msgr2=0x5621044d75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a2affd640 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a2affd640 1 -- 172.21.15.37:0/2100303528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562104146210 con 0x5621044cf410 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a2affd640 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044d7080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5621044d1690 tx=0x7f6a1c012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e5d17f7ea2d367c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2100303528 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c004050 con 0x5621044cf410 2024-09-19T06:51:26.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2c036070 con 0x5621044cf410 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2100303528 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f6a1c0041f0 con 0x5621044cf410 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2100303528 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6a1c0043f0 con 0x5621044cf410 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a3cbbc740 10 client.27214 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.844+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562104212620 con 0x5621044cf410 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2100303528 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c036070 con 0x5621044cf410 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2100303528 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6a1c023070 con 0x5621044cf410 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a3cbbc740 10 client.27214 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a3cbbc740 10 client.27214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a3cbbc740 20 client.27214 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621044ee070 0x5621044f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a3cbbc740 4 client.27214 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:26.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562104440a20 con 0x5621044ee070 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a33fff640 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621044ee070 0x5621044f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a33fff640 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621044ee070 0x5621044f0460 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d39953409ff6eb8 server_cookie=9b5e74a5bf0114e6 in_seq=0 out_seq=0 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.845+0000 7f6a12ffd640 10 client.27214 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2100303528 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562104440a20 con 0x5621044ee070 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a12ffd640 10 client.27214 handle_command_reply: tid=0 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 client.27214 shutdown 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 msgr2=0x5621044d7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044d7080 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x5621044d1690 tx=0x7f6a1c012b60 comp rx=0 tx=0).stop 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 shutdown_connections 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621044ee070 0x5621044f0460 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044d7080 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2100303528 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a12ffd640 0 client.27214 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 >> 172.21.15.37:0/2100303528 conn(0x5621044cb060 msgr2=0x5621044d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 shutdown_connections 2024-09-19T06:51:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2100303528 wait complete. 2024-09-19T06:51:26.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.846+0000 7f6a3cbbc740 20 client.27214 trim_cache size 0 max 16384 2024-09-19T06:51:26.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 Processor -- start 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 -- start start 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044f6270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341adf20 con 0x5621044d9cb0 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341ab7f0 con 0x5621044ceaf0 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341a9660 con 0x5621044cf410 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044f6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40414/0 (socket says 172.21.15.37:40414) 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a33fff640 1 -- 172.21.15.37:0/914294969 learned_addr learned my addr 172.21.15.37:0/914294969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a2affd640 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.862+0000 7f6a33fff640 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a33fff640 1 -- 172.21.15.37:0/914294969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 msgr2=0x5621044f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a33fff640 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a33fff640 1 -- 172.21.15.37:0/914294969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 msgr2=0x5621044d73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a33fff640 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a33fff640 1 -- 172.21.15.37:0/914294969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621044cf240 con 0x5621044d9cb0 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a2affd640 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a33fff640 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044f6270 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c004ab0 tx=0x7f6a2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=367e64216dc6d161 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a297fa640 1 -- 172.21.15.37:0/914294969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c048040 con 0x5621044d9cb0 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a297fa640 1 -- 172.21.15.37:0/914294969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c036070 con 0x5621044d9cb0 2024-09-19T06:51:26.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.863+0000 7f6a297fa640 1 -- 172.21.15.37:0/914294969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c0305a0 con 0x5621044d9cb0 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/914294969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 msgr2=0x5621044f6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044f6270 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c004ab0 tx=0x7f6a2c004a40 comp rx=0 tx=0).stop 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/914294969 shutdown_connections 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044f6270 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044cf410 0x5621044f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/914294969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044ceaf0 0x5621044d73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/914294969 >> 172.21.15.37:0/914294969 conn(0x5621044cb060 msgr2=0x5621044d8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/914294969 shutdown_connections 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/914294969 wait complete. 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 -- start start 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 Processor -- start 2024-09-19T06:51:26.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.864+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044ceaf0 0x5621044e6460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044cf410 0x5621044e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a3cbbc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341a9cb0 con 0x5621044d9cb0 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341ab120 con 0x5621044cf410 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a3cbbc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6a341aa380 con 0x5621044ceaf0 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a3cbbc740 10 client.0 _ll_register_callbacks cb 0x7f6a1c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a2affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044cf410 0x5621044e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40426/0 (socket says 172.21.15.37:40426) 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a2affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044cf410 0x5621044e69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:50754/0 (socket says 172.21.15.37:50754) 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 -- 172.21.15.37:0/2884726947 learned_addr learned my addr 172.21.15.37:0/2884726947 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044ceaf0 0x5621044e6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 -- 172.21.15.37:0/2884726947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044ceaf0 msgr2=0x5621044e6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044ceaf0 0x5621044e6460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 -- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044cf410 msgr2=0x5621044e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044cf410 0x5621044e69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 -- 172.21.15.37:0/2884726947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621044d3f80 con 0x5621044d9cb0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a2affd640 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044cf410 0x5621044e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a33fff640 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d3750 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009960 tx=0x7f6a2c056040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30dcf1802f73046b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.865+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2884726947 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a2c05f020 con 0x5621044d9cb0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a2c036070 con 0x5621044d9cb0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2884726947 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f6a2c004040 con 0x5621044d9cb0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2884726947 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6a2c06d050 con 0x5621044d9cb0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 10 client.18135 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5621045589e0 con 0x5621044d9cb0 2024-09-19T06:51:26.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2884726947 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6a2c036070 con 0x5621044d9cb0 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2884726947 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6a2c04e040 con 0x5621044d9cb0 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 10 client.18135 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 10 client.18135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 20 client.18135 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562104558cc0 0x56210455b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 4 client.18135 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56210443ad70 con 0x562104558cc0 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.866+0000 7f6a2affd640 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562104558cc0 0x56210455b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.867+0000 7f6a12ffd640 10 client.18135 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:26.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:26.867+0000 7f6a2affd640 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562104558cc0 0x56210455b0b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da45a18f39d0925d server_cookie=94031001da1609f2 in_seq=0 out_seq=0 2024-09-19T06:51:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.001+0000 7f6a12ffd640 1 -- 172.21.15.37:0/2884726947 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56210443ad70 con 0x562104558cc0 2024-09-19T06:51:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.001+0000 7f6a12ffd640 10 client.18135 handle_command_reply: tid=0 2024-09-19T06:51:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.001+0000 7f6a3cbbc740 1 client.18135 shutdown 2024-09-19T06:51:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 msgr2=0x5621044d3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d3750 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c009960 tx=0x7f6a2c056040 comp rx=0 tx=0).stop 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 shutdown_connections 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562104558cc0 0x56210455b0b0 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a12ffd640 0 client.18135 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621044d9cb0 0x5621044d3750 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621044cf410 0x5621044e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 --2- 172.21.15.37:0/2884726947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621044ceaf0 0x5621044e6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 >> 172.21.15.37:0/2884726947 conn(0x5621044cb060 msgr2=0x5621044c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 shutdown_connections 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 1 -- 172.21.15.37:0/2884726947 wait complete. 2024-09-19T06:51:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.002+0000 7f6a3cbbc740 20 client.18135 trim_cache size 0 max 16384 2024-09-19T06:51:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:51:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:51:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:51:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:51:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a200a34e0 msgr2=0x7f6a200a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a200a34e0 0x7f6a200a59a0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6a240cd810 tx=0x7f6a240d3b90 comp rx=0 tx=0).stop 2024-09-19T06:51:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a341009b0 msgr2=0x7f6a341982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a341009b0 0x7f6a341982f0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400e9f0 tx=0x7f6a2400edf0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 shutdown_connections 2024-09-19T06:51:27.007 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6a341069c0 0x7f6a3419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6a34101350 0x7f6a34198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6a200a34e0 0x7f6a200a59a0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 --2- 172.21.15.37:0/3793294490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6a341009b0 0x7f6a341982f0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 >> 172.21.15.37:0/3793294490 conn(0x7f6a34077990 msgr2=0x7f6a340ff100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 shutdown_connections 2024-09-19T06:51:27.008 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:27.003+0000 7f6a39d49640 1 -- 172.21.15.37:0/3793294490 wait complete. 2024-09-19T06:51:27.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 754378 8.11 MB/sec execute 354 sec latency 84.503 ms 2024-09-19T06:51:27.437 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:51:27.701 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:28.097 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:27 smithi037 ceph-mon[31211]: pgmap v303: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-09-19T06:51:28.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 -- 172.21.15.37:0/2598740572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8100a80 msgr2=0x7fddc8100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 --2- 172.21.15.37:0/2598740572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8100a80 0x7fddc8100ee0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fddb0009930 tx=0x7fddb002a420 comp rx=0 tx=0).stop 2024-09-19T06:51:28.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 -- 172.21.15.37:0/2598740572 shutdown_connections 2024-09-19T06:51:28.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 --2- 172.21.15.37:0/2598740572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddc8101420 0x7fddc810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 --2- 172.21.15.37:0/2598740572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8100a80 0x7fddc8100ee0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 --2- 172.21.15.37:0/2598740572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc8106a80 0x7fddc8106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.190 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 -- 172.21.15.37:0/2598740572 >> 172.21.15.37:0/2598740572 conn(0x7fddc80fc7a0 msgr2=0x7fddc80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 -- 172.21.15.37:0/2598740572 shutdown_connections 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.183+0000 7fddccdd4640 1 -- 172.21.15.37:0/2598740572 wait complete. 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 Processor -- start 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 -- start start 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddc8100a80 0x7fddc8197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8101420 0x7fddc81978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc8106a80 0x7fddc819bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8110ed0 con 0x7fddc8101420 2024-09-19T06:51:28.191 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8110d50 con 0x7fddc8106a80 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddccdd4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fddc8111050 con 0x7fddc8100a80 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddc5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8101420 0x7fddc81978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddc659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddc8100a80 0x7fddc8197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddc5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8101420 0x7fddc81978b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40454/0 (socket says 172.21.15.37:40454) 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddc5d9b640 1 -- 172.21.15.37:0/1496922230 learned_addr learned my addr 172.21.15.37:0/1496922230 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.184+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc8106a80 0x7fddc819bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc5d9b640 1 -- 172.21.15.37:0/1496922230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddc8100a80 msgr2=0x7fddc8197370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc5d9b640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddc8100a80 0x7fddc8197370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc5d9b640 1 -- 172.21.15.37:0/1496922230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc8106a80 msgr2=0x7fddc819bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc5d9b640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc8106a80 0x7fddc819bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc5d9b640 1 -- 172.21.15.37:0/1496922230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddc819c320 con 0x7fddc8101420 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc659c640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddc8100a80 0x7fddc8197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc8106a80 0x7fddc819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddc5d9b640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8101420 0x7fddc81978b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fddb002a940 tx=0x7fddb0003b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddbf7fe640 1 -- 172.21.15.37:0/1496922230 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb003f070 con 0x7fddc8101420 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddbf7fe640 1 -- 172.21.15.37:0/1496922230 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddb0031c90 con 0x7fddc8101420 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddc819c5b0 con 0x7fddc8101420 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddbf7fe640 1 -- 172.21.15.37:0/1496922230 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddb0035030 con 0x7fddc8101420 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.185+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddc806a790 con 0x7fddc8101420 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.187+0000 7fddcf1ad740 1 Processor -- start 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.188+0000 7fddcf1ad740 1 -- start start 2024-09-19T06:51:28.193 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.188+0000 7fddbf7fe640 1 -- 172.21.15.37:0/1496922230 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fddb0042070 con 0x7fddc8101420 2024-09-19T06:51:28.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b04090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b045d0 0x559676b04a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b15b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559676a6a2f0 con 0x559676b13780 2024-09-19T06:51:28.198 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559676a11cf0 con 0x559676b045d0 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559676a662b0 con 0x559676b05b40 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b04090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc5d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b15b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b04090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:49598/0 (socket says 172.21.15.37:49598) 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3319140593 learned_addr learned my addr 172.21.15.37:0/3319140593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc659c640 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b045d0 0x559676b04a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3319140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b045d0 msgr2=0x559676b04a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b045d0 0x559676b04a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3319140593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 msgr2=0x559676b15b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b15b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.189+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3319140593 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596769f63a0 con 0x559676b05b40 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.190+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b04090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fddc00117e0 tx=0x7fddc0011ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f70b7cea4a62bde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.190+0000 7fddbf7fe640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fddb40a3430 0x7fddb40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.199 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.190+0000 7fddc659c640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fddb40a3430 0x7fddb40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.190+0000 7fddbcff9640 1 -- 172.21.15.37:0/3319140593 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0011310 con 0x559676b05b40 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.190+0000 7fddbf7fe640 1 -- 172.21.15.37:0/1496922230 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fddb00f0300 con 0x7fddc8101420 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.190+0000 7fddbcff9640 1 -- 172.21.15.37:0/3319140593 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc0003e50 con 0x559676b05b40 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.190+0000 7fddbcff9640 1 -- 172.21.15.37:0/3319140593 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0004620 con 0x559676b05b40 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3319140593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 msgr2=0x559676b04090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b04090 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fddc00117e0 tx=0x7fddc0011ca0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddc659c640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fddb40a3430 0x7fddb40a58f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x559676b01ac0 tx=0x7fddb8005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3319140593 shutdown_connections 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b15b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b045d0 0x559676b04a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3319140593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b04090 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3319140593 >> 172.21.15.37:0/3319140593 conn(0x559676acd310 msgr2=0x559676b00c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.200 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3319140593 shutdown_connections 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3319140593 wait complete. 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.191+0000 7fddcf1ad740 1 -- start start 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 1 Processor -- start 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b045d0 0x559676b131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b10450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b13780 0x559676b10a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559676b16480 con 0x559676b045d0 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559676a25060 con 0x559676b13780 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559676b16600 con 0x559676b05b40 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddcf1ad740 10 client.0 _ll_register_callbacks cb 0x5596767788c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b13780 0x559676b10a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b045d0 0x559676b131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.201 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b13780 0x559676b10a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:50806/0 (socket says 172.21.15.37:50806) 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3124072296 learned_addr learned my addr 172.21.15.37:0/3124072296 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc5d9b640 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b10450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3124072296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 msgr2=0x559676b10450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b10450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3124072296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b045d0 msgr2=0x559676b131a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b045d0 0x559676b131a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3124072296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596767db3c0 con 0x559676b13780 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.192+0000 7fddc659c640 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b045d0 0x559676b131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:28.202 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b13780 0x559676b10a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fddc000f790 tx=0x7fddc0011340 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2ecc6b3622315b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddadffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddadffb640 1 -- 172.21.15.37:0/3124072296 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0004a40 con 0x559676b13780 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc0003e50 con 0x559676b13780 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddadffb640 1 -- 172.21.15.37:0/3124072296 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fddc0004be0 con 0x559676b13780 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddadffb640 1 -- 172.21.15.37:0/3124072296 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fddc0004de0 con 0x559676b13780 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddcf1ad740 10 client.27488 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559676b21f50 con 0x559676b13780 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddadffb640 1 -- 172.21.15.37:0/3124072296 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc0003e50 con 0x559676b13780 2024-09-19T06:51:28.203 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.193+0000 7fddadffb640 1 -- 172.21.15.37:0/3124072296 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fddc0018020 con 0x559676b13780 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddcf1ad740 10 client.27488 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddcf1ad740 10 client.27488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddcf1ad740 20 client.27488 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b232a0 0x559676b25690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddcf1ad740 4 client.27488 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559676a77860 con 0x559676b232a0 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddc659c640 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b232a0 0x559676b25690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddadffb640 10 client.27488 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.194+0000 7fddc659c640 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b232a0 0x559676b25690 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7738e530970577 server_cookie=792176eaa388656d in_seq=0 out_seq=0 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddadffb640 1 -- 172.21.15.37:0/3124072296 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559676a77860 con 0x559676b232a0 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddadffb640 10 client.27488 handle_command_reply: tid=0 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 client.27488 shutdown 2024-09-19T06:51:28.204 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b13780 msgr2=0x559676b10a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b13780 0x559676b10a60 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fddc000f790 tx=0x7fddc0011340 comp rx=0 tx=0).stop 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 shutdown_connections 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b232a0 0x559676b25690 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b13780 0x559676b10a60 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b05b40 0x559676b10450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddadffb640 0 client.27488 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3124072296 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b045d0 0x559676b131a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 >> 172.21.15.37:0/3124072296 conn(0x559676acd310 msgr2=0x559676b06960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 shutdown_connections 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3124072296 wait complete. 2024-09-19T06:51:28.205 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.195+0000 7fddcf1ad740 20 client.27488 trim_cache size 0 max 16384 2024-09-19T06:51:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 Processor -- start 2024-09-19T06:51:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 -- start start 2024-09-19T06:51:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b2ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b2b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fddc81ae010 con 0x559676b13780 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fddc81ab8e0 con 0x559676b05b40 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fddc81a9750 con 0x559676b045d0 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b2b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.209+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b2b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40492/0 (socket says 172.21.15.37:40492) 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3104353793 learned_addr learned my addr 172.21.15.37:0/3104353793 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc5d9b640 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b2ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc659c640 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3104353793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 msgr2=0x559676b1f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3104353793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 msgr2=0x559676b2ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b2ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc6d9d640 1 -- 172.21.15.37:0/3104353793 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559676b07120 con 0x559676b13780 2024-09-19T06:51:28.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b2b340 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fddc0005450 tx=0x7fddc00058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bb984e932b7e44 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddaffff640 1 -- 172.21.15.37:0/3104353793 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0004f80 con 0x559676b13780 2024-09-19T06:51:28.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddaffff640 1 -- 172.21.15.37:0/3104353793 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc001e040 con 0x559676b13780 2024-09-19T06:51:28.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.210+0000 7fddaffff640 1 -- 172.21.15.37:0/3104353793 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc000fe10 con 0x559676b13780 2024-09-19T06:51:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3104353793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 msgr2=0x559676b2b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b2b340 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fddc0005450 tx=0x7fddc00058c0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3104353793 shutdown_connections 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b2b340 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b2ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/3104353793 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3104353793 >> 172.21.15.37:0/3104353793 conn(0x559676acd310 msgr2=0x559676b10410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3104353793 shutdown_connections 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 -- 172.21.15.37:0/3104353793 wait complete. 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 -- start start 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.211+0000 7fddcf1ad740 1 Processor -- start 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddcf1ad740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b17c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fddc81a9da0 con 0x559676b13780 2024-09-19T06:51:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fddc81ab210 con 0x559676b05b40 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddcf1ad740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fddc81aa470 con 0x559676b045d0 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddcf1ad740 10 client.0 _ll_register_callbacks cb 0x7fddb00f3d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b17c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b17c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40506/0 (socket says 172.21.15.37:40506) 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc5d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b174d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:50820/0 (socket says 172.21.15.37:50820) 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 -- 172.21.15.37:0/1455781818 learned_addr learned my addr 172.21.15.37:0/1455781818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc659c640 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 -- 172.21.15.37:0/1455781818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 msgr2=0x559676b1c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 -- 172.21.15.37:0/1455781818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 msgr2=0x559676b174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 -- 172.21.15.37:0/1455781818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559676b8cdd0 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddc6d9d640 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b17c60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fddc000f790 tx=0x7fddc0005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fb35bf4521f6ea server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.212+0000 7fddadffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddadffb640 1 -- 172.21.15.37:0/1455781818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0005230 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddadffb640 1 -- 172.21.15.37:0/1455781818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7fddc000fbb0 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fddc001e040 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddadffb640 1 -- 172.21.15.37:0/1455781818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fddc0045050 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 10 client.18153 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559676b8df50 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddadffb640 1 -- 172.21.15.37:0/1455781818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fddc001e040 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddadffb640 1 -- 172.21.15.37:0/1455781818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fddc0018070 con 0x559676b13780 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 10 client.18153 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 10 client.18153 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 20 client.18153 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b8e4b0 0x559676b908a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 4 client.18153 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559676a71d70 con 0x559676b8e4b0 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.213+0000 7fddc659c640 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b8e4b0 0x559676b908a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddc659c640 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b8e4b0 0x559676b908a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e4cddbcb7b1475f server_cookie=d892cbe7b8ca17b4 in_seq=0 out_seq=0 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddadffb640 10 client.18153 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddadffb640 1 -- 172.21.15.37:0/1455781818 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559676a71d70 con 0x559676b8e4b0 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddadffb640 10 client.18153 handle_command_reply: tid=0 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 client.18153 shutdown 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 msgr2=0x559676b17c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b17c60 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fddc000f790 tx=0x7fddc0005a60 comp rx=0 tx=0).stop 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 shutdown_connections 2024-09-19T06:51:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559676b8e4b0 0x559676b908a0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559676b13780 0x559676b17c60 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559676b05b40 0x559676b174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddadffb640 0 client.18153 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 --2- 172.21.15.37:0/1455781818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559676b045d0 0x559676b1c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 >> 172.21.15.37:0/1455781818 conn(0x559676acd310 msgr2=0x559676b11cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 shutdown_connections 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.214+0000 7fddcf1ad740 1 -- 172.21.15.37:0/1455781818 wait complete. 2024-09-19T06:51:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddcf1ad740 20 client.18153 trim_cache size 0 max 16384 2024-09-19T06:51:28.224 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:51:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fddb40a3430 msgr2=0x7fddb40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fddb40a3430 0x7fddb40a58f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x559676b01ac0 tx=0x7fddb8005f90 comp rx=0 tx=0).stop 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8101420 msgr2=0x7fddc81978b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8101420 0x7fddc81978b0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fddb002a940 tx=0x7fddb0003b20 comp rx=0 tx=0).stop 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 shutdown_connections 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fddc8106a80 0x7fddc819bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fddb40a3430 0x7fddb40a58f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fddc8101420 0x7fddc81978b0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 --2- 172.21.15.37:0/1496922230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddc8100a80 0x7fddc8197370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 >> 172.21.15.37:0/1496922230 conn(0x7fddc80fc7a0 msgr2=0x7fddc80fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 shutdown_connections 2024-09-19T06:51:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:28.215+0000 7fddccdd4640 1 -- 172.21.15.37:0/1496922230 wait complete. 2024-09-19T06:51:28.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 756605 8.12 MB/sec execute 355 sec latency 24.936 ms 2024-09-19T06:51:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:27 smithi096 ceph-mon[35004]: pgmap v303: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-09-19T06:51:28.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:27 smithi151 ceph-mon[35236]: pgmap v303: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 285 op/s 2024-09-19T06:51:29.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 758927 8.12 MB/sec execute 356 sec latency 16.791 ms 2024-09-19T06:51:29.899 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:51:29.899 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:51:30.159 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:30.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 760548 8.13 MB/sec execute 357 sec latency 42.839 ms 2024-09-19T06:51:30.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:29 smithi096 ceph-mon[35004]: pgmap v304: 97 pgs: 97 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 473 op/s 2024-09-19T06:51:30.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:29 smithi037 ceph-mon[31211]: pgmap v304: 97 pgs: 97 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 473 op/s 2024-09-19T06:51:30.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:29 smithi151 ceph-mon[35236]: pgmap v304: 97 pgs: 97 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 MiB/s rd, 4.9 MiB/s wr, 473 op/s 2024-09-19T06:51:30.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 -- 172.21.15.37:0/2605455550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074eb0 msgr2=0x7f18c8112650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 --2- 172.21.15.37:0/2605455550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074eb0 0x7f18c8112650 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009900 tx=0x7f18b802a480 comp rx=0 tx=0).stop 2024-09-19T06:51:30.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 -- 172.21.15.37:0/2605455550 shutdown_connections 2024-09-19T06:51:30.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 --2- 172.21.15.37:0/2605455550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18c8112b90 0x7f18c8114f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 --2- 172.21.15.37:0/2605455550 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074eb0 0x7f18c8112650 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 --2- 172.21.15.37:0/2605455550 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c8074590 0x7f18c8074970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 -- 172.21.15.37:0/2605455550 >> 172.21.15.37:0/2605455550 conn(0x7f18c80fd8c0 msgr2=0x7f18c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 -- 172.21.15.37:0/2605455550 shutdown_connections 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.605+0000 7f18d069c640 1 -- 172.21.15.37:0/2605455550 wait complete. 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 Processor -- start 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 -- start start 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074590 0x7f18c81a4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c8074eb0 0x7f18c81a4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18c8112b90 0x7f18c819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18c81185c0 con 0x7f18c8074590 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18c8118440 con 0x7f18c8112b90 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.606+0000 7f18d069c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18c8118740 con 0x7f18c8074eb0 2024-09-19T06:51:30.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074590 0x7f18c81a4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074590 0x7f18c81a4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39442/0 (socket says 172.21.15.37:39442) 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 -- 172.21.15.37:0/3745454143 learned_addr learned my addr 172.21.15.37:0/3745454143 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18cec39640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18c8112b90 0x7f18c819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18cdc37640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c8074eb0 0x7f18c81a4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 -- 172.21.15.37:0/3745454143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c8074eb0 msgr2=0x7f18c81a4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c8074eb0 0x7f18c81a4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 -- 172.21.15.37:0/3745454143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18c8112b90 msgr2=0x7f18c819e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18c8112b90 0x7f18c819e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 -- 172.21.15.37:0/3745454143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c819f060 con 0x7f18c8074590 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18ce438640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074590 0x7f18c81a4770 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f18c400b4a0 tx=0x7f18c400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18bf7fe640 1 -- 172.21.15.37:0/3745454143 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4014070 con 0x7f18c8074590 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18bf7fe640 1 -- 172.21.15.37:0/3745454143 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18c4010af0 con 0x7f18c8074590 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c819f350 con 0x7f18c8074590 2024-09-19T06:51:30.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18bf7fe640 1 -- 172.21.15.37:0/3745454143 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c4017d10 con 0x7f18c8074590 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.607+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c81ab640 con 0x7f18c8074590 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.610+0000 7f18bf7fe640 1 -- 172.21.15.37:0/3745454143 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f18c4017070 con 0x7f18c8074590 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.610+0000 7f18d2a75740 1 Processor -- start 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18d2a75740 1 -- start start 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db09df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0ab130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18d2a75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5591db003280 con 0x5591db09db60 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18d2a75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5591dafaab10 con 0x5591db0a8d40 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18d2a75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591dafff240 con 0x5591db09e480 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18bf7fe640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18b40a3430 0x7f18b40a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cec39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db09df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0ab130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0ab130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47366/0 (socket says 172.21.15.37:47366) 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 -- 172.21.15.37:0/3327856511 learned_addr learned my addr 172.21.15.37:0/3327856511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18ce438640 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18bf7fe640 1 -- 172.21.15.37:0/3745454143 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f18c40cef30 con 0x7f18c8074590 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cec39640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18b40a3430 0x7f18b40a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 -- 172.21.15.37:0/3327856511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 msgr2=0x5591db0a8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 -- 172.21.15.37:0/3327856511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 msgr2=0x5591db09df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db09df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cdc37640 1 -- 172.21.15.37:0/3327856511 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591dafb71f0 con 0x5591db0a8d40 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.611+0000 7f18cec39640 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db09df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:30.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.612+0000 7f18cdc37640 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0ab130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009900 tx=0x7f18b8032ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52d9edc277aa9d4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.612+0000 7f18bcff9640 1 -- 172.21.15.37:0/3327856511 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b803f070 con 0x5591db0a8d40 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.612+0000 7f18cec39640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18b40a3430 0x7f18b40a58f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f18c8076600 tx=0x7f18c0004ab0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.612+0000 7f18bcff9640 1 -- 172.21.15.37:0/3327856511 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b80317b0 con 0x5591db0a8d40 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.612+0000 7f18bcff9640 1 -- 172.21.15.37:0/3327856511 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8035070 con 0x5591db0a8d40 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 -- 172.21.15.37:0/3327856511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 msgr2=0x5591db0ab130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0ab130 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f18b8009900 tx=0x7f18b8032ce0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 -- 172.21.15.37:0/3327856511 shutdown_connections 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0ab130 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3327856511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db09df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 -- 172.21.15.37:0/3327856511 >> 172.21.15.37:0/3327856511 conn(0x5591db09a0f0 msgr2=0x5591db09a4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 -- 172.21.15.37:0/3327856511 shutdown_connections 2024-09-19T06:51:30.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 -- 172.21.15.37:0/3327856511 wait complete. 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 -- start start 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 Processor -- start 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.613+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0a6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18d2a75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5591db0ab910 con 0x5591db09db60 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18d2a75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5591dafbdf10 con 0x5591db0a8d40 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18d2a75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5591db0aba90 con 0x5591db09e480 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18d2a75740 10 client.0 _ll_register_callbacks cb 0x5591daf86a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.631 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39446/0 (socket says 172.21.15.37:39446) 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 -- 172.21.15.37:0/19607647 learned_addr learned my addr 172.21.15.37:0/19607647 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18cdc37640 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18cec39640 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0a6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 -- 172.21.15.37:0/19607647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 msgr2=0x5591db0a6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 -- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 msgr2=0x5591db0a6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0a6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.632 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 -- 172.21.15.37:0/19607647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591dad746a0 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18ce438640 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a5bf0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f18c400be60 tx=0x7f18c4009230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b271376c04190d72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18b1ffb640 1 -- 172.21.15.37:0/19607647 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18c401e070 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18b1ffb640 1 -- 172.21.15.37:0/19607647 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f18c40cc8d0 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b80317b0 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18b1ffb640 1 -- 172.21.15.37:0/19607647 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18c4095e60 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18d2a75740 10 client.18165 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.614+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5591db0b7540 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18b1ffb640 1 -- 172.21.15.37:0/19607647 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b80317b0 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18b1ffb640 1 -- 172.21.15.37:0/19607647 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f18c4097960 con 0x5591db09db60 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18d2a75740 10 client.18165 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:30.633 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18d2a75740 10 client.18165 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18d2a75740 20 client.18165 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18d2a75740 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db0b8950 0x5591db0badc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18d2a75740 4 client.18165 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591db0107f0 con 0x5591db0b8950 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18cdc37640 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db0b8950 0x5591db0badc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18cdc37640 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db0b8950 0x5591db0badc0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=499afd90168f4db9 server_cookie=3f81f72a5d0cf63a in_seq=0 out_seq=0 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.615+0000 7f18b1ffb640 10 client.18165 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18b1ffb640 1 -- 172.21.15.37:0/19607647 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5591db0107f0 con 0x5591db0b8950 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18b1ffb640 10 client.18165 handle_command_reply: tid=0 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 client.18165 shutdown 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 msgr2=0x5591db0a5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a5bf0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f18c400be60 tx=0x7f18c4009230 comp rx=0 tx=0).stop 2024-09-19T06:51:30.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 shutdown_connections 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db0b8950 0x5591db0badc0 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db0a8d40 0x5591db0a6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db09e480 0x5591db0a6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 --2- 172.21.15.37:0/19607647 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a5bf0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18b1ffb640 0 client.18165 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 >> 172.21.15.37:0/19607647 conn(0x5591db09a0f0 msgr2=0x5591db09fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 shutdown_connections 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 1 -- 172.21.15.37:0/19607647 wait complete. 2024-09-19T06:51:30.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.617+0000 7f18d2a75740 20 client.18165 trim_cache size 0 max 16384 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.640+0000 7f18d2a75740 1 Processor -- start 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.640+0000 7f18d2a75740 1 -- start start 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09e480 0x5591db0a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0c0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18d2a75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18c81b6820 con 0x5591db09db60 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18d2a75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18c81b40f0 con 0x5591db09e480 2024-09-19T06:51:30.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18d2a75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18c81b1f60 con 0x5591db0a8d40 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18ce438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09e480 0x5591db0a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09e480 0x5591db0a7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47394/0 (socket says 172.21.15.37:47394) 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 -- 172.21.15.37:0/3545269739 learned_addr learned my addr 172.21.15.37:0/3545269739 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18ce438640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39448/0 (socket says 172.21.15.37:39448) 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cec39640 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 -- 172.21.15.37:0/3545269739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 msgr2=0x5591db0c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0c0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 -- 172.21.15.37:0/3545269739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 msgr2=0x5591db0a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 -- 172.21.15.37:0/3545269739 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591db09e2b0 con 0x5591db09e480 2024-09-19T06:51:30.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18ce438640 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.641+0000 7f18cdc37640 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09e480 0x5591db0a7a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f18b8005310 tx=0x7f18b80053e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b68e46959341b029 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18b3fff640 1 -- 172.21.15.37:0/3545269739 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b803f040 con 0x5591db09e480 2024-09-19T06:51:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18b3fff640 1 -- 172.21.15.37:0/3545269739 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b8005450 con 0x5591db09e480 2024-09-19T06:51:30.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18b3fff640 1 -- 172.21.15.37:0/3545269739 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8035040 con 0x5591db09e480 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 -- 172.21.15.37:0/3545269739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09e480 msgr2=0x5591db0a7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09e480 0x5591db0a7a50 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f18b8005310 tx=0x7f18b80053e0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 -- 172.21.15.37:0/3545269739 shutdown_connections 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0c0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09e480 0x5591db0a7a50 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 --2- 172.21.15.37:0/3545269739 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09db60 0x5591db0a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 -- 172.21.15.37:0/3545269739 >> 172.21.15.37:0/3545269739 conn(0x5591db09a0f0 msgr2=0x5591db0a7340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 -- 172.21.15.37:0/3545269739 shutdown_connections 2024-09-19T06:51:30.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.642+0000 7f18d2a75740 1 -- 172.21.15.37:0/3545269739 wait complete. 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 -- start start 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 Processor -- start 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09db60 0x5591db0acb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09e480 0x5591db0ad080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18c81b25b0 con 0x5591db09e480 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18c81b3a20 con 0x5591db09db60 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18c81b2c80 con 0x5591db0a8d40 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18d2a75740 10 client.0 _ll_register_callbacks cb 0x7f18c4017eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18cdc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09e480 0x5591db0ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18ce438640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09db60 0x5591db0acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18cec39640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18cdc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09e480 0x5591db0ad080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39458/0 (socket says 172.21.15.37:39458) 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.643+0000 7f18cdc37640 1 -- 172.21.15.37:0/726134003 learned_addr learned my addr 172.21.15.37:0/726134003 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18cdc37640 1 -- 172.21.15.37:0/726134003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 msgr2=0x5591db0ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18cdc37640 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18cdc37640 1 -- 172.21.15.37:0/726134003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09db60 msgr2=0x5591db0acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18cdc37640 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09db60 0x5591db0acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18cdc37640 1 -- 172.21.15.37:0/726134003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591db122750 con 0x5591db09e480 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18cdc37640 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09e480 0x5591db0ad080 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f18b802a4c0 tx=0x7f18b8005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d7c50de83689fb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18b1ffb640 1 -- 172.21.15.37:0/726134003 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8051020 con 0x5591db09e480 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18b1ffb640 1 -- 172.21.15.37:0/726134003 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f18b8004430 con 0x5591db09e480 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b8005450 con 0x5591db09e480 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18b1ffb640 1 -- 172.21.15.37:0/726134003 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18b805c050 con 0x5591db09e480 2024-09-19T06:51:30.650 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18d2a75740 10 client.18171 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5591db123a70 con 0x5591db09e480 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.644+0000 7f18b1ffb640 1 -- 172.21.15.37:0/726134003 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b8005450 con 0x5591db09e480 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18b1ffb640 1 -- 172.21.15.37:0/726134003 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f18b804d7b0 con 0x5591db09e480 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18d2a75740 10 client.18171 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18d2a75740 10 client.18171 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18d2a75740 20 client.18171 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18d2a75740 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db123db0 0x5591db1261a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18d2a75740 4 client.18171 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5591db00ad00 con 0x5591db123db0 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18ce438640 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db123db0 0x5591db1261a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18ce438640 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db123db0 0x5591db1261a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43241e22e416d89 server_cookie=9750d8cd9c523aa8 in_seq=0 out_seq=0 2024-09-19T06:51:30.651 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.645+0000 7f18b1ffb640 10 client.18171 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:30.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18b1ffb640 1 -- 172.21.15.37:0/726134003 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5591db00ad00 con 0x5591db123db0 2024-09-19T06:51:30.652 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18b1ffb640 10 client.18171 handle_command_reply: tid=0 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 client.18171 shutdown 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09e480 msgr2=0x5591db0ad080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09e480 0x5591db0ad080 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f18b802a4c0 tx=0x7f18b8005710 comp rx=0 tx=0).stop 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 shutdown_connections 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5591db123db0 0x5591db1261a0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18b1ffb640 0 client.18171 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5591db0a8d40 0x5591db0ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5591db09e480 0x5591db0ad080 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 --2- 172.21.15.37:0/726134003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5591db09db60 0x5591db0acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 >> 172.21.15.37:0/726134003 conn(0x5591db09a0f0 msgr2=0x5591db099270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.647+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 shutdown_connections 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d2a75740 1 -- 172.21.15.37:0/726134003 wait complete. 2024-09-19T06:51:30.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d2a75740 20 client.18171 trim_cache size 0 max 16384 2024-09-19T06:51:30.654 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:51:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:51:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "5ed6b459-9b9a-4e8d-8043-07c8e545d165", 2024-09-19T06:51:30.654 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:51:30.654 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18b40a3430 msgr2=0x7f18b40a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18b40a3430 0x7f18b40a58f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f18c8076600 tx=0x7f18c0004ab0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074590 msgr2=0x7f18c81a4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074590 0x7f18c81a4770 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f18c400b4a0 tx=0x7f18c400b960 comp rx=0 tx=0).stop 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 shutdown_connections 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18c8112b90 0x7f18c819e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18c8074eb0 0x7f18c81a4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18b40a3430 0x7f18b40a58f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 --2- 172.21.15.37:0/3745454143 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18c8074590 0x7f18c81a4770 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:30.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 >> 172.21.15.37:0/3745454143 conn(0x7f18c80fd8c0 msgr2=0x7f18c80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:30.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 shutdown_connections 2024-09-19T06:51:30.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:30.648+0000 7f18d069c640 1 -- 172.21.15.37:0/3745454143 wait complete. 2024-09-19T06:51:31.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 762875 8.14 MB/sec execute 358 sec latency 46.024 ms 2024-09-19T06:51:31.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:30 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:51:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:30 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:30 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:51:31.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:30 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:31.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:30 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:51:31.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:30 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:30 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:51:31.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:30 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:31.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:30 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:51:31.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:30 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:31.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:30 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:51:31.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:30 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:31.510 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:51:31.774 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:32.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:31 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:51:32.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:31 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:32.045 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:31 smithi037 ceph-mon[31211]: pgmap v305: 97 pgs: 97 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 286 op/s 2024-09-19T06:51:32.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 764229 8.14 MB/sec execute 359 sec latency 37.457 ms 2024-09-19T06:51:32.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.232+0000 7f18c375a640 1 -- 172.21.15.37:0/783402099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 msgr2=0x7f18bc110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.232+0000 7f18c375a640 1 --2- 172.21.15.37:0/783402099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 0x7f18bc110910 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009ca0 tx=0x7f18b402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.232+0000 7f18c375a640 1 -- 172.21.15.37:0/783402099 shutdown_connections 2024-09-19T06:51:32.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.232+0000 7f18c375a640 1 --2- 172.21.15.37:0/783402099 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 0x7f18bc110910 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.232+0000 7f18c375a640 1 --2- 172.21.15.37:0/783402099 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc101440 0x7f18bc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.232+0000 7f18c375a640 1 --2- 172.21.15.37:0/783402099 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18bc100b20 0x7f18bc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.232+0000 7f18c375a640 1 -- 172.21.15.37:0/783402099 >> 172.21.15.37:0/783402099 conn(0x7f18bc0fc7d0 msgr2=0x7f18bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.233+0000 7f18c375a640 1 -- 172.21.15.37:0/783402099 shutdown_connections 2024-09-19T06:51:32.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.233+0000 7f18c375a640 1 -- 172.21.15.37:0/783402099 wait complete. 2024-09-19T06:51:32.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.233+0000 7f18c375a640 1 Processor -- start 2024-09-19T06:51:32.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.233+0000 7f18c375a640 1 -- start start 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c375a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc100b20 0x7f18bc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c375a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18bc101440 0x7f18bc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c375a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 0x7f18bc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c375a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc114250 con 0x7f18bc10e500 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c375a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1140d0 con 0x7f18bc100b20 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c375a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1143d0 con 0x7f18bc101440 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc100b20 0x7f18bc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 0x7f18bc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc100b20 0x7f18bc19a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47414/0 (socket says 172.21.15.37:47414) 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 -- 172.21.15.37:0/1337264740 learned_addr learned my addr 172.21.15.37:0/1337264740 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18bc101440 0x7f18bc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 -- 172.21.15.37:0/1337264740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18bc101440 msgr2=0x7f18bc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18bc101440 0x7f18bc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 -- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 msgr2=0x7f18bc19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 0x7f18bc19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 -- 172.21.15.37:0/1337264740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18bc19f6a0 con 0x7f18bc100b20 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18bc101440 0x7f18bc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.234+0000 7f18c14f6640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc100b20 0x7f18bc19a6f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e9f0 tx=0x7f18b800edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.235+0000 7f18b27fc640 1 -- 172.21.15.37:0/1337264740 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b800cd80 con 0x7f18bc100b20 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.235+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18bc19f990 con 0x7f18bc100b20 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.235+0000 7f18b27fc640 1 -- 172.21.15.37:0/1337264740 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b8005500 con 0x7f18bc100b20 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.235+0000 7f18b27fc640 1 -- 172.21.15.37:0/1337264740 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8010690 con 0x7f18bc100b20 2024-09-19T06:51:32.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.235+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18bc1a7270 con 0x7f18bc100b20 2024-09-19T06:51:32.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.237+0000 7f18b27fc640 1 -- 172.21.15.37:0/1337264740 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f18b80046c0 con 0x7f18bc100b20 2024-09-19T06:51:32.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.237+0000 7f18c5b33740 1 Processor -- start 2024-09-19T06:51:32.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.237+0000 7f18c5b33740 1 -- start start 2024-09-19T06:51:32.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320a4a0 0x5577a320a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320adc0 0x5577a3215140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3217a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c5b33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5577a31702e0 con 0x5577a320adc0 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c5b33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5577a3117ce0 con 0x5577a3215680 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c5b33740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577a316c2a0 con 0x5577a320a4a0 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c14f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320adc0 0x5577a3215140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320a4a0 0x5577a320a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c14f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320adc0 0x5577a3215140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39488/0 (socket says 172.21.15.37:39488) 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c14f6640 1 -- 172.21.15.37:0/372574412 learned_addr learned my addr 172.21.15.37:0/372574412 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3217a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c1cf7640 1 -- 172.21.15.37:0/372574412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 msgr2=0x5577a3217a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3217a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c1cf7640 1 -- 172.21.15.37:0/372574412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320adc0 msgr2=0x5577a3215140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320adc0 0x5577a3215140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.238+0000 7f18c1cf7640 1 -- 172.21.15.37:0/372574412 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a30fc440 con 0x5577a320a4a0 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.239+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320a4a0 0x5577a320a880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009bf0 tx=0x7f18b4005ef0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32b3e93a385b11b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.239+0000 7f18b27fc640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18a80a33b0 0x7f18a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.239+0000 7f18a77fe640 1 -- 172.21.15.37:0/372574412 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4031db0 con 0x5577a320a4a0 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.239+0000 7f18c14f6640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18a80a33b0 0x7f18a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.239+0000 7f18a77fe640 1 -- 172.21.15.37:0/372574412 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b4036040 con 0x5577a320a4a0 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.239+0000 7f18a77fe640 1 -- 172.21.15.37:0/372574412 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40042a0 con 0x5577a320a4a0 2024-09-19T06:51:32.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.239+0000 7f18b27fc640 1 -- 172.21.15.37:0/1337264740 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f18b80d0140 con 0x7f18bc100b20 2024-09-19T06:51:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 -- 172.21.15.37:0/372574412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320a4a0 msgr2=0x5577a320a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c14f6640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18a80a33b0 0x7f18a80a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18b80988f0 tx=0x7f18b80d3d20 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320a4a0 0x5577a320a880 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f18b4009bf0 tx=0x7f18b4005ef0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 -- 172.21.15.37:0/372574412 shutdown_connections 2024-09-19T06:51:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3217a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320adc0 0x5577a3215140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 --2- 172.21.15.37:0/372574412 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320a4a0 0x5577a320a880 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 -- 172.21.15.37:0/372574412 >> 172.21.15.37:0/372574412 conn(0x5577a3206a30 msgr2=0x5577a3206e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 -- 172.21.15.37:0/372574412 shutdown_connections 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 -- 172.21.15.37:0/372574412 wait complete. 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 -- start start 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.240+0000 7f18c5b33740 1 Processor -- start 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3214f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a3212220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3212760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c5b33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5577a32180d0 con 0x5577a320a4a0 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c5b33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5577a312b050 con 0x5577a3215680 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c5b33740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5577a3218250 con 0x5577a320adc0 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c5b33740 10 client.0 _ll_register_callbacks cb 0x5577a316b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3214f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3214f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39498/0 (socket says 172.21.15.37:39498) 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c1cf7640 1 -- 172.21.15.37:0/1938127014 learned_addr learned my addr 172.21.15.37:0/1938127014 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a3212220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3212760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c0cf5640 1 -- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 msgr2=0x5577a3212760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3212760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c0cf5640 1 -- 172.21.15.37:0/1938127014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 msgr2=0x5577a3214f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3214f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c0cf5640 1 -- 172.21.15.37:0/1938127014 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a2ee13c0 con 0x5577a320adc0 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3214f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a3212220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18bc10e920 tx=0x7f18ac012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ec76589ee9b664d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.241+0000 7f18a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1938127014 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac011c40 con 0x5577a320adc0 2024-09-19T06:51:32.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1938127014 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f18ac011de0 con 0x5577a320adc0 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18b4036040 con 0x5577a320adc0 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1938127014 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18ac004020 con 0x5577a320adc0 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 10 client.27265 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5577a3223aa0 con 0x5577a320adc0 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1938127014 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18b4036040 con 0x5577a320adc0 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1938127014 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f18ac01c470 con 0x5577a320adc0 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 10 client.27265 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 10 client.27265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 20 client.27265 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a3224fb0 0x5577a32273d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 4 client.27265 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.242+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577a317d850 con 0x5577a3224fb0 2024-09-19T06:51:32.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.243+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a3224fb0 0x5577a32273d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.243+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a3224fb0 0x5577a32273d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e88916b85e44bb7e server_cookie=e4f0f9aa7d6e27c2 in_seq=0 out_seq=0 2024-09-19T06:51:32.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.243+0000 7f18a4ff9640 10 client.27265 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:32.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1938127014 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5577a317d850 con 0x5577a3224fb0 2024-09-19T06:51:32.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18a4ff9640 10 client.27265 handle_command_reply: tid=0 2024-09-19T06:51:32.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 client.27265 shutdown 2024-09-19T06:51:32.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 msgr2=0x5577a3212220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a3212220 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f18bc10e920 tx=0x7f18ac012b60 comp rx=0 tx=0).stop 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 shutdown_connections 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a3224fb0 0x5577a32273d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a3212760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18a4ff9640 0 client.27265 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a3212220 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1938127014 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3214f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 >> 172.21.15.37:0/1938127014 conn(0x5577a3206a30 msgr2=0x5577a3205b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 shutdown_connections 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 1 -- 172.21.15.37:0/1938127014 wait complete. 2024-09-19T06:51:32.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.244+0000 7f18c5b33740 20 client.27265 trim_cache size 0 max 16384 2024-09-19T06:51:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 Processor -- start 2024-09-19T06:51:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 -- start start 2024-09-19T06:51:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3221330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a322cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a322d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1b2390 con 0x5577a320a4a0 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1afc60 con 0x5577a3215680 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c5b33740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1adad0 con 0x5577a320adc0 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3221330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c0cf5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a322cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3221330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39500/0 (socket says 172.21.15.37:39500) 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c1cf7640 1 -- 172.21.15.37:0/2770024489 learned_addr learned my addr 172.21.15.37:0/2770024489 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a322d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c0cf5640 1 -- 172.21.15.37:0/2770024489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 msgr2=0x5577a322d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a322d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c0cf5640 1 -- 172.21.15.37:0/2770024489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 msgr2=0x5577a3221330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3221330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c0cf5640 1 -- 172.21.15.37:0/2770024489 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577a320a2d0 con 0x5577a320adc0 2024-09-19T06:51:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.262+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3221330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.263+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a322cca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac01cc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d8c6513aa4e81ae7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.263+0000 7f18a6ffd640 1 -- 172.21.15.37:0/2770024489 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac028040 con 0x5577a320adc0 2024-09-19T06:51:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.263+0000 7f18a6ffd640 1 -- 172.21.15.37:0/2770024489 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18ac005520 con 0x5577a320adc0 2024-09-19T06:51:32.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.263+0000 7f18a6ffd640 1 -- 172.21.15.37:0/2770024489 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0041c0 con 0x5577a320adc0 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.263+0000 7f18c5b33740 1 -- 172.21.15.37:0/2770024489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 msgr2=0x5577a322cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.263+0000 7f18c5b33740 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a322cca0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f18ac000c00 tx=0x7f18ac01cc90 comp rx=0 tx=0).stop 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- 172.21.15.37:0/2770024489 shutdown_connections 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a3215680 0x5577a322d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a320adc0 0x5577a322cca0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 --2- 172.21.15.37:0/2770024489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a3221330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- 172.21.15.37:0/2770024489 >> 172.21.15.37:0/2770024489 conn(0x5577a3206a30 msgr2=0x5577a3211310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- 172.21.15.37:0/2770024489 shutdown_connections 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- 172.21.15.37:0/2770024489 wait complete. 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- start start 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 Processor -- start 2024-09-19T06:51:32.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a321dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a320adc0 0x5577a3218f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a3215680 0x5577a3219460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1ae120 con 0x5577a320a4a0 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1af590 con 0x5577a320adc0 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.264+0000 7f18c5b33740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f18bc1ae7f0 con 0x5577a3215680 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c5b33740 10 client.0 _ll_register_callbacks cb 0x7f18ac0097e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a321dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a320adc0 0x5577a3218f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a320adc0 0x5577a3218f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:47450/0 (socket says 172.21.15.37:47450) 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 -- 172.21.15.37:0/1976764633 learned_addr learned my addr 172.21.15.37:0/1976764633 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c1cf7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a321dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39512/0 (socket says 172.21.15.37:39512) 2024-09-19T06:51:32.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a3215680 0x5577a3219460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 -- 172.21.15.37:0/1976764633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a3215680 msgr2=0x5577a3219460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a3215680 0x5577a3219460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 -- 172.21.15.37:0/1976764633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 msgr2=0x5577a321dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a321dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 -- 172.21.15.37:0/1976764633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577a32076c0 con 0x5577a320adc0 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a3215680 0x5577a3219460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18c0cf5640 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a320adc0 0x5577a3218f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18ac012ba0 tx=0x7f18ac01c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5316f793c555a89c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18a4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.265+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1976764633 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac039020 con 0x5577a320adc0 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac005520 con 0x5577a320adc0 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1976764633 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f18ac011c40 con 0x5577a320adc0 2024-09-19T06:51:32.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1976764633 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f18ac044050 con 0x5577a320adc0 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 10 client.27530 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5577a3207ae0 con 0x5577a320adc0 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1976764633 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f18ac005520 con 0x5577a320adc0 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1976764633 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f18ac02b070 con 0x5577a320adc0 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 10 client.27530 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 10 client.27530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 20 client.27530 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a328ffb0 0x5577a32923d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 4 client.27530 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5577a3177d60 con 0x5577a328ffb0 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.266+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a328ffb0 0x5577a32923d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18a4ff9640 10 client.27530 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:32.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c1cf7640 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a328ffb0 0x5577a32923d0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6267d84915b38d49 server_cookie=c9848c789435a387 in_seq=0 out_seq=0 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18a4ff9640 1 -- 172.21.15.37:0/1976764633 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5577a3177d60 con 0x5577a328ffb0 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18a4ff9640 10 client.27530 handle_command_reply: tid=0 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 client.27530 shutdown 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a320adc0 msgr2=0x5577a3218f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a320adc0 0x5577a3218f20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f18ac012ba0 tx=0x7f18ac01c990 comp rx=0 tx=0).stop 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 shutdown_connections 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5577a328ffb0 0x5577a32923d0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5577a3215680 0x5577a3219460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5577a320adc0 0x5577a3218f20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18a4ff9640 0 client.27530 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 --2- 172.21.15.37:0/1976764633 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5577a320a4a0 0x5577a321dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.267+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 >> 172.21.15.37:0/1976764633 conn(0x5577a3206a30 msgr2=0x5577a3213a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 shutdown_connections 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c5b33740 1 -- 172.21.15.37:0/1976764633 wait complete. 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c5b33740 20 client.27530 trim_cache size 0 max 16384 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stdout: "5ed6b459-9b9a-4e8d-8043-07c8e545d165": { 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "5ed6b459-9b9a-4e8d-8043-07c8e545d165", 2024-09-19T06:51:32.275 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:51:32.276 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:51:32.276 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:51:32.276 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:51:32.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18a80a33b0 msgr2=0x7f18a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18a80a33b0 0x7f18a80a5870 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f18b80988f0 tx=0x7f18b80d3d20 comp rx=0 tx=0).stop 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc100b20 msgr2=0x7f18bc19a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc100b20 0x7f18bc19a6f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f18b800e9f0 tx=0x7f18b800edf0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 shutdown_connections 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f18a80a33b0 0x7f18a80a5870 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f18bc10e500 0x7f18bc19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f18bc101440 0x7f18bc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 --2- 172.21.15.37:0/1337264740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f18bc100b20 0x7f18bc19a6f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 >> 172.21.15.37:0/1337264740 conn(0x7f18bc0fc7d0 msgr2=0x7f18bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 shutdown_connections 2024-09-19T06:51:32.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:32.268+0000 7f18c375a640 1 -- 172.21.15.37:0/1337264740 wait complete. 2024-09-19T06:51:32.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:31 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:51:32.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:31 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:32.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:31 smithi096 ceph-mon[35004]: pgmap v305: 97 pgs: 97 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 286 op/s 2024-09-19T06:51:32.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:31 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:51:32.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:31 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:51:32.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:31 smithi151 ceph-mon[35236]: pgmap v305: 97 pgs: 97 active+clean; 71 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 286 op/s 2024-09-19T06:51:32.982 INFO:tasks.cephfs.filesystem:scrub status for tag:5ed6b459-9b9a-4e8d-8043-07c8e545d165 - {'path': '/', 'tag': '5ed6b459-9b9a-4e8d-8043-07c8e545d165', 'options': 'recursive,force'} 2024-09-19T06:51:33.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 767032 8.17 MB/sec execute 360 sec latency 14.001 ms 2024-09-19T06:51:34.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 768721 8.16 MB/sec execute 361 sec latency 14.280 ms 2024-09-19T06:51:34.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:33 smithi096 ceph-mon[35004]: pgmap v306: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 390 op/s 2024-09-19T06:51:34.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:33 smithi037 ceph-mon[31211]: pgmap v306: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 390 op/s 2024-09-19T06:51:34.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:33 smithi151 ceph-mon[35236]: pgmap v306: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.8 MiB/s wr, 390 op/s 2024-09-19T06:51:35.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 770902 8.18 MB/sec execute 362 sec latency 23.791 ms 2024-09-19T06:51:36.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 772978 8.18 MB/sec execute 363 sec latency 54.655 ms 2024-09-19T06:51:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:35 smithi096 ceph-mon[35004]: pgmap v307: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-19T06:51:36.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:35 smithi037 ceph-mon[31211]: pgmap v307: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-19T06:51:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:35 smithi151 ceph-mon[35236]: pgmap v307: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-19T06:51:37.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 774650 8.19 MB/sec execute 364 sec latency 28.156 ms 2024-09-19T06:51:37.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:36 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:51:37.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:36 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:51:37.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:36 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:51:38.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 777556 8.21 MB/sec execute 365 sec latency 15.571 ms 2024-09-19T06:51:38.263 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:37 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:51:38.264 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:37 smithi151 ceph-mon[35236]: pgmap v308: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-19T06:51:38.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:37 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:51:38.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:37 smithi096 ceph-mon[35004]: pgmap v308: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-19T06:51:38.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:37 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:51:38.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:37 smithi037 ceph-mon[31211]: pgmap v308: 97 pgs: 97 active+clean; 77 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 292 op/s 2024-09-19T06:51:39.223 INFO:tasks.workunit.client.0.smithi037.stdout: 1 778870 8.20 MB/sec execute 366 sec latency 14.001 ms 2024-09-19T06:51:39.792 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:51:39.793 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T06:51:39.793 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:51:40.069 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:40.110 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 -- 172.21.15.37:0/3379172081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69681009d0 msgr2=0x7f6968100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 --2- 172.21.15.37:0/3379172081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69681009d0 0x7f6968100e30 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f695c009930 tx=0x7f695c02a420 comp rx=0 tx=0).stop 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 -- 172.21.15.37:0/3379172081 shutdown_connections 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 --2- 172.21.15.37:0/3379172081 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6968101370 0x7f696810d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 --2- 172.21.15.37:0/3379172081 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f69681009d0 0x7f6968100e30 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 --2- 172.21.15.37:0/3379172081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69681069e0 0x7f6968106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 -- 172.21.15.37:0/3379172081 >> 172.21.15.37:0/3379172081 conn(0x7f69680fc6b0 msgr2=0x7f69680fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 -- 172.21.15.37:0/3379172081 shutdown_connections 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.106+0000 7f696c978640 1 -- 172.21.15.37:0/3379172081 wait complete. 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 Processor -- start 2024-09-19T06:51:40.111 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 -- start start 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69681009d0 0x7f6968199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968101370 0x7f6968199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69681069e0 0x7f69681a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6968110e20 con 0x7f6968101370 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6968110ca0 con 0x7f69681009d0 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696c978640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6968110fa0 con 0x7f69681069e0 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f6965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968101370 0x7f6968199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f6965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968101370 0x7f6968199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54576/0 (socket says 172.21.15.37:54576) 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f6965d9b640 1 -- 172.21.15.37:0/1085027444 learned_addr learned my addr 172.21.15.37:0/1085027444 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.107+0000 7f696659c640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69681009d0 0x7f6968199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f6966d9d640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69681069e0 0x7f69681a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.112 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f6965d9b640 1 -- 172.21.15.37:0/1085027444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69681069e0 msgr2=0x7f69681a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f6965d9b640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69681069e0 0x7f69681a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f6965d9b640 1 -- 172.21.15.37:0/1085027444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69681009d0 msgr2=0x7f6968199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f6965d9b640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69681009d0 0x7f6968199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f6965d9b640 1 -- 172.21.15.37:0/1085027444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69681a47d0 con 0x7f6968101370 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f696659c640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69681009d0 0x7f6968199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f6965d9b640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968101370 0x7f6968199750 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f695c009230 tx=0x7f695c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f69577fe640 1 -- 172.21.15.37:0/1085027444 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c03f030 con 0x7f6968101370 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f69577fe640 1 -- 172.21.15.37:0/1085027444 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695c004440 con 0x7f6968101370 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69681a49a0 con 0x7f6968101370 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f69577fe640 1 -- 172.21.15.37:0/1085027444 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c035070 con 0x7f6968101370 2024-09-19T06:51:40.113 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.108+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69681a4e60 con 0x7f6968101370 2024-09-19T06:51:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.110+0000 7f69577fe640 1 -- 172.21.15.37:0/1085027444 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f695c042030 con 0x7f6968101370 2024-09-19T06:51:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 Processor -- start 2024-09-19T06:51:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 -- start start 2024-09-19T06:51:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405dfaf0 0x5648405dfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ea770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405eacb0 0x5648405ed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648405442f0 con 0x5648405e0410 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5648404ec270 con 0x5648405dfaf0 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696f7f0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648405402b0 con 0x5648405eacb0 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ea770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f6966d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405dfaf0 0x5648405dfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f6966d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405dfaf0 0x5648405dfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43010/0 (socket says 172.21.15.37:43010) 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f696659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ea770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54592/0 (socket says 172.21.15.37:54592) 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f6966d9d640 1 -- 172.21.15.37:0/1620956240 learned_addr learned my addr 172.21.15.37:0/1620956240 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.111+0000 7f6965d9b640 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405eacb0 0x5648405ed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f69577fe640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f69500a34e0 0x7f69500a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f696659c640 1 -- 172.21.15.37:0/1620956240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405eacb0 msgr2=0x5648405ed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f696659c640 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405eacb0 0x5648405ed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f696659c640 1 -- 172.21.15.37:0/1620956240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405dfaf0 msgr2=0x5648405dfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f696659c640 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405dfaf0 0x5648405dfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f696659c640 1 -- 172.21.15.37:0/1620956240 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648404eb340 con 0x5648405e0410 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f6966d9d640 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405dfaf0 0x5648405dfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f6966d9d640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f69500a34e0 0x7f69500a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f69577fe640 1 -- 172.21.15.37:0/1085027444 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f695c0f0070 con 0x7f6968101370 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f696659c640 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ea770 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6958009f50 tx=0x7f6958013c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15f2608f5fb6353 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f6954ff9640 1 -- 172.21.15.37:0/1620956240 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958004030 con 0x5648405e0410 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f6954ff9640 1 -- 172.21.15.37:0/1620956240 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695800ee30 con 0x5648405e0410 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.112+0000 7f6954ff9640 1 -- 172.21.15.37:0/1620956240 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958003d30 con 0x5648405e0410 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 -- 172.21.15.37:0/1620956240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 msgr2=0x5648405ea770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f6966d9d640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f69500a34e0 0x7f69500a59a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69600107f0 tx=0x7f6960004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ea770 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f6958009f50 tx=0x7f6958013c60 comp rx=0 tx=0).stop 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 -- 172.21.15.37:0/1620956240 shutdown_connections 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405eacb0 0x5648405ed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ea770 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1620956240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405dfaf0 0x5648405dfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 -- 172.21.15.37:0/1620956240 >> 172.21.15.37:0/1620956240 conn(0x5648405dc060 msgr2=0x5648405dc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 -- 172.21.15.37:0/1620956240 shutdown_connections 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 -- 172.21.15.37:0/1620956240 wait complete. 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.113+0000 7f696f7f0740 1 -- start start 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 1 Processor -- start 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405e7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405e80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405e8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5648405ed880 con 0x5648405e0410 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5648404cd290 con 0x5648405eacb0 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5648405eda00 con 0x5648405dfaf0 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696f7f0740 10 client.0 _ll_register_callbacks cb 0x5648404fe140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f6965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405e80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f6966d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405e8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f6966d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405e8620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43020/0 (socket says 172.21.15.37:43020) 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f6965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405e80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54596/0 (socket says 172.21.15.37:54596) 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f6966d9d640 1 -- 172.21.15.37:0/1406966378 learned_addr learned my addr 172.21.15.37:0/1406966378 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.114+0000 7f696659c640 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405e7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f6966d9d640 1 -- 172.21.15.37:0/1406966378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 msgr2=0x5648405e7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f6966d9d640 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405e7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.123 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f6966d9d640 1 -- 172.21.15.37:0/1406966378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 msgr2=0x5648405e80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f6966d9d640 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405e80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f6966d9d640 1 -- 172.21.15.37:0/1406966378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564840257210 con 0x5648405eacb0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f6965d9b640 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f6966d9d640 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405e8620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6960013d10 tx=0x7f6960004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=59162cf2196a35ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f694dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f694dffb640 1 -- 172.21.15.37:0/1406966378 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6960003e00 con 0x5648405eacb0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f695800ee30 con 0x5648405eacb0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f694dffb640 1 -- 172.21.15.37:0/1406966378 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f6960004550 con 0x5648405eacb0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f694dffb640 1 -- 172.21.15.37:0/1406966378 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f6960030050 con 0x5648405eacb0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f696f7f0740 10 client.27542 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564840323620 con 0x5648405eacb0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.115+0000 7f694dffb640 1 -- 172.21.15.37:0/1406966378 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f695800ee30 con 0x5648405eacb0 2024-09-19T06:51:40.124 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f694dffb640 1 -- 172.21.15.37:0/1406966378 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f6960020400 con 0x5648405eacb0 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696f7f0740 10 client.27542 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696f7f0740 10 client.27542 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696f7f0740 20 client.27542 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648405fac60 0x5648405fd070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696f7f0740 4 client.27542 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564840551a20 con 0x5648405fac60 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696659c640 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648405fac60 0x5648405fd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f694dffb640 10 client.27542 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:40.125 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.116+0000 7f696659c640 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648405fac60 0x5648405fd070 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6292998482f0db28 server_cookie=b5ba7d5bf6111831 in_seq=0 out_seq=0 2024-09-19T06:51:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f694dffb640 1 -- 172.21.15.37:0/1406966378 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564840551a20 con 0x5648405fac60 2024-09-19T06:51:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f694dffb640 10 client.27542 handle_command_reply: tid=0 2024-09-19T06:51:40.126 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 client.27542 shutdown 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 msgr2=0x5648405e8620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405e8620 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6960013d10 tx=0x7f6960004ab0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 shutdown_connections 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5648405fac60 0x5648405fd070 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405e8620 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405e80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f694dffb640 0 client.27542 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 --2- 172.21.15.37:0/1406966378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405e7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 >> 172.21.15.37:0/1406966378 conn(0x5648405dc060 msgr2=0x5648405e1dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 shutdown_connections 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.118+0000 7f696f7f0740 1 -- 172.21.15.37:0/1406966378 wait complete. 2024-09-19T06:51:40.127 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.119+0000 7f696f7f0740 20 client.27542 trim_cache size 0 max 16384 2024-09-19T06:51:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.133+0000 7f696f7f0740 1 Processor -- start 2024-09-19T06:51:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.133+0000 7f696f7f0740 1 -- start start 2024-09-19T06:51:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405dfaf0 0x5648405e25b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405e0410 0x5648405e8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648406029d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696f7f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69681adfa0 con 0x5648405dfaf0 2024-09-19T06:51:40.136 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696f7f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ab870 con 0x5648405eacb0 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696f7f0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69681a96e0 con 0x5648405e0410 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405dfaf0 0x5648405e25b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405dfaf0 0x5648405e25b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54600/0 (socket says 172.21.15.37:54600) 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 -- 172.21.15.37:0/213573574 learned_addr learned my addr 172.21.15.37:0/213573574 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f6965d9b640 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405e0410 0x5648405e8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f6966d9d640 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648406029d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 -- 172.21.15.37:0/213573574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405e0410 msgr2=0x5648405e8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405e0410 0x5648405e8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 -- 172.21.15.37:0/213573574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 msgr2=0x5648406029d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648406029d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 -- 172.21.15.37:0/213573574 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648405f9850 con 0x5648405dfaf0 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f696659c640 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405dfaf0 0x5648405e25b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6958009f50 tx=0x7f6958013c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0440662630240e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.137 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.134+0000 7f694ffff640 1 -- 172.21.15.37:0/213573574 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695800e9d0 con 0x5648405dfaf0 2024-09-19T06:51:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f694ffff640 1 -- 172.21.15.37:0/213573574 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6958005500 con 0x5648405dfaf0 2024-09-19T06:51:40.138 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f694ffff640 1 -- 172.21.15.37:0/213573574 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6958004910 con 0x5648405dfaf0 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 -- 172.21.15.37:0/213573574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405dfaf0 msgr2=0x5648405e25b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405dfaf0 0x5648405e25b0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f6958009f50 tx=0x7f6958013c60 comp rx=0 tx=0).stop 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 -- 172.21.15.37:0/213573574 shutdown_connections 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648406029d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405e0410 0x5648405e8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 --2- 172.21.15.37:0/213573574 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405dfaf0 0x5648405e25b0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 -- 172.21.15.37:0/213573574 >> 172.21.15.37:0/213573574 conn(0x5648405dc060 msgr2=0x5648405e9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 -- 172.21.15.37:0/213573574 shutdown_connections 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 -- 172.21.15.37:0/213573574 wait complete. 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.135+0000 7f696f7f0740 1 -- start start 2024-09-19T06:51:40.141 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 1 Processor -- start 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405f3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f69681a9d30 con 0x5648405e0410 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f69681ab1a0 con 0x5648405eacb0 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f69681aa400 con 0x5648405dfaf0 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696f7f0740 10 client.0 _ll_register_callbacks cb 0x7f6960010d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ee580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54602/0 (socket says 172.21.15.37:54602) 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405f3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 -- 172.21.15.37:0/3566933497 learned_addr learned my addr 172.21.15.37:0/3566933497 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6966d9d640 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.142 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 -- 172.21.15.37:0/3566933497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 msgr2=0x5648405f3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405f3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 -- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 msgr2=0x5648405eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405eeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 -- 172.21.15.37:0/3566933497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648405ef160 con 0x5648405e0410 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f696659c640 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f6965d9b640 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ee580 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f695c0edd90 tx=0x7f695c0045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f37989e5948019 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.136+0000 7f694dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f694dffb640 1 -- 172.21.15.37:0/3566933497 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f695c049040 con 0x5648405e0410 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f694dffb640 1 -- 172.21.15.37:0/3566933497 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f695c031bd0 con 0x5648405e0410 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6958005500 con 0x5648405e0410 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f694dffb640 1 -- 172.21.15.37:0/3566933497 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f695c0bbe60 con 0x5648405e0410 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 10 client.18207 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x564840665880 con 0x5648405e0410 2024-09-19T06:51:40.143 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f694dffb640 1 -- 172.21.15.37:0/3566933497 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6958005500 con 0x5648405e0410 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f694dffb640 1 -- 172.21.15.37:0/3566933497 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f695c0d50f0 con 0x5648405e0410 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 10 client.18207 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 10 client.18207 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 20 client.18207 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564840666080 0x564840668470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 4 client.18207 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56484054bd70 con 0x564840666080 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.137+0000 7f696659c640 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564840666080 0x564840668470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.138+0000 7f696659c640 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564840666080 0x564840668470 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353fe17694d789ac server_cookie=3884f3475eeca671 in_seq=0 out_seq=0 2024-09-19T06:51:40.144 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.138+0000 7f694dffb640 10 client.18207 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:40.197 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:39 smithi037 ceph-mon[31211]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 384 op/s 2024-09-19T06:51:40.224 INFO:tasks.workunit.client.0.smithi037.stdout: 1 781253 8.22 MB/sec execute 367 sec latency 84.712 ms 2024-09-19T06:51:40.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:39 smithi096 ceph-mon[35004]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 384 op/s 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.304+0000 7f694dffb640 1 -- 172.21.15.37:0/3566933497 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56484054bd70 con 0x564840666080 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.304+0000 7f694dffb640 10 client.18207 handle_command_reply: tid=0 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.304+0000 7f696f7f0740 1 client.18207 shutdown 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.304+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 msgr2=0x5648405ee580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.304+0000 7f696f7f0740 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ee580 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f695c0edd90 tx=0x7f695c0045f0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 shutdown_connections 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x564840666080 0x564840668470 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.307 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f694dffb640 0 client.18207 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648405eacb0 0x5648405eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5648405e0410 0x5648405ee580 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 --2- 172.21.15.37:0/3566933497 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5648405dfaf0 0x5648405f3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 >> 172.21.15.37:0/3566933497 conn(0x5648405dc060 msgr2=0x5648405db1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 shutdown_connections 2024-09-19T06:51:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 1 -- 172.21.15.37:0/3566933497 wait complete. 2024-09-19T06:51:40.308 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.305+0000 7f696f7f0740 20 client.18207 trim_cache size 0 max 16384 2024-09-19T06:51:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:51:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:51:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:51:40.309 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.306+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f69500a34e0 msgr2=0x7f69500a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.306+0000 7f696c978640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f69500a34e0 0x7f69500a59a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f69600107f0 tx=0x7f6960004a40 comp rx=0 tx=0).stop 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.306+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968101370 msgr2=0x7f6968199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.306+0000 7f696c978640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968101370 0x7f6968199750 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f695c009230 tx=0x7f695c004040 comp rx=0 tx=0).stop 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 shutdown_connections 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f69681069e0 0x7f69681a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f69500a34e0 0x7f69500a59a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6968101370 0x7f6968199750 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 --2- 172.21.15.37:0/1085027444 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f69681009d0 0x7f6968199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 >> 172.21.15.37:0/1085027444 conn(0x7f69680fc6b0 msgr2=0x7f69680fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.310 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 shutdown_connections 2024-09-19T06:51:40.311 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:40.307+0000 7f696c978640 1 -- 172.21.15.37:0/1085027444 wait complete. 2024-09-19T06:51:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:39 smithi151 ceph-mon[35236]: pgmap v309: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 4.7 MiB/s wr, 384 op/s 2024-09-19T06:51:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 -- 172.21.15.37:0/4250933619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4102d90 msgr2=0x7f5bb4103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.573 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/4250933619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4102d90 0x7f5bb4103170 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009930 tx=0x7f5ba402a420 comp rx=0 tx=0).stop 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 -- 172.21.15.37:0/4250933619 shutdown_connections 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/4250933619 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4108290 0x7f5bb410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/4250933619 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb4103740 0x7f5bb4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/4250933619 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4102d90 0x7f5bb4103170 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 -- 172.21.15.37:0/4250933619 >> 172.21.15.37:0/4250933619 conn(0x7f5bb40fc7d0 msgr2=0x7f5bb40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 -- 172.21.15.37:0/4250933619 shutdown_connections 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.543+0000 7f5bbaa53640 1 -- 172.21.15.37:0/4250933619 wait complete. 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.544+0000 7f5bbaa53640 1 Processor -- start 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.544+0000 7f5bbaa53640 1 -- start start 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.544+0000 7f5bbaa53640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb4102d90 0x7f5bb41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.544+0000 7f5bbaa53640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4103740 0x7f5bb4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bbaa53640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4108290 0x7f5bb419cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bbaa53640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4112090 con 0x7f5bb4108290 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bbaa53640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4111f10 con 0x7f5bb4103740 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bbaa53640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bb4112210 con 0x7f5bb4102d90 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4103740 0x7f5bb4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb8ff0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4108290 0x7f5bb419cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4103740 0x7f5bb4198b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43050/0 (socket says 172.21.15.37:43050) 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 -- 172.21.15.37:0/3751503095 learned_addr learned my addr 172.21.15.37:0/3751503095 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb3fff640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb4102d90 0x7f5bb41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 -- 172.21.15.37:0/3751503095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb4102d90 msgr2=0x7f5bb41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb4102d90 0x7f5bb41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 -- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4108290 msgr2=0x7f5bb419cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4108290 0x7f5bb419cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 -- 172.21.15.37:0/3751503095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bb419d5a0 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb37fe640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4103740 0x7f5bb4198b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800d740 tx=0x7f5ba800db40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3751503095 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8017070 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3751503095 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ba8009820 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.545+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bb419d890 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.546+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3751503095 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8004e00 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.546+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bb4100920 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.547+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bb406a7f0 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.548+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3751503095 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5ba80099d0 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.549+0000 7f5bb17fa640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5b9c0a33b0 0x7f5b9c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.549+0000 7f5bb3fff640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5b9c0a33b0 0x7f5b9c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.549+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3751503095 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f5ba80cfa50 con 0x7f5bb4103740 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.549+0000 7f5bb3fff640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5b9c0a33b0 0x7f5b9c0a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009900 tx=0x7f5ba40058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:40.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.552+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3751503095 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5ba809c400 con 0x7f5bb4103740 2024-09-19T06:51:40.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.824+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5bb4199950 con 0x7f5bb4103740 2024-09-19T06:51:40.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.826+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3751503095 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7f5ba809bd00 con 0x7f5bb4103740 2024-09-19T06:51:40.828 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:51:40.828 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:51:40.828 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5b9c0a33b0 msgr2=0x7f5b9c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5b9c0a33b0 0x7f5b9c0a5870 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4009900 tx=0x7f5ba40058d0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4103740 msgr2=0x7f5bb4198b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4103740 0x7f5bb4198b30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f5ba800d740 tx=0x7f5ba800db40 comp rx=0 tx=0).stop 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 shutdown_connections 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5b9c0a33b0 0x7f5b9c0a5870 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bb4108290 0x7f5bb419cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bb4103740 0x7f5bb4198b30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.830 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 --2- 172.21.15.37:0/3751503095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bb4102d90 0x7f5bb41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 >> 172.21.15.37:0/3751503095 conn(0x7f5bb40fc7d0 msgr2=0x7f5bb4105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 shutdown_connections 2024-09-19T06:51:40.831 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:40.828+0000 7f5bbaa53640 1 -- 172.21.15.37:0/3751503095 wait complete. 2024-09-19T06:51:41.224 INFO:tasks.workunit.client.0.smithi037.stdout: 1 782574 8.21 MB/sec execute 368 sec latency 115.849 ms 2024-09-19T06:51:41.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:40 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3751503095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:41.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:40 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3751503095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:41.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:40 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3751503095' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:51:41.710 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T06:51:41.710 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T06:51:42.022 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:51:42.224 INFO:tasks.workunit.client.0.smithi037.stdout: 1 784841 8.23 MB/sec execute 369 sec latency 18.262 ms 2024-09-19T06:51:42.254 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:41 smithi037 ceph-mon[31211]: pgmap v310: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-19T06:51:42.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:41 smithi096 ceph-mon[35004]: pgmap v310: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-19T06:51:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:41 smithi151 ceph-mon[35236]: pgmap v310: 97 pgs: 97 active+clean; 84 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.1 MiB/s wr, 196 op/s 2024-09-19T06:51:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 -- 172.21.15.37:0/201064358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c100e20 msgr2=0x7f421c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 --2- 172.21.15.37:0/201064358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c100e20 0x7f421c101200 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f420c009900 tx=0x7f420c02a480 comp rx=0 tx=0).stop 2024-09-19T06:51:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 -- 172.21.15.37:0/201064358 shutdown_connections 2024-09-19T06:51:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 --2- 172.21.15.37:0/201064358 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f421c102ef0 0x7f421c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 --2- 172.21.15.37:0/201064358 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c102570 0x7f421c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 --2- 172.21.15.37:0/201064358 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c100e20 0x7f421c101200 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 -- 172.21.15.37:0/201064358 >> 172.21.15.37:0/201064358 conn(0x7f421c0fc7d0 msgr2=0x7f421c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 -- 172.21.15.37:0/201064358 shutdown_connections 2024-09-19T06:51:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 -- 172.21.15.37:0/201064358 wait complete. 2024-09-19T06:51:42.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.468+0000 7f4221efe640 1 Processor -- start 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f4221efe640 1 -- start start 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f4221efe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c100e20 0x7f421c197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f4221efe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c102570 0x7f421c197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f4221efe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f421c102ef0 0x7f421c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f4221efe640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f421c110f50 con 0x7f421c102570 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f4221efe640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f421c110dd0 con 0x7f421c100e20 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f4221efe640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1110d0 con 0x7f421c102ef0 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f421affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c102570 0x7f421c197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c100e20 0x7f421c197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f421affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c102570 0x7f421c197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54640/0 (socket says 172.21.15.37:54640) 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c100e20 0x7f421c197410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43072/0 (socket says 172.21.15.37:43072) 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f421affd640 1 -- 172.21.15.37:0/971681137 learned_addr learned my addr 172.21.15.37:0/971681137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:42.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.469+0000 7f421bfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f421c102ef0 0x7f421c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f421affd640 1 -- 172.21.15.37:0/971681137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f421c102ef0 msgr2=0x7f421c19bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f421affd640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f421c102ef0 0x7f421c19bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f421affd640 1 -- 172.21.15.37:0/971681137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c100e20 msgr2=0x7f421c197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f421affd640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c100e20 0x7f421c197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f421affd640 1 -- 172.21.15.37:0/971681137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f421c19c3c0 con 0x7f421c102570 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f421b7fe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c100e20 0x7f421c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f421affd640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c102570 0x7f421c197950 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f421000eb70 tx=0x7f421000b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f4218ff9640 1 -- 172.21.15.37:0/971681137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421000be60 con 0x7f421c102570 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f4218ff9640 1 -- 172.21.15.37:0/971681137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42100035f0 con 0x7f421c102570 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f421c19c6b0 con 0x7f421c102570 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f4218ff9640 1 -- 172.21.15.37:0/971681137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42100146e0 con 0x7f421c102570 2024-09-19T06:51:42.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.470+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f421c069f70 con 0x7f421c102570 2024-09-19T06:51:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.473+0000 7f4218ff9640 1 -- 172.21.15.37:0/971681137 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f42100037a0 con 0x7f421c102570 2024-09-19T06:51:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.473+0000 7f42242d7740 1 Processor -- start 2024-09-19T06:51:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.473+0000 7f42242d7740 1 -- start start 2024-09-19T06:51:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.473+0000 7f42242d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e64a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.473+0000 7f42242d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64f60 0x563691e73810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f42242d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e73d50 0x563691e76140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f42242d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563691dca2a0 con 0x563691e64640 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f42242d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563691d71b10 con 0x563691e73d50 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f42242d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563691dc6260 con 0x563691e64f60 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e64a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e73d50 0x563691e76140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e64a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54642/0 (socket says 172.21.15.37:54642) 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 -- 172.21.15.37:0/1547730442 learned_addr learned my addr 172.21.15.37:0/1547730442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421b7fe640 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64f60 0x563691e73810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 -- 172.21.15.37:0/1547730442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64f60 msgr2=0x563691e73810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64f60 0x563691e73810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 -- 172.21.15.37:0/1547730442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e73d50 msgr2=0x563691e76140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e73d50 0x563691e76140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 -- 172.21.15.37:0/1547730442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563691b3bb90 con 0x563691e64640 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f4218ff9640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42040a3430 0x7f42040a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421b7fe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42040a3430 0x7f42040a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f4218ff9640 1 -- 172.21.15.37:0/971681137 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f42100d1780 con 0x7f421c102570 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.474+0000 7f421bfff640 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e64a20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f4214007ec0 tx=0x7f421400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6a9f50a45aef7b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.475+0000 7f4201ffb640 1 -- 172.21.15.37:0/1547730442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214010840 con 0x563691e64640 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.475+0000 7f4201ffb640 1 -- 172.21.15.37:0/1547730442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42140183f0 con 0x563691e64640 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.475+0000 7f4201ffb640 1 -- 172.21.15.37:0/1547730442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214017600 con 0x563691e64640 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 -- 172.21.15.37:0/1547730442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 msgr2=0x563691e64a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e64a20 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f4214007ec0 tx=0x7f421400f790 comp rx=0 tx=0).stop 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f421b7fe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42040a3430 0x7f42040a58f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f420c009230 tx=0x7f420c0058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:42.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 -- 172.21.15.37:0/1547730442 shutdown_connections 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e73d50 0x563691e76140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64f60 0x563691e73810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 --2- 172.21.15.37:0/1547730442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e64a20 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 -- 172.21.15.37:0/1547730442 >> 172.21.15.37:0/1547730442 conn(0x563691e60cf0 msgr2=0x563691e610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 -- 172.21.15.37:0/1547730442 shutdown_connections 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 -- 172.21.15.37:0/1547730442 wait complete. 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 -- start start 2024-09-19T06:51:42.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.476+0000 7f42242d7740 1 Processor -- start 2024-09-19T06:51:42.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f42242d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64640 0x563691e70c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f42242d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64f60 0x563691e71150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f42242d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e71690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.483 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f42242d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563691e76920 con 0x563691e64f60 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f42242d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563691d526e0 con 0x563691e64640 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f42242d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563691e76aa0 con 0x563691e73d50 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f42242d7740 10 client.0 _ll_register_callbacks cb 0x563691d76a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64f60 0x563691e71150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64640 0x563691e70c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64f60 0x563691e71150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54648/0 (socket says 172.21.15.37:54648) 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 -- 172.21.15.37:0/4266647664 learned_addr learned my addr 172.21.15.37:0/4266647664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421bfff640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e71690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 -- 172.21.15.37:0/4266647664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 msgr2=0x563691e71690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e71690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 -- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64640 msgr2=0x563691e70c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64640 0x563691e70c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 -- 172.21.15.37:0/4266647664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563691e7fe20 con 0x563691e64f60 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421b7fe640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64640 0x563691e70c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421bfff640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e71690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f421affd640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64f60 0x563691e71150 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4210005860 tx=0x7f42100d2c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fe931f40e00e3b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.477+0000 7f41faffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f41faffd640 1 -- 172.21.15.37:0/4266647664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f421009d440 con 0x563691e64f60 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f41faffd640 1 -- 172.21.15.37:0/4266647664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f42100cf120 con 0x563691e64f60 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42140183f0 con 0x563691e64f60 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f41faffd640 1 -- 172.21.15.37:0/4266647664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f4210009470 con 0x563691e64f60 2024-09-19T06:51:42.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 10 client.18225 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563691e82860 con 0x563691e64f60 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f41faffd640 1 -- 172.21.15.37:0/4266647664 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f42140183f0 con 0x563691e64f60 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f41faffd640 1 -- 172.21.15.37:0/4266647664 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f42100a6210 con 0x563691e64f60 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 10 client.18225 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 10 client.18225 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 20 client.18225 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691e83d20 0x563691e86190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 4 client.18225 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563691dd79d0 con 0x563691e83d20 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.478+0000 7f421b7fe640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691e83d20 0x563691e86190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.479+0000 7f421b7fe640 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691e83d20 0x563691e86190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad2f81c55f50a27 server_cookie=3faf3f4f960f01c0 in_seq=0 out_seq=0 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.479+0000 7f41faffd640 10 client.18225 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f41faffd640 1 -- 172.21.15.37:0/4266647664 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563691dd79d0 con 0x563691e83d20 2024-09-19T06:51:42.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f41faffd640 10 client.18225 handle_command_reply: tid=0 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 client.18225 shutdown 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64f60 msgr2=0x563691e71150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64f60 0x563691e71150 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f4210005860 tx=0x7f42100d2c80 comp rx=0 tx=0).stop 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 shutdown_connections 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691e83d20 0x563691e86190 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e71690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64f60 0x563691e71150 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f41faffd640 0 client.18225 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 --2- 172.21.15.37:0/4266647664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64640 0x563691e70c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 >> 172.21.15.37:0/4266647664 conn(0x563691e60cf0 msgr2=0x563691e66a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 shutdown_connections 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 1 -- 172.21.15.37:0/4266647664 wait complete. 2024-09-19T06:51:42.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.480+0000 7f42242d7740 20 client.18225 trim_cache size 0 max 16384 2024-09-19T06:51:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.497+0000 7f42242d7740 1 Processor -- start 2024-09-19T06:51:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.497+0000 7f42242d7740 1 -- start start 2024-09-19T06:51:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42242d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64640 0x563691e5fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42242d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e8baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42242d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e73d50 0x563691e8bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42242d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1adfe0 con 0x563691e73d50 2024-09-19T06:51:42.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42242d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1ab8b0 con 0x563691e64f60 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42242d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1a9720 con 0x563691e64640 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e8baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64640 0x563691e5fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e8baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43112/0 (socket says 172.21.15.37:43112) 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421affd640 1 -- 172.21.15.37:0/3923419529 learned_addr learned my addr 172.21.15.37:0/3923419529 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421bfff640 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e73d50 0x563691e8bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421bfff640 1 -- 172.21.15.37:0/3923419529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64640 msgr2=0x563691e5fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421bfff640 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64640 0x563691e5fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421bfff640 1 -- 172.21.15.37:0/3923419529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 msgr2=0x563691e8baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421bfff640 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e8baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421bfff640 1 -- 172.21.15.37:0/3923419529 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563691e6fbc0 con 0x563691e73d50 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f421bfff640 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e73d50 0x563691e8bfe0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4214000f80 tx=0x7f4214007dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3dd4b513b864657 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:42.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42017fa640 1 -- 172.21.15.37:0/3923419529 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214026020 con 0x563691e73d50 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.498+0000 7f42017fa640 1 -- 172.21.15.37:0/3923419529 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4214003e50 con 0x563691e73d50 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42017fa640 1 -- 172.21.15.37:0/3923419529 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4214005310 con 0x563691e73d50 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 -- 172.21.15.37:0/3923419529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e73d50 msgr2=0x563691e8bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e73d50 0x563691e8bfe0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f4214000f80 tx=0x7f4214007dc0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 -- 172.21.15.37:0/3923419529 shutdown_connections 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e73d50 0x563691e8bfe0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e8baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 --2- 172.21.15.37:0/3923419529 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e64640 0x563691e5fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 -- 172.21.15.37:0/3923419529 >> 172.21.15.37:0/3923419529 conn(0x563691e60cf0 msgr2=0x563691e5f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 -- 172.21.15.37:0/3923419529 shutdown_connections 2024-09-19T06:51:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 -- 172.21.15.37:0/3923419529 wait complete. 2024-09-19T06:51:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.499+0000 7f42242d7740 1 -- start start 2024-09-19T06:51:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 1 Processor -- start 2024-09-19T06:51:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e7c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e78d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e792d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1a9d70 con 0x563691e64640 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1ab1e0 con 0x563691e64f60 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f421c1aa440 con 0x563691e73d50 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f42242d7740 10 client.0 _ll_register_callbacks cb 0x7f4210003e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e7c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f421b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e7c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:54670/0 (socket says 172.21.15.37:54670) 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f421b7fe640 1 -- 172.21.15.37:0/4256850917 learned_addr learned my addr 172.21.15.37:0/4256850917 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f421bfff640 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e792d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.500+0000 7f421affd640 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e78d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f421b7fe640 1 -- 172.21.15.37:0/4256850917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 msgr2=0x563691e792d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f421b7fe640 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e792d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f421b7fe640 1 -- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 msgr2=0x563691e78d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f421b7fe640 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e78d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f421b7fe640 1 -- 172.21.15.37:0/4256850917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563691e79b00 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f421b7fe640 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e7c580 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f420c0314e0 tx=0x7f420c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd648a2b31c70289 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f41faffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f41faffd640 1 -- 172.21.15.37:0/4256850917 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420c035050 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f41faffd640 1 -- 172.21.15.37:0/4256850917 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f420c042030 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4214003e50 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f41faffd640 1 -- 172.21.15.37:0/4256850917 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f420c003e10 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f42242d7740 10 client.18231 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563691eefb60 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.501+0000 7f41faffd640 1 -- 172.21.15.37:0/4256850917 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4214003e50 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f41faffd640 1 -- 172.21.15.37:0/4256850917 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f420c04a430 con 0x563691e64640 2024-09-19T06:51:42.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f42242d7740 10 client.18231 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f42242d7740 10 client.18231 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f42242d7740 20 client.18231 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f42242d7740 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691ef0320 0x563691ef2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f42242d7740 4 client.18231 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x563691dd1d20 con 0x563691ef0320 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f421affd640 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691ef0320 0x563691ef2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f421affd640 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691ef0320 0x563691ef2740 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9d48458972aea99 server_cookie=cd1ce9ab95f1b0f5 in_seq=0 out_seq=0 2024-09-19T06:51:42.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.502+0000 7f41faffd640 10 client.18231 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f41faffd640 1 -- 172.21.15.37:0/4256850917 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563691dd1d20 con 0x563691ef0320 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f41faffd640 10 client.18231 handle_command_reply: tid=0 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 client.18231 shutdown 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 msgr2=0x563691e7c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e7c580 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f420c0314e0 tx=0x7f420c02a480 comp rx=0 tx=0).stop 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 shutdown_connections 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563691ef0320 0x563691ef2740 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563691e73d50 0x563691e792d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563691e64f60 0x563691e78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f41faffd640 0 client.18231 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:42.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 --2- 172.21.15.37:0/4256850917 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563691e64640 0x563691e7c580 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 >> 172.21.15.37:0/4256850917 conn(0x563691e60cf0 msgr2=0x563691e713f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 shutdown_connections 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 1 -- 172.21.15.37:0/4256850917 wait complete. 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.609+0000 7f42242d7740 20 client.18231 trim_cache size 0 max 16384 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 39, 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 34, 2024-09-19T06:51:42.616 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "d51d7a7a": { 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 34, 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:51:42.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:51:42.618 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:51:42.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:51:42.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:51:42.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:51:42.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:51:42.618 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:51:42.618 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42040a3430 msgr2=0x7f42040a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42040a3430 0x7f42040a58f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f420c009230 tx=0x7f420c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c102570 msgr2=0x7f421c197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c102570 0x7f421c197950 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f421000eb70 tx=0x7f421000b690 comp rx=0 tx=0).stop 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 shutdown_connections 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f421c102ef0 0x7f421c19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f42040a3430 0x7f42040a58f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f421c102570 0x7f421c197950 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 --2- 172.21.15.37:0/971681137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f421c100e20 0x7f421c197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 >> 172.21.15.37:0/971681137 conn(0x7f421c0fc7d0 msgr2=0x7f421c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 shutdown_connections 2024-09-19T06:51:42.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:51:42.610+0000 7f4221efe640 1 -- 172.21.15.37:0/971681137 wait complete. 2024-09-19T06:51:43.224 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.21 MB/sec execute 370 sec latency 709.520 ms 2024-09-19T06:51:43.465 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd51d7a7a', quiesce took 3.7 seconds. Will release after: 26.3 2024-09-19T06:51:44.224 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.19 MB/sec execute 371 sec latency 1709.567 ms 2024-09-19T06:51:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:43 smithi096 ceph-mon[35004]: pgmap v311: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-19T06:51:44.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:43 smithi037 ceph-mon[31211]: pgmap v311: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-19T06:51:44.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:43 smithi151 ceph-mon[35236]: pgmap v311: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 4.5 MiB/s wr, 397 op/s 2024-09-19T06:51:45.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.16 MB/sec execute 372 sec latency 2709.677 ms 2024-09-19T06:51:46.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.14 MB/sec execute 373 sec latency 3709.786 ms 2024-09-19T06:51:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:45 smithi096 ceph-mon[35004]: pgmap v312: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-19T06:51:46.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:45 smithi037 ceph-mon[31211]: pgmap v312: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-19T06:51:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:45 smithi151 ceph-mon[35236]: pgmap v312: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-19T06:51:47.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.12 MB/sec execute 374 sec latency 4709.892 ms 2024-09-19T06:51:48.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.10 MB/sec execute 375 sec latency 5709.971 ms 2024-09-19T06:51:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:48 smithi096 ceph-mon[35004]: pgmap v313: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-19T06:51:48.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:48 smithi037 ceph-mon[31211]: pgmap v313: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-19T06:51:48.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:48 smithi151 ceph-mon[35236]: pgmap v313: 97 pgs: 97 active+clean; 75 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 293 op/s 2024-09-19T06:51:49.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.08 MB/sec execute 376 sec latency 6710.076 ms 2024-09-19T06:51:50.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.06 MB/sec execute 377 sec latency 7710.188 ms 2024-09-19T06:51:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:50 smithi096 ceph-mon[35004]: pgmap v314: 97 pgs: 97 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-09-19T06:51:50.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:50 smithi037 ceph-mon[31211]: pgmap v314: 97 pgs: 97 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-09-19T06:51:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:50 smithi151 ceph-mon[35236]: pgmap v314: 97 pgs: 97 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.8 MiB/s wr, 324 op/s 2024-09-19T06:51:51.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.03 MB/sec execute 378 sec latency 8710.306 ms 2024-09-19T06:51:52.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 8.01 MB/sec execute 379 sec latency 9710.421 ms 2024-09-19T06:51:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:52 smithi096 ceph-mon[35004]: pgmap v315: 97 pgs: 97 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-09-19T06:51:52.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:52 smithi037 ceph-mon[31211]: pgmap v315: 97 pgs: 97 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-09-19T06:51:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:52 smithi151 ceph-mon[35236]: pgmap v315: 97 pgs: 97 active+clean; 73 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 232 op/s 2024-09-19T06:51:53.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.99 MB/sec execute 380 sec latency 10710.473 ms 2024-09-19T06:51:53.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 -- 172.21.15.37:0/1141016774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac101250 msgr2=0x7f16ac107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 --2- 172.21.15.37:0/1141016774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac101250 0x7f16ac107990 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f169c009900 tx=0x7f169c02a410 comp rx=0 tx=0).stop 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 -- 172.21.15.37:0/1141016774 shutdown_connections 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 --2- 172.21.15.37:0/1141016774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16ac1080f0 0x7f16ac10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 --2- 172.21.15.37:0/1141016774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac101250 0x7f16ac107990 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 --2- 172.21.15.37:0/1141016774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16ac1008a0 0x7f16ac100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 -- 172.21.15.37:0/1141016774 >> 172.21.15.37:0/1141016774 conn(0x7f16ac075ef0 msgr2=0x7f16ac076300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 -- 172.21.15.37:0/1141016774 shutdown_connections 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.411+0000 7f16b4477640 1 -- 172.21.15.37:0/1141016774 wait complete. 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 Processor -- start 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 -- start start 2024-09-19T06:51:53.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16ac1008a0 0x7f16ac101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac1080f0 0x7f16ac101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16ac102290 0x7f16ac1a6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac111f20 con 0x7f16ac1080f0 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac111da0 con 0x7f16ac1008a0 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b4477640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1120a0 con 0x7f16ac102290 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b1a12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac1080f0 0x7f16ac101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b1a12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac1080f0 0x7f16ac101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60042/0 (socket says 172.21.15.37:60042) 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b1a12640 1 -- 172.21.15.37:0/1748325672 learned_addr learned my addr 172.21.15.37:0/1748325672 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b2a14640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16ac102290 0x7f16ac1a6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.412+0000 7f16b2213640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16ac1008a0 0x7f16ac101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b1a12640 1 -- 172.21.15.37:0/1748325672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16ac102290 msgr2=0x7f16ac1a6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b1a12640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16ac102290 0x7f16ac1a6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b1a12640 1 -- 172.21.15.37:0/1748325672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16ac1008a0 msgr2=0x7f16ac101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b1a12640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16ac1008a0 0x7f16ac101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b1a12640 1 -- 172.21.15.37:0/1748325672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16ac1a6c40 con 0x7f16ac1080f0 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b1a12640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac1080f0 0x7f16ac101d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f169c005940 tx=0x7f169c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16a37fe640 1 -- 172.21.15.37:0/1748325672 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c03f030 con 0x7f16ac1080f0 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16a37fe640 1 -- 172.21.15.37:0/1748325672 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169c003ce0 con 0x7f16ac1080f0 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16ac1a6ed0 con 0x7f16ac1080f0 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16a37fe640 1 -- 172.21.15.37:0/1748325672 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c035070 con 0x7f16ac1080f0 2024-09-19T06:51:53.416 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.413+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16ac1a7330 con 0x7f16ac1080f0 2024-09-19T06:51:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.416+0000 7f16a37fe640 1 -- 172.21.15.37:0/1748325672 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f169c042030 con 0x7f16ac1080f0 2024-09-19T06:51:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.416+0000 7f16b720a740 1 Processor -- start 2024-09-19T06:51:53.418 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.416+0000 7f16b720a740 1 -- start start 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b720a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd72103ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b720a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b720a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd721110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b720a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd720682f0 con 0x55dd7210ecb0 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b720a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dd72010270 con 0x55dd72103af0 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b720a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dd720642b0 con 0x55dd72104410 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16a37fe640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f16980a3490 0x7f16980a5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd721110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b2a14640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f16980a3490 0x7f16980a5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd721110a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60046/0 (socket says 172.21.15.37:60046) 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 -- 172.21.15.37:0/1725657899 learned_addr learned my addr 172.21.15.37:0/1725657899 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b2a14640 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd72103ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16a37fe640 1 -- 172.21.15.37:0/1748325672 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f169c0f0170 con 0x7f16ac1080f0 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b2213640 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 -- 172.21.15.37:0/1725657899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 msgr2=0x55dd7210e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 -- 172.21.15.37:0/1725657899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 msgr2=0x55dd72103ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd72103ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b1a12640 1 -- 172.21.15.37:0/1725657899 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd7200f340 con 0x55dd7210ecb0 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.417+0000 7f16b2a14640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f16980a3490 0x7f16980a5950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009fd0 tx=0x7f16a4009d60 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.418+0000 7f16b1a12640 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd721110a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f169c0bc210 tx=0x7f169c08f340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9a524a90b81cf0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.418+0000 7f16a0bfe640 1 -- 172.21.15.37:0/1725657899 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c049070 con 0x55dd7210ecb0 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.418+0000 7f16a0bfe640 1 -- 172.21.15.37:0/1725657899 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169c0bc510 con 0x55dd7210ecb0 2024-09-19T06:51:53.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.418+0000 7f16a0bfe640 1 -- 172.21.15.37:0/1725657899 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169c0b8b80 con 0x55dd7210ecb0 2024-09-19T06:51:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.418+0000 7f16b720a740 1 -- 172.21.15.37:0/1725657899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 msgr2=0x55dd721110a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.418+0000 7f16b720a740 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd721110a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f169c0bc210 tx=0x7f169c08f340 comp rx=0 tx=0).stop 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- 172.21.15.37:0/1725657899 shutdown_connections 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd721110a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 --2- 172.21.15.37:0/1725657899 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd72103ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- 172.21.15.37:0/1725657899 >> 172.21.15.37:0/1725657899 conn(0x55dd72100060 msgr2=0x55dd72100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- 172.21.15.37:0/1725657899 shutdown_connections 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- 172.21.15.37:0/1725657899 wait complete. 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- start start 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 Processor -- start 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd7210bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd7210c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55dd72111640 con 0x55dd72103af0 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55dd71ff1290 con 0x55dd7210ecb0 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55dd721117c0 con 0x55dd72104410 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b720a740 10 client.0 _ll_register_callbacks cb 0x55dd72022140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b2213640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd7210bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.419+0000 7f16b2213640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd7210bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60060/0 (socket says 172.21.15.37:60060) 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2213640 1 -- 172.21.15.37:0/2660017503 learned_addr learned my addr 172.21.15.37:0/2660017503 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b1a12640 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2a14640 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd7210c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2213640 1 -- 172.21.15.37:0/2660017503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 msgr2=0x55dd7210c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2213640 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2213640 1 -- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 msgr2=0x55dd7210c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2213640 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd7210c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2213640 1 -- 172.21.15.37:0/2660017503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd71d7b210 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b2213640 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd7210bb90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f16a800b9c0 tx=0x7f16a800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d2b6a1f6607a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f1695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f1695ffb640 1 -- 172.21.15.37:0/2660017503 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8011440 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f1695ffb640 1 -- 172.21.15.37:0/2660017503 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f16a80115e0 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f169c0bc510 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f1695ffb640 1 -- 172.21.15.37:0/2660017503 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f16a80117e0 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b720a740 10 client.18243 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.420+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd71e47620 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f1695ffb640 1 -- 172.21.15.37:0/2660017503 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f169c0bc510 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f1695ffb640 1 -- 172.21.15.37:0/2660017503 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f16a80239c0 con 0x55dd72103af0 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f16b720a740 10 client.18243 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f16b720a740 10 client.18243 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:53.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f16b720a740 20 client.18243 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f16b720a740 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7211e990 0x55dd72120dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f16b720a740 4 client.18243 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd72075a20 con 0x55dd7211e990 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.421+0000 7f16b1a12640 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7211e990 0x55dd72120dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.422+0000 7f16b1a12640 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7211e990 0x55dd72120dc0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17732ee138494b64 server_cookie=a2ac302da26fe9b4 in_seq=0 out_seq=0 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.422+0000 7f1695ffb640 10 client.18243 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.422+0000 7f1695ffb640 1 -- 172.21.15.37:0/2660017503 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dd72075a20 con 0x55dd7211e990 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.422+0000 7f1695ffb640 10 client.18243 handle_command_reply: tid=0 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.422+0000 7f16b720a740 1 client.18243 shutdown 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 msgr2=0x55dd7210bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd7210bb90 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f16a800b9c0 tx=0x7f16a800be80 comp rx=0 tx=0).stop 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 shutdown_connections 2024-09-19T06:51:53.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7211e990 0x55dd72120dc0 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd7210c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd7210c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f1695ffb640 0 client.18243 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 --2- 172.21.15.37:0/2660017503 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd7210bb90 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 >> 172.21.15.37:0/2660017503 conn(0x55dd72100060 msgr2=0x55dd72105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 shutdown_connections 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 1 -- 172.21.15.37:0/2660017503 wait complete. 2024-09-19T06:51:53.429 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.423+0000 7f16b720a740 20 client.18243 trim_cache size 0 max 16384 2024-09-19T06:51:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 Processor -- start 2024-09-19T06:51:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 -- start start 2024-09-19T06:51:53.443 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd7210c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd72126720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd72126c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1b2400 con 0x55dd7210ecb0 2024-09-19T06:51:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1afcd0 con 0x55dd72103af0 2024-09-19T06:51:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b720a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1adb40 con 0x55dd72104410 2024-09-19T06:51:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd72126c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.444 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd72126c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60074/0 (socket says 172.21.15.37:60074) 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 -- 172.21.15.37:0/2430794298 learned_addr learned my addr 172.21.15.37:0/2430794298 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2213640 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd7210c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b1a12640 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd72126720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 -- 172.21.15.37:0/2430794298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 msgr2=0x55dd72126720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd72126720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 -- 172.21.15.37:0/2430794298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 msgr2=0x55dd7210c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd7210c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.438+0000 7f16b2a14640 1 -- 172.21.15.37:0/2430794298 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd72103920 con 0x55dd7210ecb0 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b2213640 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd7210c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b2a14640 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd72126c60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f16a4012110 tx=0x7f16a401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bd13b2d5bedf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f1697fff640 1 -- 172.21.15.37:0/2430794298 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4012140 con 0x55dd7210ecb0 2024-09-19T06:51:53.445 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f1697fff640 1 -- 172.21.15.37:0/2430794298 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4004520 con 0x55dd7210ecb0 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f1697fff640 1 -- 172.21.15.37:0/2430794298 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a4020a60 con 0x55dd7210ecb0 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b720a740 1 -- 172.21.15.37:0/2430794298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 msgr2=0x55dd72126c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b720a740 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd72126c60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f16a4012110 tx=0x7f16a401f000 comp rx=0 tx=0).stop 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b720a740 1 -- 172.21.15.37:0/2430794298 shutdown_connections 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b720a740 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd7210ecb0 0x55dd72126c60 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b720a740 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd72126720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b720a740 1 --2- 172.21.15.37:0/2430794298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd72103af0 0x55dd7210c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.439+0000 7f16b720a740 1 -- 172.21.15.37:0/2430794298 >> 172.21.15.37:0/2430794298 conn(0x55dd72100060 msgr2=0x55dd7210dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 -- 172.21.15.37:0/2430794298 shutdown_connections 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 -- 172.21.15.37:0/2430794298 wait complete. 2024-09-19T06:51:53.446 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 -- start start 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 Processor -- start 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd72108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd721091f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd72109730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1ae190 con 0x55dd72103af0 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1af600 con 0x55dd7210ecb0 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f16ac1ae860 con 0x55dd72104410 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b720a740 10 client.0 _ll_register_callbacks cb 0x7f16a80042b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b2213640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd72108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b2213640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd72108cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60088/0 (socket says 172.21.15.37:60088) 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b2213640 1 -- 172.21.15.37:0/3046164912 learned_addr learned my addr 172.21.15.37:0/3046164912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b1a12640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd721091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.447 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.440+0000 7f16b2a14640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd72109730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b2213640 1 -- 172.21.15.37:0/3046164912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 msgr2=0x55dd721091f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b2213640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd721091f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b2213640 1 -- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 msgr2=0x55dd72109730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b2213640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd72109730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b2213640 1 -- 172.21.15.37:0/3046164912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd721011f0 con 0x55dd72103af0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b2a14640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd72109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b2213640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd72108cb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f16a8011770 tx=0x7f16a8003fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44bfdb060cc212f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f1695ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f1695ffb640 1 -- 172.21.15.37:0/3046164912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800b4a0 con 0x55dd72103af0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f1695ffb640 1 -- 172.21.15.37:0/3046164912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8571+0+0 (secure 0 0 0) 0x7f16a8011980 con 0x55dd72103af0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16a4004520 con 0x55dd72103af0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f1695ffb640 1 -- 172.21.15.37:0/3046164912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f16a8011b80 con 0x55dd72103af0 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b720a740 10 client.18249 fetch_fsmap learned FSMap version 39 2024-09-19T06:51:53.448 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55dd72189990 con 0x55dd72103af0 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f1695ffb640 1 -- 172.21.15.37:0/3046164912 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f16a4004520 con 0x55dd72103af0 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f1695ffb640 1 -- 172.21.15.37:0/3046164912 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f16a802a070 con 0x55dd72103af0 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b720a740 10 client.18249 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.441+0000 7f16b720a740 10 client.18249 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.442+0000 7f16b720a740 20 client.18249 populate_metadata read hostname 'smithi037' 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.442+0000 7f16b720a740 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7218a130 0x55dd7218c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.442+0000 7f16b720a740 4 client.18249 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.442+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd7206fd70 con 0x55dd7218a130 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.442+0000 7f16b1a12640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7218a130 0x55dd7218c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.442+0000 7f1695ffb640 10 client.18249 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:53.449 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.442+0000 7f16b1a12640 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7218a130 0x55dd7218c550 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e57884ad00c89ed server_cookie=92a32deee3aef60b in_seq=0 out_seq=0 2024-09-19T06:51:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f1695ffb640 1 -- 172.21.15.37:0/3046164912 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd7206fd70 con 0x55dd7218a130 2024-09-19T06:51:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f1695ffb640 10 client.18249 handle_command_reply: tid=0 2024-09-19T06:51:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 client.18249 shutdown 2024-09-19T06:51:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 msgr2=0x55dd72108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd72108cb0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f16a8011770 tx=0x7f16a8003fd0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 shutdown_connections 2024-09-19T06:51:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55dd7218a130 0x55dd7218c550 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55dd7210ecb0 0x55dd72109730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55dd72104410 0x55dd721091f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 --2- 172.21.15.37:0/3046164912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55dd72103af0 0x55dd72108cb0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 >> 172.21.15.37:0/3046164912 conn(0x55dd72100060 msgr2=0x55dd720fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f1695ffb640 0 client.18249 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 shutdown_connections 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 1 -- 172.21.15.37:0/3046164912 wait complete. 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.525+0000 7f16b720a740 20 client.18249 trim_cache size 0 max 16384 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:51:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:51:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:51:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:51:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f16980a3490 msgr2=0x7f16980a5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f16980a3490 0x7f16980a5950 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f16a4009fd0 tx=0x7f16a4009d60 comp rx=0 tx=0).stop 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac1080f0 msgr2=0x7f16ac101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac1080f0 0x7f16ac101d50 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f169c005940 tx=0x7f169c0042f0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 shutdown_connections 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16ac102290 0x7f16ac1a6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f16980a3490 0x7f16980a5950 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f16ac1080f0 0x7f16ac101d50 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 --2- 172.21.15.37:0/1748325672 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f16ac1008a0 0x7f16ac101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 >> 172.21.15.37:0/1748325672 conn(0x7f16ac075ef0 msgr2=0x7f16ac0ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 shutdown_connections 2024-09-19T06:51:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:51:53.526+0000 7f16b4477640 1 -- 172.21.15.37:0/1748325672 wait complete. 2024-09-19T06:51:54.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.97 MB/sec execute 381 sec latency 11710.534 ms 2024-09-19T06:51:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:54 smithi096 ceph-mon[35004]: pgmap v316: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 262 op/s 2024-09-19T06:51:54.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:54 smithi037 ceph-mon[31211]: pgmap v316: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 262 op/s 2024-09-19T06:51:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:54 smithi151 ceph-mon[35236]: pgmap v316: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.9 MiB/s wr, 262 op/s 2024-09-19T06:51:55.225 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.95 MB/sec execute 382 sec latency 12710.646 ms 2024-09-19T06:51:56.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.93 MB/sec execute 383 sec latency 13710.747 ms 2024-09-19T06:51:56.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:56 smithi096 ceph-mon[35004]: pgmap v317: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-09-19T06:51:56.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:56 smithi037 ceph-mon[31211]: pgmap v317: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-09-19T06:51:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:56 smithi151 ceph-mon[35236]: pgmap v317: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-09-19T06:51:57.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.91 MB/sec execute 384 sec latency 14710.845 ms 2024-09-19T06:51:58.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.89 MB/sec execute 385 sec latency 15710.985 ms 2024-09-19T06:51:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:51:58 smithi096 ceph-mon[35004]: pgmap v318: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-09-19T06:51:58.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:51:58 smithi037 ceph-mon[31211]: pgmap v318: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-09-19T06:51:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:51:58 smithi151 ceph-mon[35236]: pgmap v318: 97 pgs: 97 active+clean; 72 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.4 MiB/s wr, 60 op/s 2024-09-19T06:51:59.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.87 MB/sec execute 386 sec latency 16711.096 ms 2024-09-19T06:52:00.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.85 MB/sec execute 387 sec latency 17711.145 ms 2024-09-19T06:52:00.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:00 smithi037 ceph-mon[31211]: pgmap v319: 97 pgs: 97 active+clean; 67 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-09-19T06:52:00.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:00 smithi151 ceph-mon[35236]: pgmap v319: 97 pgs: 97 active+clean; 67 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-09-19T06:52:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:00 smithi096 ceph-mon[35004]: pgmap v319: 97 pgs: 97 active+clean; 67 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 142 KiB/s rd, 2.2 MiB/s wr, 125 op/s 2024-09-19T06:52:01.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.83 MB/sec execute 388 sec latency 18711.188 ms 2024-09-19T06:52:02.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.81 MB/sec execute 389 sec latency 19711.236 ms 2024-09-19T06:52:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:02 smithi151 ceph-mon[35236]: pgmap v320: 97 pgs: 97 active+clean; 67 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-09-19T06:52:02.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:02 smithi096 ceph-mon[35004]: pgmap v320: 97 pgs: 97 active+clean; 67 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-09-19T06:52:02.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:02 smithi037 ceph-mon[31211]: pgmap v320: 97 pgs: 97 active+clean; 67 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 1.5 MiB/s wr, 94 op/s 2024-09-19T06:52:02.983 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:52:03.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.79 MB/sec execute 390 sec latency 20711.280 ms 2024-09-19T06:52:03.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:03 smithi037 ceph-mon[31211]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T06:52:03.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:03 smithi151 ceph-mon[35236]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T06:52:03.391 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:03.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:03 smithi096 ceph-mon[35004]: osdmap e95: 12 total, 12 up, 12 in 2024-09-19T06:52:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 -- 172.21.15.37:0/2203759344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 msgr2=0x7efe8810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/2203759344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8810eaa0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7efe80009ca0 tx=0x7efe8002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 -- 172.21.15.37:0/2203759344 shutdown_connections 2024-09-19T06:52:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/2203759344 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8810eaa0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/2203759344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe881027a0 0x7efe881067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/2203759344 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe88101df0 0x7efe881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 -- 172.21.15.37:0/2203759344 >> 172.21.15.37:0/2203759344 conn(0x7efe880fd8f0 msgr2=0x7efe880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 -- 172.21.15.37:0/2203759344 shutdown_connections 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.056+0000 7efe8e4c6640 1 -- 172.21.15.37:0/2203759344 wait complete. 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 Processor -- start 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 -- start start 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe88101df0 0x7efe881983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe881027a0 0x7efe88198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe88111f80 con 0x7efe88106f20 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe88111e00 con 0x7efe88101df0 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8e4c6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe88112100 con 0x7efe881027a0 2024-09-19T06:52:04.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8ca63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8ca63640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8819ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56530/0 (socket says 172.21.15.37:56530) 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe8ca63640 1 -- 172.21.15.37:0/1256193226 learned_addr learned my addr 172.21.15.37:0/1256193226 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe87fff640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe88101df0 0x7efe881983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.057+0000 7efe7ffff640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe881027a0 0x7efe88198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8ca63640 1 -- 172.21.15.37:0/1256193226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe881027a0 msgr2=0x7efe88198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8ca63640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe881027a0 0x7efe88198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8ca63640 1 -- 172.21.15.37:0/1256193226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe88101df0 msgr2=0x7efe881983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8ca63640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe88101df0 0x7efe881983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8ca63640 1 -- 172.21.15.37:0/1256193226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8819d390 con 0x7efe88106f20 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8ca63640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8819ccb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7efe80031a20 tx=0x7efe80031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe85ffb640 1 -- 172.21.15.37:0/1256193226 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe80003b70 con 0x7efe88106f20 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8819d620 con 0x7efe88106f20 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe85ffb640 1 -- 172.21.15.37:0/1256193226 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe80003d10 con 0x7efe88106f20 2024-09-19T06:52:04.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe85ffb640 1 -- 172.21.15.37:0/1256193226 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe8002fc40 con 0x7efe88106f20 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.058+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe881a4f60 con 0x7efe88106f20 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.060+0000 7efe85ffb640 1 -- 172.21.15.37:0/1256193226 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efe80004050 con 0x7efe88106f20 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.060+0000 7efe9089f740 1 Processor -- start 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.060+0000 7efe9089f740 1 -- start start 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe9089f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1deea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe9089f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1def5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe9089f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1def7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe9089f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e1de502e0 con 0x560e1def5680 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe9089f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e1ddf7ce0 con 0x560e1deeadc0 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe9089f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e1de4c2a0 con 0x560e1deea4a0 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1def7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1def7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56538/0 (socket says 172.21.15.37:56538) 2024-09-19T06:52:04.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 -- 172.21.15.37:0/1860767967 learned_addr learned my addr 172.21.15.37:0/1860767967 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1deea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe87fff640 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1def5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe85ffb640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efe740a33b0 0x7efe740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe8ca63640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efe740a33b0 0x7efe740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 -- 172.21.15.37:0/1860767967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 msgr2=0x560e1deea880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1deea880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 -- 172.21.15.37:0/1860767967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 msgr2=0x560e1def5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1def5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 -- 172.21.15.37:0/1860767967 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1dddc440 con 0x560e1def5680 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe85ffb640 1 -- 172.21.15.37:0/1256193226 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7efe800f0d50 con 0x7efe88106f20 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.061+0000 7efe7ffff640 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1def7a70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7efe70011780 tx=0x7efe70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b9884c5860f3f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe7e7fc640 1 -- 172.21.15.37:0/1860767967 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70004050 con 0x560e1def5680 2024-09-19T06:52:04.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe7e7fc640 1 -- 172.21.15.37:0/1860767967 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70005500 con 0x560e1def5680 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe7e7fc640 1 -- 172.21.15.37:0/1860767967 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70004d30 con 0x560e1def5680 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe8ca63640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efe740a33b0 0x7efe740a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efe800eea40 tx=0x7efe80033040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 -- 172.21.15.37:0/1860767967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 msgr2=0x560e1def7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1def7a70 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7efe70011780 tx=0x7efe70004730 comp rx=0 tx=0).stop 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 -- 172.21.15.37:0/1860767967 shutdown_connections 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1def7a70 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1def5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 --2- 172.21.15.37:0/1860767967 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1deea880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 -- 172.21.15.37:0/1860767967 >> 172.21.15.37:0/1860767967 conn(0x560e1dee6a30 msgr2=0x560e1dee6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 -- 172.21.15.37:0/1860767967 shutdown_connections 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.062+0000 7efe9089f740 1 -- 172.21.15.37:0/1860767967 wait complete. 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 -- start start 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 Processor -- start 2024-09-19T06:52:04.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1def4f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1def2200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1def2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560e1def8250 con 0x560e1deeadc0 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x560e1de0b050 con 0x560e1deea4a0 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x560e1def83d0 con 0x560e1def5680 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe9089f740 10 client.0 _ll_register_callbacks cb 0x560e1de4b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1def2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1def2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56554/0 (socket says 172.21.15.37:56554) 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.063+0000 7efe87fff640 1 -- 172.21.15.37:0/2248512109 learned_addr learned my addr 172.21.15.37:0/2248512109 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1def4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1def2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe87fff640 1 -- 172.21.15.37:0/2248512109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 msgr2=0x560e1def2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe87fff640 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1def2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe87fff640 1 -- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 msgr2=0x560e1def4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe87fff640 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1def4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe87fff640 1 -- 172.21.15.37:0/2248512109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1dbc13c0 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe87fff640 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1def2200 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efe78012740 tx=0x7efe78012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a56a0bc594b02e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe677fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe677fe640 1 -- 172.21.15.37:0/2248512109 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7800f920 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe677fe640 1 -- 172.21.15.37:0/2248512109 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7efe7800fac0 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe70005500 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe677fe640 1 -- 172.21.15.37:0/2248512109 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe7800fcc0 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe9089f740 10 client.18261 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.064+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560e1df03be0 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe677fe640 1 -- 172.21.15.37:0/2248512109 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70005500 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe677fe640 1 -- 172.21.15.37:0/2248512109 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efe7801c420 con 0x560e1deeadc0 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe9089f740 10 client.18261 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:04.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe9089f740 10 client.18261 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe9089f740 20 client.18261 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe9089f740 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df05140 0x560e1df075c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe9089f740 4 client.18261 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e1de5d850 con 0x560e1df05140 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df05140 0x560e1df075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df05140 0x560e1df075c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=484c22df5bdb8237 server_cookie=e6572e4db1e8c94f in_seq=0 out_seq=0 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.065+0000 7efe677fe640 10 client.18261 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe677fe640 1 -- 172.21.15.37:0/2248512109 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560e1de5d850 con 0x560e1df05140 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe677fe640 10 client.18261 handle_command_reply: tid=0 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 client.18261 shutdown 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 msgr2=0x560e1def2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1def2200 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7efe78012740 tx=0x7efe78012c00 comp rx=0 tx=0).stop 2024-09-19T06:52:04.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 shutdown_connections 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df05140 0x560e1df075c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1def2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1def2200 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 --2- 172.21.15.37:0/2248512109 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1def4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 >> 172.21.15.37:0/2248512109 conn(0x560e1dee6a30 msgr2=0x560e1deec710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe677fe640 0 client.18261 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 shutdown_connections 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.066+0000 7efe9089f740 1 -- 172.21.15.37:0/2248512109 wait complete. 2024-09-19T06:52:04.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.067+0000 7efe9089f740 20 client.18261 trim_cache size 0 max 16384 2024-09-19T06:52:04.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 Processor -- start 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 -- start start 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1df014b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1df0ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1df0d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe881b0330 con 0x560e1def5680 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe881adc00 con 0x560e1deeadc0 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.093+0000 7efe9089f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe881aba70 con 0x560e1deea4a0 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1df0d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1df0d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56568/0 (socket says 172.21.15.37:56568) 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 -- 172.21.15.37:0/2151572811 learned_addr learned my addr 172.21.15.37:0/2151572811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1df014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe87fff640 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1df0ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 -- 172.21.15.37:0/2151572811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 msgr2=0x560e1df014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1df014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 -- 172.21.15.37:0/2151572811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 msgr2=0x560e1df0ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1df0ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 -- 172.21.15.37:0/2151572811 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e1dee6860 con 0x560e1def5680 2024-09-19T06:52:04.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7ffff640 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1df0d3d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ab0 tx=0x7efe700048d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e783b48d543774 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7dffb640 1 -- 172.21.15.37:0/2151572811 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70033070 con 0x560e1def5680 2024-09-19T06:52:04.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7dffb640 1 -- 172.21.15.37:0/2151572811 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70034040 con 0x560e1def5680 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.094+0000 7efe7dffb640 1 -- 172.21.15.37:0/2151572811 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70030a60 con 0x560e1def5680 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 -- 172.21.15.37:0/2151572811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 msgr2=0x560e1df0d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1df0d3d0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ab0 tx=0x7efe700048d0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 -- 172.21.15.37:0/2151572811 shutdown_connections 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1def5680 0x560e1df0d3d0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deeadc0 0x560e1df0ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 --2- 172.21.15.37:0/2151572811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1deea4a0 0x560e1df014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 -- 172.21.15.37:0/2151572811 >> 172.21.15.37:0/2151572811 conn(0x560e1dee6a30 msgr2=0x560e1deed8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.095+0000 7efe9089f740 1 -- 172.21.15.37:0/2151572811 shutdown_connections 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.096+0000 7efe9089f740 1 -- 172.21.15.37:0/2151572811 wait complete. 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.096+0000 7efe9089f740 1 -- start start 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.096+0000 7efe9089f740 1 Processor -- start 2024-09-19T06:52:04.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.096+0000 7efe9089f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1df6b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.096+0000 7efe9089f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1df6bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe9089f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1df0cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe9089f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efe881ac0c0 con 0x560e1deeadc0 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe9089f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe881ad530 con 0x560e1deea4a0 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe9089f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efe881ac790 con 0x560e1def5680 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1df6bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1df6bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56576/0 (socket says 172.21.15.37:56576) 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 -- 172.21.15.37:0/3801195931 learned_addr learned my addr 172.21.15.37:0/3801195931 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe9089f740 10 client.0 _ll_register_callbacks cb 0x7efe78009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe7ffff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1df6b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe7ffff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1df0cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 -- 172.21.15.37:0/3801195931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 msgr2=0x560e1df0cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1df0cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 -- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 msgr2=0x560e1df6b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:04.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1df6b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe87fff640 1 -- 172.21.15.37:0/3801195931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e1df0d1b0 con 0x560e1deeadc0 2024-09-19T06:52:04.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.097+0000 7efe7ffff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1df0cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe87fff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1df6bc30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7efe78000c00 tx=0x7efe7800fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148d188e245a19c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe677fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe677fe640 1 -- 172.21.15.37:0/3801195931 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7801cab0 con 0x560e1deeadc0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe70034040 con 0x560e1deeadc0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe677fe640 1 -- 172.21.15.37:0/3801195931 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7efe7801cc50 con 0x560e1deeadc0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe677fe640 1 -- 172.21.15.37:0/3801195931 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efe7801ce50 con 0x560e1deeadc0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe9089f740 10 client.18267 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x560e1df74020 con 0x560e1deeadc0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.098+0000 7efe677fe640 1 -- 172.21.15.37:0/3801195931 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe70034040 con 0x560e1deeadc0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe677fe640 1 -- 172.21.15.37:0/3801195931 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efe780039d0 con 0x560e1deeadc0 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe9089f740 10 client.18267 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:04.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe9089f740 10 client.18267 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe9089f740 20 client.18267 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe9089f740 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df747d0 0x560e1df76bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe9089f740 4 client.18267 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e1de57d60 con 0x560e1df747d0 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.099+0000 7efe7ffff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df747d0 0x560e1df76bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe677fe640 10 client.18267 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe7ffff640 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df747d0 0x560e1df76bf0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e014c6cae7bece05 server_cookie=be66f2c4b7ffcc71 in_seq=0 out_seq=0 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe677fe640 1 -- 172.21.15.37:0/3801195931 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560e1de57d60 con 0x560e1df747d0 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe677fe640 10 client.18267 handle_command_reply: tid=0 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 client.18267 shutdown 2024-09-19T06:52:04.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 msgr2=0x560e1df6bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1df6bc30 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7efe78000c00 tx=0x7efe7800fed0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 shutdown_connections 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560e1df747d0 0x560e1df76bf0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x560e1def5680 0x560e1df0cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe677fe640 0 client.18267 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x560e1deeadc0 0x560e1df6bc30 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 --2- 172.21.15.37:0/3801195931 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x560e1deea4a0 0x560e1df6b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 >> 172.21.15.37:0/3801195931 conn(0x560e1dee6a30 msgr2=0x560e1dee5350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 shutdown_connections 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.100+0000 7efe9089f740 1 -- 172.21.15.37:0/3801195931 wait complete. 2024-09-19T06:52:04.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe9089f740 20 client.18267 trim_cache size 0 max 16384 2024-09-19T06:52:04.105 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:04.105 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:52:04.105 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:52:04.105 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efe740a33b0 msgr2=0x7efe740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efe740a33b0 0x7efe740a5870 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7efe800eea40 tx=0x7efe80033040 comp rx=0 tx=0).stop 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 msgr2=0x7efe8819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8819ccb0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7efe80031a20 tx=0x7efe80031a50 comp rx=0 tx=0).stop 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 shutdown_connections 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efe740a33b0 0x7efe740a5870 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efe88106f20 0x7efe8819ccb0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.106 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe881027a0 0x7efe88198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 --2- 172.21.15.37:0/1256193226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efe88101df0 0x7efe881983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:04.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.101+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 >> 172.21.15.37:0/1256193226 conn(0x7efe880fd8f0 msgr2=0x7efe880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:04.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.102+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 shutdown_connections 2024-09-19T06:52:04.107 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:04.102+0000 7efe8e4c6640 1 -- 172.21.15.37:0/1256193226 wait complete. 2024-09-19T06:52:04.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.77 MB/sec execute 391 sec latency 21711.387 ms 2024-09-19T06:52:04.272 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:04 smithi151 ceph-mon[35236]: pgmap v322: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:04.272 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:04 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:52:04.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:04 smithi037 ceph-mon[31211]: pgmap v322: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:04.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:04 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:52:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:04 smithi096 ceph-mon[35004]: pgmap v322: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:04 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:52:05.184 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:52:05.185 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:05.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.75 MB/sec execute 392 sec latency 22711.433 ms 2024-09-19T06:52:05.960 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:06.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: pgmap v323: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:06.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:06.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:06.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T06:52:06.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:05 smithi037 ceph-mon[31211]: Unable to set osd_memory_target on smithi151 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T06:52:06.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.73 MB/sec execute 393 sec latency 23711.473 ms 2024-09-19T06:52:06.241 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: pgmap v323: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T06:52:06.242 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:05 smithi096 ceph-mon[35004]: Unable to set osd_memory_target on smithi151 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: pgmap v323: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi151 to 176.0M 2024-09-19T06:52:06.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:05 smithi151 ceph-mon[35236]: Unable to set osd_memory_target on smithi151 to 184591769: error parsing value: Value '184591769' is below minimum 939524096 2024-09-19T06:52:06.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 -- 172.21.15.37:0/802663238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 msgr2=0x7ff3cc1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/802663238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 0x7ff3cc1067e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009900 tx=0x7ff3c002a410 comp rx=0 tx=0).stop 2024-09-19T06:52:06.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 -- 172.21.15.37:0/802663238 shutdown_connections 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/802663238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc106f10 0x7ff3cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/802663238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 0x7ff3cc1067e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/802663238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc101de0 0x7ff3cc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 -- 172.21.15.37:0/802663238 >> 172.21.15.37:0/802663238 conn(0x7ff3cc0fd8c0 msgr2=0x7ff3cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 -- 172.21.15.37:0/802663238 shutdown_connections 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.480+0000 7ff3d1f25640 1 -- 172.21.15.37:0/802663238 wait complete. 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 Processor -- start 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 -- start start 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc101de0 0x7ff3cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 0x7ff3cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc106f10 0x7ff3cc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc111f30 con 0x7ff3cc102790 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc111db0 con 0x7ff3cc101de0 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3d1f25640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ff3cc1120b0 con 0x7ff3cc106f10 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc101de0 0x7ff3cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3caffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 0x7ff3cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3cb7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc101de0 0x7ff3cc0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:55088/0 (socket says 172.21.15.37:55088) 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.481+0000 7ff3cb7fe640 1 -- 172.21.15.37:0/1470946983 learned_addr learned my addr 172.21.15.37:0/1470946983 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3cbfff640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc106f10 0x7ff3cc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3cb7fe640 1 -- 172.21.15.37:0/1470946983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc106f10 msgr2=0x7ff3cc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3cb7fe640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc106f10 0x7ff3cc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3cb7fe640 1 -- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 msgr2=0x7ff3cc074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3cb7fe640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 0x7ff3cc074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3cb7fe640 1 -- 172.21.15.37:0/1470946983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3cc1a6890 con 0x7ff3cc101de0 2024-09-19T06:52:06.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3caffd640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 0x7ff3cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:06.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3cb7fe640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc101de0 0x7ff3cc0744f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c920 tx=0x7ff3bc00cde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3c8ff9640 1 -- 172.21.15.37:0/1470946983 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc007c70 con 0x7ff3cc101de0 2024-09-19T06:52:06.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3cc1a6b80 con 0x7ff3cc101de0 2024-09-19T06:52:06.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3c8ff9640 1 -- 172.21.15.37:0/1470946983 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff3bc007e10 con 0x7ff3cc101de0 2024-09-19T06:52:06.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3c8ff9640 1 -- 172.21.15.37:0/1470946983 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3bc018890 con 0x7ff3cc101de0 2024-09-19T06:52:06.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.482+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3cc1a70e0 con 0x7ff3cc101de0 2024-09-19T06:52:06.494 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.483+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3cc1021c0 con 0x7ff3cc101de0 2024-09-19T06:52:06.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.485+0000 7ff3c8ff9640 1 -- 172.21.15.37:0/1470946983 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ff3bc018a70 con 0x7ff3cc101de0 2024-09-19T06:52:06.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.485+0000 7ff3c8ff9640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3b40a33b0 0x7ff3b40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.485+0000 7ff3c8ff9640 1 -- 172.21.15.37:0/1470946983 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7ff3bc0d0200 con 0x7ff3cc101de0 2024-09-19T06:52:06.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.485+0000 7ff3caffd640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3b40a33b0 0x7ff3b40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.486+0000 7ff3caffd640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3b40a33b0 0x7ff3b40a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009230 tx=0x7ff3c00058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.495 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.488+0000 7ff3c8ff9640 1 -- 172.21.15.37:0/1470946983 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7ff3bc014030 con 0x7ff3cc101de0 2024-09-19T06:52:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.634+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1438940677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 msgr2=0x7f0f88110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1438940677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f88110870 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009ca0 tx=0x7f0f8002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1438940677 shutdown_connections 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1438940677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f88110870 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1438940677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88101280 0x7f0f8810deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1438940677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f881008d0 0x7f0f88100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1438940677 >> 172.21.15.37:0/1438940677 conn(0x7f0f880fc6b0 msgr2=0x7f0f880fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1438940677 shutdown_connections 2024-09-19T06:52:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1438940677 wait complete. 2024-09-19T06:52:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.635+0000 7f0f8cbb0640 1 Processor -- start 2024-09-19T06:52:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8cbb0640 1 -- start start 2024-09-19T06:52:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8cbb0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f881008d0 0x7f0f8819a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8cbb0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88101280 0x7f0f8819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8cbb0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f8819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8cbb0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88114140 con 0x7f0f8810e480 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8cbb0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88113fc0 con 0x7f0f88101280 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8cbb0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881142c0 con 0x7f0f881008d0 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f8819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f8819ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56622/0 (socket says 172.21.15.37:56622) 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 -- 172.21.15.37:0/1138261698 learned_addr learned my addr 172.21.15.37:0/1138261698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88101280 0x7f0f8819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f8659c640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f881008d0 0x7f0f8819a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 -- 172.21.15.37:0/1138261698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f881008d0 msgr2=0x7f0f8819a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f881008d0 0x7f0f8819a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 -- 172.21.15.37:0/1138261698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88101280 msgr2=0x7f0f8819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88101280 0x7f0f8819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f86d9d640 1 -- 172.21.15.37:0/1138261698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f8819f540 con 0x7f0f8810e480 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.636+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88101280 0x7f0f8819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.637+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f8819ee10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005e30 tx=0x7f0f80003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.637+0000 7f0f777fe640 1 -- 172.21.15.37:0/1138261698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80031650 con 0x7f0f8810e480 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.637+0000 7f0f777fe640 1 -- 172.21.15.37:0/1138261698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f800317f0 con 0x7f0f8810e480 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.637+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8819f7d0 con 0x7f0f8810e480 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.637+0000 7f0f777fe640 1 -- 172.21.15.37:0/1138261698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8002fa40 con 0x7f0f8810e480 2024-09-19T06:52:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.637+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f881a70c0 con 0x7f0f8810e480 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f777fe640 1 -- 172.21.15.37:0/1138261698 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0f8002fc00 con 0x7f0f8810e480 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 Processor -- start 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 -- start start 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a52df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a65f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a688b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eec89b72f0 con 0x55eec8a52a10 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eec895f270 con 0x55eec8a53330 2024-09-19T06:52:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eec89b32b0 con 0x55eec8a664c0 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f85d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a52df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f85d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a52df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56628/0 (socket says 172.21.15.37:56628) 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.641+0000 7f0f85d9b640 1 -- 172.21.15.37:0/1076694570 learned_addr learned my addr 172.21.15.37:0/1076694570 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a688b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f85d9b640 1 -- 172.21.15.37:0/1076694570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 msgr2=0x55eec8a688b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a688b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f85d9b640 1 -- 172.21.15.37:0/1076694570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 msgr2=0x55eec8a65f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a65f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f85d9b640 1 -- 172.21.15.37:0/1076694570 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec895e340 con 0x55eec8a52a10 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f777fe640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f700a34e0 0x7f0f700a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a52df0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b620 tx=0x7f0f7c00bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27f6f2ccebcee4b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f777fe640 1 -- 172.21.15.37:0/1138261698 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f0f800f10a0 con 0x7f0f8810e480 2024-09-19T06:52:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f8659c640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f700a34e0 0x7f0f700a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f74ff9640 1 -- 172.21.15.37:0/1076694570 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c004020 con 0x55eec8a52a10 2024-09-19T06:52:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f74ff9640 1 -- 172.21.15.37:0/1076694570 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7c00edb0 con 0x55eec8a52a10 2024-09-19T06:52:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.642+0000 7f0f74ff9640 1 -- 172.21.15.37:0/1076694570 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c003ab0 con 0x55eec8a52a10 2024-09-19T06:52:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1076694570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 msgr2=0x55eec8a52df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a52df0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c00b620 tx=0x7f0f7c00bb90 comp rx=0 tx=0).stop 2024-09-19T06:52:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1076694570 shutdown_connections 2024-09-19T06:52:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a688b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a65f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1076694570 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a52df0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1076694570 >> 172.21.15.37:0/1076694570 conn(0x55eec8a4f060 msgr2=0x55eec8a4f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8659c640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f700a34e0 0x7f0f700a59a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009ca0 tx=0x7f0f78009840 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1076694570 shutdown_connections 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.643+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1076694570 wait complete. 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.644+0000 7f0f8fa25740 1 -- start start 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.644+0000 7f0f8fa25740 1 Processor -- start 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.644+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a52a10 0x55eec8a63440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.644+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a63980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.644+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a63ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eec8a69090 con 0x55eec8a53330 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eec8940290 con 0x55eec8a52a10 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eec8a69210 con 0x55eec8a664c0 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a52a10 0x55eec8a63440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f85d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a63980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a52a10 0x55eec8a63440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:55132/0 (socket says 172.21.15.37:55132) 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 -- 172.21.15.37:0/2060042651 learned_addr learned my addr 172.21.15.37:0/2060042651 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8fa25740 10 client.0 _ll_register_callbacks cb 0x55eec8971140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a63ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 -- 172.21.15.37:0/2060042651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 msgr2=0x55eec8a63ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a63ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 -- 172.21.15.37:0/2060042651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 msgr2=0x55eec8a63980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a63980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 -- 172.21.15.37:0/2060042651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec86ca210 con 0x55eec8a52a10 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a63ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8659c640 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a52a10 0x55eec8a63440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0f7801caf0 tx=0x7f0f78005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44f25758f0c1180c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f6dffb640 1 -- 172.21.15.37:0/2060042651 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7801d8e0 con 0x55eec8a52a10 2024-09-19T06:52:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f6dffb640 1 -- 172.21.15.37:0/2060042651 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f0f7801da80 con 0x55eec8a52a10 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.645+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7c00edb0 con 0x55eec8a52a10 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f6dffb640 1 -- 172.21.15.37:0/2060042651 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0f7801dc80 con 0x55eec8a52a10 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 10 client.27623 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eec8796620 con 0x55eec8a52a10 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f6dffb640 1 -- 172.21.15.37:0/2060042651 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7c00edb0 con 0x55eec8a52a10 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f6dffb640 1 -- 172.21.15.37:0/2060042651 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f0f7802f4e0 con 0x55eec8a52a10 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 10 client.27623 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 10 client.27623 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 20 client.27623 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8a78980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 4 client.27623 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eec89c4a20 con 0x55eec8a76510 2024-09-19T06:52:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.646+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8a78980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.647+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8a78980 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d67a07a5713352d4 server_cookie=cdf046be7bab524c in_seq=0 out_seq=0 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.647+0000 7f0f6dffb640 10 client.27623 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.647+0000 7f0f6dffb640 1 -- 172.21.15.37:0/2060042651 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eec89c4a20 con 0x55eec8a76510 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.647+0000 7f0f6dffb640 10 client.27623 handle_command_reply: tid=0 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.647+0000 7f0f8fa25740 1 client.27623 shutdown 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.647+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a52a10 msgr2=0x55eec8a63440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.647+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a52a10 0x55eec8a63440 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0f7801caf0 tx=0x7f0f78005f90 comp rx=0 tx=0).stop 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 shutdown_connections 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8a78980 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a63ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f6dffb640 0 client.27623 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a63980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2060042651 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a52a10 0x55eec8a63440 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 >> 172.21.15.37:0/2060042651 conn(0x55eec8a4f060 msgr2=0x55eec8a54d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 shutdown_connections 2024-09-19T06:52:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2060042651 wait complete. 2024-09-19T06:52:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.648+0000 7f0f8fa25740 20 client.27623 trim_cache size 0 max 16384 2024-09-19T06:52:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.668+0000 7f0f8fa25740 1 Processor -- start 2024-09-19T06:52:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.668+0000 7f0f8fa25740 1 -- start start 2024-09-19T06:52:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a7e290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a7e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a60680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ae040 con 0x55eec8a52a10 2024-09-19T06:52:06.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ab910 con 0x55eec8a53330 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a97c0 con 0x55eec8a664c0 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a7e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f85d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a7e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a7e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56650/0 (socket says 172.21.15.37:56650) 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8659c640 1 -- 172.21.15.37:0/2274135771 learned_addr learned my addr 172.21.15.37:0/2274135771 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a60680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f85d9b640 1 -- 172.21.15.37:0/2274135771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 msgr2=0x55eec8a60680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a60680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f85d9b640 1 -- 172.21.15.37:0/2274135771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 msgr2=0x55eec8a7e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a7e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f85d9b640 1 -- 172.21.15.37:0/2274135771 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eec8a61970 con 0x55eec8a53330 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f8659c640 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a7e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.669+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a7e7d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c003790 tx=0x7f0f7c0035f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b76dde023e716c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f6ffff640 1 -- 172.21.15.37:0/2274135771 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c00e8d0 con 0x55eec8a53330 2024-09-19T06:52:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f6ffff640 1 -- 172.21.15.37:0/2274135771 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f7c005040 con 0x55eec8a53330 2024-09-19T06:52:06.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f6ffff640 1 -- 172.21.15.37:0/2274135771 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c0048a0 con 0x55eec8a53330 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2274135771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 msgr2=0x55eec8a7e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a7e7d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c003790 tx=0x7f0f7c0035f0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2274135771 shutdown_connections 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a664c0 0x55eec8a60680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a53330 0x55eec8a7e7d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/2274135771 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a52a10 0x55eec8a7e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2274135771 >> 172.21.15.37:0/2274135771 conn(0x55eec8a4f060 msgr2=0x55eec8a654c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2274135771 shutdown_connections 2024-09-19T06:52:06.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.670+0000 7f0f8fa25740 1 -- 172.21.15.37:0/2274135771 wait complete. 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 -- start start 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 Processor -- start 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a52a10 0x55eec8a6aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a664c0 0x55eec8ada2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a9e10 con 0x55eec8a53330 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ab240 con 0x55eec8a664c0 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881aa4a0 con 0x55eec8a52a10 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8fa25740 10 client.0 _ll_register_callbacks cb 0x7f0f78009a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f85d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a6b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f86d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a664c0 0x55eec8ada2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f8659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a52a10 0x55eec8a6aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f85d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a6b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56658/0 (socket says 172.21.15.37:56658) 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f86d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a664c0 0x55eec8ada2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:55150/0 (socket says 172.21.15.37:55150) 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.671+0000 7f0f85d9b640 1 -- 172.21.15.37:0/1395964789 learned_addr learned my addr 172.21.15.37:0/1395964789 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f86d9d640 1 -- 172.21.15.37:0/1395964789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a52a10 msgr2=0x55eec8a6aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a52a10 0x55eec8a6aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f86d9d640 1 -- 172.21.15.37:0/1395964789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 msgr2=0x55eec8a6b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a6b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f86d9d640 1 -- 172.21.15.37:0/1395964789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eec8ada8e0 con 0x55eec8a664c0 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f85d9b640 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f86d9d640 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a664c0 0x55eec8ada2a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0f800ee8a0 tx=0x7f0f800bce50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e7de4f9dc90d141 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f6dffb640 1 -- 172.21.15.37:0/1395964789 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800c17e0 con 0x55eec8a664c0 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f6dffb640 1 -- 172.21.15.37:0/1395964789 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7f0f800b9aa0 con 0x55eec8a664c0 2024-09-19T06:52:06.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f7c005040 con 0x55eec8a664c0 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f6dffb640 1 -- 172.21.15.37:0/1395964789 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0f800c60a0 con 0x55eec8a664c0 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f8fa25740 10 client.27629 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eec8ae1c80 con 0x55eec8a664c0 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.672+0000 7f0f6dffb640 1 -- 172.21.15.37:0/1395964789 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f800a0020 con 0x55eec8a664c0 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f6dffb640 1 -- 172.21.15.37:0/1395964789 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f0f80004b40 con 0x55eec8a664c0 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8fa25740 10 client.27629 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8fa25740 10 client.27629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8fa25740 20 client.27629 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8ae4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8fa25740 4 client.27629 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:06.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eec89bed70 con 0x55eec8a76510 2024-09-19T06:52:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8659c640 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8ae4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f8659c640 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8ae4360 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75ad6205bd4928b0 server_cookie=1938e687686a554 in_seq=0 out_seq=0 2024-09-19T06:52:06.679 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.673+0000 7f0f6dffb640 10 client.27629 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.717+0000 7f0f6dffb640 1 -- 172.21.15.37:0/1395964789 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eec89bed70 con 0x55eec8a76510 2024-09-19T06:52:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.717+0000 7f0f6dffb640 10 client.27629 handle_command_reply: tid=0 2024-09-19T06:52:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 client.27629 shutdown 2024-09-19T06:52:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a664c0 msgr2=0x55eec8ada2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a664c0 0x55eec8ada2a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f0f800ee8a0 tx=0x7f0f800bce50 comp rx=0 tx=0).stop 2024-09-19T06:52:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 shutdown_connections 2024-09-19T06:52:06.719 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eec8a76510 0x55eec8ae4360 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eec8a664c0 0x55eec8ada2a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eec8a53330 0x55eec8a6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 --2- 172.21.15.37:0/1395964789 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eec8a52a10 0x55eec8a6aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 >> 172.21.15.37:0/1395964789 conn(0x55eec8a4f060 msgr2=0x55eec8a4d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f6dffb640 0 client.27629 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 shutdown_connections 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 1 -- 172.21.15.37:0/1395964789 wait complete. 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.718+0000 7f0f8fa25740 20 client.27629 trim_cache size 0 max 16384 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:52:06.720 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:52:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:52:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:52:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f700a34e0 msgr2=0x7f0f700a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.721 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f700a34e0 0x7f0f700a59a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f0f78009ca0 tx=0x7f0f78009840 comp rx=0 tx=0).stop 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 msgr2=0x7f0f8819ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f8819ee10 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f0f80005e30 tx=0x7f0f80003f10 comp rx=0 tx=0).stop 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 shutdown_connections 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f700a34e0 0x7f0f700a59a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f8810e480 0x7f0f8819ee10 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88101280 0x7f0f8819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 --2- 172.21.15.37:0/1138261698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f881008d0 0x7f0f8819a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 >> 172.21.15.37:0/1138261698 conn(0x7f0f880fc6b0 msgr2=0x7f0f880feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 shutdown_connections 2024-09-19T06:52:06.722 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:06.719+0000 7f0f8cbb0640 1 -- 172.21.15.37:0/1138261698 wait complete. 2024-09-19T06:52:06.763 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.763+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3cc02d7c0 con 0x7ff3cc101de0 2024-09-19T06:52:06.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.764+0000 7ff3c8ff9640 1 -- 172.21.15.37:0/1470946983 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7ff3bc09cb80 con 0x7ff3cc101de0 2024-09-19T06:52:06.766 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:52:06.766 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:52:06.766 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3b40a33b0 msgr2=0x7ff3b40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3b40a33b0 0x7ff3b40a5870 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009230 tx=0x7ff3c00058d0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc101de0 msgr2=0x7ff3cc0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc101de0 0x7ff3cc0744f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ff3bc00c920 tx=0x7ff3bc00cde0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 shutdown_connections 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff3cc106f10 0x7ff3cc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ff3b40a33b0 0x7ff3b40a5870 secure :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff3c0009230 tx=0x7ff3c00058d0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ff3cc102790 0x7ff3cc074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 --2- 172.21.15.37:0/1470946983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff3cc101de0 0x7ff3cc0744f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:06.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 >> 172.21.15.37:0/1470946983 conn(0x7ff3cc0fd8c0 msgr2=0x7ff3cc0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:06.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 shutdown_connections 2024-09-19T06:52:06.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:06.768+0000 7ff3d1f25640 1 -- 172.21.15.37:0/1470946983 wait complete. 2024-09-19T06:52:07.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:06 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1470946983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:07.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.71 MB/sec execute 394 sec latency 24711.518 ms 2024-09-19T06:52:07.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:06 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1470946983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:07.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:06 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1470946983' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:07.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:52:08.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:07 smithi037 ceph-mon[31211]: pgmap v324: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:08.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:07 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:08.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:07 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:08.108 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:08.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:07 smithi096 ceph-mon[35004]: pgmap v324: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:08.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:07 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:08.175 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:07 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:08.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.69 MB/sec execute 395 sec latency 25711.560 ms 2024-09-19T06:52:08.264 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:07 smithi151 ceph-mon[35236]: pgmap v324: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 65 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 1.8 MiB/s wr, 133 op/s 2024-09-19T06:52:08.264 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:07 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:08.264 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:07 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 -- 172.21.15.37:0/2710473072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba010c170 msgr2=0x7fcba010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 --2- 172.21.15.37:0/2710473072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba010c170 0x7fcba010e610 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcb98009ca0 tx=0x7fcb9802a950 comp rx=0 tx=0).stop 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 -- 172.21.15.37:0/2710473072 shutdown_connections 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 --2- 172.21.15.37:0/2710473072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba010c170 0x7fcba010e610 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 --2- 172.21.15.37:0/2710473072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba0103660 0x7fcba010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 --2- 172.21.15.37:0/2710473072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcba0102d40 0x7fcba0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 -- 172.21.15.37:0/2710473072 >> 172.21.15.37:0/2710473072 conn(0x7fcba00fc7a0 msgr2=0x7fcba00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 -- 172.21.15.37:0/2710473072 shutdown_connections 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.644+0000 7fcba513c640 1 -- 172.21.15.37:0/2710473072 wait complete. 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.645+0000 7fcba513c640 1 Processor -- start 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.645+0000 7fcba513c640 1 -- start start 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcba513c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102d40 0x7fcba0198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcba513c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcba0103660 0x7fcba0198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcba513c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba010c170 0x7fcba019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcba513c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0111f70 con 0x7fcba0102d40 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcba513c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0111df0 con 0x7fcba010c170 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcba513c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01120f0 con 0x7fcba0103660 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102d40 0x7fcba0198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102d40 0x7fcba0198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56678/0 (socket says 172.21.15.37:56678) 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1294300275 learned_addr learned my addr 172.21.15.37:0/1294300275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcba0103660 0x7fcba0198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba010c170 0x7fcba019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1294300275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcba0103660 msgr2=0x7fcba0198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcba0103660 0x7fcba0198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1294300275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba010c170 msgr2=0x7fcba019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba010c170 0x7fcba019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1294300275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba019d3c0 con 0x7fcba0102d40 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba010c170 0x7fcba019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102d40 0x7fcba0198410 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d770 tx=0x7fcb9000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcb87fff640 1 -- 172.21.15.37:0/1294300275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90017070 con 0x7fcba0102d40 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.646+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba019d6b0 con 0x7fcba0102d40 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.647+0000 7fcb87fff640 1 -- 172.21.15.37:0/1294300275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb90005500 con 0x7fcba0102d40 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.647+0000 7fcb87fff640 1 -- 172.21.15.37:0/1294300275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90004e00 con 0x7fcba0102d40 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.647+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba01a4f90 con 0x7fcba0102d40 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcb87fff640 1 -- 172.21.15.37:0/1294300275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcb90004fa0 con 0x7fcba0102d40 2024-09-19T06:52:08.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 Processor -- start 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 -- start start 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab6986b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986b5d0 0x55ab6986ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab69876370 0x55ab69878760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab697d12f0 con 0x55ab6986b5d0 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab69778cf0 con 0x55ab6986cb40 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcba7515740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ab697cd2b0 con 0x55ab69876370 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.649+0000 7fcb9ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986b5d0 0x55ab6986ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986b5d0 0x55ab6986ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56684/0 (socket says 172.21.15.37:56684) 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1280540643 learned_addr learned my addr 172.21.15.37:0/1280540643 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb87fff640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcb880a33b0 0x7fcb880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab69876370 0x55ab69878760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.658 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab6986b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcb880a33b0 0x7fcb880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb87fff640 1 -- 172.21.15.37:0/1294300275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fcb900cf800 con 0x7fcba0102d40 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1280540643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab69876370 msgr2=0x55ab69878760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab69876370 0x55ab69878760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1280540643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 msgr2=0x55ab6986b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab6986b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 -- 172.21.15.37:0/1280540643 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6975d3a0 con 0x55ab6986b5d0 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986b5d0 0x55ab6986ba10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcb9009b780 tx=0x7fcb90009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=434cf9a257ff5a75 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb857fa640 1 -- 172.21.15.37:0/1280540643 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90020070 con 0x55ab6986b5d0 2024-09-19T06:52:08.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb857fa640 1 -- 172.21.15.37:0/1280540643 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9009c180 con 0x55ab6986b5d0 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb857fa640 1 -- 172.21.15.37:0/1280540643 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb900d2e60 con 0x55ab6986b5d0 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.650+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcb880a33b0 0x7fcb880a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005fc0 tx=0x7fcb980095c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 -- 172.21.15.37:0/1280540643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986b5d0 msgr2=0x55ab6986ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986b5d0 0x55ab6986ba10 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcb9009b780 tx=0x7fcb90009c30 comp rx=0 tx=0).stop 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 -- 172.21.15.37:0/1280540643 shutdown_connections 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab69876370 0x55ab69878760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986b5d0 0x55ab6986ba10 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 --2- 172.21.15.37:0/1280540643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab6986b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 -- 172.21.15.37:0/1280540643 >> 172.21.15.37:0/1280540643 conn(0x55ab69834310 msgr2=0x55ab69867c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 -- 172.21.15.37:0/1280540643 shutdown_connections 2024-09-19T06:52:08.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.651+0000 7fcba7515740 1 -- 172.21.15.37:0/1280540643 wait complete. 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 -- start start 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 Processor -- start 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69875d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab69872ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698735e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ab69879010 con 0x55ab69876370 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ab6978c060 con 0x55ab6986cb40 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ab69879190 con 0x55ab6986b5d0 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcba7515740 10 client.0 _ll_register_callbacks cb 0x55ab694df8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698735e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698735e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56688/0 (socket says 172.21.15.37:56688) 2024-09-19T06:52:08.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9f59e640 1 -- 172.21.15.37:0/3445032123 learned_addr learned my addr 172.21.15.37:0/3445032123 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69875d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab69872ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9f59e640 1 -- 172.21.15.37:0/3445032123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 msgr2=0x55ab69875d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69875d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9f59e640 1 -- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 msgr2=0x55ab69872ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.652+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab69872ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb9f59e640 1 -- 172.21.15.37:0/3445032123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab695423c0 con 0x55ab69876370 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab69872ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:08.662 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698735e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fcb98030040 tx=0x7fcb9802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6c5282438b72c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb82ff5640 1 -- 172.21.15.37:0/3445032123 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9804e030 con 0x55ab69876370 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb82ff5640 1 -- 172.21.15.37:0/3445032123 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fcb98044070 con 0x55ab69876370 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb9009c180 con 0x55ab69876370 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb82ff5640 1 -- 172.21.15.37:0/3445032123 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb98034040 con 0x55ab69876370 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcba7515740 10 client.18303 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ab69884ad0 con 0x55ab69876370 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb82ff5640 1 -- 172.21.15.37:0/3445032123 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb9009c180 con 0x55ab69876370 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.653+0000 7fcb82ff5640 1 -- 172.21.15.37:0/3445032123 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcb98047050 con 0x55ab69876370 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcba7515740 10 client.18303 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcba7515740 10 client.18303 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcba7515740 20 client.18303 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:08.663 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcba7515740 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab69886030 0x55ab69888490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcba7515740 4 client.18303 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ab697de860 con 0x55ab69886030 2024-09-19T06:52:08.664 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab69886030 0x55ab69888490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcb82ff5640 10 client.18303 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.654+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab69886030 0x55ab69888490 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f94d2158885fbef0 server_cookie=b57bb895c53ae4b5 in_seq=0 out_seq=0 2024-09-19T06:52:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcb82ff5640 1 -- 172.21.15.37:0/3445032123 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ab697de860 con 0x55ab69886030 2024-09-19T06:52:08.665 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcb82ff5640 10 client.18303 handle_command_reply: tid=0 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 client.18303 shutdown 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 msgr2=0x55ab698735e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698735e0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fcb98030040 tx=0x7fcb9802a950 comp rx=0 tx=0).stop 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 shutdown_connections 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab69886030 0x55ab69888490 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698735e0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986cb40 0x55ab69872ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcb82ff5640 0 client.18303 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 --2- 172.21.15.37:0/3445032123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69875d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 >> 172.21.15.37:0/3445032123 conn(0x55ab69834310 msgr2=0x55ab6986d990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.666 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 shutdown_connections 2024-09-19T06:52:08.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 1 -- 172.21.15.37:0/3445032123 wait complete. 2024-09-19T06:52:08.667 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.655+0000 7fcba7515740 20 client.18303 trim_cache size 0 max 16384 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.668+0000 7fcba7515740 1 Processor -- start 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.668+0000 7fcba7515740 1 -- start start 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcba7515740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69867e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcba7515740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986cb40 0x55ab698822b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcba7515740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab69876370 0x55ab6987e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcba7515740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01b0290 con 0x55ab6986cb40 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcba7515740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01adb60 con 0x55ab69876370 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcba7515740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01ab9d0 con 0x55ab6986b5d0 2024-09-19T06:52:08.671 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986cb40 0x55ab698822b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986cb40 0x55ab698822b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56702/0 (socket says 172.21.15.37:56702) 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 -- 172.21.15.37:0/4160909582 learned_addr learned my addr 172.21.15.37:0/4160909582 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab69876370 0x55ab6987e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69867e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 -- 172.21.15.37:0/4160909582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 msgr2=0x55ab69867e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69867e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 -- 172.21.15.37:0/4160909582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab69876370 msgr2=0x55ab6987e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab69876370 0x55ab6987e800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 -- 172.21.15.37:0/4160909582 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab69875960 con 0x55ab6986cb40 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986cb40 0x55ab698822b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcb94004650 tx=0x7fcb9400bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae19cebb8868ab server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb84ff9640 1 -- 172.21.15.37:0/4160909582 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94003d50 con 0x55ab6986cb40 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb84ff9640 1 -- 172.21.15.37:0/4160909582 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb94005500 con 0x55ab6986cb40 2024-09-19T06:52:08.672 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.669+0000 7fcb84ff9640 1 -- 172.21.15.37:0/4160909582 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb94011440 con 0x55ab6986cb40 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 -- 172.21.15.37:0/4160909582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986cb40 msgr2=0x55ab698822b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986cb40 0x55ab698822b0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fcb94004650 tx=0x7fcb9400bbc0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 -- 172.21.15.37:0/4160909582 shutdown_connections 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab69876370 0x55ab6987e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab6986cb40 0x55ab698822b0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 --2- 172.21.15.37:0/4160909582 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986b5d0 0x55ab69867e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 -- 172.21.15.37:0/4160909582 >> 172.21.15.37:0/4160909582 conn(0x55ab69834310 msgr2=0x55ab69867700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 -- 172.21.15.37:0/4160909582 shutdown_connections 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 -- 172.21.15.37:0/4160909582 wait complete. 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 -- start start 2024-09-19T06:52:08.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.670+0000 7fcba7515740 1 Processor -- start 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcba7515740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986b5d0 0x55ab698e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcba7515740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986cb40 0x55ab698e9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcba7515740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcba7515740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01ac020 con 0x55ab69876370 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcba7515740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01ad490 con 0x55ab6986b5d0 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcba7515740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcba01ac6f0 con 0x55ab6986cb40 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcba7515740 10 client.0 _ll_register_callbacks cb 0x7fcb98009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:08.677 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698f3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56714/0 (socket says 172.21.15.37:56714) 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 -- 172.21.15.37:0/2666800758 learned_addr learned my addr 172.21.15.37:0/2666800758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986b5d0 0x55ab698e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986cb40 0x55ab698e9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 -- 172.21.15.37:0/2666800758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986cb40 msgr2=0x55ab698e9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986cb40 0x55ab698e9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 -- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986b5d0 msgr2=0x55ab698e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986b5d0 0x55ab698e8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 -- 172.21.15.37:0/2666800758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab698f42b0 con 0x55ab69876370 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986b5d0 0x55ab698e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9e59c640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986cb40 0x55ab698e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:08.678 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb9f59e640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698f3d70 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fcb98004010 tx=0x7fcb98003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98c91e4194a0abb2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.671+0000 7fcb82ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcb82ff5640 1 -- 172.21.15.37:0/2666800758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9805f030 con 0x55ab69876370 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcb82ff5640 1 -- 172.21.15.37:0/2666800758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8615+0+0 (secure 0 0 0) 0x7fcb98044040 con 0x55ab69876370 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcb94005500 con 0x55ab69876370 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcb82ff5640 1 -- 172.21.15.37:0/2666800758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fcb98069050 con 0x55ab69876370 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 10 client.18309 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ab698f5580 con 0x55ab69876370 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcb82ff5640 1 -- 172.21.15.37:0/2666800758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb94005500 con 0x55ab69876370 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcb82ff5640 1 -- 172.21.15.37:0/2666800758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fcb98047070 con 0x55ab69876370 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 10 client.18309 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 10 client.18309 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 20 client.18309 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:08.679 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab698f5d80 0x55ab698f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 4 client.18309 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.672+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ab697d8d70 con 0x55ab698f5d80 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab698f5d80 0x55ab698f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcb9ed9d640 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab698f5d80 0x55ab698f81a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b5e069e4692cc1b server_cookie=f51e28eac9abe27f in_seq=0 out_seq=0 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcb82ff5640 10 client.18309 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcb82ff5640 1 -- 172.21.15.37:0/2666800758 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ab697d8d70 con 0x55ab698f5d80 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcb82ff5640 10 client.18309 handle_command_reply: tid=0 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 client.18309 shutdown 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 msgr2=0x55ab698f3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698f3d70 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fcb98004010 tx=0x7fcb98003e80 comp rx=0 tx=0).stop 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 shutdown_connections 2024-09-19T06:52:08.680 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55ab698f5d80 0x55ab698f81a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ab69876370 0x55ab698f3d70 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ab6986cb40 0x55ab698e9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 --2- 172.21.15.37:0/2666800758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ab6986b5d0 0x55ab698e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 >> 172.21.15.37:0/2666800758 conn(0x55ab69834310 msgr2=0x55ab69874850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcb82ff5640 0 client.18309 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 shutdown_connections 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 1 -- 172.21.15.37:0/2666800758 wait complete. 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.673+0000 7fcba7515740 20 client.18309 trim_cache size 0 max 16384 2024-09-19T06:52:08.681 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:52:08.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcb880a33b0 msgr2=0x7fcb880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcb880a33b0 0x7fcb880a5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005fc0 tx=0x7fcb980095c0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102d40 msgr2=0x7fcba0198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:08.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102d40 0x7fcba0198410 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcb9000d770 tx=0x7fcb9000dc30 comp rx=0 tx=0).stop 2024-09-19T06:52:08.682 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 shutdown_connections 2024-09-19T06:52:08.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcba010c170 0x7fcba019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcba0103660 0x7fcba0198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcb880a33b0 0x7fcb880a5870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 --2- 172.21.15.37:0/1294300275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcba0102d40 0x7fcba0198410 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:08.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 >> 172.21.15.37:0/1294300275 conn(0x7fcba00fc7a0 msgr2=0x7fcba00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:08.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 shutdown_connections 2024-09-19T06:52:08.683 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:08.674+0000 7fcba513c640 1 -- 172.21.15.37:0/1294300275 wait complete. 2024-09-19T06:52:09.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.67 MB/sec execute 396 sec latency 26711.659 ms 2024-09-19T06:52:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi037 to 177.0M 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: Unable to set osd_memory_target on smithi037 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: Unable to set osd_memory_target on smithi096 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:52:09.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi037 to 177.0M 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: Unable to set osd_memory_target on smithi037 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T06:52:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: Unable to set osd_memory_target on smithi096 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-19T06:52:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:52:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:52:09.277 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi037 to 177.0M 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: Unable to set osd_memory_target on smithi037 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-19T06:52:09.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: Adjusting osd_memory_target on smithi096 to 176.0M 2024-09-19T06:52:09.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: Unable to set osd_memory_target on smithi096 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-19T06:52:09.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:52:09.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:52:09.365 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:52:09.537 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:52:09.767 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:09.803 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:10.226 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.65 MB/sec execute 397 sec latency 27711.707 ms 2024-09-19T06:52:10.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 -- 172.21.15.37:0/1254654901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa010c170 msgr2=0x7ffaa010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/1254654901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa010c170 0x7ffaa010e610 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ffa98009ca0 tx=0x7ffa9802a950 comp rx=0 tx=0).stop 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 -- 172.21.15.37:0/1254654901 shutdown_connections 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/1254654901 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa010c170 0x7ffaa010e610 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/1254654901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa0103660 0x7ffaa010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/1254654901 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa0102d40 0x7ffaa0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 -- 172.21.15.37:0/1254654901 >> 172.21.15.37:0/1254654901 conn(0x7ffaa00fc7a0 msgr2=0x7ffaa00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.262+0000 7ffaa5d59640 1 -- 172.21.15.37:0/1254654901 shutdown_connections 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.263+0000 7ffaa5d59640 1 -- 172.21.15.37:0/1254654901 wait complete. 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.263+0000 7ffaa5d59640 1 Processor -- start 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.263+0000 7ffaa5d59640 1 -- start start 2024-09-19T06:52:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffaa5d59640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa0102d40 0x7ffaa0198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffaa5d59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa0103660 0x7ffaa01989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffaa5d59640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa010c170 0x7ffaa019cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffaa5d59640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0111fb0 con 0x7ffaa0103660 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffaa5d59640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0111e30 con 0x7ffaa010c170 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffaa5d59640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0112130 con 0x7ffaa0102d40 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa0103660 0x7ffaa01989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa0103660 0x7ffaa01989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40670/0 (socket says 172.21.15.37:40670) 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 -- 172.21.15.37:0/3404694853 learned_addr learned my addr 172.21.15.37:0/3404694853 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa0102d40 0x7ffaa0198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa010c170 0x7ffaa019cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 -- 172.21.15.37:0/3404694853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa0102d40 msgr2=0x7ffaa0198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa0102d40 0x7ffaa0198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 -- 172.21.15.37:0/3404694853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa010c170 msgr2=0x7ffaa019cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa010c170 0x7ffaa019cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 -- 172.21.15.37:0/3404694853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa019d420 con 0x7ffaa0103660 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa0102d40 0x7ffaa0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa010c170 0x7ffaa019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.264+0000 7ffa9effd640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa0103660 0x7ffaa01989b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d740 tx=0x7ffa9400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.265+0000 7ffa9cff9640 1 -- 172.21.15.37:0/3404694853 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94017070 con 0x7ffaa0103660 2024-09-19T06:52:10.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.265+0000 7ffa9cff9640 1 -- 172.21.15.37:0/3404694853 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9400de50 con 0x7ffaa0103660 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.265+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa019d710 con 0x7ffaa0103660 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.265+0000 7ffa9cff9640 1 -- 172.21.15.37:0/3404694853 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400f620 con 0x7ffaa0103660 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.265+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa01a4ff0 con 0x7ffaa0103660 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 Processor -- start 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffa9cff9640 1 -- 172.21.15.37:0/3404694853 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa94009c30 con 0x7ffaa0103660 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 -- start start 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e58f6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e58f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e5907b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2e585c2f0 con 0x55d2e58f65d0 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2e5803cf0 con 0x55d2e58f7b40 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.267+0000 7ffaa8132740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d2e58582b0 con 0x55d2e5905780 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e58f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e58f6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40684/0 (socket says 172.21.15.37:40684) 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1592087320 learned_addr learned my addr 172.21.15.37:0/1592087320 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e58f6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9effd640 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e5907b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1592087320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 msgr2=0x55d2e5907b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e5907b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1592087320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 msgr2=0x55d2e58f6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e58f6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1592087320 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e57e83a0 con 0x55d2e58f65d0 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e58f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e58f6a10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007850 tx=0x7ffa90007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f34d3495563d7f6b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa85ffb640 1 -- 172.21.15.37:0/1592087320 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000d950 con 0x55d2e58f65d0 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9cff9640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa880a3430 0x7ffa880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa85ffb640 1 -- 172.21.15.37:0/1592087320 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa90018070 con 0x55d2e58f65d0 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa85ffb640 1 -- 172.21.15.37:0/1592087320 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa900123e0 con 0x55d2e58f65d0 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.268+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa880a3430 0x7ffa880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffa9cff9640 1 -- 172.21.15.37:0/3404694853 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ffa940cf980 con 0x7ffaa0103660 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 -- 172.21.15.37:0/1592087320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 msgr2=0x55d2e58f6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa880a3430 0x7ffa880a58f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006010 tx=0x7ffa98005d80 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e58f6a10 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007850 tx=0x7ffa90007d10 comp rx=0 tx=0).stop 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 -- 172.21.15.37:0/1592087320 shutdown_connections 2024-09-19T06:52:10.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e5907b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e58f6a10 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1592087320 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e58f6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 -- 172.21.15.37:0/1592087320 >> 172.21.15.37:0/1592087320 conn(0x55d2e58bf310 msgr2=0x55d2e58f2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 -- 172.21.15.37:0/1592087320 shutdown_connections 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.269+0000 7ffaa8132740 1 -- 172.21.15.37:0/1592087320 wait complete. 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 -- start start 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 Processor -- start 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e5905110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e59023c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e5902940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d2e5908480 con 0x55d2e5905780 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2e5817060 con 0x55d2e58f65d0 2024-09-19T06:52:10.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d2e5908600 con 0x55d2e58f7b40 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffaa8132740 10 client.0 _ll_register_callbacks cb 0x55d2e556a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffa9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e5902940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffa9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e5902940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40700/0 (socket says 172.21.15.37:40700) 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffa9ffff640 1 -- 172.21.15.37:0/1282139049 learned_addr learned my addr 172.21.15.37:0/1282139049 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffa9effd640 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e59023c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.270+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e5905110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa9ffff640 1 -- 172.21.15.37:0/1282139049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 msgr2=0x55d2e59023c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e59023c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa9ffff640 1 -- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 msgr2=0x55d2e5905110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e5905110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa9ffff640 1 -- 172.21.15.37:0/1282139049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e55cd3c0 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e5902940 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ffa9802f540 tx=0x7ffa9802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad9150b388816bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa7effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa7effd640 1 -- 172.21.15.37:0/1282139049 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98034040 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa90018070 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa7effd640 1 -- 172.21.15.37:0/1282139049 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ffa98003fb0 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa7effd640 1 -- 172.21.15.37:0/1282139049 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa980041b0 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffaa8132740 10 client.18321 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d2e5913f00 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa7effd640 1 -- 172.21.15.37:0/1282139049 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa90018070 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.271+0000 7ffa7effd640 1 -- 172.21.15.37:0/1282139049 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffa9804f070 con 0x55d2e5905780 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffaa8132740 10 client.18321 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:10.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffaa8132740 10 client.18321 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffaa8132740 20 client.18321 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e5915310 0x55d2e5917780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffaa8132740 4 client.18321 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2e5869860 con 0x55d2e5915310 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e5915310 0x55d2e5917780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e5915310 0x55d2e5917780 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4e6dc3c356e8f80f server_cookie=4a74fbe862ba1dae in_seq=0 out_seq=0 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.272+0000 7ffa7effd640 10 client.18321 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffa7effd640 1 -- 172.21.15.37:0/1282139049 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d2e5869860 con 0x55d2e5915310 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffa7effd640 10 client.18321 handle_command_reply: tid=0 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 client.18321 shutdown 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 msgr2=0x55d2e5902940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e5902940 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7ffa9802f540 tx=0x7ffa9802a950 comp rx=0 tx=0).stop 2024-09-19T06:52:10.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 shutdown_connections 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e5915310 0x55d2e5917780 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e5902940 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e59023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffa7effd640 0 client.18321 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1282139049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e5905110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.273+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 >> 172.21.15.37:0/1282139049 conn(0x55d2e58bf310 msgr2=0x55d2e58f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.274+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 shutdown_connections 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.274+0000 7ffaa8132740 1 -- 172.21.15.37:0/1282139049 wait complete. 2024-09-19T06:52:10.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.274+0000 7ffaa8132740 20 client.18321 trim_cache size 0 max 16384 2024-09-19T06:52:10.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 Processor -- start 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 -- start start 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e58f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e5911720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e591d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01ae050 con 0x55d2e5905780 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01ab920 con 0x55d2e58f65d0 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffaa8132740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01a9790 con 0x55d2e58f7b40 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffa9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e591d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffa9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e591d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40708/0 (socket says 172.21.15.37:40708) 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffa9ffff640 1 -- 172.21.15.37:0/3618180442 learned_addr learned my addr 172.21.15.37:0/3618180442 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e58f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.288+0000 7ffa9effd640 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e5911720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa9ffff640 1 -- 172.21.15.37:0/3618180442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 msgr2=0x55d2e5911720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e5911720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa9ffff640 1 -- 172.21.15.37:0/3618180442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 msgr2=0x55d2e58f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e58f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa9ffff640 1 -- 172.21.15.37:0/3618180442 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2e58f9120 con 0x55d2e5905780 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e591d050 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ffa9802f960 tx=0x7ffa980093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80526dfb461d8ab9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa857fa640 1 -- 172.21.15.37:0/3618180442 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98034040 con 0x55d2e5905780 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa857fa640 1 -- 172.21.15.37:0/3618180442 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa98030440 con 0x55d2e5905780 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.289+0000 7ffa857fa640 1 -- 172.21.15.37:0/3618180442 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98030760 con 0x55d2e5905780 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 -- 172.21.15.37:0/3618180442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 msgr2=0x55d2e591d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e591d050 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7ffa9802f960 tx=0x7ffa980093e0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 -- 172.21.15.37:0/3618180442 shutdown_connections 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e5905780 0x55d2e591d050 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e58f7b40 0x55d2e5911720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.296 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 --2- 172.21.15.37:0/3618180442 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f65d0 0x55d2e58f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 -- 172.21.15.37:0/3618180442 >> 172.21.15.37:0/3618180442 conn(0x55d2e58bf310 msgr2=0x55d2e58f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 -- 172.21.15.37:0/3618180442 shutdown_connections 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.290+0000 7ffaa8132740 1 -- 172.21.15.37:0/3618180442 wait complete. 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.291+0000 7ffaa8132740 1 -- start start 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.291+0000 7ffaa8132740 1 Processor -- start 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.291+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e5909710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.291+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e5909c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.291+0000 7ffaa8132740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e590a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.291+0000 7ffaa8132740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01a9de0 con 0x55d2e58f65d0 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.291+0000 7ffaa8132740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01ab250 con 0x55d2e58f7b40 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e5909710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffaa8132740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01aa4b0 con 0x55d2e5905780 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e5909710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40712/0 (socket says 172.21.15.37:40712) 2024-09-19T06:52:10.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1320179415 learned_addr learned my addr 172.21.15.37:0/1320179415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffaa8132740 10 client.0 _ll_register_callbacks cb 0x7ffa98009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9effd640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e5909c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9ffff640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e590a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1320179415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 msgr2=0x55d2e590a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e590a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 msgr2=0x55d2e5909c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e5909c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 -- 172.21.15.37:0/1320179415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2e597f180 con 0x55d2e58f65d0 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9effd640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e5909c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa9f7fe640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e5909710 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007850 tx=0x7ffa90007380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf65e8f679c8b61 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa7effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa7effd640 1 -- 172.21.15.37:0/1320179415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9000d950 con 0x55d2e58f65d0 2024-09-19T06:52:10.298 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.292+0000 7ffa7effd640 1 -- 172.21.15.37:0/1320179415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ffa9000daf0 con 0x55d2e58f65d0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa98030440 con 0x55d2e58f65d0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffa7effd640 1 -- 172.21.15.37:0/1320179415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffa9000dcf0 con 0x55d2e58f65d0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 10 client.18327 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d2e5980400 con 0x55d2e58f65d0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffa7effd640 1 -- 172.21.15.37:0/1320179415 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa98030440 con 0x55d2e58f65d0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffa7effd640 1 -- 172.21.15.37:0/1320179415 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffa90037070 con 0x55d2e58f65d0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 10 client.18327 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 10 client.18327 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 20 client.18327 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e59807f0 0x55d2e5982be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 4 client.18327 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d2e5863d70 con 0x55d2e59807f0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.293+0000 7ffa9effd640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e59807f0 0x55d2e5982be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:10.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffa7effd640 10 client.18327 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffa9effd640 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e59807f0 0x55d2e5982be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8237d237b4b77ba8 server_cookie=b34f2c7cd8a99840 in_seq=0 out_seq=0 2024-09-19T06:52:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffa7effd640 1 -- 172.21.15.37:0/1320179415 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d2e5863d70 con 0x55d2e59807f0 2024-09-19T06:52:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffa7effd640 10 client.18327 handle_command_reply: tid=0 2024-09-19T06:52:10.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffaa8132740 1 client.18327 shutdown 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 msgr2=0x55d2e5909710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e5909710 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7ffa90007850 tx=0x7ffa90007380 comp rx=0 tx=0).stop 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.294+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 shutdown_connections 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55d2e59807f0 0x55d2e5982be0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d2e5905780 0x55d2e590a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2e58f7b40 0x55d2e5909c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffa7effd640 0 client.18327 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 1 --2- 172.21.15.37:0/1320179415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d2e58f65d0 0x55d2e5909710 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 >> 172.21.15.37:0/1320179415 conn(0x55d2e58bf310 msgr2=0x55d2e58f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 shutdown_connections 2024-09-19T06:52:10.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 1 -- 172.21.15.37:0/1320179415 wait complete. 2024-09-19T06:52:10.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa8132740 20 client.18327 trim_cache size 0 max 16384 2024-09-19T06:52:10.302 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:52:10.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa880a3430 msgr2=0x7ffa880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.295+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa880a3430 0x7ffa880a58f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ffa98006010 tx=0x7ffa98005d80 comp rx=0 tx=0).stop 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa0103660 msgr2=0x7ffaa01989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa0103660 0x7ffaa01989b0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400d740 tx=0x7ffa9400dc00 comp rx=0 tx=0).stop 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 shutdown_connections 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa010c170 0x7ffaa019cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa880a3430 0x7ffa880a58f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaa0103660 0x7ffaa01989b0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 --2- 172.21.15.37:0/3404694853 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaa0102d40 0x7ffaa0198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 >> 172.21.15.37:0/3404694853 conn(0x7ffaa00fc7a0 msgr2=0x7ffaa00fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 shutdown_connections 2024-09-19T06:52:10.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:10.296+0000 7ffaa5d59640 1 -- 172.21.15.37:0/3404694853 wait complete. 2024-09-19T06:52:10.755 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:10 smithi096 ceph-mon[35004]: pgmap v325: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.8 MiB/s wr, 111 op/s 2024-09-19T06:52:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:10 smithi096 ceph-mon[35004]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T06:52:10.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:10 smithi037 ceph-mon[31211]: pgmap v325: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.8 MiB/s wr, 111 op/s 2024-09-19T06:52:10.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:10 smithi037 ceph-mon[31211]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T06:52:10.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:10 smithi151 ceph-mon[35236]: pgmap v325: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 1.8 MiB/s wr, 111 op/s 2024-09-19T06:52:10.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:10 smithi151 ceph-mon[35236]: osdmap e96: 12 total, 12 up, 12 in 2024-09-19T06:52:11.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.63 MB/sec execute 398 sec latency 28711.752 ms 2024-09-19T06:52:11.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:52:11.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 -- 172.21.15.37:0/75262295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401053e0 msgr2=0x7f994010cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 --2- 172.21.15.37:0/75262295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401053e0 0x7f994010cf60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f9934009900 tx=0x7f993402a3f0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 -- 172.21.15.37:0/75262295 shutdown_connections 2024-09-19T06:52:11.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 --2- 172.21.15.37:0/75262295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401053e0 0x7f994010cf60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 --2- 172.21.15.37:0/75262295 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9940100c60 0x7f9940104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 --2- 172.21.15.37:0/75262295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99401002b0 0x7f9940100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 -- 172.21.15.37:0/75262295 >> 172.21.15.37:0/75262295 conn(0x7f9940077070 msgr2=0x7f99400fe480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:11.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 -- 172.21.15.37:0/75262295 shutdown_connections 2024-09-19T06:52:11.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.805+0000 7f993ffff640 1 -- 172.21.15.37:0/75262295 wait complete. 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.806+0000 7f993ffff640 1 Processor -- start 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.806+0000 7f993ffff640 1 -- start start 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993ffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401002b0 0x7f99401a34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9940100c60 0x7f99401a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99401053e0 0x7f99401099c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993ffff640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f994010fbb0 con 0x7f9940100c60 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993ffff640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f994010fa30 con 0x7f99401002b0 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993ffff640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f994010fd30 con 0x7f99401053e0 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401002b0 0x7f99401a34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993effd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401002b0 0x7f99401a34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:37352/0 (socket says 172.21.15.37:37352) 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993effd640 1 -- 172.21.15.37:0/4047117691 learned_addr learned my addr 172.21.15.37:0/4047117691 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:11.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993e7fc640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9940100c60 0x7f99401a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.807+0000 7f993f7fe640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99401053e0 0x7f99401099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993effd640 1 -- 172.21.15.37:0/4047117691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99401053e0 msgr2=0x7f99401099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993effd640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99401053e0 0x7f99401099c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993effd640 1 -- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9940100c60 msgr2=0x7f99401a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993effd640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9940100c60 0x7f99401a3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993effd640 1 -- 172.21.15.37:0/4047117691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994010a250 con 0x7f99401002b0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993e7fc640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9940100c60 0x7f99401a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993f7fe640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99401053e0 0x7f99401099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993effd640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401002b0 0x7f99401a34f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f993000d950 tx=0x7f993000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f9927fff640 1 -- 172.21.15.37:0/4047117691 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930017070 con 0x7f99401002b0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994010a540 con 0x7f99401002b0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f9927fff640 1 -- 172.21.15.37:0/4047117691 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9930009c30 con 0x7f99401002b0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f994006bde0 con 0x7f99401002b0 2024-09-19T06:52:11.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.808+0000 7f9927fff640 1 -- 172.21.15.37:0/4047117691 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9930017070 con 0x7f99401002b0 2024-09-19T06:52:11.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.810+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f992c005290 con 0x7f99401002b0 2024-09-19T06:52:11.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.811+0000 7f9927fff640 1 -- 172.21.15.37:0/4047117691 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9930004020 con 0x7f99401002b0 2024-09-19T06:52:11.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.812+0000 7f9927fff640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99280a37b0 0x7f99280a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:11.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.813+0000 7f993e7fc640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99280a37b0 0x7f99280a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:11.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.813+0000 7f9927fff640 1 -- 172.21.15.37:0/4047117691 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f9930009a30 con 0x7f99401002b0 2024-09-19T06:52:11.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.813+0000 7f993e7fc640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99280a37b0 0x7f99280a5c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f994010aae0 tx=0x7f9938009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:11.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:11.815+0000 7f9927fff640 1 -- 172.21.15.37:0/4047117691 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f993009b530 con 0x7f99401002b0 2024-09-19T06:52:11.835 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.044+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f992c005580 con 0x7f99401002b0 2024-09-19T06:52:12.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.045+0000 7f9927fff640 1 -- 172.21.15.37:0/4047117691 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7f993009f830 con 0x7f99401002b0 2024-09-19T06:52:12.046 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:52:12.046 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:52:12.047 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:52:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99280a37b0 msgr2=0x7f99280a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99280a37b0 0x7f99280a5c70 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f994010aae0 tx=0x7f9938009500 comp rx=0 tx=0).stop 2024-09-19T06:52:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401002b0 msgr2=0x7f99401a34f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401002b0 0x7f99401a34f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f993000d950 tx=0x7f993000de10 comp rx=0 tx=0).stop 2024-09-19T06:52:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 shutdown_connections 2024-09-19T06:52:12.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f99401053e0 0x7f99401099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f99280a37b0 0x7f99280a5c70 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9940100c60 0x7f99401a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 --2- 172.21.15.37:0/4047117691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f99401002b0 0x7f99401a34f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.047+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 >> 172.21.15.37:0/4047117691 conn(0x7f9940077070 msgr2=0x7f9940103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.048+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 shutdown_connections 2024-09-19T06:52:12.049 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.048+0000 7f993ffff640 1 -- 172.21.15.37:0/4047117691 wait complete. 2024-09-19T06:52:12.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.61 MB/sec execute 399 sec latency 29711.827 ms 2024-09-19T06:52:12.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 -- 172.21.15.37:0/4061224892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664106a80 msgr2=0x7fa664106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 --2- 172.21.15.37:0/4061224892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664106a80 0x7fa664106e60 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa654009930 tx=0x7fa65402a420 comp rx=0 tx=0).stop 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 -- 172.21.15.37:0/4061224892 shutdown_connections 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 --2- 172.21.15.37:0/4061224892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa664101420 0x7fa66410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 --2- 172.21.15.37:0/4061224892 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa664100a80 0x7fa664100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 --2- 172.21.15.37:0/4061224892 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664106a80 0x7fa664106e60 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 -- 172.21.15.37:0/4061224892 >> 172.21.15.37:0/4061224892 conn(0x7fa6640fc7a0 msgr2=0x7fa6640febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 -- 172.21.15.37:0/4061224892 shutdown_connections 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.478+0000 7fa6697f6640 1 -- 172.21.15.37:0/4061224892 wait complete. 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 Processor -- start 2024-09-19T06:52:12.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 -- start start 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa664100a80 0x7fa664199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664101420 0x7fa664199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa664106a80 0x7fa6641a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa664110e80 con 0x7fa664101420 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa664110d00 con 0x7fa664106a80 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6697f6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa664111000 con 0x7fa664100a80 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa65a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664101420 0x7fa664199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa65a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664101420 0x7fa664199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40750/0 (socket says 172.21.15.37:40750) 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa65a7fc640 1 -- 172.21.15.37:0/1534356629 learned_addr learned my addr 172.21.15.37:0/1534356629 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa664106a80 0x7fa6641a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.479+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa664100a80 0x7fa664199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa65a7fc640 1 -- 172.21.15.37:0/1534356629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa664100a80 msgr2=0x7fa664199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa664100a80 0x7fa664199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa65a7fc640 1 -- 172.21.15.37:0/1534356629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa664106a80 msgr2=0x7fa6641a4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa664106a80 0x7fa6641a4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa65a7fc640 1 -- 172.21.15.37:0/1534356629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6641a48d0 con 0x7fa664101420 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa664106a80 0x7fa6641a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664101420 0x7fa664199850 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c970 tx=0x7fa64c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa660ff9640 1 -- 172.21.15.37:0/1534356629 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c007bb0 con 0x7fa664101420 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6641a4aa0 con 0x7fa664101420 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa660ff9640 1 -- 172.21.15.37:0/1534356629 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa64c007d50 con 0x7fa664101420 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa660ff9640 1 -- 172.21.15.37:0/1534356629 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64c00f450 con 0x7fa664101420 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.480+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6641a4f60 con 0x7fa664101420 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.482+0000 7fa660ff9640 1 -- 172.21.15.37:0/1534356629 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa64c005020 con 0x7fa664101420 2024-09-19T06:52:12.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.482+0000 7fa66bbcf740 1 Processor -- start 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa66bbcf740 1 -- start start 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce83b40 0x55d89ce82090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce82a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce8f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d89cde82f0 con 0x55d89ce83b40 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d89cd8fcf0 con 0x55d89ce8d370 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d89cde42b0 con 0x55d89ce825d0 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce83b40 0x55d89ce82090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce83b40 0x55d89ce82090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40760/0 (socket says 172.21.15.37:40760) 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa6637fe640 1 -- 172.21.15.37:0/840083505 learned_addr learned my addr 172.21.15.37:0/840083505 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce8f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.483+0000 7fa662ffd640 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce82a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6637fe640 1 -- 172.21.15.37:0/840083505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 msgr2=0x55d89ce82a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6637fe640 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce82a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6637fe640 1 -- 172.21.15.37:0/840083505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 msgr2=0x55d89ce8f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6637fe640 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce8f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6637fe640 1 -- 172.21.15.37:0/840083505 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89cd743a0 con 0x55d89ce83b40 2024-09-19T06:52:12.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa662ffd640 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce82a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:12.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa660ff9640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa6500a33b0 0x7fa6500a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa6500a33b0 0x7fa6500a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.882 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:12 smithi151 ceph-mon[35236]: pgmap v327: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.1 MiB/s wr, 127 op/s 2024-09-19T06:52:12.882 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:12 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4047117691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:12.882 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:12 smithi096 ceph-mon[35004]: pgmap v327: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.1 MiB/s wr, 127 op/s 2024-09-19T06:52:12.882 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:12 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4047117691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:12.882 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:12 smithi037 ceph-mon[31211]: pgmap v327: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 2.1 MiB/s wr, 127 op/s 2024-09-19T06:52:12.882 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:12 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4047117691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:12.883 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:52:12.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6637fe640 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce83b40 0x55d89ce82090 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009f30 tx=0x7fa65c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c2b536b5060093 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:12.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa660ff9640 1 -- 172.21.15.37:0/1534356629 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fa64c0cfc40 con 0x7fa664101420 2024-09-19T06:52:12.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6597fa640 1 -- 172.21.15.37:0/840083505 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c016070 con 0x55d89ce83b40 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6597fa640 1 -- 172.21.15.37:0/840083505 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c01bb40 con 0x55d89ce83b40 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.484+0000 7fa6597fa640 1 -- 172.21.15.37:0/840083505 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c012c50 con 0x55d89ce83b40 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 -- 172.21.15.37:0/840083505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce83b40 msgr2=0x55d89ce82090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce83b40 0x55d89ce82090 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009f30 tx=0x7fa65c010800 comp rx=0 tx=0).stop 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa6500a33b0 0x7fa6500a5870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa654000c00 tx=0x7fa6540058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 -- 172.21.15.37:0/840083505 shutdown_connections 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce8f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce82a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/840083505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce83b40 0x55d89ce82090 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 -- 172.21.15.37:0/840083505 >> 172.21.15.37:0/840083505 conn(0x55d89ce4b310 msgr2=0x55d89ce7ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 -- 172.21.15.37:0/840083505 shutdown_connections 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 -- 172.21.15.37:0/840083505 wait complete. 2024-09-19T06:52:12.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 -- start start 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.485+0000 7fa66bbcf740 1 Processor -- start 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce8cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce89fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce8d370 0x55d89ce8a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d89ce90070 con 0x55d89ce8d370 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d89cda3060 con 0x55d89ce83b40 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d89ce901f0 con 0x55d89ce825d0 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa66bbcf740 10 client.0 _ll_register_callbacks cb 0x55d89caf68c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce8d370 0x55d89ce8a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce8d370 0x55d89ce8a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40774/0 (socket says 172.21.15.37:40774) 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 -- 172.21.15.37:0/1954649468 learned_addr learned my addr 172.21.15.37:0/1954649468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce89fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 -- 172.21.15.37:0/1954649468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 msgr2=0x55d89ce8cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce8cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 -- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 msgr2=0x55d89ce89fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce89fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 -- 172.21.15.37:0/1954649468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d89cb593c0 con 0x55d89ce8d370 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce89fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce8d370 0x55d89ce8a5d0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa65c012990 tx=0x7fa65c009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff21f3e01fa0383c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa64a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.486+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1954649468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c029060 con 0x55d89ce8d370 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1954649468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fa65c004c50 con 0x55d89ce8d370 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa65c01bb40 con 0x55d89ce8d370 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1954649468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa65c016070 con 0x55d89ce8d370 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 10 client.18345 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d89ce9bad0 con 0x55d89ce8d370 2024-09-19T06:52:12.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1954649468 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa65c01bb40 con 0x55d89ce8d370 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1954649468 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa65c00f040 con 0x55d89ce8d370 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 10 client.18345 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 10 client.18345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 20 client.18345 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89ce9ced0 0x55d89ce9f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 4 client.18345 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d89cdf5860 con 0x55d89ce9ced0 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.487+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89ce9ced0 0x55d89ce9f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.488+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89ce9ced0 0x55d89ce9f2c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca73d79a4cd380ab server_cookie=c9284f71c6522f2 in_seq=0 out_seq=0 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.488+0000 7fa64a7fc640 10 client.18345 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.488+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1954649468 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d89cdf5860 con 0x55d89ce9ced0 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.488+0000 7fa64a7fc640 10 client.18345 handle_command_reply: tid=0 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.488+0000 7fa66bbcf740 1 client.18345 shutdown 2024-09-19T06:52:12.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce8d370 msgr2=0x55d89ce8a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce8d370 0x55d89ce8a5d0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa65c012990 tx=0x7fa65c009a40 comp rx=0 tx=0).stop 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 shutdown_connections 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89ce9ced0 0x55d89ce9f2c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce8d370 0x55d89ce8a5d0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce89fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1954649468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce825d0 0x55d89ce8cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 >> 172.21.15.37:0/1954649468 conn(0x55d89ce4b310 msgr2=0x55d89ce84960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa64a7fc640 0 client.18345 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 shutdown_connections 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1954649468 wait complete. 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.489+0000 7fa66bbcf740 20 client.18345 trim_cache size 0 max 16384 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 Processor -- start 2024-09-19T06:52:12.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 -- start start 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce99310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce9eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce8d370 0x55d89cea4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641b02a0 con 0x55d89ce825d0 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641adb70 con 0x55d89ce83b40 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641ab9e0 con 0x55d89ce8d370 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce99310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce99310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40784/0 (socket says 172.21.15.37:40784) 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 -- 172.21.15.37:0/1101354945 learned_addr learned my addr 172.21.15.37:0/1101354945 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce9eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 -- 172.21.15.37:0/1101354945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce8d370 msgr2=0x55d89cea4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce8d370 0x55d89cea4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 -- 172.21.15.37:0/1101354945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 msgr2=0x55d89ce9eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce9eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.502+0000 7fa662ffd640 1 -- 172.21.15.37:0/1101354945 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d89ce7e510 con 0x55d89ce825d0 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce99310 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa65402a940 tx=0x7fa65403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2c08d961428e08 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa658ff9640 1 -- 172.21.15.37:0/1101354945 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa654035050 con 0x55d89ce825d0 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa658ff9640 1 -- 172.21.15.37:0/1101354945 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa654054070 con 0x55d89ce825d0 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa658ff9640 1 -- 172.21.15.37:0/1101354945 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65404e2d0 con 0x55d89ce825d0 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1101354945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 msgr2=0x55d89ce99310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce99310 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa65402a940 tx=0x7fa65403d000 comp rx=0 tx=0).stop 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1101354945 shutdown_connections 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce8d370 0x55d89cea4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce83b40 0x55d89ce9eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1101354945 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce99310 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.503+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1101354945 >> 172.21.15.37:0/1101354945 conn(0x55d89ce4b310 msgr2=0x55d89ce888b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1101354945 shutdown_connections 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1101354945 wait complete. 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 -- start start 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 Processor -- start 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce83b40 0x55d89ce919f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce91f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641ac030 con 0x55d89ce825d0 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641ad4a0 con 0x55d89ce8d370 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6641ac700 con 0x55d89ce83b40 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa66bbcf740 10 client.0 _ll_register_callbacks cb 0x7fa64c0d35c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:12.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa662ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce914b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40790/0 (socket says 172.21.15.37:40790) 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa662ffd640 1 -- 172.21.15.37:0/1623840281 learned_addr learned my addr 172.21.15.37:0/1623840281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce91f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.504+0000 7fa65a7fc640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce83b40 0x55d89ce919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa6637fe640 1 -- 172.21.15.37:0/1623840281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce83b40 msgr2=0x55d89ce919f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce83b40 0x55d89ce919f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa6637fe640 1 -- 172.21.15.37:0/1623840281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 msgr2=0x55d89ce914b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce914b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa6637fe640 1 -- 172.21.15.37:0/1623840281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d89cf06a80 con 0x55d89ce8d370 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa6637fe640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce91f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005d50 tx=0x7fa65c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93797eb1ccf61845 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa64a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:12.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1623840281 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa65c029070 con 0x55d89ce8d370 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa654054070 con 0x55d89ce8d370 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1623840281 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fa65c021040 con 0x55d89ce8d370 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1623840281 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa65c03c050 con 0x55d89ce8d370 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa66bbcf740 10 client.27686 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d89cf07d40 con 0x55d89ce8d370 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.505+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1623840281 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa654054070 con 0x55d89ce8d370 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1623840281 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa65c0129c0 con 0x55d89ce8d370 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 10 client.27686 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 10 client.27686 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 20 client.27686 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89cf083d0 0x55d89cf0a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 4 client.27686 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d89cdefd70 con 0x55d89cf083d0 2024-09-19T06:52:12.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89cf083d0 0x55d89cf0a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa64a7fc640 10 client.27686 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa662ffd640 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89cf083d0 0x55d89cf0a7f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=372c169e02521f90 server_cookie=fff56df1943876a4 in_seq=0 out_seq=0 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa64a7fc640 1 -- 172.21.15.37:0/1623840281 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d89cdefd70 con 0x55d89cf083d0 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa64a7fc640 10 client.27686 handle_command_reply: tid=0 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 1 client.27686 shutdown 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 msgr2=0x55d89ce91f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce91f30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa65c005d50 tx=0x7fa65c005fd0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 shutdown_connections 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.506+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d89cf083d0 0x55d89cf0a7f0 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d89ce8d370 0x55d89ce91f30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa64a7fc640 0 client.27686 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d89ce83b40 0x55d89ce919f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa66bbcf740 1 --2- 172.21.15.37:0/1623840281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d89ce825d0 0x55d89ce914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 >> 172.21.15.37:0/1623840281 conn(0x55d89ce4b310 msgr2=0x55d89ce8b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 shutdown_connections 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa66bbcf740 1 -- 172.21.15.37:0/1623840281 wait complete. 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa66bbcf740 20 client.27686 trim_cache size 0 max 16384 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa6500a33b0 msgr2=0x7fa6500a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa6500a33b0 0x7fa6500a5870 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa654000c00 tx=0x7fa6540058d0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664101420 msgr2=0x7fa664199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664101420 0x7fa664199850 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa64c00c970 tx=0x7fa64c00ce30 comp rx=0 tx=0).stop 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 shutdown_connections 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa664106a80 0x7fa6641a4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa6500a33b0 0x7fa6500a5870 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa664101420 0x7fa664199850 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 --2- 172.21.15.37:0/1534356629 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa664100a80 0x7fa664199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 >> 172.21.15.37:0/1534356629 conn(0x7fa6640fc7a0 msgr2=0x7fa6640fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:12.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 shutdown_connections 2024-09-19T06:52:12.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:12.507+0000 7fa6697f6640 1 -- 172.21.15.37:0/1534356629 wait complete. 2024-09-19T06:52:13.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.59 MB/sec execute 400 sec latency 30711.925 ms 2024-09-19T06:52:13.512 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id d51d7a7a --release --await' 2024-09-19T06:52:13.512 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id d51d7a7a --release --await 2024-09-19T06:52:14.105 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:14.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 785152 7.57 MB/sec execute 401 sec latency 31711.970 ms 2024-09-19T06:52:14.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 -- 172.21.15.37:0/1982676343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 msgr2=0x7f815410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 --2- 172.21.15.37:0/1982676343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f815410d950 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f814c009cd0 tx=0x7f814c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 -- 172.21.15.37:0/1982676343 shutdown_connections 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 --2- 172.21.15.37:0/1982676343 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f815410d950 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 --2- 172.21.15.37:0/1982676343 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8154100a80 0x7f8154100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 --2- 172.21.15.37:0/1982676343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8154106a80 0x7f8154106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 -- 172.21.15.37:0/1982676343 >> 172.21.15.37:0/1982676343 conn(0x7f81540fc7a0 msgr2=0x7f81540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 -- 172.21.15.37:0/1982676343 shutdown_connections 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 -- 172.21.15.37:0/1982676343 wait complete. 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.547+0000 7f8159293640 1 Processor -- start 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8159293640 1 -- start start 2024-09-19T06:52:14.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8159293640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8154100a80 0x7f8154199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8159293640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f81541997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8159293640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8154106a80 0x7f81541a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8159293640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8154110ef0 con 0x7f8154101420 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8159293640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8154110d70 con 0x7f8154106a80 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8159293640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8154111070 con 0x7f8154100a80 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f81541997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f81541997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40806/0 (socket says 172.21.15.37:40806) 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 -- 172.21.15.37:0/869584835 learned_addr learned my addr 172.21.15.37:0/869584835 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f8152ffd640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8154100a80 0x7f8154199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81537fe640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8154106a80 0x7f81541a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 -- 172.21.15.37:0/869584835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8154100a80 msgr2=0x7f8154199280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8154100a80 0x7f8154199280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 -- 172.21.15.37:0/869584835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8154106a80 msgr2=0x7f81541a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8154106a80 0x7f81541a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81527fc640 1 -- 172.21.15.37:0/869584835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81541a4840 con 0x7f8154101420 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.548+0000 7f81537fe640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8154106a80 0x7f81541a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.549+0000 7f81527fc640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f81541997c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f814800c920 tx=0x7f814800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.549+0000 7f813bfff640 1 -- 172.21.15.37:0/869584835 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148007c70 con 0x7f8154101420 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.549+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81541a4a10 con 0x7f8154101420 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.549+0000 7f813bfff640 1 -- 172.21.15.37:0/869584835 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8148007e10 con 0x7f8154101420 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.549+0000 7f813bfff640 1 -- 172.21.15.37:0/869584835 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8148018890 con 0x7f8154101420 2024-09-19T06:52:14.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.549+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81541a4f50 con 0x7f8154101420 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f813bfff640 1 -- 172.21.15.37:0/869584835 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8148018b30 con 0x7f8154101420 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 Processor -- start 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 -- start start 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936a6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c29360b620 con 0x55c2936a6980 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c2935b2e50 con 0x55c2936a6060 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f815b66c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c2936075e0 con 0x55c2936b1240 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.552+0000 7f8152ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.564 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40812/0 (socket says 172.21.15.37:40812) 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 -- 172.21.15.37:0/1836845612 learned_addr learned my addr 172.21.15.37:0/1836845612 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f81537fe640 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f813bfff640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f813c0a33b0 0x7f813c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f81527fc640 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 -- 172.21.15.37:0/1836845612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 msgr2=0x55c2936b3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f81537fe640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f813c0a33b0 0x7f813c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 -- 172.21.15.37:0/1836845612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 msgr2=0x55c2936a6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936a6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 -- 172.21.15.37:0/1836845612 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2935a3d40 con 0x55c2936a6980 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f813bfff640 1 -- 172.21.15.37:0/869584835 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f81480cffd0 con 0x7f8154101420 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b0d00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f814400b820 tx=0x7f814400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d885a8c8fc21cc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f81397fa640 1 -- 172.21.15.37:0/1836845612 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814400d950 con 0x55c2936a6980 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f81397fa640 1 -- 172.21.15.37:0/1836845612 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8144019070 con 0x55c2936a6980 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f81537fe640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f813c0a33b0 0x7f813c0a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f814c02ae00 tx=0x7f814c02a8e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.553+0000 7f81397fa640 1 -- 172.21.15.37:0/1836845612 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81440123e0 con 0x55c2936a6980 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 -- 172.21.15.37:0/1836845612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 msgr2=0x55c2936b0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b0d00 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f814400b820 tx=0x7f814400bce0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 -- 172.21.15.37:0/1836845612 shutdown_connections 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b0d00 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 --2- 172.21.15.37:0/1836845612 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936a6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 -- 172.21.15.37:0/1836845612 >> 172.21.15.37:0/1836845612 conn(0x55c2936a25f0 msgr2=0x55c2936a29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 -- 172.21.15.37:0/1836845612 shutdown_connections 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.554+0000 7f815b66c740 1 -- 172.21.15.37:0/1836845612 wait complete. 2024-09-19T06:52:14.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 -- start start 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 Processor -- start 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ae060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936ae5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936aeae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c2936b3e10 con 0x55c2936b1240 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c293593f10 con 0x55c2936a6980 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c2936b3f90 con 0x55c2936a6060 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f815b66c740 10 client.0 _ll_register_callbacks cb 0x55c2935b7db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936aeae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936aeae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40820/0 (socket says 172.21.15.37:40820) 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81537fe640 1 -- 172.21.15.37:0/1244542362 learned_addr learned my addr 172.21.15.37:0/1244542362 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ae060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81527fc640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936ae5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81537fe640 1 -- 172.21.15.37:0/1244542362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 msgr2=0x55c2936ae060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81537fe640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ae060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81537fe640 1 -- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 msgr2=0x55c2936ae5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.555+0000 7f81537fe640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936ae5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f81537fe640 1 -- 172.21.15.37:0/1244542362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c29331a580 con 0x55c2936b1240 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f81527fc640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f81537fe640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936aeae0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f814c003cf0 tx=0x7f814c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66da04af4b6feed server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f8136ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f8136ff5640 1 -- 172.21.15.37:0/1244542362 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c0305a0 con 0x55c2936b1240 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8144019070 con 0x55c2936b1240 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f8136ff5640 1 -- 172.21.15.37:0/1244542362 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f814c030740 con 0x55c2936b1240 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f8136ff5640 1 -- 172.21.15.37:0/1244542362 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f814c058050 con 0x55c2936b1240 2024-09-19T06:52:14.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f815b66c740 10 client.18363 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c2933e4960 con 0x55c2936b1240 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f8136ff5640 1 -- 172.21.15.37:0/1244542362 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8144019070 con 0x55c2936b1240 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.556+0000 7f8136ff5640 1 -- 172.21.15.37:0/1244542362 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f814c030070 con 0x55c2936b1240 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f815b66c740 10 client.18363 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f815b66c740 10 client.18363 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f815b66c740 20 client.18363 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f815b66c740 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c2936c3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f815b66c740 4 client.18363 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c293618b90 con 0x55c2936c0cb0 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c2936c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c2936c3150 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc7e7fd97800b408 server_cookie=4a405dcae69b5071 in_seq=0 out_seq=0 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.557+0000 7f8136ff5640 10 client.18363 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f8136ff5640 1 -- 172.21.15.37:0/1244542362 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c293618b90 con 0x55c2936c0cb0 2024-09-19T06:52:14.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f8136ff5640 10 client.18363 handle_command_reply: tid=0 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 client.18363 shutdown 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 msgr2=0x55c2936aeae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936aeae0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f814c003cf0 tx=0x7f814c0095c0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 shutdown_connections 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c2936c3150 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f8136ff5640 0 client.18363 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936aeae0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936ae5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 --2- 172.21.15.37:0/1244542362 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ae060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.558+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 >> 172.21.15.37:0/1244542362 conn(0x55c2936a25f0 msgr2=0x55c2936a8320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.559+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 shutdown_connections 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.559+0000 7f815b66c740 1 -- 172.21.15.37:0/1244542362 wait complete. 2024-09-19T06:52:14.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.559+0000 7f815b66c740 20 client.18363 trim_cache size 0 max 16384 2024-09-19T06:52:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 Processor -- start 2024-09-19T06:52:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 -- start start 2024-09-19T06:52:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.578 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936a1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b02c0 con 0x55c2936b1240 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81541adb90 con 0x55c2936a6980 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f815b66c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81541aba00 con 0x55c2936a6060 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40830/0 (socket says 172.21.15.37:40830) 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 -- 172.21.15.37:0/4012196223 learned_addr learned my addr 172.21.15.37:0/4012196223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81527fc640 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936a1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f8152ffd640 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 -- 172.21.15.37:0/4012196223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 msgr2=0x55c2936ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ac930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 -- 172.21.15.37:0/4012196223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 msgr2=0x55c2936a1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936a1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.579 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 -- 172.21.15.37:0/4012196223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2936a2420 con 0x55c2936b1240 2024-09-19T06:52:14.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f8152ffd640 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:14.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.576+0000 7f81537fe640 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936c8a70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f814c009bf0 tx=0x7f814c009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a552d927af14b2b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f8138ff9640 1 -- 172.21.15.37:0/4012196223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c02af20 con 0x55c2936b1240 2024-09-19T06:52:14.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f8138ff9640 1 -- 172.21.15.37:0/4012196223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c036040 con 0x55c2936b1240 2024-09-19T06:52:14.580 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f8138ff9640 1 -- 172.21.15.37:0/4012196223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c03f810 con 0x55c2936b1240 2024-09-19T06:52:14.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f815b66c740 1 -- 172.21.15.37:0/4012196223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 msgr2=0x55c2936c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f815b66c740 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936c8a70 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f814c009bf0 tx=0x7f814c009360 comp rx=0 tx=0).stop 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f815b66c740 1 -- 172.21.15.37:0/4012196223 shutdown_connections 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f815b66c740 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936b1240 0x55c2936c8a70 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f815b66c740 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6980 0x55c2936a1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f815b66c740 1 --2- 172.21.15.37:0/4012196223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936a6060 0x55c2936ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.577+0000 7f815b66c740 1 -- 172.21.15.37:0/4012196223 >> 172.21.15.37:0/4012196223 conn(0x55c2936a25f0 msgr2=0x55c2936a0f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 -- 172.21.15.37:0/4012196223 shutdown_connections 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 -- 172.21.15.37:0/4012196223 wait complete. 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 -- start start 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 Processor -- start 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b41d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f81541ac050 con 0x55c2936a6980 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81541ad4c0 con 0x55c2936a6060 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f81541ac720 con 0x55c2936b1240 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f815b66c740 10 client.0 _ll_register_callbacks cb 0x7f814c003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f81527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f81527fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40832/0 (socket says 172.21.15.37:40832) 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f81527fc640 1 -- 172.21.15.37:0/1599376698 learned_addr learned my addr 172.21.15.37:0/1599376698 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.578+0000 7f81537fe640 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f81527fc640 1 -- 172.21.15.37:0/1599376698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 msgr2=0x55c2936b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f81527fc640 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f81527fc640 1 -- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 msgr2=0x55c2936b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f81527fc640 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f81527fc640 1 -- 172.21.15.37:0/1599376698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2936b4d00 con 0x55c2936a6980 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f81527fc640 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b41d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f814800c8f0 tx=0x7f814809c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b88bfc077a717b28 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f8136ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f8136ff5640 1 -- 172.21.15.37:0/1599376698 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81480b5070 con 0x55c2936a6980 2024-09-19T06:52:14.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f8136ff5640 1 -- 172.21.15.37:0/1599376698 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f81480a0170 con 0x55c2936a6980 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c036040 con 0x55c2936a6980 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f8136ff5640 1 -- 172.21.15.37:0/1599376698 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f81480bae60 con 0x55c2936a6980 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f815b66c740 10 client.18369 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c29372c830 con 0x55c2936a6980 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f8136ff5640 1 -- 172.21.15.37:0/1599376698 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f814c036040 con 0x55c2936a6980 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.579+0000 7f8136ff5640 1 -- 172.21.15.37:0/1599376698 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f814809c480 con 0x55c2936a6980 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f815b66c740 10 client.18369 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f815b66c740 10 client.18369 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f815b66c740 20 client.18369 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f815b66c740 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c29372eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f815b66c740 4 client.18369 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d51d7a7a", "release": true, "await": true}] 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d51d7a7a", "release": true, "await": true}) -- 0x55c2936130a0 con 0x55c2936c0cb0 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c29372eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:14.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f8136ff5640 10 client.18369 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:14.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.580+0000 7f8152ffd640 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c29372eff0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c135d994e7eb3919 server_cookie=47893603e44ef922 in_seq=0 out_seq=0 2024-09-19T06:52:14.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f8136ff5640 1 -- 172.21.15.37:0/1599376698 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55c2936130a0 con 0x55c2936c0cb0 2024-09-19T06:52:14.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f8136ff5640 10 client.18369 handle_command_reply: tid=0 2024-09-19T06:52:14.595 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 client.18369 shutdown 2024-09-19T06:52:14.595 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:14 smithi037 ceph-mon[31211]: pgmap v328: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:14.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 msgr2=0x55c2936b41d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b41d0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f814800c8f0 tx=0x7f814809c9e0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 shutdown_connections 2024-09-19T06:52:14.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c2936c0cb0 0x55c29372eff0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c2936b1240 0x55c2936b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c2936a6980 0x55c2936b41d0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.596 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 --2- 172.21.15.37:0/1599376698 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2936a6060 0x55c2936b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 >> 172.21.15.37:0/1599376698 conn(0x55c2936a25f0 msgr2=0x55c2936ae840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.593+0000 7f8136ff5640 0 client.18369 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 shutdown_connections 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f815b66c740 1 -- 172.21.15.37:0/1599376698 wait complete. 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f815b66c740 20 client.18369 trim_cache size 0 max 16384 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 39, 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 36, 2024-09-19T06:52:14.597 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "d51d7a7a": { 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 36, 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:52:14.598 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:14.599 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f813c0a33b0 msgr2=0x7f813c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f813c0a33b0 0x7f813c0a5870 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f814c02ae00 tx=0x7f814c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 msgr2=0x7f81541997c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f81541997c0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f814800c920 tx=0x7f814800cde0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 shutdown_connections 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8154106a80 0x7f81541a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f813c0a33b0 0x7f813c0a5870 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8154101420 0x7f81541997c0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.594+0000 7f8159293640 1 --2- 172.21.15.37:0/869584835 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8154100a80 0x7f8154199280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:14.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.595+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 >> 172.21.15.37:0/869584835 conn(0x7f81540fc7a0 msgr2=0x7f81540fcd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:14.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.595+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 shutdown_connections 2024-09-19T06:52:14.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:14.595+0000 7f8159293640 1 -- 172.21.15.37:0/869584835 wait complete. 2024-09-19T06:52:14.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:14 smithi096 ceph-mon[35004]: pgmap v328: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:14.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:14 smithi151 ceph-mon[35236]: pgmap v328: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:15.009 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:52:15.009 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:52:15.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 786385 7.57 MB/sec execute 402 sec latency 32082.056 ms 2024-09-19T06:52:15.496 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd51d7a7a', total seconds elapsed: 35.7 2024-09-19T06:52:15.496 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-09-19T06:52:15.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:15.618 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 -- 172.21.15.37:0/360441237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 msgr2=0x7f1a00101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 --2- 172.21.15.37:0/360441237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a00101200 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f19f4009900 tx=0x7f19f402a480 comp rx=0 tx=0).stop 2024-09-19T06:52:16.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 -- 172.21.15.37:0/360441237 shutdown_connections 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 --2- 172.21.15.37:0/360441237 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a00102ef0 0x7f1a0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 --2- 172.21.15.37:0/360441237 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a00102570 0x7f1a001029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 --2- 172.21.15.37:0/360441237 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a00101200 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 -- 172.21.15.37:0/360441237 >> 172.21.15.37:0/360441237 conn(0x7f1a000fc7d0 msgr2=0x7f1a000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 -- 172.21.15.37:0/360441237 shutdown_connections 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.098+0000 7f1a06b03640 1 -- 172.21.15.37:0/360441237 wait complete. 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 Processor -- start 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 -- start start 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a000fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a00102570 0x7f1a000fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a00102ef0 0x7f1a000fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00110fa0 con 0x7f1a00100e20 2024-09-19T06:52:16.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00110e20 con 0x7f1a00102570 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a06b03640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a00111120 con 0x7f1a00102ef0 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a000fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a000fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40850/0 (socket says 172.21.15.37:40850) 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f1a0489f640 1 -- 172.21.15.37:0/2706164437 learned_addr learned my addr 172.21.15.37:0/2706164437 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.099+0000 7f19fffff640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a00102570 0x7f1a000fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a050a0640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a00102ef0 0x7f1a000fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a0489f640 1 -- 172.21.15.37:0/2706164437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a00102ef0 msgr2=0x7f1a000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a0489f640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a00102ef0 0x7f1a000fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a0489f640 1 -- 172.21.15.37:0/2706164437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a00102570 msgr2=0x7f1a000fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a0489f640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a00102570 0x7f1a000fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a0489f640 1 -- 172.21.15.37:0/2706164437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a001a69e0 con 0x7f1a00100e20 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a0489f640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a000fcc80 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f19fdffb640 1 -- 172.21.15.37:0/2706164437 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f403f070 con 0x7f1a00100e20 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a001a6c70 con 0x7f1a00100e20 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f19fdffb640 1 -- 172.21.15.37:0/2706164437 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f4031c30 con 0x7f1a00100e20 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f19fdffb640 1 -- 172.21.15.37:0/2706164437 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4035030 con 0x7f1a00100e20 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.100+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a001a7150 con 0x7f1a00100e20 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.103+0000 7f19fdffb640 1 -- 172.21.15.37:0/2706164437 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f19f4042070 con 0x7f1a00100e20 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.103+0000 7f1a08edc740 1 Processor -- start 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a08edc740 1 -- start start 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5bdb60 0x55d65c5bdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5be480 0x55d65c5c8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a08edc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d65c523280 con 0x55d65c5be480 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a08edc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d65c4cab10 con 0x55d65c5c8d40 2024-09-19T06:52:16.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a08edc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d65c51f240 con 0x55d65c5bdb60 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5be480 0x55d65c5c8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f19fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5be480 0x55d65c5c8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40864/0 (socket says 172.21.15.37:40864) 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 -- 172.21.15.37:0/787894174 learned_addr learned my addr 172.21.15.37:0/787894174 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a050a0640 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5bdb60 0x55d65c5bdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f19fdffb640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f19f00a33b0 0x7f19f00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 -- 172.21.15.37:0/787894174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5bdb60 msgr2=0x55d65c5bdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5bdb60 0x55d65c5bdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 -- 172.21.15.37:0/787894174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 msgr2=0x55d65c5cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a0489f640 1 -- 172.21.15.37:0/787894174 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65c4d71f0 con 0x55d65c5be480 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.104+0000 7f1a050a0640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f19f00a33b0 0x7f19f00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.105+0000 7f19fdffb640 1 -- 172.21.15.37:0/2706164437 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f19f40f0560 con 0x7f1a00100e20 2024-09-19T06:52:16.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.105+0000 7f1a0489f640 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5be480 0x55d65c5c8800 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f19f40edb60 tx=0x7f19f40e7580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e6938b322a22aa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.105+0000 7f19eaffd640 1 -- 172.21.15.37:0/787894174 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40bc1c0 con 0x55d65c5be480 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.105+0000 7f19eaffd640 1 -- 172.21.15.37:0/787894174 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40a5b60 con 0x55d65c5be480 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.105+0000 7f19eaffd640 1 -- 172.21.15.37:0/787894174 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f40843d0 con 0x55d65c5be480 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 -- 172.21.15.37:0/787894174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5be480 msgr2=0x55d65c5c8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5be480 0x55d65c5c8800 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f19f40edb60 tx=0x7f19f40e7580 comp rx=0 tx=0).stop 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a050a0640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f19f00a33b0 0x7f19f00a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009fd0 tx=0x7f19f8004b50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 -- 172.21.15.37:0/787894174 shutdown_connections 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5be480 0x55d65c5c8800 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 --2- 172.21.15.37:0/787894174 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5bdb60 0x55d65c5bdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 -- 172.21.15.37:0/787894174 >> 172.21.15.37:0/787894174 conn(0x55d65c5ba0f0 msgr2=0x55d65c5ba4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 -- 172.21.15.37:0/787894174 shutdown_connections 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 -- 172.21.15.37:0/787894174 wait complete. 2024-09-19T06:52:16.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 -- start start 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 Processor -- start 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.106+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5c6630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a08edc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d65c5cb6d0 con 0x55d65c5bdb60 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a08edc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d65c4ddf10 con 0x55d65c5be480 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a08edc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d65c5cb850 con 0x55d65c5c8d40 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a08edc740 10 client.0 _ll_register_callbacks cb 0x55d65c4a6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40872/0 (socket says 172.21.15.37:40872) 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f19fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 -- 172.21.15.37:0/3283822219 learned_addr learned my addr 172.21.15.37:0/3283822219 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a050a0640 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 -- 172.21.15.37:0/3283822219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 msgr2=0x55d65c5c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5c6630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 -- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 msgr2=0x55d65c5c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 -- 172.21.15.37:0/3283822219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65c2946a0 con 0x55d65c5bdb60 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f19fffff640 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c5bb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f19f40c4ef0 tx=0x7f19f4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1f81f5c09fe564 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f19dffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:16.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f19dffff640 1 -- 172.21.15.37:0/3283822219 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4049030 con 0x55d65c5bdb60 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f19dffff640 1 -- 172.21.15.37:0/3283822219 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f19f40b8ed0 con 0x55d65c5bdb60 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.107+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f40a5b60 con 0x55d65c5bdb60 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f19dffff640 1 -- 172.21.15.37:0/3283822219 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19f40845a0 con 0x55d65c5bdb60 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 10 client.18381 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d65c5cb6d0 con 0x55d65c5bdb60 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f19dffff640 1 -- 172.21.15.37:0/3283822219 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40a5b60 con 0x55d65c5bdb60 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f19dffff640 1 -- 172.21.15.37:0/3283822219 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f19f40edea0 con 0x55d65c5bdb60 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 10 client.18381 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 10 client.18381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 20 client.18381 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c5d84b0 0x55d65c5da900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 4 client.18381 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:16.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d65c5307f0 con 0x55d65c5d84b0 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.108+0000 7f19fffff640 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c5d84b0 0x55d65c5da900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.109+0000 7f19dffff640 10 client.18381 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.109+0000 7f19fffff640 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c5d84b0 0x55d65c5da900 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=610c9077fb9094d5 server_cookie=27f2625e94677717 in_seq=0 out_seq=0 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.110+0000 7f19dffff640 1 -- 172.21.15.37:0/3283822219 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d65c5307f0 con 0x55d65c5d84b0 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.110+0000 7f19dffff640 10 client.18381 handle_command_reply: tid=0 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.110+0000 7f1a08edc740 1 client.18381 shutdown 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 msgr2=0x55d65c5c5bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c5bb0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f19f40c4ef0 tx=0x7f19f4005c90 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 shutdown_connections 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c5d84b0 0x55d65c5da900 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3283822219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c5bb0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f19dffff640 0 client.18381 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:16.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 >> 172.21.15.37:0/3283822219 conn(0x55d65c5ba0f0 msgr2=0x55d65c5bfe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 shutdown_connections 2024-09-19T06:52:16.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.111+0000 7f1a08edc740 1 -- 172.21.15.37:0/3283822219 wait complete. 2024-09-19T06:52:16.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.112+0000 7f1a08edc740 20 client.18381 trim_cache size 0 max 16384 2024-09-19T06:52:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.133+0000 7f1a08edc740 1 Processor -- start 2024-09-19T06:52:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.133+0000 7f1a08edc740 1 -- start start 2024-09-19T06:52:16.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5be480 0x55d65c5e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5e0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a08edc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001b2490 con 0x55d65c5bdb60 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a08edc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001afd60 con 0x55d65c5c8d40 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a08edc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001adbd0 con 0x55d65c5be480 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40884/0 (socket says 172.21.15.37:40884) 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 -- 172.21.15.37:0/3284504747 learned_addr learned my addr 172.21.15.37:0/3284504747 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a050a0640 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5e0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f19fffff640 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5be480 0x55d65c5e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 -- 172.21.15.37:0/3284504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5be480 msgr2=0x55d65c5e01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5be480 0x55d65c5e01d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 -- 172.21.15.37:0/3284504747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 msgr2=0x55d65c5e0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5e0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 -- 172.21.15.37:0/3284504747 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d65c5be2b0 con 0x55d65c5bdb60 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a050a0640 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5e0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f19fffff640 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5be480 0x55d65c5e01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.134+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c7a10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f19f40b8cc0 tx=0x7f19f40bad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d61f83b6059fc8e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.135+0000 7f19ea7fc640 1 -- 172.21.15.37:0/3284504747 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4049070 con 0x55d65c5bdb60 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.135+0000 7f19ea7fc640 1 -- 172.21.15.37:0/3284504747 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40d3150 con 0x55d65c5bdb60 2024-09-19T06:52:16.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.135+0000 7f19ea7fc640 1 -- 172.21.15.37:0/3284504747 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f408fc50 con 0x55d65c5bdb60 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.135+0000 7f1a08edc740 1 -- 172.21.15.37:0/3284504747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 msgr2=0x55d65c5c7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.135+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c7a10 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f19f40b8cc0 tx=0x7f19f40bad70 comp rx=0 tx=0).stop 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 -- 172.21.15.37:0/3284504747 shutdown_connections 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5c8d40 0x55d65c5e0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5be480 0x55d65c5e01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3284504747 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5c7a10 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 -- 172.21.15.37:0/3284504747 >> 172.21.15.37:0/3284504747 conn(0x55d65c5ba0f0 msgr2=0x55d65c5c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 -- 172.21.15.37:0/3284504747 shutdown_connections 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 -- 172.21.15.37:0/3284504747 wait complete. 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 -- start start 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.136+0000 7f1a08edc740 1 Processor -- start 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5ccdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a08edc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a08edc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ae220 con 0x55d65c5bdb60 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a08edc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001af690 con 0x55d65c5be480 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a08edc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a001ae8f0 con 0x55d65c5c8d40 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a08edc740 10 client.0 _ll_register_callbacks cb 0x7f19ec00b810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f19fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5ccdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40900/0 (socket says 172.21.15.37:40900) 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 -- 172.21.15.37:0/3405114093 learned_addr learned my addr 172.21.15.37:0/3405114093 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a050a0640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 -- 172.21.15.37:0/3405114093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 msgr2=0x55d65c5cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 -- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 msgr2=0x55d65c5ccdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5ccdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a0489f640 1 -- 172.21.15.37:0/3405114093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d65c642300 con 0x55d65c5bdb60 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f1a050a0640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.137+0000 7f19fffff640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f1a0489f640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5cc880 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f19f40bc7f0 tx=0x7f19f4005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d6e19d8f943ce3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f19dffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f19dffff640 1 -- 172.21.15.37:0/3405114093 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19f4049070 con 0x55d65c5bdb60 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f19dffff640 1 -- 172.21.15.37:0/3405114093 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f19f40d0130 con 0x55d65c5bdb60 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19f40d3150 con 0x55d65c5bdb60 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f19dffff640 1 -- 172.21.15.37:0/3405114093 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f19f40bc1c0 con 0x55d65c5bdb60 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f1a08edc740 10 client.18387 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d65c642300 con 0x55d65c5bdb60 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f19dffff640 1 -- 172.21.15.37:0/3405114093 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19f40d3150 con 0x55d65c5bdb60 2024-09-19T06:52:16.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.138+0000 7f19dffff640 1 -- 172.21.15.37:0/3405114093 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f19f40bcac0 con 0x55d65c5bdb60 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f1a08edc740 10 client.18387 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f1a08edc740 10 client.18387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f1a08edc740 20 client.18387 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c643850 0x55d65c645c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f1a08edc740 4 client.18387 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d65c52ad00 con 0x55d65c643850 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f19fffff640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c643850 0x55d65c645c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f19fffff640 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c643850 0x55d65c645c40 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f4b561c78deb7a server_cookie=d4f80149f85ca855 in_seq=0 out_seq=0 2024-09-19T06:52:16.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.139+0000 7f19dffff640 10 client.18387 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:16.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f19dffff640 1 -- 172.21.15.37:0/3405114093 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d65c52ad00 con 0x55d65c643850 2024-09-19T06:52:16.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f19dffff640 10 client.18387 handle_command_reply: tid=0 2024-09-19T06:52:16.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 client.18387 shutdown 2024-09-19T06:52:16.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 msgr2=0x55d65c5cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5cc880 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f19f40bc7f0 tx=0x7f19f4005c90 comp rx=0 tx=0).stop 2024-09-19T06:52:16.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 shutdown_connections 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d65c643850 0x55d65c645c40 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d65c5c8d40 0x55d65c5cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d65c5be480 0x55d65c5ccdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 --2- 172.21.15.37:0/3405114093 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d65c5bdb60 0x55d65c5cc880 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 >> 172.21.15.37:0/3405114093 conn(0x55d65c5ba0f0 msgr2=0x55d65c5b9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.141+0000 7f19dffff640 0 client.18387 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 shutdown_connections 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a08edc740 1 -- 172.21.15.37:0/3405114093 wait complete. 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a08edc740 20 client.18387 trim_cache size 0 max 16384 2024-09-19T06:52:16.147 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:16.148 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:52:16.148 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "b265b637-b702-4901-8d4e-6fb974c22a8d", 2024-09-19T06:52:16.148 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:52:16.148 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:16.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f19f00a33b0 msgr2=0x7f19f00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f19f00a33b0 0x7f19f00a5870 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f19f8009fd0 tx=0x7f19f8004b50 comp rx=0 tx=0).stop 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 msgr2=0x7f1a000fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a000fcc80 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f19f4006010 tx=0x7f19f4003c10 comp rx=0 tx=0).stop 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 shutdown_connections 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a00102ef0 0x7f1a000fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a00102570 0x7f1a000fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f19f00a33b0 0x7f19f00a5870 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.142+0000 7f1a06b03640 1 --2- 172.21.15.37:0/2706164437 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a00100e20 0x7f1a000fcc80 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.143+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 >> 172.21.15.37:0/2706164437 conn(0x7f1a000fc7d0 msgr2=0x7f1a0006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.143+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 shutdown_connections 2024-09-19T06:52:16.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:16.143+0000 7f1a06b03640 1 -- 172.21.15.37:0/2706164437 wait complete. 2024-09-19T06:52:16.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 788471 7.58 MB/sec execute 403 sec latency 42.232 ms 2024-09-19T06:52:16.498 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:16.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:16 smithi096 ceph-mon[35004]: pgmap v329: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:16.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:16 smithi037 ceph-mon[31211]: pgmap v329: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:16.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:16 smithi151 ceph-mon[35236]: pgmap v329: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:16.913 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:52:17.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 790607 7.58 MB/sec execute 404 sec latency 12.806 ms 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2523552489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298106a80 msgr2=0x7fd298106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2523552489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298106a80 0x7fd298106e60 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd288009930 tx=0x7fd28802a420 comp rx=0 tx=0).stop 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2523552489 shutdown_connections 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2523552489 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd298101420 0x7fd29810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2523552489 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd298100a80 0x7fd298100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2523552489 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298106a80 0x7fd298106e60 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2523552489 >> 172.21.15.37:0/2523552489 conn(0x7fd2980fc7a0 msgr2=0x7fd2980febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2523552489 shutdown_connections 2024-09-19T06:52:17.431 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.426+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2523552489 wait complete. 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.427+0000 7fd29cfc4640 1 Processor -- start 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.427+0000 7fd29cfc4640 1 -- start start 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.427+0000 7fd29cfc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298100a80 0x7fd298198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.427+0000 7fd29cfc4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd298101420 0x7fd298198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.427+0000 7fd29cfc4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd298106a80 0x7fd29819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29cfc4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd298110e80 con 0x7fd298100a80 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29cfc4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd298110d00 con 0x7fd298101420 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29cfc4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd298111000 con 0x7fd298106a80 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298100a80 0x7fd298198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298100a80 0x7fd298198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40918/0 (socket says 172.21.15.37:40918) 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 -- 172.21.15.37:0/2069313275 learned_addr learned my addr 172.21.15.37:0/2069313275 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd28ffff640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd298101420 0x7fd298198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd296d9d640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd298106a80 0x7fd29819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 -- 172.21.15.37:0/2069313275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd298106a80 msgr2=0x7fd29819cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd298106a80 0x7fd29819cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 -- 172.21.15.37:0/2069313275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd298101420 msgr2=0x7fd298198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd298101420 0x7fd298198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 -- 172.21.15.37:0/2069313275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd29819d3c0 con 0x7fd298100a80 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29659c640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298100a80 0x7fd298198410 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd288009230 tx=0x7fd288004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd28f7fe640 1 -- 172.21.15.37:0/2069313275 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28803f030 con 0x7fd298100a80 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd28f7fe640 1 -- 172.21.15.37:0/2069313275 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd288004440 con 0x7fd298100a80 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd29819d650 con 0x7fd298100a80 2024-09-19T06:52:17.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd28f7fe640 1 -- 172.21.15.37:0/2069313275 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288035070 con 0x7fd298100a80 2024-09-19T06:52:17.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.428+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981a4f90 con 0x7fd298100a80 2024-09-19T06:52:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.430+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2981021c0 con 0x7fd298100a80 2024-09-19T06:52:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.432+0000 7fd28f7fe640 1 -- 172.21.15.37:0/2069313275 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd288042030 con 0x7fd298100a80 2024-09-19T06:52:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.432+0000 7fd28f7fe640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.432+0000 7fd28ffff640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.432+0000 7fd28f7fe640 1 -- 172.21.15.37:0/2069313275 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd2880f0500 con 0x7fd298100a80 2024-09-19T06:52:17.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.433+0000 7fd28ffff640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2840a3430 0x7fd2840a58f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd298105210 tx=0x7fd280005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:17.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.436+0000 7fd28f7fe640 1 -- 172.21.15.37:0/2069313275 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd2880bceb0 con 0x7fd298100a80 2024-09-19T06:52:17.473 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:17 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:52:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:17 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:17 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:52:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:17 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:52:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:17 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:17 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.669 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.667+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd298100ee0 con 0x7fd298100a80 2024-09-19T06:52:17.670 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.668+0000 7fd28f7fe640 1 -- 172.21.15.37:0/2069313275 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7fd298100ee0 con 0x7fd298100a80 2024-09-19T06:52:17.671 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:52:17.671 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:52:17.671 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.670+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2840a3430 msgr2=0x7fd2840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.670+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2840a3430 0x7fd2840a58f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd298105210 tx=0x7fd280005f90 comp rx=0 tx=0).stop 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298100a80 msgr2=0x7fd298198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298100a80 0x7fd298198410 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd288009230 tx=0x7fd288004040 comp rx=0 tx=0).stop 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 shutdown_connections 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd298106a80 0x7fd29819cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd298101420 0x7fd298198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd2840a3430 0x7fd2840a58f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 --2- 172.21.15.37:0/2069313275 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd298100a80 0x7fd298198410 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 >> 172.21.15.37:0/2069313275 conn(0x7fd2980fc7a0 msgr2=0x7fd29806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:17.673 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 shutdown_connections 2024-09-19T06:52:17.674 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:17.671+0000 7fd29cfc4640 1 -- 172.21.15.37:0/2069313275 wait complete. 2024-09-19T06:52:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:17 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:52:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:17 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:17 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:52:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:17 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:52:17.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:17 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.775 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:17 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:17 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:52:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:17 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:17 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:52:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:17 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:52:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:17 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:17.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:17 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:18.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 792250 7.59 MB/sec execute 405 sec latency 16.123 ms 2024-09-19T06:52:18.350 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0' 2024-09-19T06:52:18.350 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 150.0 --await-for 45.0 2024-09-19T06:52:18.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:18 smithi037 ceph-mon[31211]: pgmap v330: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:18.599 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:18 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2069313275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:18.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 -- 172.21.15.37:0/3083611816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c100e20 msgr2=0x7fb87c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 --2- 172.21.15.37:0/3083611816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c100e20 0x7fb87c101200 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7fb86c009900 tx=0x7fb86c02a480 comp rx=0 tx=0).stop 2024-09-19T06:52:18.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 -- 172.21.15.37:0/3083611816 shutdown_connections 2024-09-19T06:52:18.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 --2- 172.21.15.37:0/3083611816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb87c102ef0 0x7fb87c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 --2- 172.21.15.37:0/3083611816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c102570 0x7fb87c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 --2- 172.21.15.37:0/3083611816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c100e20 0x7fb87c101200 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 -- 172.21.15.37:0/3083611816 >> 172.21.15.37:0/3083611816 conn(0x7fb87c0fc7d0 msgr2=0x7fb87c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 -- 172.21.15.37:0/3083611816 shutdown_connections 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.597+0000 7fb880d2a640 1 -- 172.21.15.37:0/3083611816 wait complete. 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 Processor -- start 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 -- start start 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb87c100e20 0x7fb87c19a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c102570 0x7fb87c196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c102ef0 0x7fb87c196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c110f80 con 0x7fb87c102570 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c110e00 con 0x7fb87c100e20 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.598+0000 7fb880d2a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c111100 con 0x7fb87c102ef0 2024-09-19T06:52:18.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb879d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c102570 0x7fb87c196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb87c100e20 0x7fb87c19a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb879d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c102570 0x7fb87c196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40930/0 (socket says 172.21.15.37:40930) 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb879d9b640 1 -- 172.21.15.37:0/4001922879 learned_addr learned my addr 172.21.15.37:0/4001922879 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87ad9d640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c102ef0 0x7fb87c196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87a59c640 1 -- 172.21.15.37:0/4001922879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c102ef0 msgr2=0x7fb87c196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87a59c640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c102ef0 0x7fb87c196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87a59c640 1 -- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c102570 msgr2=0x7fb87c196410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87a59c640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c102570 0x7fb87c196410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87a59c640 1 -- 172.21.15.37:0/4001922879 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb87c197120 con 0x7fb87c100e20 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb879d9b640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c102570 0x7fb87c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb87a59c640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb87c100e20 0x7fb87c19a0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005940 tx=0x7fb86c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb86b7fe640 1 -- 172.21.15.37:0/4001922879 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c03f070 con 0x7fb87c100e20 2024-09-19T06:52:18.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb87c1ab080 con 0x7fb87c100e20 2024-09-19T06:52:18.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb86b7fe640 1 -- 172.21.15.37:0/4001922879 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c031c30 con 0x7fb87c100e20 2024-09-19T06:52:18.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.599+0000 7fb86b7fe640 1 -- 172.21.15.37:0/4001922879 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c035030 con 0x7fb87c100e20 2024-09-19T06:52:18.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.600+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb87c1ab560 con 0x7fb87c100e20 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.602+0000 7fb883103740 1 Processor -- start 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.602+0000 7fb86b7fe640 1 -- 172.21.15.37:0/4001922879 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb86c042070 con 0x7fb87c100e20 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.602+0000 7fb883103740 1 -- start start 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb883103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95aa5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb883103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb883103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ab2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb883103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558e95a0b2e0 con 0x558e95ab0680 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb883103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e959b2ce0 con 0x558e95aa5dc0 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb883103740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558e95a072a0 con 0x558e95aa54a0 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ab2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb87a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ab2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40934/0 (socket says 172.21.15.37:40934) 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 -- 172.21.15.37:0/3753958671 learned_addr learned my addr 172.21.15.37:0/3753958671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb87ad9d640 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95aa5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 -- 172.21.15.37:0/3753958671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 msgr2=0x558e95aa5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95aa5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb86b7fe640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8640a33b0 0x7fb8640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 -- 172.21.15.37:0/3753958671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 msgr2=0x558e95ab0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 -- 172.21.15.37:0/3753958671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e95997440 con 0x558e95ab0680 2024-09-19T06:52:18.608 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb87ad9d640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8640a33b0 0x7fb8640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb86b7fe640 1 -- 172.21.15.37:0/4001922879 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fb86c0f0400 con 0x7fb87c100e20 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.603+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ab2a70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb870011810 tx=0x7fb870011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=150d699172e64fce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.604+0000 7fb868ff9640 1 -- 172.21.15.37:0/3753958671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870004d20 con 0x558e95ab0680 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.604+0000 7fb868ff9640 1 -- 172.21.15.37:0/3753958671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87000fe20 con 0x558e95ab0680 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.604+0000 7fb868ff9640 1 -- 172.21.15.37:0/3753958671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870012450 con 0x558e95ab0680 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb87ad9d640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8640a33b0 0x7fb8640a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb874009cf0 tx=0x7fb874005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 -- 172.21.15.37:0/3753958671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 msgr2=0x558e95ab2a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ab2a70 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fb870011810 tx=0x7fb870011cd0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 -- 172.21.15.37:0/3753958671 shutdown_connections 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ab2a70 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.609 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 --2- 172.21.15.37:0/3753958671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95aa5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 -- 172.21.15.37:0/3753958671 >> 172.21.15.37:0/3753958671 conn(0x558e95aa1a30 msgr2=0x558e95aa1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 -- 172.21.15.37:0/3753958671 shutdown_connections 2024-09-19T06:52:18.610 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.605+0000 7fb883103740 1 -- 172.21.15.37:0/3753958671 wait complete. 2024-09-19T06:52:18.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.606+0000 7fb883103740 1 -- start start 2024-09-19T06:52:18.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.606+0000 7fb883103740 1 Processor -- start 2024-09-19T06:52:18.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.606+0000 7fb883103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95aaffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.606+0000 7fb883103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95aa5dc0 0x558e95aad250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.606+0000 7fb883103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95ab0680 0x558e95aad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb883103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558e95ab3250 con 0x558e95aa5dc0 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb87a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95aaffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb87a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95aaffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:37580/0 (socket says 172.21.15.37:37580) 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb87a59c640 1 -- 172.21.15.37:0/2234520886 learned_addr learned my addr 172.21.15.37:0/2234520886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb879d9b640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95aa5dc0 0x558e95aad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb883103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e959c6050 con 0x558e95aa54a0 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558e95ab33d0 con 0x558e95ab0680 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb87ad9d640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95ab0680 0x558e95aad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb883103740 10 client.0 _ll_register_callbacks cb 0x558e95a06160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb879d9b640 1 -- 172.21.15.37:0/2234520886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95ab0680 msgr2=0x558e95aad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb879d9b640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95ab0680 0x558e95aad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb879d9b640 1 -- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 msgr2=0x558e95aaffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb879d9b640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95aaffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb879d9b640 1 -- 172.21.15.37:0/2234520886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9577c3c0 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.607+0000 7fb87a59c640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95aaffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb879d9b640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95aa5dc0 0x558e95aad250 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fb8700117e0 tx=0x7fb87000fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5e98f6679e8a519 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb861ffb640 1 -- 172.21.15.37:0/2234520886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87002a060 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb861ffb640 1 -- 172.21.15.37:0/2234520886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fb870004a40 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb87000fe20 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb861ffb640 1 -- 172.21.15.37:0/2234520886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb87002a060 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb883103740 10 client.18405 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558e95abec10 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.608+0000 7fb861ffb640 1 -- 172.21.15.37:0/2234520886 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb87000fe20 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb861ffb640 1 -- 172.21.15.37:0/2234520886 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb87000c070 con 0x558e95aa5dc0 2024-09-19T06:52:18.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb883103740 10 client.18405 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb883103740 10 client.18405 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb883103740 20 client.18405 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb883103740 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95ac00d0 0x558e95ac24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb883103740 4 client.18405 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e95a18850 con 0x558e95ac00d0 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb87a59c640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95ac00d0 0x558e95ac24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb87a59c640 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95ac00d0 0x558e95ac24c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3bf82b1b5404725 server_cookie=577ecb18ac95e093 in_seq=0 out_seq=0 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.609+0000 7fb861ffb640 10 client.18405 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb861ffb640 1 -- 172.21.15.37:0/2234520886 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558e95a18850 con 0x558e95ac00d0 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb861ffb640 10 client.18405 handle_command_reply: tid=0 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 client.18405 shutdown 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95aa5dc0 msgr2=0x558e95aad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95aa5dc0 0x558e95aad250 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fb8700117e0 tx=0x7fb87000fb70 comp rx=0 tx=0).stop 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 shutdown_connections 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95ac00d0 0x558e95ac24c0 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95ab0680 0x558e95aad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95aa5dc0 0x558e95aad250 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 --2- 172.21.15.37:0/2234520886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95aaffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 >> 172.21.15.37:0/2234520886 conn(0x558e95aa1a30 msgr2=0x558e95aa7710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb861ffb640 0 client.18405 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 shutdown_connections 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.610+0000 7fb883103740 1 -- 172.21.15.37:0/2234520886 wait complete. 2024-09-19T06:52:18.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.611+0000 7fb883103740 20 client.18405 trim_cache size 0 max 16384 2024-09-19T06:52:18.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 Processor -- start 2024-09-19T06:52:18.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 -- start start 2024-09-19T06:52:18.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95abc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.634 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa5dc0 0x558e95aacb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ac7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1b6a50 con 0x558e95ab0680 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1b4320 con 0x558e95aa54a0 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb883103740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1b2110 con 0x558e95aa5dc0 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb87a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95abc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb87a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95abc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:37584/0 (socket says 172.21.15.37:37584) 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb87a59c640 1 -- 172.21.15.37:0/3186713652 learned_addr learned my addr 172.21.15.37:0/3186713652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa5dc0 0x558e95aacb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.631+0000 7fb87ad9d640 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ac7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.635 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb87a59c640 1 -- 172.21.15.37:0/3186713652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa5dc0 msgr2=0x558e95aacb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb87a59c640 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa5dc0 0x558e95aacb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb87a59c640 1 -- 172.21.15.37:0/3186713652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 msgr2=0x558e95ac7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb87a59c640 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ac7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb87a59c640 1 -- 172.21.15.37:0/3186713652 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e95aa14d0 con 0x558e95aa54a0 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa5dc0 0x558e95aacb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb87a59c640 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95abc4b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0bc640 tx=0x7fb86c0f2910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0eb18d371ac6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb863fff640 1 -- 172.21.15.37:0/3186713652 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c049040 con 0x558e95aa54a0 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb863fff640 1 -- 172.21.15.37:0/3186713652 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c031de0 con 0x558e95aa54a0 2024-09-19T06:52:18.636 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.632+0000 7fb863fff640 1 -- 172.21.15.37:0/3186713652 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb86c0c0a60 con 0x558e95aa54a0 2024-09-19T06:52:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 -- 172.21.15.37:0/3186713652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 msgr2=0x558e95abc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95abc4b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb86c0bc640 tx=0x7fb86c0f2910 comp rx=0 tx=0).stop 2024-09-19T06:52:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 -- 172.21.15.37:0/3186713652 shutdown_connections 2024-09-19T06:52:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95ac7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa5dc0 0x558e95aacb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 --2- 172.21.15.37:0/3186713652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa54a0 0x558e95abc4b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 -- 172.21.15.37:0/3186713652 >> 172.21.15.37:0/3186713652 conn(0x558e95aa1a30 msgr2=0x558e95aa8870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 -- 172.21.15.37:0/3186713652 shutdown_connections 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.633+0000 7fb883103740 1 -- 172.21.15.37:0/3186713652 wait complete. 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 -- start start 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 Processor -- start 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95ab37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95b28a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1b27e0 con 0x558e95ab0680 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1b3c50 con 0x558e95aa5dc0 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb87c1b2eb0 con 0x558e95aa54a0 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.634+0000 7fb883103740 10 client.0 _ll_register_callbacks cb 0x7fb86c0f3e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:18.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb87ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95b28a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb879d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb87ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95b28a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40942/0 (socket says 172.21.15.37:40942) 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb87ad9d640 1 -- 172.21.15.37:0/3689223187 learned_addr learned my addr 172.21.15.37:0/3689223187 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb87a59c640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95ab37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb879d9b640 1 -- 172.21.15.37:0/3689223187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 msgr2=0x558e95ab37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95ab37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb879d9b640 1 -- 172.21.15.37:0/3689223187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 msgr2=0x558e95b28a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95b28a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb879d9b640 1 -- 172.21.15.37:0/3689223187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e95b28fc0 con 0x558e95aa5dc0 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb87a59c640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95ab37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb87ad9d640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95b28a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb879d9b640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab3d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e7c0 tx=0x7fb8700093c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=152717eb9e8ac3ec server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb861ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:18.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb861ffb640 1 -- 172.21.15.37:0/3689223187 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87002a060 con 0x558e95aa5dc0 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb86c031de0 con 0x558e95aa5dc0 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.635+0000 7fb861ffb640 1 -- 172.21.15.37:0/3689223187 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fb870004a40 con 0x558e95aa5dc0 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb861ffb640 1 -- 172.21.15.37:0/3689223187 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb87003e050 con 0x558e95aa5dc0 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 10 client.27740 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x558e95b2a170 con 0x558e95aa5dc0 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb861ffb640 1 -- 172.21.15.37:0/3689223187 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb86c031de0 con 0x558e95aa5dc0 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb861ffb640 1 -- 172.21.15.37:0/3689223187 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fb87000c070 con 0x558e95aa5dc0 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 10 client.27740 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 10 client.27740 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 20 client.27740 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95b2a660 0x558e95b2ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 4 client.27740 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:52:18.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.636+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558e95a12d60 con 0x558e95b2a660 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb87a59c640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95b2a660 0x558e95b2ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb861ffb640 10 client.27740 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb87a59c640 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95b2a660 0x558e95b2ca80 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=340f60baeaf46d0 server_cookie=ad59f39fa3adc99c in_seq=0 out_seq=0 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb861ffb640 1 -- 172.21.15.37:0/3689223187 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558e95a12d60 con 0x558e95b2a660 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb861ffb640 10 client.27740 handle_command_reply: tid=0 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb883103740 1 client.27740 shutdown 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 msgr2=0x558e95ab3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb883103740 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab3d00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb87000e7c0 tx=0x7fb8700093c0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 shutdown_connections 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.637+0000 7fb883103740 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558e95b2a660 0x558e95b2ca80 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb861ffb640 0 client.27740 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb883103740 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558e95ab0680 0x558e95b28a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb883103740 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e95aa5dc0 0x558e95ab3d00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb883103740 1 --2- 172.21.15.37:0/3689223187 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558e95aa54a0 0x558e95ab37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 >> 172.21.15.37:0/3689223187 conn(0x558e95aa1a30 msgr2=0x558e95aa0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 shutdown_connections 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb883103740 1 -- 172.21.15.37:0/3689223187 wait complete. 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb883103740 20 client.27740 trim_cache size 0 max 16384 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:52:18.645 INFO:teuthology.orchestra.run.smithi037.stdout: "b265b637-b702-4901-8d4e-6fb974c22a8d": { 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "b265b637-b702-4901-8d4e-6fb974c22a8d", 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8640a33b0 msgr2=0x7fb8640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8640a33b0 0x7fb8640a5870 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb874009cf0 tx=0x7fb874005f90 comp rx=0 tx=0).stop 2024-09-19T06:52:18.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb87c100e20 msgr2=0x7fb87c19a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb87c100e20 0x7fb87c19a0d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb86c005940 tx=0x7fb86c003c10 comp rx=0 tx=0).stop 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 shutdown_connections 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb87c102ef0 0x7fb87c196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8640a33b0 0x7fb8640a5870 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb87c102570 0x7fb87c196410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 --2- 172.21.15.37:0/4001922879 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb87c100e20 0x7fb87c19a0d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.638+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 >> 172.21.15.37:0/4001922879 conn(0x7fb87c0fc7d0 msgr2=0x7fb87c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.639+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 shutdown_connections 2024-09-19T06:52:18.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:18.639+0000 7fb880d2a640 1 -- 172.21.15.37:0/4001922879 wait complete. 2024-09-19T06:52:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:18 smithi096 ceph-mon[35004]: pgmap v330: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:18 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2069313275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:18.855 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:18.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:18 smithi151 ceph-mon[35236]: pgmap v330: 97 pgs: 97 active+clean; 57 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 1.7 MiB/s wr, 87 op/s 2024-09-19T06:52:18.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:18 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2069313275' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:19.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 794776 7.61 MB/sec execute 406 sec latency 18.161 ms 2024-09-19T06:52:19.383 INFO:tasks.cephfs.filesystem:scrub status for tag:b265b637-b702-4901-8d4e-6fb974c22a8d - {'path': '/', 'tag': 'b265b637-b702-4901-8d4e-6fb974c22a8d', 'options': 'recursive,force'} 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.811+0000 7fa491eb7640 1 -- 172.21.15.37:0/244032673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c1009b0 msgr2=0x7fa48c100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.811+0000 7fa491eb7640 1 --2- 172.21.15.37:0/244032673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c1009b0 0x7fa48c100e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa474009890 tx=0x7fa47402a380 comp rx=0 tx=0).stop 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.812+0000 7fa491eb7640 1 -- 172.21.15.37:0/244032673 shutdown_connections 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.812+0000 7fa491eb7640 1 --2- 172.21.15.37:0/244032673 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c101350 0x7fa48c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.812+0000 7fa491eb7640 1 --2- 172.21.15.37:0/244032673 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c1009b0 0x7fa48c100e10 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.812+0000 7fa491eb7640 1 --2- 172.21.15.37:0/244032673 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1069c0 0x7fa48c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.812+0000 7fa491eb7640 1 -- 172.21.15.37:0/244032673 >> 172.21.15.37:0/244032673 conn(0x7fa48c077990 msgr2=0x7fa48c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.812+0000 7fa491eb7640 1 -- 172.21.15.37:0/244032673 shutdown_connections 2024-09-19T06:52:19.814 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.812+0000 7fa491eb7640 1 -- 172.21.15.37:0/244032673 wait complete. 2024-09-19T06:52:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 Processor -- start 2024-09-19T06:52:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 -- start start 2024-09-19T06:52:19.816 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1009b0 0x7fa48c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c101350 0x7fa48c198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c1069c0 0x7fa48c19cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c110dc0 con 0x7fa48c1009b0 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c110c40 con 0x7fa48c1069c0 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa491eb7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c110f40 con 0x7fa48c101350 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1009b0 0x7fa48c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1009b0 0x7fa48c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42994/0 (socket says 172.21.15.37:42994) 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 -- 172.21.15.37:0/3667541913 learned_addr learned my addr 172.21.15.37:0/3667541913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48bfff640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c1069c0 0x7fa48c19cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa483fff640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c101350 0x7fa48c198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.817 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 -- 172.21.15.37:0/3667541913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c101350 msgr2=0x7fa48c198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c101350 0x7fa48c198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 -- 172.21.15.37:0/3667541913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c1069c0 msgr2=0x7fa48c19cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c1069c0 0x7fa48c19cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 -- 172.21.15.37:0/3667541913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa48c19d3b0 con 0x7fa48c1009b0 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.813+0000 7fa48b7fe640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1009b0 0x7fa48c1983c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00e9c0 tx=0x7fa47c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.814+0000 7fa4897fa640 1 -- 172.21.15.37:0/3667541913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c00ccc0 con 0x7fa48c1009b0 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.814+0000 7fa4897fa640 1 -- 172.21.15.37:0/3667541913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa47c005500 con 0x7fa48c1009b0 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.814+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa48c19d6a0 con 0x7fa48c1009b0 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.814+0000 7fa4897fa640 1 -- 172.21.15.37:0/3667541913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa47c010430 con 0x7fa48c1009b0 2024-09-19T06:52:19.818 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.814+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa48c1a4f30 con 0x7fa48c1009b0 2024-09-19T06:52:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa4897fa640 1 -- 172.21.15.37:0/3667541913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa47c0046c0 con 0x7fa48c1009b0 2024-09-19T06:52:19.824 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 Processor -- start 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 -- start start 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64382db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643832f0 0x55ae64391b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643920c0 0x55ae643944b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae642e72f0 con 0x55ae643832f0 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae6428f270 con 0x55ae643829d0 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.816+0000 7fa494c50740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ae642e32b0 con 0x55ae643920c0 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643832f0 0x55ae64391b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64382db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.825 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643832f0 0x55ae64391b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43004/0 (socket says 172.21.15.37:43004) 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48b7fe640 1 -- 172.21.15.37:0/4221708410 learned_addr learned my addr 172.21.15.37:0/4221708410 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64382db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35396/0 (socket says 172.21.15.37:35396) 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa483fff640 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643920c0 0x55ae643944b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 -- 172.21.15.37:0/4221708410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643920c0 msgr2=0x55ae643944b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643920c0 0x55ae643944b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 -- 172.21.15.37:0/4221708410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643832f0 msgr2=0x55ae64391b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643832f0 0x55ae64391b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 -- 172.21.15.37:0/4221708410 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae6428e340 con 0x55ae643829d0 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48b7fe640 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643832f0 0x55ae64391b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa483fff640 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643920c0 0x55ae643944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa4897fa640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4780a3460 0x7fa4780a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64382db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa484009f30 tx=0x7fa4840118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80ba7914a1decd03 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.826 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa48b7fe640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4780a3460 0x7fa4780a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa4897fa640 1 -- 172.21.15.37:0/3667541913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fa47c0d0040 con 0x7fa48c1009b0 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.817+0000 7fa481ffb640 1 -- 172.21.15.37:0/4221708410 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484004d20 con 0x55ae643829d0 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa481ffb640 1 -- 172.21.15.37:0/4221708410 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484004050 con 0x55ae643829d0 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa481ffb640 1 -- 172.21.15.37:0/4221708410 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484005220 con 0x55ae643829d0 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 -- 172.21.15.37:0/4221708410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 msgr2=0x55ae64382db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64382db0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa484009f30 tx=0x7fa4840118d0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa48b7fe640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4780a3460 0x7fa4780a5920 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0a4a30 tx=0x7fa47c0d3b90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 -- 172.21.15.37:0/4221708410 shutdown_connections 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643920c0 0x55ae643944b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643832f0 0x55ae64391b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 --2- 172.21.15.37:0/4221708410 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64382db0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 -- 172.21.15.37:0/4221708410 >> 172.21.15.37:0/4221708410 conn(0x55ae6437f060 msgr2=0x55ae6437f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 -- 172.21.15.37:0/4221708410 shutdown_connections 2024-09-19T06:52:19.827 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.818+0000 7fa494c50740 1 -- 172.21.15.37:0/4221708410 wait complete. 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 -- start start 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 Processor -- start 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643829d0 0x55ae6438f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643832f0 0x55ae6438f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae6438fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ae64394cf0 con 0x55ae643920c0 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ae64270290 con 0x55ae643832f0 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ae64394e70 con 0x55ae643829d0 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa494c50740 10 client.0 _ll_register_callbacks cb 0x55ae642a1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.819+0000 7fa483fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643832f0 0x55ae6438f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643832f0 0x55ae6438f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35410/0 (socket says 172.21.15.37:35410) 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae6438fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.828 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 -- 172.21.15.37:0/4069442485 learned_addr learned my addr 172.21.15.37:0/4069442485 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643829d0 0x55ae6438f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 -- 172.21.15.37:0/4069442485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643829d0 msgr2=0x55ae6438f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643829d0 0x55ae6438f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 -- 172.21.15.37:0/4069442485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 msgr2=0x55ae6438fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae6438fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 -- 172.21.15.37:0/4069442485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae63ffa210 con 0x55ae643832f0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa483fff640 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643832f0 0x55ae6438f590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa474009860 tx=0x7fa474004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e79df713c91c25da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa46affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa46affd640 1 -- 172.21.15.37:0/4069442485 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474037070 con 0x55ae643832f0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa484004050 con 0x55ae643832f0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa46affd640 1 -- 172.21.15.37:0/4069442485 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fa474030750 con 0x55ae643832f0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa46affd640 1 -- 172.21.15.37:0/4069442485 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa47404a050 con 0x55ae643832f0 2024-09-19T06:52:19.829 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa494c50740 10 client.27752 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.820+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae640c6620 con 0x55ae643832f0 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa46affd640 1 -- 172.21.15.37:0/4069442485 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484004050 con 0x55ae643832f0 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa46affd640 1 -- 172.21.15.37:0/4069442485 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa474004840 con 0x55ae643832f0 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa494c50740 10 client.27752 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa494c50740 10 client.27752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa494c50740 20 client.27752 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa494c50740 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae643a2150 0x55ae643a4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa494c50740 4 client.27752 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae642f4a20 con 0x55ae643a2150 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae643a2150 0x55ae643a4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa46affd640 10 client.27752 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:19.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.821+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae643a2150 0x55ae643a4540 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=580c421bb960d102 server_cookie=1dce17031359d1bd in_seq=0 out_seq=0 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa46affd640 1 -- 172.21.15.37:0/4069442485 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ae642f4a20 con 0x55ae643a2150 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa46affd640 10 client.27752 handle_command_reply: tid=0 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 client.27752 shutdown 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643832f0 msgr2=0x55ae6438f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643832f0 0x55ae6438f590 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa474009860 tx=0x7fa474004270 comp rx=0 tx=0).stop 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 shutdown_connections 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae643a2150 0x55ae643a4540 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae6438fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643832f0 0x55ae6438f590 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 --2- 172.21.15.37:0/4069442485 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643829d0 0x55ae6438f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa46affd640 0 client.27752 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 >> 172.21.15.37:0/4069442485 conn(0x55ae6437f060 msgr2=0x55ae64384d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 shutdown_connections 2024-09-19T06:52:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 1 -- 172.21.15.37:0/4069442485 wait complete. 2024-09-19T06:52:19.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.827+0000 7fa494c50740 20 client.27752 trim_cache size 0 max 16384 2024-09-19T06:52:19.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 Processor -- start 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 -- start start 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643829d0 0x55ae6438f900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae643a9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643920c0 0x55ae643aa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ae010 con 0x55ae643829d0 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ab8e0 con 0x55ae643920c0 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa494c50740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1a9770 con 0x55ae643832f0 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643829d0 0x55ae6438f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643829d0 0x55ae6438f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43030/0 (socket says 172.21.15.37:43030) 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa48bfff640 1 -- 172.21.15.37:0/4268637200 learned_addr learned my addr 172.21.15.37:0/4268637200 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa483fff640 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae643a9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.842+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643920c0 0x55ae643aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa48bfff640 1 -- 172.21.15.37:0/4268637200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 msgr2=0x55ae643a9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae643a9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa48bfff640 1 -- 172.21.15.37:0/4268637200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643920c0 msgr2=0x55ae643aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643920c0 0x55ae643aa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa48bfff640 1 -- 172.21.15.37:0/4268637200 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae643a4ba0 con 0x55ae643829d0 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643829d0 0x55ae6438f900 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa484009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21dc98f72c0b426e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa4817fa640 1 -- 172.21.15.37:0/4268637200 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa484003a30 con 0x55ae643829d0 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa4817fa640 1 -- 172.21.15.37:0/4268637200 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484028450 con 0x55ae643829d0 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.843+0000 7fa4817fa640 1 -- 172.21.15.37:0/4268637200 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4840040b0 con 0x55ae643829d0 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 -- 172.21.15.37:0/4268637200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643829d0 msgr2=0x55ae6438f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643829d0 0x55ae6438f900 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fa484005fd0 tx=0x7fa484009500 comp rx=0 tx=0).stop 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 -- 172.21.15.37:0/4268637200 shutdown_connections 2024-09-19T06:52:19.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643920c0 0x55ae643aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae643a9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 --2- 172.21.15.37:0/4268637200 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643829d0 0x55ae6438f900 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 -- 172.21.15.37:0/4268637200 >> 172.21.15.37:0/4268637200 conn(0x55ae6437f060 msgr2=0x55ae643910d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 -- 172.21.15.37:0/4268637200 shutdown_connections 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 -- 172.21.15.37:0/4268637200 wait complete. 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 -- start start 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.844+0000 7fa494c50740 1 Processor -- start 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa494c50740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64395e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa494c50740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae64396360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa494c50740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae643968a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa494c50740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1a9dc0 con 0x55ae643920c0 2024-09-19T06:52:19.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa494c50740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1ab210 con 0x55ae643829d0 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa494c50740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa48c1aa470 con 0x55ae643832f0 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa494c50740 10 client.0 _ll_register_callbacks cb 0x7fa47c005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae643968a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa483fff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae64396360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa48bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae643968a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43034/0 (socket says 172.21.15.37:43034) 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa48bfff640 1 -- 172.21.15.37:0/4210175111 learned_addr learned my addr 172.21.15.37:0/4210175111 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64395e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa483fff640 1 -- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 msgr2=0x55ae64395e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa483fff640 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64395e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa483fff640 1 -- 172.21.15.37:0/4210175111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 msgr2=0x55ae643968a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa483fff640 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae643968a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa483fff640 1 -- 172.21.15.37:0/4210175111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae6440bdd0 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae643968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa483fff640 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae64396360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa474004c60 tx=0x7fa474009740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae7aeaafd5a2a70a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa46affd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.845+0000 7fa46affd640 1 -- 172.21.15.37:0/4210175111 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa474052020 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa484028450 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa46affd640 1 -- 172.21.15.37:0/4210175111 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fa474030550 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa46affd640 1 -- 172.21.15.37:0/4210175111 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa474037040 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 10 client.27457 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55ae6440d0d0 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa46affd640 1 -- 172.21.15.37:0/4210175111 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa484028450 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa46affd640 1 -- 172.21.15.37:0/4210175111 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa474030750 con 0x55ae643832f0 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 10 client.27457 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:19.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 10 client.27457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 20 client.27457 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae6440d8a0 0x55ae6440fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 4 client.27457 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ae642eed70 con 0x55ae6440d8a0 2024-09-19T06:52:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.846+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae6440d8a0 0x55ae6440fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.847+0000 7fa46affd640 10 client.27457 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:19.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.847+0000 7fa48bfff640 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae6440d8a0 0x55ae6440fcc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2603c55feb17d80b server_cookie=589888afaa0e083 in_seq=0 out_seq=0 2024-09-19T06:52:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 -- 172.21.15.37:0/3452432572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 msgr2=0x7f5a9810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/3452432572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9810ea90 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f5a90009ca0 tx=0x7f5a9002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 -- 172.21.15.37:0/3452432572 shutdown_connections 2024-09-19T06:52:19.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/3452432572 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9810ea90 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/3452432572 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a98102790 0x7f5a981067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/3452432572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a98101de0 0x7f5a981021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 -- 172.21.15.37:0/3452432572 >> 172.21.15.37:0/3452432572 conn(0x7f5a980fd8c0 msgr2=0x7f5a980ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 -- 172.21.15.37:0/3452432572 shutdown_connections 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.863+0000 7f5a9fd42640 1 -- 172.21.15.37:0/3452432572 wait complete. 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 Processor -- start 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 -- start start 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a98101de0 0x7f5a981983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a98102790 0x7f5a98198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98111f70 con 0x7f5a98106f10 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98111df0 con 0x7f5a98101de0 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9fd42640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981120f0 con 0x7f5a98102790 2024-09-19T06:52:19.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9dade640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a98101de0 0x7f5a981983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43048/0 (socket says 172.21.15.37:43048) 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9d2dd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a98102790 0x7f5a98198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 -- 172.21.15.37:0/883639291 learned_addr learned my addr 172.21.15.37:0/883639291 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9dade640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a98101de0 0x7f5a981983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35452/0 (socket says 172.21.15.37:35452) 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 -- 172.21.15.37:0/883639291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a98102790 msgr2=0x7f5a98198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a98102790 0x7f5a98198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 -- 172.21.15.37:0/883639291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a98101de0 msgr2=0x7f5a981983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a98101de0 0x7f5a981983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9e2df640 1 -- 172.21.15.37:0/883639291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a9819d380 con 0x7f5a98106f10 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.864+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a98102790 0x7f5a98198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:19.868 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.865+0000 7f5a9e2df640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9819cca0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5a90031a20 tx=0x7f5a90031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.865+0000 7f5a8affd640 1 -- 172.21.15.37:0/883639291 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90003c30 con 0x7f5a98106f10 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.865+0000 7f5a8affd640 1 -- 172.21.15.37:0/883639291 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a90003dd0 con 0x7f5a98106f10 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.865+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a9819d610 con 0x7f5a98106f10 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.865+0000 7f5a8affd640 1 -- 172.21.15.37:0/883639291 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9002fc40 con 0x7f5a98106f10 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.865+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a981a4f50 con 0x7f5a98106f10 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.868+0000 7f5a8affd640 1 -- 172.21.15.37:0/883639291 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5a90004050 con 0x7f5a98106f10 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.868+0000 7f5aa211b740 1 Processor -- start 2024-09-19T06:52:19.870 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.868+0000 7f5aa211b740 1 -- start start 2024-09-19T06:52:19.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.868+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdcc99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.868+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcc9f20 0x562cbdcd8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.871 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.869+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd8d50 0x562cbdcdb140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.869+0000 7f5aa211b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562cbdc2f2a0 con 0x562cbdcc9600 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.869+0000 7f5aa211b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562cbdbd6b10 con 0x562cbdcd8d50 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.869+0000 7f5aa211b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cbdc2b260 con 0x562cbdcc9f20 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.869+0000 7f5a8affd640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5a840a3430 0x7f5a840a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.869+0000 7f5a8affd640 1 -- 172.21.15.37:0/883639291 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f5a900f0d30 con 0x7f5a98106f10 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.870+0000 7f5a9dade640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcc9f20 0x562cbdcd8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.870+0000 7f5a9dade640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcc9f20 0x562cbdcd8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:41934/0 (socket says 172.21.15.37:41934) 2024-09-19T06:52:19.872 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.870+0000 7f5a9dade640 1 -- 172.21.15.37:0/2595274808 learned_addr learned my addr 172.21.15.37:0/2595274808 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.871+0000 7f5a9dade640 1 -- 172.21.15.37:0/2595274808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd8d50 msgr2=0x562cbdcdb140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:52:19.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.871+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd8d50 0x562cbdcdb140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.871+0000 7f5a9dade640 1 -- 172.21.15.37:0/2595274808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 msgr2=0x562cbdcc99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.871+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdcc99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.873 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.871+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdcc99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.871+0000 7f5a9dade640 1 -- 172.21.15.37:0/2595274808 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbd9a0b90 con 0x562cbdcc9f20 2024-09-19T06:52:19.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.871+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdcc99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:19.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.872+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcc9f20 0x562cbdcd8810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009d20 tx=0x7f5a8c011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13f4c78f06e03275 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.873+0000 7f5a83fff640 1 -- 172.21.15.37:0/2595274808 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c01e070 con 0x562cbdcc9f20 2024-09-19T06:52:19.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.873+0000 7f5a83fff640 1 -- 172.21.15.37:0/2595274808 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c019b10 con 0x562cbdcc9f20 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.873+0000 7f5a83fff640 1 -- 172.21.15.37:0/2595274808 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00ebb0 con 0x562cbdcc9f20 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.873+0000 7f5a9e2df640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5a840a3430 0x7f5a840a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5aa211b740 1 -- 172.21.15.37:0/2595274808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcc9f20 msgr2=0x562cbdcd8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcc9f20 0x562cbdcd8810 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009d20 tx=0x7f5a8c011810 comp rx=0 tx=0).stop 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5a9e2df640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5a840a3430 0x7f5a840a58f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a900c5950 tx=0x7f5a90033040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5aa211b740 1 -- 172.21.15.37:0/2595274808 shutdown_connections 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd8d50 0x562cbdcdb140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcc9f20 0x562cbdcd8810 secure :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009d20 tx=0x7f5a8c011810 comp rx=0 tx=0).stop 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2595274808 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdcc99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.874+0000 7f5aa211b740 1 -- 172.21.15.37:0/2595274808 >> 172.21.15.37:0/2595274808 conn(0x562cbdcc5cf0 msgr2=0x562cbdcc60d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.878+0000 7f5aa211b740 1 -- 172.21.15.37:0/2595274808 shutdown_connections 2024-09-19T06:52:19.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.878+0000 7f5aa211b740 1 -- 172.21.15.37:0/2595274808 wait complete. 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.878+0000 7f5aa211b740 1 -- start start 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 1 Processor -- start 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd8d50 0x562cbdcd2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd2640 0x562cbdcd6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562cbdcdb920 con 0x562cbdcd8d50 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562cbdbb76e0 con 0x562cbdcc9600 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562cbdcdbaa0 con 0x562cbdcd2640 2024-09-19T06:52:19.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.879+0000 7f5aa211b740 10 client.0 _ll_register_callbacks cb 0x562cbdbdba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.882+0000 7f5a9d2dd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd2640 0x562cbdcd6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.882+0000 7f5a9d2dd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd2640 0x562cbdcd6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:41940/0 (socket says 172.21.15.37:41940) 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.882+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2468345984 learned_addr learned my addr 172.21.15.37:0/2468345984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.883+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.883+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd8d50 0x562cbdcd2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.884+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 msgr2=0x562cbdcd1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.884+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.884+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2468345984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd8d50 msgr2=0x562cbdcd2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.884+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd8d50 0x562cbdcd2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.884+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2468345984 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbdcd6990 con 0x562cbdcd2640 2024-09-19T06:52:19.886 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.884+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd2640 0x562cbdcd6450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a94013f00 tx=0x7f5a94005d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7725639b92c91fe8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.888+0000 7f5a817fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:52:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.888+0000 7f5a817fa640 1 -- 172.21.15.37:0/2468345984 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94011e80 con 0x562cbdcd2640 2024-09-19T06:52:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.888+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a8c019b10 con 0x562cbdcd2640 2024-09-19T06:52:19.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.888+0000 7f5a817fa640 1 -- 172.21.15.37:0/2468345984 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f5a94004020 con 0x562cbdcd2640 2024-09-19T06:52:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.890+0000 7f5a817fa640 1 -- 172.21.15.37:0/2468345984 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a94004220 con 0x562cbdcd2640 2024-09-19T06:52:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.890+0000 7f5aa211b740 10 client.27469 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:19.891 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.890+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562cbdcfd320 con 0x562cbdcd2640 2024-09-19T06:52:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.890+0000 7f5a817fa640 1 -- 172.21.15.37:0/2468345984 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c019b10 con 0x562cbdcd2640 2024-09-19T06:52:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.892+0000 7f5a817fa640 1 -- 172.21.15.37:0/2468345984 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5a940047a0 con 0x562cbdcd2640 2024-09-19T06:52:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.892+0000 7f5aa211b740 10 client.27469 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.892+0000 7f5aa211b740 10 client.27469 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:52:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.892+0000 7f5aa211b740 20 client.27469 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.892+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdcfeac0 0x562cbdd00eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.892+0000 7f5aa211b740 4 client.27469 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:19.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.892+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562cbdc3c9d0 con 0x562cbdcfeac0 2024-09-19T06:52:19.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.893+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdcfeac0 0x562cbdd00eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.894+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdcfeac0 0x562cbdd00eb0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=273a4b362b570b25 server_cookie=550e2bfe569d50e9 in_seq=0 out_seq=0 2024-09-19T06:52:19.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.894+0000 7f5a817fa640 10 client.27469 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:19.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.901+0000 7f5a817fa640 1 -- 172.21.15.37:0/2468345984 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562cbdc3c9d0 con 0x562cbdcfeac0 2024-09-19T06:52:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.901+0000 7f5a817fa640 10 client.27469 handle_command_reply: tid=0 2024-09-19T06:52:19.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.901+0000 7f5aa211b740 1 client.27469 shutdown 2024-09-19T06:52:19.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.902+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd2640 msgr2=0x562cbdcd6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.906 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.902+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd2640 0x562cbdcd6450 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5a94013f00 tx=0x7f5a94005d70 comp rx=0 tx=0).stop 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.903+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 shutdown_connections 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.903+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdcfeac0 0x562cbdd00eb0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.903+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd2640 0x562cbdcd6450 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.903+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd8d50 0x562cbdcd2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.903+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2468345984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.903+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 >> 172.21.15.37:0/2468345984 conn(0x562cbdcc5cf0 msgr2=0x562cbdccba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.903+0000 7f5a817fa640 0 client.27469 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:19.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.906+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 shutdown_connections 2024-09-19T06:52:19.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.906+0000 7f5aa211b740 1 -- 172.21.15.37:0/2468345984 wait complete. 2024-09-19T06:52:19.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.906+0000 7f5aa211b740 20 client.27469 trim_cache size 0 max 16384 2024-09-19T06:52:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.936+0000 7f5aa211b740 1 Processor -- start 2024-09-19T06:52:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.936+0000 7f5aa211b740 1 -- start start 2024-09-19T06:52:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd2640 0x562cbdccb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdd06760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5aa211b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a981a5770 con 0x562cbdcd2640 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5aa211b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98074540 con 0x562cbdcc9600 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5aa211b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98072440 con 0x562cbdcd8d50 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9d2dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9d2dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:35460/0 (socket says 172.21.15.37:35460) 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/4237163723 learned_addr learned my addr 172.21.15.37:0/4237163723 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9dade640 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd2640 0x562cbdccb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdd06760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9dade640 1 -- 172.21.15.37:0/4237163723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 msgr2=0x562cbdd06760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9dade640 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdd06760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9dade640 1 -- 172.21.15.37:0/4237163723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 msgr2=0x562cbdcd6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9dade640 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9dade640 1 -- 172.21.15.37:0/4237163723 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cbdcc1260 con 0x562cbdcd2640 2024-09-19T06:52:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.937+0000 7f5a9dade640 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd2640 0x562cbdccb8b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x562cbdccc2e0 tx=0x7f5a8c0051f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54feb6f0e974661 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.938+0000 7f5a837fe640 1 -- 172.21.15.37:0/4237163723 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c029070 con 0x562cbdcd2640 2024-09-19T06:52:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.938+0000 7f5a837fe640 1 -- 172.21.15.37:0/4237163723 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c005560 con 0x562cbdcd2640 2024-09-19T06:52:19.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.938+0000 7f5a837fe640 1 -- 172.21.15.37:0/4237163723 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c01e040 con 0x562cbdcd2640 2024-09-19T06:52:19.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.938+0000 7f5aa211b740 1 -- 172.21.15.37:0/4237163723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd2640 msgr2=0x562cbdccb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.938+0000 7f5aa211b740 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd2640 0x562cbdccb8b0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x562cbdccc2e0 tx=0x7f5a8c0051f0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.938+0000 7f5aa211b740 1 -- 172.21.15.37:0/4237163723 shutdown_connections 2024-09-19T06:52:19.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdd06760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcd2640 0x562cbdccb8b0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 --2- 172.21.15.37:0/4237163723 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcc9600 0x562cbdcd6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 -- 172.21.15.37:0/4237163723 >> 172.21.15.37:0/4237163723 conn(0x562cbdcc5cf0 msgr2=0x562cbdccb1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 -- 172.21.15.37:0/4237163723 shutdown_connections 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 -- 172.21.15.37:0/4237163723 wait complete. 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 -- start start 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.939+0000 7f5aa211b740 1 Processor -- start 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdce9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd2640 0x562cbdce9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5aa211b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdce6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5aa211b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98072a00 con 0x562cbdcc9600 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5aa211b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a98073e70 con 0x562cbdcd2640 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5aa211b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5a980730d0 con 0x562cbdcd8d50 2024-09-19T06:52:19.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5aa211b740 10 client.0 _ll_register_callbacks cb 0x7f5a94013b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:19.944 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.940+0000 7fa46affd640 1 -- 172.21.15.37:0/4210175111 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ae642eed70 con 0x55ae6440d8a0 2024-09-19T06:52:19.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9d2dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdce9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9dade640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd2640 0x562cbdce9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.940+0000 7fa46affd640 10 client.27457 handle_command_reply: tid=0 2024-09-19T06:52:19.945 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.940+0000 7fa494c50740 1 client.27457 shutdown 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9d2dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdce9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43080/0 (socket says 172.21.15.37:43080) 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2240378759 learned_addr learned my addr 172.21.15.37:0/2240378759 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdce6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2240378759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd2640 msgr2=0x562cbdce9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd2640 0x562cbdce9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.940+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2240378759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 msgr2=0x562cbdce9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdce9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5a9d2dd640 1 -- 172.21.15.37:0/2240378759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cbdce6d90 con 0x562cbdcd8d50 2024-09-19T06:52:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 msgr2=0x55ae64396360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae64396360 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa474004c60 tx=0x7fa474009740 comp rx=0 tx=0).stop 2024-09-19T06:52:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 shutdown_connections 2024-09-19T06:52:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ae6440d8a0 0x55ae6440fcc0 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ae643920c0 0x55ae643968a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ae643832f0 0x55ae64396360 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.947 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 --2- 172.21.15.37:0/4210175111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ae643829d0 0x55ae64395e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5a9d2dd640 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdce6590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c010 tx=0x7f5a94015040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9b426e83dffb8eb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 >> 172.21.15.37:0/4210175111 conn(0x55ae6437f060 msgr2=0x55ae643844f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.948 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa46affd640 0 client.27457 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:19.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5a817fa640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:52:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 shutdown_connections 2024-09-19T06:52:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 1 -- 172.21.15.37:0/4210175111 wait complete. 2024-09-19T06:52:19.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5a817fa640 1 -- 172.21.15.37:0/2240378759 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9402d040 con 0x562cbdcd8d50 2024-09-19T06:52:19.949 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.941+0000 7fa494c50740 20 client.27457 trim_cache size 0 max 16384 2024-09-19T06:52:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5a817fa640 1 -- 172.21.15.37:0/2240378759 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f5a94004f50 con 0x562cbdcd8d50 2024-09-19T06:52:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a8c005560 con 0x562cbdcd8d50 2024-09-19T06:52:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5a817fa640 1 -- 172.21.15.37:0/2240378759 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5a94005150 con 0x562cbdcd8d50 2024-09-19T06:52:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5aa211b740 10 client.27475 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:19.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.941+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562cbdd6a140 con 0x562cbdcd8d50 2024-09-19T06:52:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:52:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:52:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:52:19.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:52:19.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5a817fa640 1 -- 172.21.15.37:0/2240378759 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5a8c005560 con 0x562cbdcd8d50 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4780a3460 msgr2=0x7fa4780a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4780a3460 0x7fa4780a5920 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fa47c0a4a30 tx=0x7fa47c0d3b90 comp rx=0 tx=0).stop 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1009b0 msgr2=0x7fa48c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1009b0 0x7fa48c1983c0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00e9c0 tx=0x7fa47c00ee80 comp rx=0 tx=0).stop 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 shutdown_connections 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa48c1069c0 0x7fa48c19cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa48c101350 0x7fa48c198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa4780a3460 0x7fa4780a5920 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 --2- 172.21.15.37:0/3667541913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa48c1009b0 0x7fa48c1983c0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 >> 172.21.15.37:0/3667541913 conn(0x7fa48c077990 msgr2=0x7fa48c0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 shutdown_connections 2024-09-19T06:52:19.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:19.942+0000 7fa491eb7640 1 -- 172.21.15.37:0/3667541913 wait complete. 2024-09-19T06:52:19.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5a817fa640 1 -- 172.21.15.37:0/2240378759 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f5a94020b10 con 0x562cbdcd8d50 2024-09-19T06:52:19.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5aa211b740 10 client.27475 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:19.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5aa211b740 10 client.27475 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:52:19.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5aa211b740 20 client.27475 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:19.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdd6a820 0x562cbdd6cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:19.953 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5aa211b740 4 client.27475 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}] 2024-09-19T06:52:19.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.942+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.0, "await_for": 45.0}) -- 0x562cbdc36d20 con 0x562cbdd6a820 2024-09-19T06:52:19.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.943+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdd6a820 0x562cbdd6cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:19.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.946+0000 7f5a9dade640 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdd6a820 0x562cbdd6cc10 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=185da27975357e44 server_cookie=1419c677000299ab in_seq=0 out_seq=0 2024-09-19T06:52:19.954 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:19.946+0000 7f5a817fa640 10 client.27475 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:20.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5a817fa640 1 -- 172.21.15.37:0/2240378759 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x562cbdc36d20 con 0x562cbdd6a820 2024-09-19T06:52:20.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5a817fa640 10 client.27475 handle_command_reply: tid=0 2024-09-19T06:52:20.054 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 client.27475 shutdown 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 msgr2=0x562cbdce6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdce6590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400c010 tx=0x7f5a94015040 comp rx=0 tx=0).stop 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 shutdown_connections 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562cbdd6a820 0x562cbdd6cc10 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562cbdcd8d50 0x562cbdce6590 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562cbdcd2640 0x562cbdce9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5a817fa640 0 client.27475 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 --2- 172.21.15.37:0/2240378759 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562cbdcc9600 0x562cbdce9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 >> 172.21.15.37:0/2240378759 conn(0x562cbdcc5cf0 msgr2=0x562cbdccb9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 shutdown_connections 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 1 -- 172.21.15.37:0/2240378759 wait complete. 2024-09-19T06:52:20.055 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.052+0000 7f5aa211b740 20 client.27475 trim_cache size 0 max 16384 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 39, 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 38, 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout: "3aae14d0": { 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 38, 2024-09-19T06:52:20.056 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:20.057 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:20.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:20.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:20.058 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:20.058 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:20.058 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5a840a3430 msgr2=0x7f5a840a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5a840a3430 0x7f5a840a58f0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f5a900c5950 tx=0x7f5a90033040 comp rx=0 tx=0).stop 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 msgr2=0x7f5a9819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9819cca0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f5a90031a20 tx=0x7f5a90031a50 comp rx=0 tx=0).stop 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 shutdown_connections 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5a840a3430 0x7f5a840a58f0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5a98106f10 0x7f5a9819cca0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5a98102790 0x7f5a98198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 --2- 172.21.15.37:0/883639291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a98101de0 0x7f5a981983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 >> 172.21.15.37:0/883639291 conn(0x7f5a980fd8c0 msgr2=0x7f5a980fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 shutdown_connections 2024-09-19T06:52:20.059 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:20.053+0000 7f5a9fd42640 1 -- 172.21.15.37:0/883639291 wait complete. 2024-09-19T06:52:20.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.60 MB/sec execute 407 sec latency 254.478 ms 2024-09-19T06:52:20.715 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:20 smithi037 ceph-mon[31211]: pgmap v331: 97 pgs: 97 active+clean; 66 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 107 op/s 2024-09-19T06:52:20.746 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3aae14d0', quiesce took 5.2 seconds. Will release after: 24.8 2024-09-19T06:52:20.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:20 smithi096 ceph-mon[35004]: pgmap v331: 97 pgs: 97 active+clean; 66 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 107 op/s 2024-09-19T06:52:20.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:20 smithi151 ceph-mon[35236]: pgmap v331: 97 pgs: 97 active+clean; 66 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 1.7 MiB/s wr, 107 op/s 2024-09-19T06:52:21.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.58 MB/sec execute 408 sec latency 1254.596 ms 2024-09-19T06:52:22.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.56 MB/sec execute 409 sec latency 2254.673 ms 2024-09-19T06:52:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:22 smithi096 ceph-mon[35004]: pgmap v332: 97 pgs: 97 active+clean; 66 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-09-19T06:52:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:22 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:52:22.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:22 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:52:22.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:22 smithi037 ceph-mon[31211]: pgmap v332: 97 pgs: 97 active+clean; 66 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-09-19T06:52:22.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:22 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:52:22.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:22 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:52:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:22 smithi151 ceph-mon[35236]: pgmap v332: 97 pgs: 97 active+clean; 66 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 1.6 MiB/s wr, 98 op/s 2024-09-19T06:52:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:22 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:52:22.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:22 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:52:23.227 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.55 MB/sec execute 410 sec latency 3254.783 ms 2024-09-19T06:52:24.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.53 MB/sec execute 411 sec latency 4254.895 ms 2024-09-19T06:52:24.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:24 smithi096 ceph-mon[35004]: pgmap v333: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-19T06:52:24.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:24 smithi037 ceph-mon[31211]: pgmap v333: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-19T06:52:24.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:24 smithi151 ceph-mon[35236]: pgmap v333: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 179 op/s 2024-09-19T06:52:25.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.51 MB/sec execute 412 sec latency 5255.014 ms 2024-09-19T06:52:26.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.49 MB/sec execute 413 sec latency 6255.144 ms 2024-09-19T06:52:26.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:26 smithi096 ceph-mon[35004]: pgmap v334: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T06:52:26.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:26 smithi037 ceph-mon[31211]: pgmap v334: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T06:52:26.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:26 smithi151 ceph-mon[35236]: pgmap v334: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T06:52:27.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.47 MB/sec execute 414 sec latency 7255.260 ms 2024-09-19T06:52:28.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.46 MB/sec execute 415 sec latency 8255.351 ms 2024-09-19T06:52:28.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:28 smithi096 ceph-mon[35004]: pgmap v335: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T06:52:28.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:28 smithi037 ceph-mon[31211]: pgmap v335: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T06:52:28.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:28 smithi151 ceph-mon[35236]: pgmap v335: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 153 op/s 2024-09-19T06:52:29.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.44 MB/sec execute 416 sec latency 9255.472 ms 2024-09-19T06:52:30.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.42 MB/sec execute 417 sec latency 10255.589 ms 2024-09-19T06:52:30.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:30 smithi096 ceph-mon[35004]: pgmap v336: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-19T06:52:30.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:30 smithi037 ceph-mon[31211]: pgmap v336: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-19T06:52:30.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:30 smithi151 ceph-mon[35236]: pgmap v336: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 1.2 MiB/s wr, 158 op/s 2024-09-19T06:52:31.228 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.40 MB/sec execute 418 sec latency 11255.731 ms 2024-09-19T06:52:32.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.38 MB/sec execute 419 sec latency 12255.833 ms 2024-09-19T06:52:32.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:32 smithi096 ceph-mon[35004]: pgmap v337: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 425 KiB/s wr, 95 op/s 2024-09-19T06:52:32.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:32 smithi037 ceph-mon[31211]: pgmap v337: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 425 KiB/s wr, 95 op/s 2024-09-19T06:52:32.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:32 smithi151 ceph-mon[35236]: pgmap v337: 97 pgs: 97 active+clean; 62 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 989 KiB/s rd, 425 KiB/s wr, 95 op/s 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 -- 172.21.15.37:0/1447631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd418105ee0 msgr2=0x7fd418106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 --2- 172.21.15.37:0/1447631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd418105ee0 0x7fd418106350 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fd410009ca0 tx=0x7fd41002a950 comp rx=0 tx=0).stop 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 -- 172.21.15.37:0/1447631736 shutdown_connections 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 --2- 172.21.15.37:0/1447631736 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd418105ee0 0x7fd418106350 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 --2- 172.21.15.37:0/1447631736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd4181013d0 0x7fd418105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 --2- 172.21.15.37:0/1447631736 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd418100ab0 0x7fd418100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 -- 172.21.15.37:0/1447631736 >> 172.21.15.37:0/1447631736 conn(0x7fd4180fc6b0 msgr2=0x7fd4180fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 -- 172.21.15.37:0/1447631736 shutdown_connections 2024-09-19T06:52:33.046 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.044+0000 7fd41f926640 1 -- 172.21.15.37:0/1447631736 wait complete. 2024-09-19T06:52:33.048 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 Processor -- start 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 -- start start 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd418100ab0 0x7fd4181a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4181013d0 0x7fd4181a4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd418105ee0 0x7fd41819e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181184d0 con 0x7fd4181013d0 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd418118350 con 0x7fd418105ee0 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41f926640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd418118650 con 0x7fd418100ab0 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41cec1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4181013d0 0x7fd4181a4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41cec1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4181013d0 0x7fd4181a4b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57950/0 (socket says 172.21.15.37:57950) 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41cec1640 1 -- 172.21.15.37:0/3841471663 learned_addr learned my addr 172.21.15.37:0/3841471663 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41dec3640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd418105ee0 0x7fd41819e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd418100ab0 0x7fd4181a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.049 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41cec1640 1 -- 172.21.15.37:0/3841471663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd418100ab0 msgr2=0x7fd4181a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41cec1640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd418100ab0 0x7fd4181a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41cec1640 1 -- 172.21.15.37:0/3841471663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd418105ee0 msgr2=0x7fd41819e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.045+0000 7fd41cec1640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd418105ee0 0x7fd41819e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.046+0000 7fd41cec1640 1 -- 172.21.15.37:0/3841471663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41819efa0 con 0x7fd4181013d0 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.046+0000 7fd41cec1640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4181013d0 0x7fd4181a4b60 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d740 tx=0x7fd40800db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.046+0000 7fd40e7fc640 1 -- 172.21.15.37:0/3841471663 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408017070 con 0x7fd4181013d0 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.046+0000 7fd40e7fc640 1 -- 172.21.15.37:0/3841471663 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd408010460 con 0x7fd4181013d0 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.046+0000 7fd40e7fc640 1 -- 172.21.15.37:0/3841471663 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd40800f620 con 0x7fd4181013d0 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.046+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41819f290 con 0x7fd4181013d0 2024-09-19T06:52:33.050 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.046+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4181ab5a0 con 0x7fd4181013d0 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.048+0000 7fd40e7fc640 1 -- 172.21.15.37:0/3841471663 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd408010610 con 0x7fd4181013d0 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 Processor -- start 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 -- start start 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89c8af0 0x5640e89c8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89d3cb0 0x5640e89d60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640e892d2f0 con 0x5640e89c9410 2024-09-19T06:52:33.053 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640e88d5270 con 0x5640e89c8af0 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd4226b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5640e89292b0 con 0x5640e89d3cb0 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd41d6c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd40e7fc640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4040a34e0 0x7fd4040a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd41d6c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57954/0 (socket says 172.21.15.37:57954) 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd41d6c2640 1 -- 172.21.15.37:0/3853292429 learned_addr learned my addr 172.21.15.37:0/3853292429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.049+0000 7fd41dec3640 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89c8af0 0x5640e89c8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41cec1640 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89d3cb0 0x5640e89d60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41dec3640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4040a34e0 0x7fd4040a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd40e7fc640 1 -- 172.21.15.37:0/3841471663 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd4080cf720 con 0x7fd4181013d0 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41d6c2640 1 -- 172.21.15.37:0/3853292429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89d3cb0 msgr2=0x5640e89d60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89d3cb0 0x5640e89d60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41d6c2640 1 -- 172.21.15.37:0/3853292429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89c8af0 msgr2=0x5640e89c8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.054 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89c8af0 0x5640e89c8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41d6c2640 1 -- 172.21.15.37:0/3853292429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e88d4340 con 0x5640e89c9410 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d3770 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ad00 tx=0x7fd41400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce08fde828b40bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd41dec3640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4040a34e0 0x7fd4040a59a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd410005e30 tx=0x7fd410005c50 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd4037fe640 1 -- 172.21.15.37:0/3853292429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41401d070 con 0x5640e89c9410 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd4037fe640 1 -- 172.21.15.37:0/3853292429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd414018b20 con 0x5640e89c9410 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.050+0000 7fd4037fe640 1 -- 172.21.15.37:0/3853292429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd414010bc0 con 0x5640e89c9410 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 -- 172.21.15.37:0/3853292429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 msgr2=0x5640e89d3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d3770 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fd41400ad00 tx=0x7fd41400f790 comp rx=0 tx=0).stop 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 -- 172.21.15.37:0/3853292429 shutdown_connections 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89d3cb0 0x5640e89d60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d3770 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3853292429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89c8af0 0x5640e89c8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 -- 172.21.15.37:0/3853292429 >> 172.21.15.37:0/3853292429 conn(0x5640e89c5060 msgr2=0x5640e89c5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:33.055 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 -- 172.21.15.37:0/3853292429 shutdown_connections 2024-09-19T06:52:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 -- 172.21.15.37:0/3853292429 wait complete. 2024-09-19T06:52:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.051+0000 7fd4226b8740 1 -- start start 2024-09-19T06:52:33.056 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 1 Processor -- start 2024-09-19T06:52:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c8af0 0x5640e89d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c9410 0x5640e89d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640e89d6880 con 0x5640e89c8af0 2024-09-19T06:52:33.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640e88b6290 con 0x5640e89d3cb0 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5640e89d6a00 con 0x5640e89c9410 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd4226b8740 10 client.0 _ll_register_callbacks cb 0x5640e88e7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd41d6c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c8af0 0x5640e89d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd41d6c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c8af0 0x5640e89d0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57958/0 (socket says 172.21.15.37:57958) 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd41d6c2640 1 -- 172.21.15.37:0/31285951 learned_addr learned my addr 172.21.15.37:0/31285951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd41dec3640 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.052+0000 7fd41cec1640 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c9410 0x5640e89d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd41d6c2640 1 -- 172.21.15.37:0/31285951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c9410 msgr2=0x5640e89d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c9410 0x5640e89d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd41d6c2640 1 -- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 msgr2=0x5640e89d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd41d6c2640 1 -- 172.21.15.37:0/31285951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e8640210 con 0x5640e89c8af0 2024-09-19T06:52:33.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c8af0 0x5640e89d0b20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd41401b040 tx=0x7fd414010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58b31265462ea36c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd400ff9640 1 -- 172.21.15.37:0/31285951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd41401d040 con 0x5640e89c8af0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd400ff9640 1 -- 172.21.15.37:0/31285951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd414018440 con 0x5640e89c8af0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd414018b20 con 0x5640e89c8af0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd400ff9640 1 -- 172.21.15.37:0/31285951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd414031050 con 0x5640e89c8af0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd4226b8740 10 client.18456 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5640e870c620 con 0x5640e89c8af0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.053+0000 7fd400ff9640 1 -- 172.21.15.37:0/31285951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd414018b20 con 0x5640e89c8af0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd400ff9640 1 -- 172.21.15.37:0/31285951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd414020030 con 0x5640e89c8af0 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd4226b8740 10 client.18456 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd4226b8740 10 client.18456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd4226b8740 20 client.18456 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:33.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd4226b8740 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e89e3b20 0x5640e89e5f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd4226b8740 4 client.18456 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640e893aa20 con 0x5640e89e3b20 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd41cec1640 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e89e3b20 0x5640e89e5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd41cec1640 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e89e3b20 0x5640e89e5f10 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ac9707c12cbbe09 server_cookie=99ce321ef942c345 in_seq=0 out_seq=0 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.054+0000 7fd400ff9640 10 client.18456 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd400ff9640 1 -- 172.21.15.37:0/31285951 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640e893aa20 con 0x5640e89e3b20 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd400ff9640 10 client.18456 handle_command_reply: tid=0 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 client.18456 shutdown 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c8af0 msgr2=0x5640e89d0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c8af0 0x5640e89d0b20 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fd41401b040 tx=0x7fd414010bd0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 shutdown_connections 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e89e3b20 0x5640e89e5f10 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.062 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd400ff9640 0 client.18456 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c9410 0x5640e89d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 --2- 172.21.15.37:0/31285951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c8af0 0x5640e89d0b20 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 >> 172.21.15.37:0/31285951 conn(0x5640e89c5060 msgr2=0x5640e89cadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 shutdown_connections 2024-09-19T06:52:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.055+0000 7fd4226b8740 1 -- 172.21.15.37:0/31285951 wait complete. 2024-09-19T06:52:33.063 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.056+0000 7fd4226b8740 20 client.18456 trim_cache size 0 max 16384 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.070+0000 7fd4226b8740 1 Processor -- start 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.070+0000 7fd4226b8740 1 -- start start 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.070+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89d1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.070+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89eb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd4226b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181b67c0 con 0x5640e89c9410 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd4226b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181b4090 con 0x5640e89d3cb0 2024-09-19T06:52:33.074 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd4226b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181b1f00 con 0x5640e89c8af0 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57962/0 (socket says 172.21.15.37:57962) 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 -- 172.21.15.37:0/3284532863 learned_addr learned my addr 172.21.15.37:0/3284532863 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41dec3640 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89eb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 -- 172.21.15.37:0/3284532863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 msgr2=0x5640e89d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 -- 172.21.15.37:0/3284532863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 msgr2=0x5640e89eb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89eb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 -- 172.21.15.37:0/3284532863 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640e89c4e90 con 0x5640e89c9410 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41dec3640 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:33.075 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd41cec1640 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d1590 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd40809b6d0 tx=0x7fd408086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c559d9100611da1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd402ffd640 1 -- 172.21.15.37:0/3284532863 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408020040 con 0x5640e89c9410 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd402ffd640 1 -- 172.21.15.37:0/3284532863 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40809fd80 con 0x5640e89c9410 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.071+0000 7fd402ffd640 1 -- 172.21.15.37:0/3284532863 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd408018d60 con 0x5640e89c9410 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 -- 172.21.15.37:0/3284532863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 msgr2=0x5640e89d1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d1590 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fd40809b6d0 tx=0x7fd408086040 comp rx=0 tx=0).stop 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 -- 172.21.15.37:0/3284532863 shutdown_connections 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89eb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d1590 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 --2- 172.21.15.37:0/3284532863 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 -- 172.21.15.37:0/3284532863 >> 172.21.15.37:0/3284532863 conn(0x5640e89c5060 msgr2=0x5640e89d2ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 -- 172.21.15.37:0/3284532863 shutdown_connections 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 -- 172.21.15.37:0/3284532863 wait complete. 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 -- start start 2024-09-19T06:52:33.076 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.072+0000 7fd4226b8740 1 Processor -- start 2024-09-19T06:52:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89dc210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.079 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd4226b8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd4226b8740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181b2550 con 0x5640e89c9410 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd4226b8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181b39c0 con 0x5640e89d3cb0 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd4226b8740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd4181b2c20 con 0x5640e89c8af0 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd4226b8740 10 client.0 _ll_register_callbacks cb 0x7fd41401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:57978/0 (socket says 172.21.15.37:57978) 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 -- 172.21.15.37:0/397002390 learned_addr learned my addr 172.21.15.37:0/397002390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89dc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41dec3640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 -- 172.21.15.37:0/397002390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 msgr2=0x5640e89dc210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89dc210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 -- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 msgr2=0x5640e89d7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:33.080 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 -- 172.21.15.37:0/397002390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640e89d80b0 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41dec3640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd41cec1640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d7530 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e010 tx=0x7fd40809b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86e02d18b3e2e879 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.073+0000 7fd400ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd400ff9640 1 -- 172.21.15.37:0/397002390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4080b3070 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd400ff9640 1 -- 172.21.15.37:0/397002390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd4080ba340 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd40809fd80 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd400ff9640 1 -- 172.21.15.37:0/397002390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd408020070 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 10 client.18462 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5640e8a4e2c0 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd400ff9640 1 -- 172.21.15.37:0/397002390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd40809fd80 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd400ff9640 1 -- 172.21.15.37:0/397002390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd414019ce0 con 0x5640e89c9410 2024-09-19T06:52:33.081 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 10 client.18462 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 10 client.18462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 20 client.18462 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e8a4ea30 0x5640e8a50e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 4 client.18462 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5640e8934d70 con 0x5640e8a4ea30 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.074+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e8a4ea30 0x5640e8a50e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.075+0000 7fd41d6c2640 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e8a4ea30 0x5640e8a50e50 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40286a8b3a353596 server_cookie=e458874fced887d7 in_seq=0 out_seq=0 2024-09-19T06:52:33.082 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.075+0000 7fd400ff9640 10 client.18462 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd400ff9640 1 -- 172.21.15.37:0/397002390 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5640e8934d70 con 0x5640e8a4ea30 2024-09-19T06:52:33.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd400ff9640 10 client.18462 handle_command_reply: tid=0 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 client.18462 shutdown 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 msgr2=0x5640e89d7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d7530 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fd40800e010 tx=0x7fd40809b850 comp rx=0 tx=0).stop 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 shutdown_connections 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640e8a4ea30 0x5640e8a50e50 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640e89d3cb0 0x5640e89d7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640e89c9410 0x5640e89d7530 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd400ff9640 0 client.18462 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 --2- 172.21.15.37:0/397002390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640e89c8af0 0x5640e89dc210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 >> 172.21.15.37:0/397002390 conn(0x5640e89c5060 msgr2=0x5640e89c39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 shutdown_connections 2024-09-19T06:52:33.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.113+0000 7fd4226b8740 1 -- 172.21.15.37:0/397002390 wait complete. 2024-09-19T06:52:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd4226b8740 20 client.18462 trim_cache size 0 max 16384 2024-09-19T06:52:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:52:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:52:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:52:33.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4040a34e0 msgr2=0x7fd4040a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4040a34e0 0x7fd4040a59a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd410005e30 tx=0x7fd410005c50 comp rx=0 tx=0).stop 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4181013d0 msgr2=0x7fd4181a4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4181013d0 0x7fd4181a4b60 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fd40800d740 tx=0x7fd40800db40 comp rx=0 tx=0).stop 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 shutdown_connections 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd418105ee0 0x7fd41819e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd4040a34e0 0x7fd4040a59a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd4181013d0 0x7fd4181a4b60 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.118 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 --2- 172.21.15.37:0/3841471663 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd418100ab0 0x7fd4181a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 >> 172.21.15.37:0/3841471663 conn(0x7fd4180fc6b0 msgr2=0x7fd418103840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 shutdown_connections 2024-09-19T06:52:33.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:33.114+0000 7fd41f926640 1 -- 172.21.15.37:0/3841471663 wait complete. 2024-09-19T06:52:33.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.37 MB/sec execute 420 sec latency 13255.922 ms 2024-09-19T06:52:34.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.35 MB/sec execute 421 sec latency 14256.029 ms 2024-09-19T06:52:34.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:34 smithi096 ceph-mon[35004]: pgmap v338: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 523 KiB/s wr, 98 op/s 2024-09-19T06:52:34.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:34 smithi037 ceph-mon[31211]: pgmap v338: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 523 KiB/s wr, 98 op/s 2024-09-19T06:52:34.865 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:34 smithi151 ceph-mon[35236]: pgmap v338: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1014 KiB/s rd, 523 KiB/s wr, 98 op/s 2024-09-19T06:52:35.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.33 MB/sec execute 422 sec latency 15256.103 ms 2024-09-19T06:52:36.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.31 MB/sec execute 423 sec latency 16256.217 ms 2024-09-19T06:52:36.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:36 smithi096 ceph-mon[35004]: pgmap v339: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 113 KiB/s wr, 8 op/s 2024-09-19T06:52:36.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:36 smithi037 ceph-mon[31211]: pgmap v339: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 113 KiB/s wr, 8 op/s 2024-09-19T06:52:36.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:36 smithi151 ceph-mon[35236]: pgmap v339: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 113 KiB/s wr, 8 op/s 2024-09-19T06:52:37.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.30 MB/sec execute 424 sec latency 17256.314 ms 2024-09-19T06:52:38.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.28 MB/sec execute 425 sec latency 18256.420 ms 2024-09-19T06:52:38.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:38 smithi096 ceph-mon[35004]: pgmap v340: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 113 KiB/s wr, 8 op/s 2024-09-19T06:52:38.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:38 smithi037 ceph-mon[31211]: pgmap v340: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 113 KiB/s wr, 8 op/s 2024-09-19T06:52:38.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:38 smithi151 ceph-mon[35236]: pgmap v340: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 113 KiB/s wr, 8 op/s 2024-09-19T06:52:39.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.26 MB/sec execute 426 sec latency 19256.542 ms 2024-09-19T06:52:40.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.25 MB/sec execute 427 sec latency 20256.648 ms 2024-09-19T06:52:40.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:40 smithi096 ceph-mon[35004]: pgmap v341: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 KiB/s wr, 22 op/s 2024-09-19T06:52:40.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:40 smithi037 ceph-mon[31211]: pgmap v341: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 KiB/s wr, 22 op/s 2024-09-19T06:52:40.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:40 smithi151 ceph-mon[35236]: pgmap v341: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 134 KiB/s wr, 22 op/s 2024-09-19T06:52:41.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.23 MB/sec execute 428 sec latency 21256.753 ms 2024-09-19T06:52:42.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.21 MB/sec execute 429 sec latency 22256.799 ms 2024-09-19T06:52:42.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:42 smithi096 ceph-mon[35004]: pgmap v342: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 119 KiB/s wr, 17 op/s 2024-09-19T06:52:42.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:42 smithi037 ceph-mon[31211]: pgmap v342: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 119 KiB/s wr, 17 op/s 2024-09-19T06:52:42.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:42 smithi151 ceph-mon[35236]: pgmap v342: 97 pgs: 97 active+clean; 63 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 119 KiB/s wr, 17 op/s 2024-09-19T06:52:43.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.20 MB/sec execute 430 sec latency 23256.899 ms 2024-09-19T06:52:44.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.18 MB/sec execute 431 sec latency 24257.009 ms 2024-09-19T06:52:44.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:44 smithi096 ceph-mon[35004]: pgmap v343: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 805 KiB/s wr, 33 op/s 2024-09-19T06:52:44.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:44 smithi037 ceph-mon[31211]: pgmap v343: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 805 KiB/s wr, 33 op/s 2024-09-19T06:52:44.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:44 smithi151 ceph-mon[35236]: pgmap v343: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 805 KiB/s wr, 33 op/s 2024-09-19T06:52:45.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.16 MB/sec execute 432 sec latency 25257.109 ms 2024-09-19T06:52:45.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:45.819 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 -- 172.21.15.37:0/545031016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41069c0 msgr2=0x7ffbd4106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/545031016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41069c0 0x7ffbd4106da0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009930 tx=0x7ffbc402a420 comp rx=0 tx=0).stop 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 -- 172.21.15.37:0/545031016 shutdown_connections 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/545031016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd4101350 0x7ffbd410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/545031016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd41009b0 0x7ffbd4100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/545031016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41069c0 0x7ffbd4106da0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 -- 172.21.15.37:0/545031016 >> 172.21.15.37:0/545031016 conn(0x7ffbd4077990 msgr2=0x7ffbd40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.207 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 -- 172.21.15.37:0/545031016 shutdown_connections 2024-09-19T06:52:46.208 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.205+0000 7ffbd89fd640 1 -- 172.21.15.37:0/545031016 wait complete. 2024-09-19T06:52:46.209 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 Processor -- start 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 -- start start 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41009b0 0x7ffbd40fcde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd4101350 0x7ffbd40fd320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd41069c0 0x7ffbd40fd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4110e60 con 0x7ffbd41009b0 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4110ce0 con 0x7ffbd41069c0 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd89fd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd4110fe0 con 0x7ffbd4101350 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41009b0 0x7ffbd40fcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41009b0 0x7ffbd40fcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40206/0 (socket says 172.21.15.37:40206) 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd259c640 1 -- 172.21.15.37:0/3544461310 learned_addr learned my addr 172.21.15.37:0/3544461310 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:46.210 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd41069c0 0x7ffbd40fd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.206+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd4101350 0x7ffbd40fd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd259c640 1 -- 172.21.15.37:0/3544461310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd4101350 msgr2=0x7ffbd40fd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd259c640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd4101350 0x7ffbd40fd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd259c640 1 -- 172.21.15.37:0/3544461310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd41069c0 msgr2=0x7ffbd40fd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd259c640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd41069c0 0x7ffbd40fd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd259c640 1 -- 172.21.15.37:0/3544461310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd41a6a80 con 0x7ffbd41009b0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd259c640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41009b0 0x7ffbd40fcde0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009a50 tx=0x7ffbc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbc37fe640 1 -- 172.21.15.37:0/3544461310 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc403f030 con 0x7ffbd41009b0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbc37fe640 1 -- 172.21.15.37:0/3544461310 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbc4003ce0 con 0x7ffbd41009b0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd41a6d10 con 0x7ffbd41009b0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbc37fe640 1 -- 172.21.15.37:0/3544461310 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4035070 con 0x7ffbd41009b0 2024-09-19T06:52:46.211 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.207+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd41a71b0 con 0x7ffbd41009b0 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.209+0000 7ffbc37fe640 1 -- 172.21.15.37:0/3544461310 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffbc4042030 con 0x7ffbd41009b0 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 Processor -- start 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 -- start start 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c929ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c934770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c934cb0 0x56449c9370a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56449c88e2f0 con 0x56449c929af0 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56449c836270 con 0x56449c934cb0 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.210+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56449c88a2b0 con 0x56449c92a410 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c929ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c929ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40210/0 (socket says 172.21.15.37:40210) 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/1474983086 learned_addr learned my addr 172.21.15.37:0/1474983086 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbc37fe640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbbc0a3460 0x7ffbbc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c934cb0 0x56449c9370a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbbc0a3460 0x7ffbbc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd259c640 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c934770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbc37fe640 1 -- 172.21.15.37:0/3544461310 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ffbc40f02d0 con 0x7ffbd41009b0 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/1474983086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 msgr2=0x56449c934770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c934770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/1474983086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c934cb0 msgr2=0x56449c9370a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c934cb0 0x56449c9370a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/1474983086 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56449c835340 con 0x56449c929af0 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd259c640 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c934770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c929ed0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00a990 tx=0x7ffbcc00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68cf981f06db89b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbc0ff9640 1 -- 172.21.15.37:0/1474983086 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc01d030 con 0x56449c929af0 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.211+0000 7ffbc0ff9640 1 -- 172.21.15.37:0/1474983086 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc015c60 con 0x56449c929af0 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbc0ff9640 1 -- 172.21.15.37:0/1474983086 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0213c0 con 0x56449c929af0 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbbc0a3460 0x7ffbbc0a5920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc020000 tx=0x7ffbcc01e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 -- 172.21.15.37:0/1474983086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 msgr2=0x56449c929ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c929ed0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00a990 tx=0x7ffbcc00ae50 comp rx=0 tx=0).stop 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 -- 172.21.15.37:0/1474983086 shutdown_connections 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c934cb0 0x56449c9370a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c934770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/1474983086 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c929ed0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 -- 172.21.15.37:0/1474983086 >> 172.21.15.37:0/1474983086 conn(0x56449c926060 msgr2=0x56449c926440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 -- 172.21.15.37:0/1474983086 shutdown_connections 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.212+0000 7ffbdb87f740 1 -- 172.21.15.37:0/1474983086 wait complete. 2024-09-19T06:52:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 -- start start 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 Processor -- start 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c929af0 0x56449c931ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c9320e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c932620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56449c937880 con 0x56449c934cb0 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56449c817290 con 0x56449c92a410 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56449c937a00 con 0x56449c929af0 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbdb87f740 10 client.0 _ll_register_callbacks cb 0x56449c848140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c932620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c932620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40224/0 (socket says 172.21.15.37:40224) 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4222784903 learned_addr learned my addr 172.21.15.37:0/4222784903 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c929af0 0x56449c931ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c9320e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4222784903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c929af0 msgr2=0x56449c931ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c929af0 0x56449c931ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 msgr2=0x56449c9320e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.213+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c9320e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4222784903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56449c5a1210 con 0x56449c934cb0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c9320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c932620 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009690 tx=0x7ffbcc019bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb77674e4a63d5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4222784903 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc02f060 con 0x56449c934cb0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4222784903 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ffbcc0216a0 con 0x56449c934cb0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc015c60 con 0x56449c934cb0 2024-09-19T06:52:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4222784903 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffbcc01d050 con 0x56449c934cb0 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbdb87f740 10 client.18474 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56449c66d620 con 0x56449c934cb0 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4222784903 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc015c60 con 0x56449c934cb0 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.214+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4222784903 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffbcc029720 con 0x56449c934cb0 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbdb87f740 10 client.18474 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbdb87f740 10 client.18474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbdb87f740 20 client.18474 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c944b40 0x56449c946f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbdb87f740 4 client.18474 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56449c89ba20 con 0x56449c944b40 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c944b40 0x56449c946f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbb9ffb640 10 client.18474 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:46.222 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.215+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c944b40 0x56449c946f30 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f32a0ff2d071313 server_cookie=5444e3771c51026f in_seq=0 out_seq=0 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4222784903 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56449c89ba20 con 0x56449c944b40 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbb9ffb640 10 client.18474 handle_command_reply: tid=0 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 client.18474 shutdown 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 msgr2=0x56449c932620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c932620 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc009690 tx=0x7ffbcc019bd0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 shutdown_connections 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c944b40 0x56449c946f30 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c932620 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c9320e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbb9ffb640 0 client.18474 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4222784903 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c929af0 0x56449c931ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.224 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 >> 172.21.15.37:0/4222784903 conn(0x56449c926060 msgr2=0x56449c92bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 shutdown_connections 2024-09-19T06:52:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4222784903 wait complete. 2024-09-19T06:52:46.225 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.216+0000 7ffbdb87f740 20 client.18474 trim_cache size 0 max 16384 2024-09-19T06:52:46.229 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.15 MB/sec execute 433 sec latency 26257.159 ms 2024-09-19T06:52:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 Processor -- start 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 -- start start 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c929af0 0x56449c932450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c933b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c94c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41b23c0 con 0x56449c934cb0 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41afc90 con 0x56449c929af0 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41adb00 con 0x56449c92a410 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c94c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c94c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40226/0 (socket says 172.21.15.37:40226) 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4037223288 learned_addr learned my addr 172.21.15.37:0/4037223288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c929af0 0x56449c932450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c933b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4037223288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 msgr2=0x56449c933b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c933b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4037223288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c929af0 msgr2=0x56449c932450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c929af0 0x56449c932450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd2d9d640 1 -- 172.21.15.37:0/4037223288 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56449c927110 con 0x56449c934cb0 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c933b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.231+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c929af0 0x56449c932450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.232+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c94c700 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00a830 tx=0x7ffbcc00a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be40220dcff9d263 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.232+0000 7ffbbbfff640 1 -- 172.21.15.37:0/4037223288 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc02f040 con 0x56449c934cb0 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.232+0000 7ffbbbfff640 1 -- 172.21.15.37:0/4037223288 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc026070 con 0x56449c934cb0 2024-09-19T06:52:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.232+0000 7ffbbbfff640 1 -- 172.21.15.37:0/4037223288 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc0216a0 con 0x56449c934cb0 2024-09-19T06:52:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.232+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4037223288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 msgr2=0x56449c94c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.232+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c94c700 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00a830 tx=0x7ffbcc00a610 comp rx=0 tx=0).stop 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4037223288 shutdown_connections 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c934cb0 0x56449c94c700 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c92a410 0x56449c933b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4037223288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c929af0 0x56449c932450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4037223288 >> 172.21.15.37:0/4037223288 conn(0x56449c926060 msgr2=0x56449c933400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4037223288 shutdown_connections 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4037223288 wait complete. 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- start start 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 Processor -- start 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c938240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c938780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.239 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c934cb0 0x56449c9ad6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41ae150 con 0x56449c929af0 2024-09-19T06:52:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41af5c0 con 0x56449c92a410 2024-09-19T06:52:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd41ae820 con 0x56449c934cb0 2024-09-19T06:52:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbdb87f740 10 client.0 _ll_register_callbacks cb 0x7ffbcc00a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbd259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c938240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.240 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbd1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c938780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbd259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c938240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40236/0 (socket says 172.21.15.37:40236) 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbd259c640 1 -- 172.21.15.37:0/4275366966 learned_addr learned my addr 172.21.15.37:0/4275366966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.233+0000 7ffbd2d9d640 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c934cb0 0x56449c9ad6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbd259c640 1 -- 172.21.15.37:0/4275366966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c934cb0 msgr2=0x56449c9ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c934cb0 0x56449c9ad6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbd259c640 1 -- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 msgr2=0x56449c938780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c938780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbd259c640 1 -- 172.21.15.37:0/4275366966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56449c9212d0 con 0x56449c929af0 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c938780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbd259c640 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c938240 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4031550 tx=0x7ffbc402faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=850870a7c6189aeb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4275366966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc4049040 con 0x56449c929af0 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffbcc026070 con 0x56449c929af0 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.234+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4275366966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7ffbc403b4d0 con 0x56449c929af0 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4275366966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffbc404ad50 con 0x56449c929af0 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 10 client.18480 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56449c9aed10 con 0x56449c929af0 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4275366966 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbcc026070 con 0x56449c929af0 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4275366966 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffbc4031bd0 con 0x56449c929af0 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 10 client.18480 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 10 client.18480 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 20 client.18480 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c9af4e0 0x56449c9b1900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 4 client.18480 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56449c895d70 con 0x56449c9af4e0 2024-09-19T06:52:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.235+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c9af4e0 0x56449c9b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.236+0000 7ffbd1d9b640 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c9af4e0 0x56449c9b1900 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7927d39566ca6e3c server_cookie=f265b46d9cbdfaed in_seq=0 out_seq=0 2024-09-19T06:52:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.236+0000 7ffbb9ffb640 10 client.18480 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbb9ffb640 1 -- 172.21.15.37:0/4275366966 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56449c895d70 con 0x56449c9af4e0 2024-09-19T06:52:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbb9ffb640 10 client.18480 handle_command_reply: tid=0 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 client.18480 shutdown 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 msgr2=0x56449c938240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c938240 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4031550 tx=0x7ffbc402faf0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 shutdown_connections 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56449c9af4e0 0x56449c9b1900 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56449c934cb0 0x56449c9ad6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56449c92a410 0x56449c938780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbb9ffb640 0 client.18480 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 --2- 172.21.15.37:0/4275366966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56449c929af0 0x56449c938240 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 >> 172.21.15.37:0/4275366966 conn(0x56449c926060 msgr2=0x56449c930020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 shutdown_connections 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 1 -- 172.21.15.37:0/4275366966 wait complete. 2024-09-19T06:52:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.294+0000 7ffbdb87f740 20 client.18480 trim_cache size 0 max 16384 2024-09-19T06:52:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:52:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:52:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:52:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbbc0a3460 msgr2=0x7ffbbc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbbc0a3460 0x7ffbbc0a5920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc020000 tx=0x7ffbcc01e000 comp rx=0 tx=0).stop 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41009b0 msgr2=0x7ffbd40fcde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41009b0 0x7ffbd40fcde0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4009a50 tx=0x7ffbc4004270 comp rx=0 tx=0).stop 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 shutdown_connections 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffbd41069c0 0x7ffbd40fd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffbd4101350 0x7ffbd40fd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffbbc0a3460 0x7ffbbc0a5920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 --2- 172.21.15.37:0/3544461310 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffbd41009b0 0x7ffbd40fcde0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 >> 172.21.15.37:0/3544461310 conn(0x7ffbd4077990 msgr2=0x7ffbd4069ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 shutdown_connections 2024-09-19T06:52:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:46.295+0000 7ffbd89fd640 1 -- 172.21.15.37:0/3544461310 wait complete. 2024-09-19T06:52:46.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.271+0000 7f09a37fe640 1 -- 172.21.15.37:0/3032933032 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998035020 con 0x7f09a4102d40 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.272+0000 7f09aabdd640 1 -- 172.21.15.37:0/3032933032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 msgr2=0x7f09a4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.272+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3032933032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a4103120 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f0998009900 tx=0x7f099802a4a0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.273+0000 7f09aabdd640 1 -- 172.21.15.37:0/3032933032 shutdown_connections 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.273+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3032933032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a410c170 0x7f09a410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.273+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3032933032 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a4103660 0x7f09a410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.273+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3032933032 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a4103120 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.273+0000 7f09aabdd640 1 -- 172.21.15.37:0/3032933032 >> 172.21.15.37:0/3032933032 conn(0x7f09a40fc7a0 msgr2=0x7f09a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.273+0000 7f09aabdd640 1 -- 172.21.15.37:0/3032933032 shutdown_connections 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.273+0000 7f09aabdd640 1 -- 172.21.15.37:0/3032933032 wait complete. 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 Processor -- start 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 -- start start 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.421 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4103660 0x7f09a4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a410c170 0x7f09a4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4112020 con 0x7f09a4102d40 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4111ea0 con 0x7f09a410c170 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09aabdd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f09a41121a0 con 0x7f09a4103660 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a8979640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a8979640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a41009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40256/0 (socket says 172.21.15.37:40256) 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a8979640 1 -- 172.21.15.37:0/3234624028 learned_addr learned my addr 172.21.15.37:0/3234624028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a917a640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a410c170 0x7f09a4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a3fff640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4103660 0x7f09a4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a8979640 1 -- 172.21.15.37:0/3234624028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4103660 msgr2=0x7f09a4100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a8979640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4103660 0x7f09a4100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a8979640 1 -- 172.21.15.37:0/3234624028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a410c170 msgr2=0x7f09a4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.275+0000 7f09a8979640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a410c170 0x7f09a4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09a8979640 1 -- 172.21.15.37:0/3234624028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a41aae80 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09a917a640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a410c170 0x7f09a4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09a3fff640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4103660 0x7f09a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09a8979640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a41009a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0998000c00 tx=0x7f0998032b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09a1ffb640 1 -- 172.21.15.37:0/3234624028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099803f030 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09a1ffb640 1 -- 172.21.15.37:0/3234624028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0998003ad0 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09a1ffb640 1 -- 172.21.15.37:0/3234624028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0998035070 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a41ab050 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.276+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41ab540 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.278+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a406b070 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.279+0000 7f09a1ffb640 1 -- 172.21.15.37:0/3234624028 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0998042030 con 0x7f09a4102d40 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.279+0000 7f09a1ffb640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09940a3430 0x7f09940a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.280+0000 7f09a3fff640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09940a3430 0x7f09940a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:46.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.280+0000 7f09a1ffb640 1 -- 172.21.15.37:0/3234624028 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f09980f05e0 con 0x7f09a4102d40 2024-09-19T06:52:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.280+0000 7f09a3fff640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09940a3430 0x7f09940a58f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f09a4102160 tx=0x7f09900092a0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:46.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.284+0000 7f09a1ffb640 1 -- 172.21.15.37:0/3234624028 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f09980bcf30 con 0x7f09a4102d40 2024-09-19T06:52:46.500 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.499+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f09a4107240 con 0x7f09a4102d40 2024-09-19T06:52:46.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.499+0000 7f09a1ffb640 1 -- 172.21.15.37:0/3234624028 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7f09980bc830 con 0x7f09a4102d40 2024-09-19T06:52:46.501 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:52:46.502 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:52:46.502 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09940a3430 msgr2=0x7f09940a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09940a3430 0x7f09940a58f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f09a4102160 tx=0x7f09900092a0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 msgr2=0x7f09a41009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a41009a0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0998000c00 tx=0x7f0998032b20 comp rx=0 tx=0).stop 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 shutdown_connections 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09a410c170 0x7f09a4101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f09a4103660 0x7f09a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f09940a3430 0x7f09940a58f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 --2- 172.21.15.37:0/3234624028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f09a4102d40 0x7f09a41009a0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 >> 172.21.15.37:0/3234624028 conn(0x7f09a40fc7a0 msgr2=0x7f09a40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 shutdown_connections 2024-09-19T06:52:46.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:46.502+0000 7f09aabdd640 1 -- 172.21.15.37:0/3234624028 wait complete. 2024-09-19T06:52:46.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:46 smithi037 ceph-mon[31211]: pgmap v344: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 707 KiB/s wr, 30 op/s 2024-09-19T06:52:46.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:46 smithi096 ceph-mon[35004]: pgmap v344: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 707 KiB/s wr, 30 op/s 2024-09-19T06:52:46.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:46 smithi151 ceph-mon[35236]: pgmap v344: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 707 KiB/s wr, 30 op/s 2024-09-19T06:52:47.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795634 7.13 MB/sec execute 434 sec latency 27257.266 ms 2024-09-19T06:52:47.306 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id 3aae14d0 --release --await' 2024-09-19T06:52:47.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id 3aae14d0 --release --await 2024-09-19T06:52:47.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:47 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3234624028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:47.570 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:47.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:47 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3234624028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:47.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:47 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3234624028' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.025+0000 7f7b4fe01640 1 -- 172.21.15.37:0/4048328164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b4810e480 msgr2=0x7f7b48110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.038 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.025+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/4048328164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b4810e480 0x7f7b48110870 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f7b40009ca0 tx=0x7f7b4002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 -- 172.21.15.37:0/4048328164 shutdown_connections 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/4048328164 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b4810e480 0x7f7b48110870 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/4048328164 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b48101280 0x7f7b4810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/4048328164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b481008d0 0x7f7b48100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 -- 172.21.15.37:0/4048328164 >> 172.21.15.37:0/4048328164 conn(0x7f7b480fc7a0 msgr2=0x7f7b480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 -- 172.21.15.37:0/4048328164 shutdown_connections 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 -- 172.21.15.37:0/4048328164 wait complete. 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 Processor -- start 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.026+0000 7f7b4fe01640 1 -- start start 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4fe01640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b481008d0 0x7f7b4819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4fe01640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48101280 0x7f7b4819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4fe01640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4810e480 0x7f7b4819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.039 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4fe01640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48114180 con 0x7f7b481008d0 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4fe01640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48114000 con 0x7f7b48101280 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4fe01640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b48114300 con 0x7f7b4810e480 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b481008d0 0x7f7b4819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b481008d0 0x7f7b4819a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40268/0 (socket says 172.21.15.37:40268) 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 -- 172.21.15.37:0/856392369 learned_addr learned my addr 172.21.15.37:0/856392369 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4810e480 0x7f7b4819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48101280 0x7f7b4819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 -- 172.21.15.37:0/856392369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4810e480 msgr2=0x7f7b4819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4810e480 0x7f7b4819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 -- 172.21.15.37:0/856392369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48101280 msgr2=0x7f7b4819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48101280 0x7f7b4819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.040 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 -- 172.21.15.37:0/856392369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b4819f630 con 0x7f7b481008d0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48101280 0x7f7b4819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b481008d0 0x7f7b4819a680 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400d950 tx=0x7f7b4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b3effd640 1 -- 172.21.15.37:0/856392369 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44017070 con 0x7f7b481008d0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b4819f920 con 0x7f7b481008d0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.027+0000 7f7b3effd640 1 -- 172.21.15.37:0/856392369 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b44009c30 con 0x7f7b481008d0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.028+0000 7f7b3effd640 1 -- 172.21.15.37:0/856392369 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4400f6b0 con 0x7f7b481008d0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.028+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b481a7200 con 0x7f7b481008d0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b3effd640 1 -- 172.21.15.37:0/856392369 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7b440109d0 con 0x7f7b481008d0 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 Processor -- start 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 -- start start 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f25440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25980 0x55be90f2fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f32630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.041 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55be90e8a620 con 0x55be90f25060 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be90e31e50 con 0x55be90f30240 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.030+0000 7f7b521da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55be90e865e0 con 0x55be90f25980 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f25440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f25440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40270/0 (socket says 172.21.15.37:40270) 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 -- 172.21.15.37:0/3166870415 learned_addr learned my addr 172.21.15.37:0/3166870415 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b3effd640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b340a33b0 0x7f7b340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f32630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25980 0x55be90f2fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 -- 172.21.15.37:0/3166870415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25980 msgr2=0x55be90f2fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25980 0x55be90f2fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 -- 172.21.15.37:0/3166870415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 msgr2=0x55be90f32630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f32630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.042 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 -- 172.21.15.37:0/3166870415 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be90e22d40 con 0x55be90f25060 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f32630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b3effd640 1 -- 172.21.15.37:0/856392369 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f7b440cf9c0 con 0x7f7b481008d0 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b340a33b0 0x7f7b340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f25440 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7b40004af0 tx=0x7f7b400099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4753abea6f850b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b33fff640 1 -- 172.21.15.37:0/3166870415 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b40042040 con 0x55be90f25060 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b33fff640 1 -- 172.21.15.37:0/3166870415 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b40036040 con 0x55be90f25060 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.031+0000 7f7b33fff640 1 -- 172.21.15.37:0/3166870415 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b4002f9d0 con 0x55be90f25060 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 -- 172.21.15.37:0/3166870415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 msgr2=0x55be90f25440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b340a33b0 0x7f7b340a5870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b40049000 tx=0x7f7b4004e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f25440 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f7b40004af0 tx=0x7f7b400099d0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 -- 172.21.15.37:0/3166870415 shutdown_connections 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f32630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25980 0x55be90f2fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.043 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 --2- 172.21.15.37:0/3166870415 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f25440 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 -- 172.21.15.37:0/3166870415 >> 172.21.15.37:0/3166870415 conn(0x55be90f215f0 msgr2=0x55be90f219d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 -- 172.21.15.37:0/3166870415 shutdown_connections 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.032+0000 7f7b521da740 1 -- 172.21.15.37:0/3166870415 wait complete. 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 -- start start 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 Processor -- start 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f2d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25980 0x55be90f2d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f2dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55be90f32e60 con 0x55be90f25060 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55be90e12f10 con 0x55be90f25980 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55be90f32fe0 con 0x55be90f30240 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b521da740 10 client.0 _ll_register_callbacks cb 0x55be90e36db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:48.044 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b4db9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f2d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b4db9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f2d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40278/0 (socket says 172.21.15.37:40278) 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.033+0000 7f7b4db9d640 1 -- 172.21.15.37:0/2022744390 learned_addr learned my addr 172.21.15.37:0/2022744390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f2dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25980 0x55be90f2d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4db9d640 1 -- 172.21.15.37:0/2022744390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 msgr2=0x55be90f2dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f2dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4db9d640 1 -- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25980 msgr2=0x55be90f2d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25980 0x55be90f2d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4db9d640 1 -- 172.21.15.37:0/2022744390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be90b99580 con 0x55be90f25060 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25980 0x55be90f2d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f2dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f2d150 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7b440cd140 tx=0x7f7b4409b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe481672647cdf9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b317fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:48.045 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b317fa640 1 -- 172.21.15.37:0/2022744390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b44020070 con 0x55be90f25060 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b317fa640 1 -- 172.21.15.37:0/2022744390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f7b4409b0c0 con 0x55be90f25060 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b40036040 con 0x55be90f25060 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b317fa640 1 -- 172.21.15.37:0/2022744390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b4409bc70 con 0x55be90f25060 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b521da740 10 client.18498 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55be90c63960 con 0x55be90f25060 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.034+0000 7f7b317fa640 1 -- 172.21.15.37:0/2022744390 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b40036040 con 0x55be90f25060 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b317fa640 1 -- 172.21.15.37:0/2022744390 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7b440979c0 con 0x55be90f25060 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b521da740 10 client.18498 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b521da740 10 client.18498 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b521da740 20 client.18498 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b521da740 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90f3fdd0 0x55be90f42290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b521da740 4 client.18498 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be90e97b90 con 0x55be90f3fdd0 2024-09-19T06:52:48.046 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90f3fdd0 0x55be90f42290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90f3fdd0 0x55be90f42290 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93e22b86fba90d42 server_cookie=a2d055892c0088cc in_seq=0 out_seq=0 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.035+0000 7f7b317fa640 10 client.18498 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.036+0000 7f7b317fa640 1 -- 172.21.15.37:0/2022744390 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55be90e97b90 con 0x55be90f3fdd0 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.036+0000 7f7b317fa640 10 client.18498 handle_command_reply: tid=0 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.036+0000 7f7b521da740 1 client.18498 shutdown 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.036+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 msgr2=0x55be90f2d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.036+0000 7f7b521da740 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f2d150 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f7b440cd140 tx=0x7f7b4409b850 comp rx=0 tx=0).stop 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.036+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 shutdown_connections 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.036+0000 7f7b521da740 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90f3fdd0 0x55be90f42290 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b521da740 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f2dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b521da740 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25980 0x55be90f2d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b521da740 1 --2- 172.21.15.37:0/2022744390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25060 0x55be90f2d150 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 >> 172.21.15.37:0/2022744390 conn(0x55be90f215f0 msgr2=0x55be90f272d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:48.047 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b317fa640 0 client.18498 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 shutdown_connections 2024-09-19T06:52:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b521da740 1 -- 172.21.15.37:0/2022744390 wait complete. 2024-09-19T06:52:48.048 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.037+0000 7f7b521da740 20 client.18498 trim_cache size 0 max 16384 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 Processor -- start 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 -- start start 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25060 0x55be90f27ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f47bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481b24b0 con 0x55be90f25980 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481afd80 con 0x55be90f25060 2024-09-19T06:52:48.056 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b521da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481adbf0 con 0x55be90f30240 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f47bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f47bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40290/0 (socket says 172.21.15.37:40290) 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 -- 172.21.15.37:0/4274596303 learned_addr learned my addr 172.21.15.37:0/4274596303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25060 0x55be90f27ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 -- 172.21.15.37:0/4274596303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 msgr2=0x55be90f480f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 -- 172.21.15.37:0/4274596303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25060 msgr2=0x55be90f27ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25060 0x55be90f27ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 -- 172.21.15.37:0/4274596303 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be90f28000 con 0x55be90f25980 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.054+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f47bb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x55be90f2ca20 tx=0x7f7b38009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c382555c357e696 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b337fe640 1 -- 172.21.15.37:0/4274596303 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b3801b5b0 con 0x55be90f25980 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b337fe640 1 -- 172.21.15.37:0/4274596303 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3801bc30 con 0x55be90f25980 2024-09-19T06:52:48.057 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b337fe640 1 -- 172.21.15.37:0/4274596303 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38011cb0 con 0x55be90f25980 2024-09-19T06:52:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 -- 172.21.15.37:0/4274596303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 msgr2=0x55be90f47bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.060 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f47bb0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x55be90f2ca20 tx=0x7f7b38009640 comp rx=0 tx=0).stop 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 -- 172.21.15.37:0/4274596303 shutdown_connections 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f30240 0x55be90f480f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f47bb0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 --2- 172.21.15.37:0/4274596303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f25060 0x55be90f27ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 -- 172.21.15.37:0/4274596303 >> 172.21.15.37:0/4274596303 conn(0x55be90f215f0 msgr2=0x55be90f2f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 -- 172.21.15.37:0/4274596303 shutdown_connections 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.055+0000 7f7b521da740 1 -- 172.21.15.37:0/4274596303 wait complete. 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 -- start start 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 Processor -- start 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25060 0x55be90f38a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f29fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f2a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.061 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481ae240 con 0x55be90f25980 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481af6b0 con 0x55be90f30240 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7b481ae910 con 0x55be90f25060 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b521da740 10 client.0 _ll_register_callbacks cb 0x7f7b4409bb30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.056+0000 7f7b4d39c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f29fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f29fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:40298/0 (socket says 172.21.15.37:40298) 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 -- 172.21.15.37:0/1010078250 learned_addr learned my addr 172.21.15.37:0/1010078250 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25060 0x55be90f38a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4e39e640 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f2a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 -- 172.21.15.37:0/1010078250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25060 msgr2=0x55be90f38a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25060 0x55be90f38a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 -- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 msgr2=0x55be90f2a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f2a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.062 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 -- 172.21.15.37:0/1010078250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be90f2adf0 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b4d39c640 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f29fd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f7b38000c00 tx=0x7f7b38011470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d674c1f93d062b17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b317fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b317fa640 1 -- 172.21.15.37:0/1010078250 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b38011990 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b317fa640 1 -- 172.21.15.37:0/1010078250 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f7b38011b30 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7b3801bc30 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b317fa640 1 -- 172.21.15.37:0/1010078250 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7b38011d30 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b521da740 10 client.18504 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.057+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55be90faaca0 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b317fa640 1 -- 172.21.15.37:0/1010078250 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7b3801bc30 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b317fa640 1 -- 172.21.15.37:0/1010078250 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7b3800f040 con 0x55be90f25980 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b521da740 10 client.18504 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b521da740 10 client.18504 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b521da740 20 client.18504 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:48.063 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b521da740 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90fab300 0x55be90fad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b521da740 4 client.18504 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3aae14d0", "release": true, "await": true}] 2024-09-19T06:52:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3aae14d0", "release": true, "await": true}) -- 0x55be90e920a0 con 0x55be90fab300 2024-09-19T06:52:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.058+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90fab300 0x55be90fad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.059+0000 7f7b4db9d640 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90fab300 0x55be90fad720 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24bfeba5d5107755 server_cookie=bee422606b8c2264 in_seq=0 out_seq=0 2024-09-19T06:52:48.064 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.059+0000 7f7b317fa640 10 client.18504 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.068+0000 7f7b317fa640 1 -- 172.21.15.37:0/1010078250 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55be90e920a0 con 0x55be90fab300 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.068+0000 7f7b317fa640 10 client.18504 handle_command_reply: tid=0 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.068+0000 7f7b521da740 1 client.18504 shutdown 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 msgr2=0x55be90f29fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f29fd0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f7b38000c00 tx=0x7f7b38011470 comp rx=0 tx=0).stop 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 shutdown_connections 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55be90fab300 0x55be90fad720 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55be90f30240 0x55be90f2a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55be90f25980 0x55be90f29fd0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 --2- 172.21.15.37:0/1010078250 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55be90f25060 0x55be90f38a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b317fa640 0 client.18504 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 >> 172.21.15.37:0/1010078250 conn(0x55be90f215f0 msgr2=0x55be90f1ff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 shutdown_connections 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 1 -- 172.21.15.37:0/1010078250 wait complete. 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b521da740 20 client.18504 trim_cache size 0 max 16384 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 39, 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:52:48.072 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 40, 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "3aae14d0": { 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 40, 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 150.0, 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:52:48.073 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:52:48.074 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.069+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b340a33b0 msgr2=0x7f7b340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b340a33b0 0x7f7b340a5870 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f7b40049000 tx=0x7f7b4004e000 comp rx=0 tx=0).stop 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b481008d0 msgr2=0x7f7b4819a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b481008d0 0x7f7b4819a680 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f7b4400d950 tx=0x7f7b4400de10 comp rx=0 tx=0).stop 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 shutdown_connections 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b4810e480 0x7f7b4819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7b48101280 0x7f7b4819abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7b340a33b0 0x7f7b340a5870 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 --2- 172.21.15.37:0/856392369 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7b481008d0 0x7f7b4819a680 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:48.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 >> 172.21.15.37:0/856392369 conn(0x7f7b480fc7a0 msgr2=0x7f7b480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:48.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 shutdown_connections 2024-09-19T06:52:48.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:48.070+0000 7f7b4fe01640 1 -- 172.21.15.37:0/856392369 wait complete. 2024-09-19T06:52:48.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 795788 7.11 MB/sec execute 435 sec latency 28101.406 ms 2024-09-19T06:52:48.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:48 smithi096 ceph-mon[35004]: pgmap v345: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 707 KiB/s wr, 30 op/s 2024-09-19T06:52:48.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:48 smithi037 ceph-mon[31211]: pgmap v345: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 707 KiB/s wr, 30 op/s 2024-09-19T06:52:48.831 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3aae14d0', total seconds elapsed: 33.3 2024-09-19T06:52:48.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:48 smithi151 ceph-mon[35236]: pgmap v345: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 707 KiB/s wr, 30 op/s 2024-09-19T06:52:49.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 798342 7.13 MB/sec execute 436 sec latency 10.062 ms 2024-09-19T06:52:49.384 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:52:49.652 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.113+0000 7fd68f58e640 1 -- 172.21.15.37:0/3553389505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100e10 msgr2=0x7fd688104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.113+0000 7fd68f58e640 1 --2- 172.21.15.37:0/3553389505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100e10 0x7fd688104e60 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd678009860 tx=0x7fd67802a340 comp rx=0 tx=0).stop 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.114+0000 7fd68f58e640 1 -- 172.21.15.37:0/3553389505 shutdown_connections 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.114+0000 7fd68f58e640 1 --2- 172.21.15.37:0/3553389505 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688105590 0x7fd68810d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.114+0000 7fd68f58e640 1 --2- 172.21.15.37:0/3553389505 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100e10 0x7fd688104e60 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.114+0000 7fd68f58e640 1 --2- 172.21.15.37:0/3553389505 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd688100460 0x7fd688100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.114+0000 7fd68f58e640 1 -- 172.21.15.37:0/3553389505 >> 172.21.15.37:0/3553389505 conn(0x7fd6880fbee0 msgr2=0x7fd6880fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.114+0000 7fd68f58e640 1 -- 172.21.15.37:0/3553389505 shutdown_connections 2024-09-19T06:52:50.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.114+0000 7fd68f58e640 1 -- 172.21.15.37:0/3553389505 wait complete. 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 Processor -- start 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 -- start start 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100460 0x7fd688198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688100e10 0x7fd688198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd688105590 0x7fd68819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd688111f80 con 0x7fd688100e10 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd688111e00 con 0x7fd688100460 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68f58e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd688112100 con 0x7fd688105590 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100460 0x7fd688198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100460 0x7fd688198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56782/0 (socket says 172.21.15.37:56782) 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68d32a640 1 -- 172.21.15.37:0/1021976303 learned_addr learned my addr 172.21.15.37:0/1021976303 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68cb29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688100e10 0x7fd688198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68db2b640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd688105590 0x7fd68819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68d32a640 1 -- 172.21.15.37:0/1021976303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd688105590 msgr2=0x7fd68819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.115+0000 7fd68d32a640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd688105590 0x7fd68819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd68d32a640 1 -- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688100e10 msgr2=0x7fd688198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd68d32a640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688100e10 0x7fd688198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd68d32a640 1 -- 172.21.15.37:0/1021976303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68819d3e0 con 0x7fd688100460 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd68db2b640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd688105590 0x7fd68819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd68d32a640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100460 0x7fd688198430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd68400e9f0 tx=0x7fd68400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd67e7fc640 1 -- 172.21.15.37:0/1021976303 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68400cd80 con 0x7fd688100460 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd67e7fc640 1 -- 172.21.15.37:0/1021976303 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd684010910 con 0x7fd688100460 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd67e7fc640 1 -- 172.21.15.37:0/1021976303 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd684018990 con 0x7fd688100460 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68819d6d0 con 0x7fd688100460 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.116+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6881a4fb0 con 0x7fd688100460 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.119+0000 7fd67e7fc640 1 -- 172.21.15.37:0/1021976303 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd684010ac0 con 0x7fd688100460 2024-09-19T06:52:50.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.119+0000 7fd691967740 1 Processor -- start 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd691967740 1 -- start start 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd691967740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc92c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd691967740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc93b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd691967740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc93de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd691967740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5605fc8922e0 con 0x5605fc92c3a0 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd691967740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5605fc839ce0 con 0x5605fc93ba90 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd691967740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5605fc88e2a0 con 0x5605fc92ccc0 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68db2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc92c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc93b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68db2b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc92c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43100/0 (socket says 172.21.15.37:43100) 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68cb29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc93de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68db2b640 1 -- 172.21.15.37:0/3011995534 learned_addr learned my addr 172.21.15.37:0/3011995534 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd67e7fc640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd6740a3430 0x7fd6740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68db2b640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd6740a3430 0x7fd6740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68d32a640 1 -- 172.21.15.37:0/3011995534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 msgr2=0x5605fc93de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68d32a640 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc93de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68d32a640 1 -- 172.21.15.37:0/3011995534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 msgr2=0x5605fc92c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.120+0000 7fd68d32a640 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc92c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.121+0000 7fd67e7fc640 1 -- 172.21.15.37:0/1021976303 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd6840d0070 con 0x7fd688100460 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.121+0000 7fd68d32a640 1 -- 172.21.15.37:0/3011995534 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605fc81e440 con 0x5605fc92ccc0 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.121+0000 7fd68db2b640 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc92c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.121+0000 7fd68d32a640 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc93b550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd6840cd790 tx=0x7fd684018420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c981a2a395cc5c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.121+0000 7fd6737fe640 1 -- 172.21.15.37:0/3011995534 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68409c320 con 0x5605fc92ccc0 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.121+0000 7fd6737fe640 1 -- 172.21.15.37:0/3011995534 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6840a06d0 con 0x5605fc92ccc0 2024-09-19T06:52:50.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.121+0000 7fd6737fe640 1 -- 172.21.15.37:0/3011995534 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6840cdba0 con 0x5605fc92ccc0 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 -- 172.21.15.37:0/3011995534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 msgr2=0x5605fc93b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd68db2b640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd6740a3430 0x7fd6740a58f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd68000b990 tx=0x7fd680004ab0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc93b550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd6840cd790 tx=0x7fd684018420 comp rx=0 tx=0).stop 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 -- 172.21.15.37:0/3011995534 shutdown_connections 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc93de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc93b550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 --2- 172.21.15.37:0/3011995534 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc92c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 -- 172.21.15.37:0/3011995534 >> 172.21.15.37:0/3011995534 conn(0x5605fc928a30 msgr2=0x5605fc928e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 -- 172.21.15.37:0/3011995534 shutdown_connections 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 -- 172.21.15.37:0/3011995534 wait complete. 2024-09-19T06:52:50.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 -- start start 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.122+0000 7fd691967740 1 Processor -- start 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd691967740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc93b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd691967740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc938650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd691967740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc938b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd691967740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5605fc93e600 con 0x5605fc92c3a0 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd691967740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5605fc84d050 con 0x5605fc93ba90 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd691967740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5605fc93e780 con 0x5605fc92ccc0 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc93b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd691967740 10 client.0 _ll_register_callbacks cb 0x5605fc88d160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc93b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43110/0 (socket says 172.21.15.37:43110) 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd68d32a640 1 -- 172.21.15.37:0/4070877105 learned_addr learned my addr 172.21.15.37:0/4070877105 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd68cb29640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc938650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.123+0000 7fd68db2b640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc938b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd68db2b640 1 -- 172.21.15.37:0/4070877105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 msgr2=0x5605fc938650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd68db2b640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc938650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd68db2b640 1 -- 172.21.15.37:0/4070877105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 msgr2=0x5605fc93b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd68db2b640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc93b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd68db2b640 1 -- 172.21.15.37:0/4070877105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605fc6033c0 con 0x5605fc93ba90 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd68d32a640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc93b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd68db2b640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc938b90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd680009ef0 tx=0x7fd680004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c598ea6bde2c20bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd670ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd670ff9640 1 -- 172.21.15.37:0/4070877105 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd680015040 con 0x5605fc93ba90 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd670ff9640 1 -- 172.21.15.37:0/4070877105 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd680004d20 con 0x5605fc93ba90 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6840a06d0 con 0x5605fc93ba90 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd670ff9640 1 -- 172.21.15.37:0/4070877105 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd68001ea30 con 0x5605fc93ba90 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd691967740 10 client.27830 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.124+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5605fc94a020 con 0x5605fc93ba90 2024-09-19T06:52:50.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd670ff9640 1 -- 172.21.15.37:0/4070877105 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd6840a06d0 con 0x5605fc93ba90 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd670ff9640 1 -- 172.21.15.37:0/4070877105 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd680025070 con 0x5605fc93ba90 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd691967740 10 client.27830 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd691967740 10 client.27830 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd691967740 20 client.27830 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd691967740 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc94b4e0 0x5605fc94d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd691967740 4 client.27830 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5605fc89f850 con 0x5605fc94b4e0 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd68d32a640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc94b4e0 0x5605fc94d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd68d32a640 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc94b4e0 0x5605fc94d910 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eff06e3463e880ee server_cookie=2c9132eb2f07a9f1 in_seq=0 out_seq=0 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.125+0000 7fd670ff9640 10 client.27830 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd670ff9640 1 -- 172.21.15.37:0/4070877105 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5605fc89f850 con 0x5605fc94b4e0 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd670ff9640 10 client.27830 handle_command_reply: tid=0 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 client.27830 shutdown 2024-09-19T06:52:50.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 msgr2=0x5605fc938b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc938b90 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd680009ef0 tx=0x7fd680004a40 comp rx=0 tx=0).stop 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 shutdown_connections 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc94b4e0 0x5605fc94d910 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc938b90 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92ccc0 0x5605fc938650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd670ff9640 0 client.27830 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 --2- 172.21.15.37:0/4070877105 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc93b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 >> 172.21.15.37:0/4070877105 conn(0x5605fc928a30 msgr2=0x5605fc92e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 shutdown_connections 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 1 -- 172.21.15.37:0/4070877105 wait complete. 2024-09-19T06:52:50.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.127+0000 7fd691967740 20 client.27830 trim_cache size 0 max 16384 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 Processor -- start 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 -- start start 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc9478a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc92ccc0 0x5605fc933c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc93ba90 0x5605fc953230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ae0b0 con 0x5605fc92c3a0 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ab980 con 0x5605fc92ccc0 2024-09-19T06:52:50.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd691967740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881a97f0 con 0x5605fc93ba90 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.146+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc9478a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68cb29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc92ccc0 0x5605fc933c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc9478a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43112/0 (socket says 172.21.15.37:43112) 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68d32a640 1 -- 172.21.15.37:0/1464782029 learned_addr learned my addr 172.21.15.37:0/1464782029 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68db2b640 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc93ba90 0x5605fc953230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68cb29640 1 -- 172.21.15.37:0/1464782029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc93ba90 msgr2=0x5605fc953230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68cb29640 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc93ba90 0x5605fc953230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68cb29640 1 -- 172.21.15.37:0/1464782029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 msgr2=0x5605fc9478a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68cb29640 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc9478a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68cb29640 1 -- 172.21.15.37:0/1464782029 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605fc936ba0 con 0x5605fc92ccc0 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd68cb29640 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc92ccc0 0x5605fc933c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd678035040 tx=0x7fd6780056d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=783ca50cc9ff2056 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd672ffd640 1 -- 172.21.15.37:0/1464782029 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678042070 con 0x5605fc92ccc0 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd672ffd640 1 -- 172.21.15.37:0/1464782029 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678030550 con 0x5605fc92ccc0 2024-09-19T06:52:50.150 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.147+0000 7fd672ffd640 1 -- 172.21.15.37:0/1464782029 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd678037040 con 0x5605fc92ccc0 2024-09-19T06:52:50.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 -- 172.21.15.37:0/1464782029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc92ccc0 msgr2=0x5605fc933c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc92ccc0 0x5605fc933c00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd678035040 tx=0x7fd6780056d0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 -- 172.21.15.37:0/1464782029 shutdown_connections 2024-09-19T06:52:50.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc93ba90 0x5605fc953230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc92ccc0 0x5605fc933c00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 --2- 172.21.15.37:0/1464782029 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92c3a0 0x5605fc9478a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 -- 172.21.15.37:0/1464782029 >> 172.21.15.37:0/1464782029 conn(0x5605fc928a30 msgr2=0x5605fc9334f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 -- 172.21.15.37:0/1464782029 shutdown_connections 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.148+0000 7fd691967740 1 -- 172.21.15.37:0/1464782029 wait complete. 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 -- start start 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 Processor -- start 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92c3a0 0x5605fc93f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92ccc0 0x5605fc93fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc940200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881a9e40 con 0x5605fc92ccc0 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881ab2b0 con 0x5605fc93ba90 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd6881aa510 con 0x5605fc92c3a0 2024-09-19T06:52:50.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd691967740 10 client.0 _ll_register_callbacks cb 0x7fd68000b540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd68cb29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92ccc0 0x5605fc93fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd68db2b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc940200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd68cb29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92ccc0 0x5605fc93fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43120/0 (socket says 172.21.15.37:43120) 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd68d32a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92c3a0 0x5605fc93f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.149+0000 7fd68cb29640 1 -- 172.21.15.37:0/2459234541 learned_addr learned my addr 172.21.15.37:0/2459234541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd68db2b640 1 -- 172.21.15.37:0/2459234541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92c3a0 msgr2=0x5605fc93f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd68db2b640 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92c3a0 0x5605fc93f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd68db2b640 1 -- 172.21.15.37:0/2459234541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92ccc0 msgr2=0x5605fc93fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd68db2b640 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92ccc0 0x5605fc93fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd68db2b640 1 -- 172.21.15.37:0/2459234541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605fc9b5230 con 0x5605fc93ba90 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd68cb29640 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92ccc0 0x5605fc93fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd68db2b640 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc940200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd680009670 tx=0x7fd68000b470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f01fd125b14f67d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd670ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:50.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd670ff9640 1 -- 172.21.15.37:0/2459234541 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd68001c6d0 con 0x5605fc93ba90 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd670ff9640 1 -- 172.21.15.37:0/2459234541 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd68001c870 con 0x5605fc93ba90 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd678030550 con 0x5605fc93ba90 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd670ff9640 1 -- 172.21.15.37:0/2459234541 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd68001ca70 con 0x5605fc93ba90 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd691967740 10 client.27836 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5605fc9b6500 con 0x5605fc93ba90 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.150+0000 7fd670ff9640 1 -- 172.21.15.37:0/2459234541 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd678030550 con 0x5605fc93ba90 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd670ff9640 1 -- 172.21.15.37:0/2459234541 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd68003e070 con 0x5605fc93ba90 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd691967740 10 client.27836 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd691967740 10 client.27836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd691967740 20 client.27836 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd691967740 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc9b6c80 0x5605fc9b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd691967740 4 client.27836 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5605fc899d60 con 0x5605fc9b6c80 2024-09-19T06:52:50.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd68d32a640 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc9b6c80 0x5605fc9b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:50.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd670ff9640 10 client.27836 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:50.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.151+0000 7fd68d32a640 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc9b6c80 0x5605fc9b90a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a73447fb15f4f841 server_cookie=ddab93fbe0562eb6 in_seq=0 out_seq=0 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd670ff9640 1 -- 172.21.15.37:0/2459234541 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5605fc899d60 con 0x5605fc9b6c80 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd670ff9640 10 client.27836 handle_command_reply: tid=0 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 client.27836 shutdown 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 msgr2=0x5605fc940200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc940200 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fd680009670 tx=0x7fd68000b470 comp rx=0 tx=0).stop 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 shutdown_connections 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605fc9b6c80 0x5605fc9b90a0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605fc93ba90 0x5605fc940200 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605fc92ccc0 0x5605fc93fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd670ff9640 0 client.27836 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 --2- 172.21.15.37:0/2459234541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605fc92c3a0 0x5605fc93f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 >> 172.21.15.37:0/2459234541 conn(0x5605fc928a30 msgr2=0x5605fc933d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.153+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 shutdown_connections 2024-09-19T06:52:50.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd691967740 1 -- 172.21.15.37:0/2459234541 wait complete. 2024-09-19T06:52:50.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd691967740 20 client.27836 trim_cache size 0 max 16384 2024-09-19T06:52:50.159 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:50.159 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:52:50.159 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:52:50.159 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd6740a3430 msgr2=0x7fd6740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd6740a3430 0x7fd6740a58f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd68000b990 tx=0x7fd680004ab0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100460 msgr2=0x7fd688198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100460 0x7fd688198430 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fd68400e9f0 tx=0x7fd68400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 shutdown_connections 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd688105590 0x7fd68819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd6740a3430 0x7fd6740a58f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd688100e10 0x7fd688198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 --2- 172.21.15.37:0/1021976303 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd688100460 0x7fd688198430 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:50.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 >> 172.21.15.37:0/1021976303 conn(0x7fd6880fbee0 msgr2=0x7fd6880fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:50.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.154+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 shutdown_connections 2024-09-19T06:52:50.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:50.155+0000 7fd68f58e640 1 -- 172.21.15.37:0/1021976303 wait complete. 2024-09-19T06:52:50.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 799935 7.13 MB/sec execute 437 sec latency 17.409 ms 2024-09-19T06:52:50.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:50 smithi096 ceph-mon[35004]: pgmap v346: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 765 KiB/s wr, 54 op/s 2024-09-19T06:52:50.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:50 smithi037 ceph-mon[31211]: pgmap v346: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 765 KiB/s wr, 54 op/s 2024-09-19T06:52:50.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:50 smithi151 ceph-mon[35236]: pgmap v346: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 765 KiB/s wr, 54 op/s 2024-09-19T06:52:50.974 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:52:50.974 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:52:51.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 802526 7.15 MB/sec execute 438 sec latency 18.624 ms 2024-09-19T06:52:51.238 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 -- 172.21.15.37:0/1462434181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64103ff0 msgr2=0x7fbc641043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 --2- 172.21.15.37:0/1462434181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64103ff0 0x7fbc641043d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009ca0 tx=0x7fbc5402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 -- 172.21.15.37:0/1462434181 shutdown_connections 2024-09-19T06:52:51.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 --2- 172.21.15.37:0/1462434181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64109120 0x7fbc64110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 --2- 172.21.15.37:0/1462434181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc641049a0 0x7fbc641089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 --2- 172.21.15.37:0/1462434181 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc64103ff0 0x7fbc641043d0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 -- 172.21.15.37:0/1462434181 >> 172.21.15.37:0/1462434181 conn(0x7fbc640ffa90 msgr2=0x7fbc64101eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 -- 172.21.15.37:0/1462434181 shutdown_connections 2024-09-19T06:52:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 -- 172.21.15.37:0/1462434181 wait complete. 2024-09-19T06:52:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.742+0000 7fbc6941a640 1 Processor -- start 2024-09-19T06:52:51.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc6941a640 1 -- start start 2024-09-19T06:52:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc6941a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64103ff0 0x7fbc6419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc6941a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc641049a0 0x7fbc6419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc6941a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc64109120 0x7fbc6419ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc6941a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64114120 con 0x7fbc641049a0 2024-09-19T06:52:51.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc6941a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbc64113fa0 con 0x7fbc64103ff0 2024-09-19T06:52:51.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc6941a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fbc641142a0 con 0x7fbc64109120 2024-09-19T06:52:51.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc641049a0 0x7fbc6419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc62ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64103ff0 0x7fbc6419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc641049a0 0x7fbc6419aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43150/0 (socket says 172.21.15.37:43150) 2024-09-19T06:52:51.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc62ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64103ff0 0x7fbc6419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56830/0 (socket says 172.21.15.37:56830) 2024-09-19T06:52:51.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 -- 172.21.15.37:0/572520348 learned_addr learned my addr 172.21.15.37:0/572520348 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:51.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc637fe640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc64109120 0x7fbc6419ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 -- 172.21.15.37:0/572520348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc64109120 msgr2=0x7fbc6419ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc64109120 0x7fbc6419ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 -- 172.21.15.37:0/572520348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64103ff0 msgr2=0x7fbc6419a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64103ff0 0x7fbc6419a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc627fc640 1 -- 172.21.15.37:0/572520348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc6419f4f0 con 0x7fbc641049a0 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.743+0000 7fbc62ffd640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64103ff0 0x7fbc6419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.744+0000 7fbc627fc640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc641049a0 0x7fbc6419aa80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c920 tx=0x7fbc5800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.744+0000 7fbc4bfff640 1 -- 172.21.15.37:0/572520348 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58007c70 con 0x7fbc641049a0 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.744+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc6419f7e0 con 0x7fbc641049a0 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.744+0000 7fbc4bfff640 1 -- 172.21.15.37:0/572520348 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbc58007e10 con 0x7fbc641049a0 2024-09-19T06:52:51.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.744+0000 7fbc4bfff640 1 -- 172.21.15.37:0/572520348 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58018890 con 0x7fbc641049a0 2024-09-19T06:52:51.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.744+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc641a70c0 con 0x7fbc641049a0 2024-09-19T06:52:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.745+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc641043d0 con 0x7fbc641049a0 2024-09-19T06:52:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.746+0000 7fbc4bfff640 1 -- 172.21.15.37:0/572520348 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fbc58018a70 con 0x7fbc641049a0 2024-09-19T06:52:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.747+0000 7fbc4bfff640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc4c0a33b0 0x7fbc4c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:51.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.747+0000 7fbc62ffd640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc4c0a33b0 0x7fbc4c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.747+0000 7fbc4bfff640 1 -- 172.21.15.37:0/572520348 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fbc580cfdf0 con 0x7fbc641049a0 2024-09-19T06:52:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.748+0000 7fbc62ffd640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc4c0a33b0 0x7fbc4c0a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006010 tx=0x7fbc54005f30 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:51.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.752+0000 7fbc4bfff640 1 -- 172.21.15.37:0/572520348 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fbc58014030 con 0x7fbc641049a0 2024-09-19T06:52:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.996+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbc6402d7c0 con 0x7fbc641049a0 2024-09-19T06:52:51.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.997+0000 7fbc4bfff640 1 -- 172.21.15.37:0/572520348 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7fbc5809c7a0 con 0x7fbc641049a0 2024-09-19T06:52:51.998 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:52:51.999 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:52:51.999 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:52:52.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc4c0a33b0 msgr2=0x7fbc4c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:52.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc4c0a33b0 0x7fbc4c0a5870 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fbc54006010 tx=0x7fbc54005f30 comp rx=0 tx=0).stop 2024-09-19T06:52:52.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc641049a0 msgr2=0x7fbc6419aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:52.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc641049a0 0x7fbc6419aa80 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800c920 tx=0x7fbc5800cde0 comp rx=0 tx=0).stop 2024-09-19T06:52:52.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 shutdown_connections 2024-09-19T06:52:52.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbc64109120 0x7fbc6419ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:52.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fbc4c0a33b0 0x7fbc4c0a5870 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:52.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fbc641049a0 0x7fbc6419aa80 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:52.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 --2- 172.21.15.37:0/572520348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbc64103ff0 0x7fbc6419a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:52.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 >> 172.21.15.37:0/572520348 conn(0x7fbc640ffa90 msgr2=0x7fbc64100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:52.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 shutdown_connections 2024-09-19T06:52:52.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:51.999+0000 7fbc6941a640 1 -- 172.21.15.37:0/572520348 wait complete. 2024-09-19T06:52:52.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 804051 7.15 MB/sec execute 439 sec latency 24.842 ms 2024-09-19T06:52:52.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:52 smithi037 ceph-mon[31211]: pgmap v347: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 744 KiB/s wr, 39 op/s 2024-09-19T06:52:52.714 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:52 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/572520348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:52.744 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:52:52.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:52 smithi096 ceph-mon[35004]: pgmap v347: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 744 KiB/s wr, 39 op/s 2024-09-19T06:52:52.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:52 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/572520348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:52.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:52 smithi151 ceph-mon[35236]: pgmap v347: 97 pgs: 97 active+clean; 64 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 744 KiB/s wr, 39 op/s 2024-09-19T06:52:52.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:52 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/572520348' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:52:53.008 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:53.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 806194 7.16 MB/sec execute 440 sec latency 17.998 ms 2024-09-19T06:52:53.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 -- 172.21.15.37:0/1600546024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0102790 msgr2=0x7f43e01067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 --2- 172.21.15.37:0/1600546024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0102790 0x7f43e01067e0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f43d4009900 tx=0x7f43d402a410 comp rx=0 tx=0).stop 2024-09-19T06:52:53.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 -- 172.21.15.37:0/1600546024 shutdown_connections 2024-09-19T06:52:53.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 --2- 172.21.15.37:0/1600546024 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43e0106f10 0x7f43e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.460 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 --2- 172.21.15.37:0/1600546024 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0102790 0x7f43e01067e0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 --2- 172.21.15.37:0/1600546024 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43e0101de0 0x7f43e01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 -- 172.21.15.37:0/1600546024 >> 172.21.15.37:0/1600546024 conn(0x7f43e00fd8c0 msgr2=0x7f43e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 -- 172.21.15.37:0/1600546024 shutdown_connections 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.451+0000 7f43e6a39640 1 -- 172.21.15.37:0/1600546024 wait complete. 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 Processor -- start 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 -- start start 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0101de0 0x7f43e00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43e0102790 0x7f43e0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43e0106f10 0x7f43e0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0111f30 con 0x7f43e0101de0 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43e0111db0 con 0x7f43e0102790 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e6a39640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01120b0 con 0x7f43e0106f10 2024-09-19T06:52:53.461 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0101de0 0x7f43e00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0101de0 0x7f43e00744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43154/0 (socket says 172.21.15.37:43154) 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 -- 172.21.15.37:0/2951657812 learned_addr learned my addr 172.21.15.37:0/2951657812 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43e0102790 0x7f43e0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43e4fd6640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43e0106f10 0x7f43e0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 -- 172.21.15.37:0/2951657812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43e0106f10 msgr2=0x7f43e0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43e0106f10 0x7f43e0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 -- 172.21.15.37:0/2951657812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43e0102790 msgr2=0x7f43e0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43e0102790 0x7f43e0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43dffff640 1 -- 172.21.15.37:0/2951657812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e01a68f0 con 0x7f43e0101de0 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.452+0000 7f43df7fe640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43e0102790 0x7f43e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.453+0000 7f43dffff640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0101de0 0x7f43e00744f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f43d000e9f0 tx=0x7f43d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.453+0000 7f43dd7fa640 1 -- 172.21.15.37:0/2951657812 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d000cd80 con 0x7f43e0101de0 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.453+0000 7f43dd7fa640 1 -- 172.21.15.37:0/2951657812 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d0010910 con 0x7f43e0101de0 2024-09-19T06:52:53.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.453+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e01a6be0 con 0x7f43e0101de0 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.453+0000 7f43dd7fa640 1 -- 172.21.15.37:0/2951657812 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0018990 con 0x7f43e0101de0 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.453+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e01a7140 con 0x7f43e0101de0 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.455+0000 7f43dd7fa640 1 -- 172.21.15.37:0/2951657812 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f43d0010ac0 con 0x7f43e0101de0 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 Processor -- start 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 -- start start 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de74a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de7f370 0x55eb1de81760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ddda2f0 con 0x55eb1de745d0 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eb1dd81cf0 con 0x55eb1de7f370 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e8e12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb1ddd62b0 con 0x55eb1de75b40 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de74a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de7f370 0x55eb1de81760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.463 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de74a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43162/0 (socket says 172.21.15.37:43162) 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43dffff640 1 -- 172.21.15.37:0/1194082567 learned_addr learned my addr 172.21.15.37:0/1194082567 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43dd7fa640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43c80a3430 0x7f43c80a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e4fd6640 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.456+0000 7f43e4fd6640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43c80a3430 0x7f43c80a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43df7fe640 1 -- 172.21.15.37:0/1194082567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 msgr2=0x55eb1de74090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43dd7fa640 1 -- 172.21.15.37:0/2951657812 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f43d00cffe0 con 0x7f43e0101de0 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43df7fe640 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de74090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43df7fe640 1 -- 172.21.15.37:0/1194082567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 msgr2=0x55eb1de74a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43df7fe640 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de74a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43df7fe640 1 -- 172.21.15.37:0/1194082567 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1dd663a0 con 0x55eb1de7f370 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43df7fe640 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de7f370 0x55eb1de81760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005940 tx=0x7f43d4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2318d07457d844d4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43e4fd6640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43c80a3430 0x7f43c80a58f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009fd0 tx=0x7f43d8009600 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.464 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43c67fc640 1 -- 172.21.15.37:0/1194082567 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4035040 con 0x55eb1de7f370 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43c67fc640 1 -- 172.21.15.37:0/1194082567 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d40318b0 con 0x55eb1de7f370 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.457+0000 7f43c67fc640 1 -- 172.21.15.37:0/1194082567 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40049b0 con 0x55eb1de7f370 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 -- 172.21.15.37:0/1194082567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de7f370 msgr2=0x55eb1de81760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de7f370 0x55eb1de81760 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f43d4005940 tx=0x7f43d4005e80 comp rx=0 tx=0).stop 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 -- 172.21.15.37:0/1194082567 shutdown_connections 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de7f370 0x55eb1de81760 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de74a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1194082567 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de74090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 -- 172.21.15.37:0/1194082567 >> 172.21.15.37:0/1194082567 conn(0x55eb1de3d310 msgr2=0x55eb1de70c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 -- 172.21.15.37:0/1194082567 shutdown_connections 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 -- 172.21.15.37:0/1194082567 wait complete. 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.458+0000 7f43e8e12740 1 -- start start 2024-09-19T06:52:53.465 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 1 Processor -- start 2024-09-19T06:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de7ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de7f370 0x55eb1de7c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.468 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eb1de82010 con 0x55eb1de745d0 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eb1dd95060 con 0x55eb1de75b40 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb1de82190 con 0x55eb1de7f370 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e8e12740 10 client.0 _ll_register_callbacks cb 0x55eb1dae88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43dffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de7ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56876/0 (socket says 172.21.15.37:56876) 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43df7fe640 1 -- 172.21.15.37:0/1022782252 learned_addr learned my addr 172.21.15.37:0/1022782252 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.459+0000 7f43e4fd6640 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de7f370 0x55eb1de7c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43df7fe640 1 -- 172.21.15.37:0/1022782252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de7f370 msgr2=0x55eb1de7c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43df7fe640 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de7f370 0x55eb1de7c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43df7fe640 1 -- 172.21.15.37:0/1022782252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 msgr2=0x55eb1de7ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43df7fe640 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de7ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.469 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43df7fe640 1 -- 172.21.15.37:0/1022782252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1db4b3c0 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43dffff640 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de7ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43df7fe640 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7bf90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f43d4004260 tx=0x7f43d4004290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8353cc2457b1c0a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43bf7fe640 1 -- 172.21.15.37:0/1022782252 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4054060 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43bf7fe640 1 -- 172.21.15.37:0/1022782252 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f43d40316b0 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d40318b0 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43bf7fe640 1 -- 172.21.15.37:0/1022782252 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f43d4035070 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43e8e12740 10 client.27854 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.460+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eb1de8da20 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43bf7fe640 1 -- 172.21.15.37:0/1022782252 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d40318b0 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43bf7fe640 1 -- 172.21.15.37:0/1022782252 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f43d4046570 con 0x55eb1de75b40 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43e8e12740 10 client.27854 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43e8e12740 10 client.27854 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:52:53.470 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43e8e12740 20 client.27854 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1de8ed70 0x55eb1de91160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43e8e12740 4 client.27854 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb1dde7860 con 0x55eb1de8ed70 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43dffff640 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1de8ed70 0x55eb1de91160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43dffff640 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1de8ed70 0x55eb1de91160 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea7b53e0e39624b2 server_cookie=334705742a54316 in_seq=0 out_seq=0 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.461+0000 7f43bf7fe640 10 client.27854 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43bf7fe640 1 -- 172.21.15.37:0/1022782252 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb1dde7860 con 0x55eb1de8ed70 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43bf7fe640 10 client.27854 handle_command_reply: tid=0 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 client.27854 shutdown 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 msgr2=0x55eb1de7bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7bf90 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f43d4004260 tx=0x7f43d4004290 comp rx=0 tx=0).stop 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 shutdown_connections 2024-09-19T06:52:53.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1de8ed70 0x55eb1de91160 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de7f370 0x55eb1de7c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7bf90 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 --2- 172.21.15.37:0/1022782252 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de745d0 0x55eb1de7ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.462+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 >> 172.21.15.37:0/1022782252 conn(0x55eb1de3d310 msgr2=0x55eb1de76960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.463+0000 7f43bf7fe640 0 client.27854 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.463+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 shutdown_connections 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.463+0000 7f43e8e12740 1 -- 172.21.15.37:0/1022782252 wait complete. 2024-09-19T06:52:53.472 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.463+0000 7f43e8e12740 20 client.27854 trim_cache size 0 max 16384 2024-09-19T06:52:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 Processor -- start 2024-09-19T06:52:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 -- start start 2024-09-19T06:52:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de745d0 0x55eb1de8b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de967f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01b23c0 con 0x55eb1de7f370 2024-09-19T06:52:53.479 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01afc90 con 0x55eb1de75b40 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e8e12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01adb00 con 0x55eb1de745d0 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43e4fd6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de967f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7df00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56882/0 (socket says 172.21.15.37:56882) 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43dffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de745d0 0x55eb1de8b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 -- 172.21.15.37:0/4292469895 learned_addr learned my addr 172.21.15.37:0/4292469895 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 -- 172.21.15.37:0/4292469895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de745d0 msgr2=0x55eb1de8b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 --2- 172.21.15.37:0/4292469895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de745d0 0x55eb1de8b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 -- 172.21.15.37:0/4292469895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 msgr2=0x55eb1de967f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 --2- 172.21.15.37:0/4292469895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de967f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 -- 172.21.15.37:0/4292469895 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb1de7e890 con 0x55eb1de75b40 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.477+0000 7f43df7fe640 1 --2- 172.21.15.37:0/4292469895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7df00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000c00 tx=0x7f43d4005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecff7ee33fcc9bda server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.480 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43c5ffb640 1 -- 172.21.15.37:0/4292469895 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4054070 con 0x55eb1de75b40 2024-09-19T06:52:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43c5ffb640 1 -- 172.21.15.37:0/4292469895 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4005550 con 0x55eb1de75b40 2024-09-19T06:52:53.481 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43c5ffb640 1 -- 172.21.15.37:0/4292469895 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4031630 con 0x55eb1de75b40 2024-09-19T06:52:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 -- 172.21.15.37:0/4292469895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 msgr2=0x55eb1de7df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 --2- 172.21.15.37:0/4292469895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7df00 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f43d4000c00 tx=0x7f43d4005e80 comp rx=0 tx=0).stop 2024-09-19T06:52:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 -- 172.21.15.37:0/4292469895 shutdown_connections 2024-09-19T06:52:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 --2- 172.21.15.37:0/4292469895 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de967f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 --2- 172.21.15.37:0/4292469895 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de75b40 0x55eb1de7df00 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 --2- 172.21.15.37:0/4292469895 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de745d0 0x55eb1de8b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 -- 172.21.15.37:0/4292469895 >> 172.21.15.37:0/4292469895 conn(0x55eb1de3d310 msgr2=0x55eb1de7d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 -- 172.21.15.37:0/4292469895 shutdown_connections 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.478+0000 7f43e8e12740 1 -- 172.21.15.37:0/4292469895 wait complete. 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 -- start start 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 Processor -- start 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de745d0 0x55eb1de78280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de787c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de78d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01ae150 con 0x55eb1de7f370 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01af5c0 con 0x55eb1de745d0 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f43e01ae820 con 0x55eb1de75b40 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e8e12740 10 client.0 _ll_register_callbacks cb 0x7f43d4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:53.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de745d0 0x55eb1de78280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43e4fd6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de78d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43dffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de745d0 0x55eb1de78280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56894/0 (socket says 172.21.15.37:56894) 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43dffff640 1 -- 172.21.15.37:0/3773428301 learned_addr learned my addr 172.21.15.37:0/3773428301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.479+0000 7f43df7fe640 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de787c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43dffff640 1 -- 172.21.15.37:0/3773428301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 msgr2=0x55eb1de787c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43dffff640 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de787c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43dffff640 1 -- 172.21.15.37:0/3773428301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 msgr2=0x55eb1de78d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43dffff640 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de78d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43dffff640 1 -- 172.21.15.37:0/3773428301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb1def78b0 con 0x55eb1de745d0 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43e4fd6640 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43dffff640 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de745d0 0x55eb1de78280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c6a0 tx=0x7f43d0098ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74c21fe5967168d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43bf7fe640 1 -- 172.21.15.37:0/3773428301 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0026040 con 0x55eb1de745d0 2024-09-19T06:52:53.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43bf7fe640 1 -- 172.21.15.37:0/3773428301 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f43d00bc0b0 con 0x55eb1de745d0 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d4005550 con 0x55eb1de745d0 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43bf7fe640 1 -- 172.21.15.37:0/3773428301 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f43d009c320 con 0x55eb1de745d0 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43e8e12740 10 client.27860 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.480+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55eb1def8bd0 con 0x55eb1de745d0 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43bf7fe640 1 -- 172.21.15.37:0/3773428301 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f43d4005550 con 0x55eb1de745d0 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43bf7fe640 1 -- 172.21.15.37:0/3773428301 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f43d00b0290 con 0x55eb1de745d0 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43e8e12740 10 client.27860 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43e8e12740 10 client.27860 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43e8e12740 20 client.27860 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43e8e12740 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1def92a0 0x55eb1defb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43e8e12740 4 client.27860 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb1dde1d70 con 0x55eb1def92a0 2024-09-19T06:52:53.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43df7fe640 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1def92a0 0x55eb1defb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43df7fe640 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1def92a0 0x55eb1defb6c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e2d45406b863582f server_cookie=13e671b92ccee08a in_seq=0 out_seq=0 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.481+0000 7f43bf7fe640 10 client.27860 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43bf7fe640 1 -- 172.21.15.37:0/3773428301 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb1dde1d70 con 0x55eb1def92a0 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43bf7fe640 10 client.27860 handle_command_reply: tid=0 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 client.27860 shutdown 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de745d0 msgr2=0x55eb1de78280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de745d0 0x55eb1de78280 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f43d000c6a0 tx=0x7f43d0098ac0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 shutdown_connections 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55eb1def92a0 0x55eb1defb6c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb1de7f370 0x55eb1de78d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43bf7fe640 0 client.27860 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb1de75b40 0x55eb1de787c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 --2- 172.21.15.37:0/3773428301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb1de745d0 0x55eb1de78280 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 >> 172.21.15.37:0/3773428301 conn(0x55eb1de3d310 msgr2=0x55eb1de7e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 shutdown_connections 2024-09-19T06:52:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 1 -- 172.21.15.37:0/3773428301 wait complete. 2024-09-19T06:52:53.490 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.482+0000 7f43e8e12740 20 client.27860 trim_cache size 0 max 16384 2024-09-19T06:52:53.490 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43c80a3430 msgr2=0x7f43c80a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43c80a3430 0x7f43c80a58f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f43d8009fd0 tx=0x7f43d8009600 comp rx=0 tx=0).stop 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0101de0 msgr2=0x7f43e00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0101de0 0x7f43e00744f0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f43d000e9f0 tx=0x7f43d000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 shutdown_connections 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f43e0106f10 0x7f43e0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43e0102790 0x7f43e0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43c80a3430 0x7f43c80a58f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 --2- 172.21.15.37:0/2951657812 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f43e0101de0 0x7f43e00744f0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 >> 172.21.15.37:0/2951657812 conn(0x7f43e00fd8c0 msgr2=0x7f43e00fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:53.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 shutdown_connections 2024-09-19T06:52:53.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:53.483+0000 7f43e6a39640 1 -- 172.21.15.37:0/2951657812 wait complete. 2024-09-19T06:52:54.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 808290 7.16 MB/sec execute 441 sec latency 21.598 ms 2024-09-19T06:52:54.360 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:52:54.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:54 smithi037 ceph-mon[31211]: pgmap v348: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-09-19T06:52:54.623 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:54.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:54 smithi096 ceph-mon[35004]: pgmap v348: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-09-19T06:52:54.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:54 smithi151 ceph-mon[35236]: pgmap v348: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1.5 MiB/s wr, 114 op/s 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 -- 172.21.15.37:0/2608560422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8100e20 msgr2=0x7f0da8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 --2- 172.21.15.37:0/2608560422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8100e20 0x7f0da8101200 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009900 tx=0x7f0d9802a480 comp rx=0 tx=0).stop 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 -- 172.21.15.37:0/2608560422 shutdown_connections 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 --2- 172.21.15.37:0/2608560422 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0da8102ef0 0x7f0da810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 --2- 172.21.15.37:0/2608560422 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0da8102570 0x7f0da81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 --2- 172.21.15.37:0/2608560422 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8100e20 0x7f0da8101200 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 -- 172.21.15.37:0/2608560422 >> 172.21.15.37:0/2608560422 conn(0x7f0da80fc7d0 msgr2=0x7f0da80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.096+0000 7f0dacae7640 1 -- 172.21.15.37:0/2608560422 shutdown_connections 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.097+0000 7f0dacae7640 1 -- 172.21.15.37:0/2608560422 wait complete. 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.097+0000 7f0dacae7640 1 Processor -- start 2024-09-19T06:52:55.113 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.097+0000 7f0dacae7640 1 -- start start 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0dacae7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0da8100e20 0x7f0da8197410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0dacae7640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0da8102570 0x7f0da8197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0dacae7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8102ef0 0x7f0da819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0dacae7640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8110f80 con 0x7f0da8102ef0 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0dacae7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8110e00 con 0x7f0da8100e20 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0dacae7640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0da8111100 con 0x7f0da8102570 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0da6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8102ef0 0x7f0da819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0da8102570 0x7f0da8197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0da6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8102ef0 0x7f0da819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43202/0 (socket says 172.21.15.37:43202) 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0da6d9d640 1 -- 172.21.15.37:0/1329960674 learned_addr learned my addr 172.21.15.37:0/1329960674 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0da659c640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0da8100e20 0x7f0da8197410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9ffff640 1 -- 172.21.15.37:0/1329960674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0da8100e20 msgr2=0x7f0da8197410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0da8100e20 0x7f0da8197410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.114 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9ffff640 1 -- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8102ef0 msgr2=0x7f0da819bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8102ef0 0x7f0da819bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9ffff640 1 -- 172.21.15.37:0/1329960674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da819c3c0 con 0x7f0da8102570 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8102ef0 0x7f0da819bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0da8102570 0x7f0da8197950 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000e9f0 tx=0x7f0d9000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0d9f7fe640 1 -- 172.21.15.37:0/1329960674 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9000ccc0 con 0x7f0da8102570 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.098+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da819c6b0 con 0x7f0da8102570 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.099+0000 7f0d9f7fe640 1 -- 172.21.15.37:0/1329960674 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d90005020 con 0x7f0da8102570 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.099+0000 7f0d9f7fe640 1 -- 172.21.15.37:0/1329960674 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900049e0 con 0x7f0da8102570 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.099+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da81a3f90 con 0x7f0da8102570 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 Processor -- start 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0d9f7fe640 1 -- 172.21.15.37:0/1329960674 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0d90005560 con 0x7f0da8102570 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 -- start start 2024-09-19T06:52:55.115 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901df5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901df55d0 0x559901df5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901e02580 0x559901e04970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559901d5b2f0 con 0x559901e02580 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559901d02cf0 con 0x559901df55d0 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0daeec0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559901d572b0 con 0x559901df6b40 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0da6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901df5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0da659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901df55d0 0x559901df5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901e02580 0x559901e04970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.116 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0da6d9d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901df5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:33114/0 (socket says 172.21.15.37:33114) 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.102+0000 7f0da6d9d640 1 -- 172.21.15.37:0/2694424781 learned_addr learned my addr 172.21.15.37:0/2694424781 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0d9f7fe640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d940a33b0 0x7f0d940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0da6d9d640 1 -- 172.21.15.37:0/2694424781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901df55d0 msgr2=0x559901df5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/2694424781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901df55d0 0x559901df5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0da6d9d640 1 -- 172.21.15.37:0/2694424781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901e02580 msgr2=0x559901e04970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/2694424781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901e02580 0x559901e04970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0da6d9d640 1 -- 172.21.15.37:0/2694424781 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559901ce73a0 con 0x559901df6b40 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0d9f7fe640 1 -- 172.21.15.37:0/1329960674 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f0d900cfcc0 con 0x7f0da8102570 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d940a33b0 0x7f0d940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/2694424781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901df5090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0da00118b0 tx=0x7f0da0011d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e31a0bd1f9bdd17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0d9cff9640 1 -- 172.21.15.37:0/2694424781 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da000fca0 con 0x559901df6b40 2024-09-19T06:52:55.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0d9cff9640 1 -- 172.21.15.37:0/2694424781 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da001bde0 con 0x559901df6b40 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.103+0000 7f0d9cff9640 1 -- 172.21.15.37:0/2694424781 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0da000e610 con 0x559901df6b40 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 -- 172.21.15.37:0/2694424781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 msgr2=0x559901df5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d940a33b0 0x7f0d940a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0da0012fd0 tx=0x7f0da0021000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2694424781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901df5090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0da00118b0 tx=0x7f0da0011d70 comp rx=0 tx=0).stop 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 -- 172.21.15.37:0/2694424781 shutdown_connections 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2694424781 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901e02580 0x559901e04970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2694424781 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901df55d0 0x559901df5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2694424781 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901df5090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 -- 172.21.15.37:0/2694424781 >> 172.21.15.37:0/2694424781 conn(0x559901dbe310 msgr2=0x559901df1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.118 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 -- 172.21.15.37:0/2694424781 shutdown_connections 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 -- 172.21.15.37:0/2694424781 wait complete. 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.104+0000 7f0daeec0740 1 -- start start 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 1 Processor -- start 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df55d0 0x559901e01e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901dff0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901dff630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559901e05280 con 0x559901df55d0 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559901d16060 con 0x559901e02580 2024-09-19T06:52:55.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559901e05400 con 0x559901df6b40 2024-09-19T06:52:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0daeec0740 10 client.0 _ll_register_callbacks cb 0x559901a698c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df55d0 0x559901e01e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df55d0 0x559901e01e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43220/0 (socket says 172.21.15.37:43220) 2024-09-19T06:52:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901dff0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 -- 172.21.15.37:0/2623652977 learned_addr learned my addr 172.21.15.37:0/2623652977 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901dff630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 -- 172.21.15.37:0/2623652977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 msgr2=0x559901dff0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901dff0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 -- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 msgr2=0x559901dff630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901dff630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 -- 172.21.15.37:0/2623652977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559901acc3c0 con 0x559901df55d0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0da659c640 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df55d0 0x559901e01e40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d980043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80315b3bcff620bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.105+0000 7f0d8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0d8dffb640 1 -- 172.21.15.37:0/2623652977 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d98035040 con 0x559901df55d0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0d8dffb640 1 -- 172.21.15.37:0/2623652977 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f0d98003a00 con 0x559901df55d0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0da001bde0 con 0x559901df55d0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0d8dffb640 1 -- 172.21.15.37:0/2623652977 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d98003c00 con 0x559901df55d0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 10 client.18558 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559901e10c10 con 0x559901df55d0 2024-09-19T06:52:55.121 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0d8dffb640 1 -- 172.21.15.37:0/2623652977 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0da001bde0 con 0x559901df55d0 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0d8dffb640 1 -- 172.21.15.37:0/2623652977 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f0d9803c4d0 con 0x559901df55d0 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 10 client.18558 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 10 client.18558 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 20 client.18558 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e120d0 0x559901e14560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 4 client.18558 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.106+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559901d68860 con 0x559901e120d0 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.107+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e120d0 0x559901e14560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.107+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e120d0 0x559901e14560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a34a163e1cd1f07 server_cookie=67b7140baa2f6074 in_seq=0 out_seq=0 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.107+0000 7f0d8dffb640 10 client.18558 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.107+0000 7f0d8dffb640 1 -- 172.21.15.37:0/2623652977 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559901d68860 con 0x559901e120d0 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.107+0000 7f0d8dffb640 10 client.18558 handle_command_reply: tid=0 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 client.18558 shutdown 2024-09-19T06:52:55.122 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df55d0 msgr2=0x559901e01e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df55d0 0x559901e01e40 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f0d98000c00 tx=0x7f0d980043b0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 shutdown_connections 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e120d0 0x559901e14560 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901dff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df6b40 0x559901dff0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0d8dffb640 0 client.18558 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 --2- 172.21.15.37:0/2623652977 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df55d0 0x559901e01e40 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 >> 172.21.15.37:0/2623652977 conn(0x559901dbe310 msgr2=0x559901df7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 shutdown_connections 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 1 -- 172.21.15.37:0/2623652977 wait complete. 2024-09-19T06:52:55.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.108+0000 7f0daeec0740 20 client.18558 trim_cache size 0 max 16384 2024-09-19T06:52:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 Processor -- start 2024-09-19T06:52:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 -- start start 2024-09-19T06:52:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901e0e520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901dfe980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901e0aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81adfc0 con 0x559901df6b40 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81ab890 con 0x559901e02580 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0daeec0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81a9700 con 0x559901df55d0 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901dfe980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901dfe980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43228/0 (socket says 172.21.15.37:43228) 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 -- 172.21.15.37:0/1031675137 learned_addr learned my addr 172.21.15.37:0/1031675137 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0da6d9d640 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901e0aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0da659c640 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901e0e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 -- 172.21.15.37:0/1031675137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 msgr2=0x559901e0e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901e0e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 -- 172.21.15.37:0/1031675137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 msgr2=0x559901e0aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901e0aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.122+0000 7f0d9ffff640 1 -- 172.21.15.37:0/1031675137 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559901dfeec0 con 0x559901df6b40 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901dfe980 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0d900cd440 tx=0x7f0d900d21d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b8c9981894bb72 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0d8ffff640 1 -- 172.21.15.37:0/1031675137 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900d1040 con 0x559901df6b40 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0d8ffff640 1 -- 172.21.15.37:0/1031675137 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d900051d0 con 0x559901df6b40 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0d8ffff640 1 -- 172.21.15.37:0/1031675137 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9001c040 con 0x559901df6b40 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0daeec0740 1 -- 172.21.15.37:0/1031675137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 msgr2=0x559901dfe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0daeec0740 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901dfe980 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f0d900cd440 tx=0x7f0d900d21d0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0daeec0740 1 -- 172.21.15.37:0/1031675137 shutdown_connections 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0daeec0740 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901e0aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0daeec0740 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901dfe980 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0daeec0740 1 --2- 172.21.15.37:0/1031675137 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901e0e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.123+0000 7f0daeec0740 1 -- 172.21.15.37:0/1031675137 >> 172.21.15.37:0/1031675137 conn(0x559901dbe310 msgr2=0x559901dfd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 -- 172.21.15.37:0/1031675137 shutdown_connections 2024-09-19T06:52:55.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 -- 172.21.15.37:0/1031675137 wait complete. 2024-09-19T06:52:55.128 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 -- start start 2024-09-19T06:52:55.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 Processor -- start 2024-09-19T06:52:55.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901df8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901df9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901df96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81a9d50 con 0x559901df6b40 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81ab1c0 con 0x559901e02580 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0da81aa420 con 0x559901df55d0 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0daeec0740 10 client.0 _ll_register_callbacks cb 0x7f0d98004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901df9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0da659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901df8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0da6d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901df96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0d9ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901df9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43234/0 (socket says 172.21.15.37:43234) 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.124+0000 7f0d9ffff640 1 -- 172.21.15.37:0/3731117169 learned_addr learned my addr 172.21.15.37:0/3731117169 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:55.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d9ffff640 1 -- 172.21.15.37:0/3731117169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 msgr2=0x559901df8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901df8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d9ffff640 1 -- 172.21.15.37:0/3731117169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 msgr2=0x559901df96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901df96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d9ffff640 1 -- 172.21.15.37:0/3731117169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559901e7de70 con 0x559901df6b40 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d9ffff640 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901df9170 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0d900cd490 tx=0x7f0d9009bbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7147c730f9af639b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d8dffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d8dffb640 1 -- 172.21.15.37:0/3731117169 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d900b6070 con 0x559901df6b40 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d8dffb640 1 -- 172.21.15.37:0/3731117169 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f0d900985e0 con 0x559901df6b40 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d900051d0 con 0x559901df6b40 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d8dffb640 1 -- 172.21.15.37:0/3731117169 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f0d900d1070 con 0x559901df6b40 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0daeec0740 10 client.18564 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:55.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559901e7f130 con 0x559901df6b40 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.125+0000 7f0d8dffb640 1 -- 172.21.15.37:0/3731117169 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d900051d0 con 0x559901df6b40 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0d8dffb640 1 -- 172.21.15.37:0/3731117169 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f0d90004c20 con 0x559901df6b40 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 10 client.18564 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 10 client.18564 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 20 client.18564 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e7f870 0x559901e81c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 4 client.18564 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559901d62d70 con 0x559901e7f870 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0da659c640 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e7f870 0x559901e81c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:55.134 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0d8dffb640 10 client.18564 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0da659c640 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e7f870 0x559901e81c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a8d28d0a8b2855e server_cookie=50098ab304bd0970 in_seq=0 out_seq=0 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0d8dffb640 1 -- 172.21.15.37:0/3731117169 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559901d62d70 con 0x559901e7f870 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0d8dffb640 10 client.18564 handle_command_reply: tid=0 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 1 client.18564 shutdown 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 msgr2=0x559901df9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.126+0000 7f0daeec0740 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901df9170 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f0d900cd490 tx=0x7f0d9009bbb0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 shutdown_connections 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559901e7f870 0x559901e81c90 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559901e02580 0x559901df96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559901df6b40 0x559901df9170 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 --2- 172.21.15.37:0/3731117169 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559901df55d0 0x559901df8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 >> 172.21.15.37:0/3731117169 conn(0x559901dbe310 msgr2=0x559901df0500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0d8dffb640 0 client.18564 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:52:55.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 shutdown_connections 2024-09-19T06:52:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 1 -- 172.21.15.37:0/3731117169 wait complete. 2024-09-19T06:52:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0daeec0740 20 client.18564 trim_cache size 0 max 16384 2024-09-19T06:52:55.136 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:52:55.136 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d940a33b0 msgr2=0x7f0d940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d940a33b0 0x7f0d940a5870 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0da0012fd0 tx=0x7f0da0021000 comp rx=0 tx=0).stop 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0da8102570 msgr2=0x7f0da8197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0da8102570 0x7f0da8197950 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0d9000e9f0 tx=0x7f0d9000eeb0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 shutdown_connections 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0d940a33b0 0x7f0d940a5870 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0da8102ef0 0x7f0da819bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0da8102570 0x7f0da8197950 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 --2- 172.21.15.37:0/1329960674 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0da8100e20 0x7f0da8197410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:55.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.127+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 >> 172.21.15.37:0/1329960674 conn(0x7f0da80fc7d0 msgr2=0x7f0da80fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:55.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.128+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 shutdown_connections 2024-09-19T06:52:55.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:55.128+0000 7f0dacae7640 1 -- 172.21.15.37:0/1329960674 wait complete. 2024-09-19T06:52:55.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 810056 7.17 MB/sec execute 442 sec latency 29.476 ms 2024-09-19T06:52:55.862 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:52:56.119 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:56.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 812864 7.19 MB/sec execute 443 sec latency 19.093 ms 2024-09-19T06:52:56.492 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:56 smithi037 ceph-mon[31211]: pgmap v349: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 98 op/s 2024-09-19T06:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 -- 172.21.15.37:0/231611401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c4103660 msgr2=0x7fd9c410bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 --2- 172.21.15.37:0/231611401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c4103660 0x7fd9c410bb50 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8009900 tx=0x7fd9b802a480 comp rx=0 tx=0).stop 2024-09-19T06:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 -- 172.21.15.37:0/231611401 shutdown_connections 2024-09-19T06:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 --2- 172.21.15.37:0/231611401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c410c170 0x7fd9c410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 --2- 172.21.15.37:0/231611401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c4103660 0x7fd9c410bb50 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 --2- 172.21.15.37:0/231611401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c4102d40 0x7fd9c4103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 -- 172.21.15.37:0/231611401 >> 172.21.15.37:0/231611401 conn(0x7fd9c40fc7a0 msgr2=0x7fd9c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 -- 172.21.15.37:0/231611401 shutdown_connections 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.574+0000 7fd9ca036640 1 -- 172.21.15.37:0/231611401 wait complete. 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 Processor -- start 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 -- start start 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c4102d40 0x7fd9c419a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c4103660 0x7fd9c419aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c410c170 0x7fd9c419ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c4111f70 con 0x7fd9c410c170 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c4111df0 con 0x7fd9c4103660 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9ca036640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41120f0 con 0x7fd9c4102d40 2024-09-19T06:52:56.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.575+0000 7fd9c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c410c170 0x7fd9c419ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c4103660 0x7fd9c419aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c410c170 0x7fd9c419ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43258/0 (socket says 172.21.15.37:43258) 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 -- 172.21.15.37:0/1429430441 learned_addr learned my addr 172.21.15.37:0/1429430441 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c4103660 0x7fd9c419aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:56968/0 (socket says 172.21.15.37:56968) 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c4102d40 0x7fd9c419a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 -- 172.21.15.37:0/1429430441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c4102d40 msgr2=0x7fd9c419a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c4102d40 0x7fd9c419a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 -- 172.21.15.37:0/1429430441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c4103660 msgr2=0x7fd9c419aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c4103660 0x7fd9c419aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 -- 172.21.15.37:0/1429430441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c419f610 con 0x7fd9c410c170 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c4103660 0x7fd9c419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c4102d40 0x7fd9c419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c3fff640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c410c170 0x7fd9c419ef30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00eb10 tx=0x7fd9bc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.599 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c0ff9640 1 -- 172.21.15.37:0/1429430441 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc004b20 con 0x7fd9c410c170 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c419f900 con 0x7fd9c410c170 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c0ff9640 1 -- 172.21.15.37:0/1429430441 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9bc004cc0 con 0x7fd9c410c170 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9c0ff9640 1 -- 172.21.15.37:0/1429430441 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc010760 con 0x7fd9c410c170 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.576+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c41a71e0 con 0x7fd9c410c170 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9c0ff9640 1 -- 172.21.15.37:0/1429430441 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd9bc005020 con 0x7fd9c410c170 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 Processor -- start 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 -- start start 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04abb40 0x55e1e04aa090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.600 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04aaa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04b5370 0x55e1e04b7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e1e04102f0 con 0x55e1e04b5370 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e1e03b7cf0 con 0x55e1e04aa5d0 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e1e040c2b0 con 0x55e1e04abb40 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04b5370 0x55e1e04b7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04aaa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04b5370 0x55e1e04b7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43262/0 (socket says 172.21.15.37:43262) 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.579+0000 7fd9c2ffd640 1 -- 172.21.15.37:0/1121281222 learned_addr learned my addr 172.21.15.37:0/1121281222 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c0ff9640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04abb40 0x55e1e04aa090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c3fff640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1121281222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04abb40 msgr2=0x55e1e04aa090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.601 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04abb40 0x55e1e04aa090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1121281222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04b5370 msgr2=0x55e1e04b7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04b5370 0x55e1e04b7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1121281222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e039c3a0 con 0x55e1e04aa5d0 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c0ff9640 1 -- 172.21.15.37:0/1429430441 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd9bc0d0240 con 0x7fd9c410c170 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04aaa10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009fd0 tx=0x7fd9b40118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7727adfe3f834989 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9c3fff640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0cdf60 tx=0x7fd9bc067380 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9a9ffb640 1 -- 172.21.15.37:0/1121281222 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4004310 con 0x55e1e04aa5d0 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9a9ffb640 1 -- 172.21.15.37:0/1121281222 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b40044b0 con 0x55e1e04aa5d0 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.580+0000 7fd9a9ffb640 1 -- 172.21.15.37:0/1121281222 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b40035f0 con 0x55e1e04aa5d0 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1121281222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 msgr2=0x55e1e04aaa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04aaa10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009fd0 tx=0x7fd9b40118d0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1121281222 shutdown_connections 2024-09-19T06:52:56.602 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04b5370 0x55e1e04b7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04aaa10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1121281222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04abb40 0x55e1e04aa090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1121281222 >> 172.21.15.37:0/1121281222 conn(0x55e1e0473310 msgr2=0x55e1e04a6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1121281222 shutdown_connections 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.581+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1121281222 wait complete. 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 -- start start 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 Processor -- start 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04aa5d0 0x55e1e04b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04b1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04b5370 0x55e1e04b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.603 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e1e04b8070 con 0x55e1e04abb40 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e1e03cb060 con 0x55e1e04b5370 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e1e04b81f0 con 0x55e1e04aa5d0 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9cc40f740 10 client.0 _ll_register_callbacks cb 0x55e1e011e8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04b1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04b1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43274/0 (socket says 172.21.15.37:43274) 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04b5370 0x55e1e04b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1522334060 learned_addr learned my addr 172.21.15.37:0/1522334060 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:56.604 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.582+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04aa5d0 0x55e1e04b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9c2ffd640 1 -- 172.21.15.37:0/1522334060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04aa5d0 msgr2=0x55e1e04b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04aa5d0 0x55e1e04b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9c2ffd640 1 -- 172.21.15.37:0/1522334060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 msgr2=0x55e1e04b1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04b1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9c2ffd640 1 -- 172.21.15.37:0/1522334060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e01813c0 con 0x55e1e04b5370 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04b5370 0x55e1e04b25e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55e1e04ab070 tx=0x7fd9b802a480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b963496d654d699 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1522334060 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8035040 con 0x55e1e04b5370 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1522334060 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd9b80055c0 con 0x55e1e04b5370 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b40044b0 con 0x55e1e04b5370 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1522334060 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd9b8054050 con 0x55e1e04b5370 2024-09-19T06:52:56.605 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9cc40f740 10 client.27890 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e1e04c3ae0 con 0x55e1e04b5370 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1522334060 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b40044b0 con 0x55e1e04b5370 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.583+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1522334060 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd9b804b070 con 0x55e1e04b5370 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9cc40f740 10 client.27890 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9cc40f740 10 client.27890 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9cc40f740 20 client.27890 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e04c4ef0 0x55e1e04c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9cc40f740 4 client.27890 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1e041d860 con 0x55e1e04c4ef0 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e04c4ef0 0x55e1e04c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e04c4ef0 0x55e1e04c72e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14aaece9f0a4907e server_cookie=c6381f98046745c0 in_seq=0 out_seq=0 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.584+0000 7fd9a2ffd640 10 client.27890 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1522334060 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e1e041d860 con 0x55e1e04c4ef0 2024-09-19T06:52:56.606 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9a2ffd640 10 client.27890 handle_command_reply: tid=0 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 client.27890 shutdown 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04b5370 msgr2=0x55e1e04b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04b5370 0x55e1e04b25e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x55e1e04ab070 tx=0x7fd9b802a480 comp rx=0 tx=0).stop 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 shutdown_connections 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e04c4ef0 0x55e1e04c72e0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04b5370 0x55e1e04b25e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04b1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1522334060 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04aa5d0 0x55e1e04b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9a2ffd640 0 client.27890 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 >> 172.21.15.37:0/1522334060 conn(0x55e1e0473310 msgr2=0x55e1e04ac960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 shutdown_connections 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1522334060 wait complete. 2024-09-19T06:52:56.607 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.585+0000 7fd9cc40f740 20 client.27890 trim_cache size 0 max 16384 2024-09-19T06:52:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 Processor -- start 2024-09-19T06:52:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 -- start start 2024-09-19T06:52:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04c1310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.613 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04ad120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04ad660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41b2400 con 0x55e1e04abb40 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41afcd0 con 0x55e1e04aa5d0 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41adb40 con 0x55e1e04b5370 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04c1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43288/0 (socket says 172.21.15.37:43288) 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.599+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1300378212 learned_addr learned my addr 172.21.15.37:0/1300378212 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04ad660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.614 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1300378212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 msgr2=0x55e1e04ad660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04ad660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1300378212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 msgr2=0x55e1e04c1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04c1310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1300378212 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e04c7910 con 0x55e1e04abb40 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04ad120 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009380 tx=0x7fd9b4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab4e47589325a5fd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9a97fa640 1 -- 172.21.15.37:0/1300378212 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4004420 con 0x55e1e04abb40 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9a97fa640 1 -- 172.21.15.37:0/1300378212 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b4003ad0 con 0x55e1e04abb40 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.600+0000 7fd9a97fa640 1 -- 172.21.15.37:0/1300378212 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b402e8e0 con 0x55e1e04abb40 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1300378212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 msgr2=0x55e1e04ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04ad120 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4009380 tx=0x7fd9b4005f90 comp rx=0 tx=0).stop 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1300378212 shutdown_connections 2024-09-19T06:52:56.615 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04ad660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04ad120 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1300378212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04c1310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1300378212 >> 172.21.15.37:0/1300378212 conn(0x55e1e0473310 msgr2=0x55e1e04b4070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1300378212 shutdown_connections 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1300378212 wait complete. 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.601+0000 7fd9cc40f740 1 -- start start 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 1 Processor -- start 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04aea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41ae190 con 0x55e1e04abb40 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41af600 con 0x55e1e04aa5d0 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41ae860 con 0x55e1e04b5370 2024-09-19T06:52:56.616 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9cc40f740 10 client.0 _ll_register_callbacks cb 0x7fd9b8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04bdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43290/0 (socket says 172.21.15.37:43290) 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c2ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04bd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57010/0 (socket says 172.21.15.37:57010) 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1865831419 learned_addr learned my addr 172.21.15.37:0/1865831419 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04aea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1865831419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 msgr2=0x55e1e04aea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04aea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1865831419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 msgr2=0x55e1e04bd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04bd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.602+0000 7fd9c37fe640 1 -- 172.21.15.37:0/1865831419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e04af260 con 0x55e1e04abb40 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9c37fe640 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04bdca0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40058c0 tx=0x7fd9b4009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d99d1576496fbde server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9a2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:56.617 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1865831419 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b4004420 con 0x55e1e04abb40 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1865831419 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fd9b4003c80 con 0x55e1e04abb40 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9b4003ad0 con 0x55e1e04abb40 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1865831419 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd9b4044050 con 0x55e1e04abb40 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9cc40f740 10 client.18582 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55e1e052f730 con 0x55e1e04abb40 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1865831419 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9b4003ad0 con 0x55e1e04abb40 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1865831419 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd9b400f4e0 con 0x55e1e04abb40 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9cc40f740 10 client.18582 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.603+0000 7fd9cc40f740 10 client.18582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.604+0000 7fd9cc40f740 20 client.18582 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.604+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e052fca0 0x55e1e05320c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.604+0000 7fd9cc40f740 4 client.18582 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:52:56.618 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.604+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e1e0417d70 con 0x55e1e052fca0 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.604+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e052fca0 0x55e1e05320c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.604+0000 7fd9c2ffd640 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e052fca0 0x55e1e05320c0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68763000acbbd50a server_cookie=c9d1cfae33ef71cd in_seq=0 out_seq=0 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.604+0000 7fd9a2ffd640 10 client.18582 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9a2ffd640 1 -- 172.21.15.37:0/1865831419 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e1e0417d70 con 0x55e1e052fca0 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9a2ffd640 10 client.18582 handle_command_reply: tid=0 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 client.18582 shutdown 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 msgr2=0x55e1e04bdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04bdca0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd9b40058c0 tx=0x7fd9b4009b30 comp rx=0 tx=0).stop 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 shutdown_connections 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e1e052fca0 0x55e1e05320c0 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e1e04b5370 0x55e1e04aea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e1e04abb40 0x55e1e04bdca0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 --2- 172.21.15.37:0/1865831419 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e1e04aa5d0 0x55e1e04bd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 >> 172.21.15.37:0/1865831419 conn(0x55e1e0473310 msgr2=0x55e1e04a54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9a2ffd640 0 client.18582 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 shutdown_connections 2024-09-19T06:52:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 1 -- 172.21.15.37:0/1865831419 wait complete. 2024-09-19T06:52:56.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.605+0000 7fd9cc40f740 20 client.18582 trim_cache size 0 max 16384 2024-09-19T06:52:56.620 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9ac0a33b0 msgr2=0x7fd9ac0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc0cdf60 tx=0x7fd9bc067380 comp rx=0 tx=0).stop 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c410c170 msgr2=0x7fd9c419ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c410c170 0x7fd9c419ef30 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc00eb10 tx=0x7fd9bc00c2d0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 shutdown_connections 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9ac0a33b0 0x7fd9ac0a5870 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9c410c170 0x7fd9c419ef30 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9c4103660 0x7fd9c419aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 --2- 172.21.15.37:0/1429430441 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9c4102d40 0x7fd9c419a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:56.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 >> 172.21.15.37:0/1429430441 conn(0x7fd9c40fc7a0 msgr2=0x7fd9c410d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:56.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 shutdown_connections 2024-09-19T06:52:56.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:56.606+0000 7fd9ca036640 1 -- 172.21.15.37:0/1429430441 wait complete. 2024-09-19T06:52:56.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:56 smithi096 ceph-mon[35004]: pgmap v349: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 98 op/s 2024-09-19T06:52:56.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:56 smithi151 ceph-mon[35236]: pgmap v349: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 98 op/s 2024-09-19T06:52:57.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 813968 7.19 MB/sec execute 444 sec latency 39.938 ms 2024-09-19T06:52:58.214 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:52:58.214 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:52:58.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 816678 7.21 MB/sec execute 445 sec latency 13.769 ms 2024-09-19T06:52:58.475 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:52:58.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:58 smithi037 ceph-mon[31211]: pgmap v350: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 98 op/s 2024-09-19T06:52:58.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:58 smithi096 ceph-mon[35004]: pgmap v350: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 98 op/s 2024-09-19T06:52:58.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:58 smithi151 ceph-mon[35236]: pgmap v350: 97 pgs: 97 active+clean; 71 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 899 KiB/s wr, 98 op/s 2024-09-19T06:52:58.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.963+0000 7f06a2670640 1 -- 172.21.15.37:0/4138979386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 msgr2=0x7f069c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.963+0000 7f06a2670640 1 --2- 172.21.15.37:0/4138979386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c1029b0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f0690009900 tx=0x7f069002a480 comp rx=0 tx=0).stop 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 -- 172.21.15.37:0/4138979386 shutdown_connections 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 --2- 172.21.15.37:0/4138979386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f069c102ef0 0x7f069c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 --2- 172.21.15.37:0/4138979386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c1029b0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 --2- 172.21.15.37:0/4138979386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f069c100e20 0x7f069c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 -- 172.21.15.37:0/4138979386 >> 172.21.15.37:0/4138979386 conn(0x7f069c0fc7d0 msgr2=0x7f069c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 -- 172.21.15.37:0/4138979386 shutdown_connections 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 -- 172.21.15.37:0/4138979386 wait complete. 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 Processor -- start 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.964+0000 7f06a2670640 1 -- start start 2024-09-19T06:52:58.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f06a2670640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f069c100e20 0x7f069c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f06a2670640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f06a2670640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f069c102ef0 0x7f069c19cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f06a2670640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f069c110f80 con 0x7f069c102570 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f06a2670640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f069c110e00 con 0x7f069c102ef0 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f06a2670640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f069c111100 con 0x7f069c100e20 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c198a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43308/0 (socket says 172.21.15.37:43308) 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 -- 172.21.15.37:0/2844764683 learned_addr learned my addr 172.21.15.37:0/2844764683 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f069c102ef0 0x7f069c19cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069bfff640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f069c100e20 0x7f069c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.983 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 -- 172.21.15.37:0/2844764683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f069c100e20 msgr2=0x7f069c1984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f069c100e20 0x7f069c1984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 -- 172.21.15.37:0/2844764683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f069c102ef0 msgr2=0x7f069c19cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f069c102ef0 0x7f069c19cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 -- 172.21.15.37:0/2844764683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c19d490 con 0x7f069c102570 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.965+0000 7f069b7fe640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c198a20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0690006010 tx=0x7f0690003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.966+0000 7f06997fa640 1 -- 172.21.15.37:0/2844764683 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f069003f070 con 0x7f069c102570 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.966+0000 7f06997fa640 1 -- 172.21.15.37:0/2844764683 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0690031c30 con 0x7f069c102570 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.966+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c19d720 con 0x7f069c102570 2024-09-19T06:52:58.984 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.966+0000 7f06997fa640 1 -- 172.21.15.37:0/2844764683 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690035030 con 0x7f069c102570 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.966+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c1a5060 con 0x7f069c102570 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.968+0000 7f06997fa640 1 -- 172.21.15.37:0/2844764683 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0690042070 con 0x7f069c102570 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 Processor -- start 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 -- start start 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06997fa640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06840a33b0 0x7f06840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec41b60 0x55a74ec41f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06997fa640 1 -- 172.21.15.37:0/2844764683 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f06900f05b0 con 0x7f069c102570 2024-09-19T06:52:58.985 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06840a33b0 0x7f06840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec4cd40 0x55a74ec4f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a74eba7280 con 0x55a74ec42480 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a74eb4eb10 con 0x55a74ec4cd40 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a4a49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a74eba3240 con 0x55a74ec41b60 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a0c0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec4cd40 0x55a74ec4f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a0c0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec4cd40 0x55a74ec4f130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57036/0 (socket says 172.21.15.37:57036) 2024-09-19T06:52:58.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f06a0c0d640 1 -- 172.21.15.37:0/1916408336 learned_addr learned my addr 172.21.15.37:0/1916408336 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43322/0 (socket says 172.21.15.37:43322) 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.969+0000 7f069bfff640 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec41b60 0x55a74ec41f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f069b7fe640 1 -- 172.21.15.37:0/1916408336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec41b60 msgr2=0x55a74ec41f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06840a33b0 0x7f06840a5870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0694007fd0 tx=0x7f0694013b60 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f069b7fe640 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec41b60 0x55a74ec41f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f069b7fe640 1 -- 172.21.15.37:0/1916408336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec4cd40 msgr2=0x55a74ec4f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f069b7fe640 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec4cd40 0x55a74ec4f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f069b7fe640 1 -- 172.21.15.37:0/1916408336 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74eb5b1f0 con 0x55a74ec42480 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec4cd40 0x55a74ec4f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f069b7fe640 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4c800 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f06900edcd0 tx=0x7f06900bc3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d0337216827fa5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:58.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f06827fc640 1 -- 172.21.15.37:0/1916408336 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0690049050 con 0x55a74ec42480 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f06827fc640 1 -- 172.21.15.37:0/1916408336 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06900bcf60 con 0x55a74ec42480 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.970+0000 7f06827fc640 1 -- 172.21.15.37:0/1916408336 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900bcac0 con 0x55a74ec42480 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 -- 172.21.15.37:0/1916408336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 msgr2=0x55a74ec4c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4c800 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f06900edcd0 tx=0x7f06900bc3a0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 -- 172.21.15.37:0/1916408336 shutdown_connections 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec4cd40 0x55a74ec4f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4c800 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 --2- 172.21.15.37:0/1916408336 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec41b60 0x55a74ec41f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 -- 172.21.15.37:0/1916408336 >> 172.21.15.37:0/1916408336 conn(0x55a74ec3e0f0 msgr2=0x55a74ec3e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 -- 172.21.15.37:0/1916408336 shutdown_connections 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 -- 172.21.15.37:0/1916408336 wait complete. 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.971+0000 7f06a4a49740 1 -- start start 2024-09-19T06:52:58.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 1 Processor -- start 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec49c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec4cd40 0x55a74ec4a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a74ec4f6d0 con 0x55a74ec42480 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a74eb61f10 con 0x55a74ec41b60 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a74ec4f850 con 0x55a74ec4cd40 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a4a49740 10 client.0 _ll_register_callbacks cb 0x55a74eb2aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec49c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4a190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43334/0 (socket says 172.21.15.37:43334) 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 -- 172.21.15.37:0/4061514875 learned_addr learned my addr 172.21.15.37:0/4061514875 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec4cd40 0x55a74ec4a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.989 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 -- 172.21.15.37:0/4061514875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec4cd40 msgr2=0x55a74ec4a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec4cd40 0x55a74ec4a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 -- 172.21.15.37:0/4061514875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 msgr2=0x55a74ec49c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec49c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 -- 172.21.15.37:0/4061514875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74e9186a0 con 0x55a74ec42480 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec4cd40 0x55a74ec4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f069b7fe640 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4a190 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f06900edd80 tx=0x7f069003b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b14b6848d6445e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.972+0000 7f067b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f067b7fe640 1 -- 172.21.15.37:0/4061514875 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06900da060 con 0x55a74ec42480 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f067b7fe640 1 -- 172.21.15.37:0/4061514875 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f06900bccd0 con 0x55a74ec42480 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06900bcf60 con 0x55a74ec42480 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f067b7fe640 1 -- 172.21.15.37:0/4061514875 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f06900ee0c0 con 0x55a74ec42480 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 10 client.18594 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a74ec4f6d0 con 0x55a74ec42480 2024-09-19T06:52:58.990 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f067b7fe640 1 -- 172.21.15.37:0/4061514875 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f06900bcf60 con 0x55a74ec42480 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f067b7fe640 1 -- 172.21.15.37:0/4061514875 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f06900bdb90 con 0x55a74ec42480 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 10 client.18594 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 10 client.18594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 20 client.18594 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec5c3f0 0x55a74ec5e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 4 client.18594 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a74ebb47f0 con 0x55a74ec5c3f0 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.973+0000 7f069bfff640 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec5c3f0 0x55a74ec5e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.974+0000 7f069bfff640 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec5c3f0 0x55a74ec5e7e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83ea511b093e95a9 server_cookie=fbd948f9fd2adabc in_seq=0 out_seq=0 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.974+0000 7f067b7fe640 10 client.18594 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f067b7fe640 1 -- 172.21.15.37:0/4061514875 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a74ebb47f0 con 0x55a74ec5c3f0 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f067b7fe640 10 client.18594 handle_command_reply: tid=0 2024-09-19T06:52:58.991 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 client.18594 shutdown 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 msgr2=0x55a74ec4a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4a190 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f06900edd80 tx=0x7f069003b870 comp rx=0 tx=0).stop 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 shutdown_connections 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec5c3f0 0x55a74ec5e7e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec4cd40 0x55a74ec4a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec4a190 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f06a4a49740 1 --2- 172.21.15.37:0/4061514875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec49c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.975+0000 7f067b7fe640 0 client.18594 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.976+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 >> 172.21.15.37:0/4061514875 conn(0x55a74ec3e0f0 msgr2=0x55a74ec3d270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.976+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 shutdown_connections 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.976+0000 7f06a4a49740 1 -- 172.21.15.37:0/4061514875 wait complete. 2024-09-19T06:52:58.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.976+0000 7f06a4a49740 20 client.18594 trim_cache size 0 max 16384 2024-09-19T06:52:59.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.996+0000 7f06a4a49740 1 Processor -- start 2024-09-19T06:52:59.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.996+0000 7f06a4a49740 1 -- start start 2024-09-19T06:52:59.004 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.996+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec3e0f0 0x55a74ec4be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.996+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec4a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec63ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a4a49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f069c1adf40 con 0x55a74ec42480 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a4a49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f069c1ab810 con 0x55a74ec41b60 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a4a49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f069c1a9680 con 0x55a74ec3e0f0 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec63ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec4a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec63ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43336/0 (socket says 172.21.15.37:43336) 2024-09-19T06:52:59.005 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 -- 172.21.15.37:0/2985464562 learned_addr learned my addr 172.21.15.37:0/2985464562 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f069bfff640 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec3e0f0 0x55a74ec4be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 -- 172.21.15.37:0/2985464562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec3e0f0 msgr2=0x55a74ec4be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec3e0f0 0x55a74ec4be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 -- 172.21.15.37:0/2985464562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 msgr2=0x55a74ec4a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec4a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 -- 172.21.15.37:0/2985464562 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a74ec3f280 con 0x55a74ec42480 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f069b7fe640 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec4a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec63ff0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0694013080 tx=0x7f069401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9844f589cd96ffa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f0681ffb640 1 -- 172.21.15.37:0/2985464562 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694020710 con 0x55a74ec42480 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.997+0000 7f0681ffb640 1 -- 172.21.15.37:0/2985464562 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694020d90 con 0x55a74ec42480 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f0681ffb640 1 -- 172.21.15.37:0/2985464562 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0694010430 con 0x55a74ec42480 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 -- 172.21.15.37:0/2985464562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 msgr2=0x55a74ec63ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec63ff0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f0694013080 tx=0x7f069401c000 comp rx=0 tx=0).stop 2024-09-19T06:52:59.006 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 -- 172.21.15.37:0/2985464562 shutdown_connections 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec42480 0x55a74ec63ff0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec4a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 --2- 172.21.15.37:0/2985464562 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec3e0f0 0x55a74ec4be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 -- 172.21.15.37:0/2985464562 >> 172.21.15.37:0/2985464562 conn(0x55a74ec5eb50 msgr2=0x55a74ec48c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 -- 172.21.15.37:0/2985464562 shutdown_connections 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.998+0000 7f06a4a49740 1 -- 172.21.15.37:0/2985464562 wait complete. 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 -- start start 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 Processor -- start 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec3e0f0 0x55a74ec50b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec51090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.007 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec42480 0x55a74ecc5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f069c1a9cd0 con 0x55a74ec3e0f0 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f069c1ab140 con 0x55a74ec41b60 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f069c1aa3a0 con 0x55a74ec42480 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f06a4a49740 10 client.0 _ll_register_callbacks cb 0x7f068c0116f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f069bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec3e0f0 0x55a74ec50b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f069b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec51090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f069bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec3e0f0 0x55a74ec50b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43348/0 (socket says 172.21.15.37:43348) 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:58.999+0000 7f069bfff640 1 -- 172.21.15.37:0/10840047 learned_addr learned my addr 172.21.15.37:0/10840047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f069bfff640 1 -- 172.21.15.37:0/10840047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec42480 msgr2=0x55a74ecc5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f06a0c0d640 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec42480 0x55a74ecc5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f069bfff640 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec42480 0x55a74ecc5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.008 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f069bfff640 1 -- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 msgr2=0x55a74ec51090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f069bfff640 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec51090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f069bfff640 1 -- 172.21.15.37:0/10840047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a74ecc6500 con 0x55a74ec3e0f0 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f069b7fe640 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec51090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f069bfff640 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec3e0f0 0x55a74ec50b50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f068c011e70 tx=0x7f068c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1bcab026c0e79ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f067b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f067b7fe640 1 -- 172.21.15.37:0/10840047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f068c005020 con 0x55a74ec3e0f0 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0694020d90 con 0x55a74ec3e0f0 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f067b7fe640 1 -- 172.21.15.37:0/10840047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7f068c0051c0 con 0x55a74ec3e0f0 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f067b7fe640 1 -- 172.21.15.37:0/10840047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f068c0053c0 con 0x55a74ec3e0f0 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f06a4a49740 10 client.18600 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:59.009 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.000+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a74ecc7740 con 0x55a74ec3e0f0 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f067b7fe640 1 -- 172.21.15.37:0/10840047 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0694020d90 con 0x55a74ec3e0f0 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f067b7fe640 1 -- 172.21.15.37:0/10840047 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f068c030070 con 0x55a74ec3e0f0 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f06a4a49740 10 client.18600 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f06a4a49740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f06a4a49740 20 client.18600 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f06a4a49740 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec4cd40 0x55a74ecc9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f06a4a49740 4 client.18600 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a74ebaed00 con 0x55a74ec4cd40 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f069b7fe640 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec4cd40 0x55a74ecc9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.010 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f067b7fe640 10 client.18600 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.001+0000 7f069b7fe640 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec4cd40 0x55a74ecc9eb0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90575d3a6df84203 server_cookie=f52b660a93c6ac27 in_seq=0 out_seq=0 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.002+0000 7f067b7fe640 1 -- 172.21.15.37:0/10840047 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a74ebaed00 con 0x55a74ec4cd40 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.002+0000 7f067b7fe640 10 client.18600 handle_command_reply: tid=0 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.002+0000 7f06a4a49740 1 client.18600 shutdown 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec3e0f0 msgr2=0x55a74ec50b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec3e0f0 0x55a74ec50b50 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f068c011e70 tx=0x7f068c0039c0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 shutdown_connections 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a74ec4cd40 0x55a74ecc9eb0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a74ec42480 0x55a74ecc5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a74ec41b60 0x55a74ec51090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 --2- 172.21.15.37:0/10840047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a74ec3e0f0 0x55a74ec50b50 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 >> 172.21.15.37:0/10840047 conn(0x55a74ec5eb50 msgr2=0x55a74ec3ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.011 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f067b7fe640 0 client.18600 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 shutdown_connections 2024-09-19T06:52:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 1 -- 172.21.15.37:0/10840047 wait complete. 2024-09-19T06:52:59.012 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a4a49740 20 client.18600 trim_cache size 0 max 16384 2024-09-19T06:52:59.012 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:52:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:52:59.012 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "d98199d3-4b90-4ebe-b7ac-61b6aadeffe6", 2024-09-19T06:52:59.013 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:52:59.013 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:52:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06840a33b0 msgr2=0x7f06840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a2670640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06840a33b0 0x7f06840a5870 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f0694007fd0 tx=0x7f0694013b60 comp rx=0 tx=0).stop 2024-09-19T06:52:59.013 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 msgr2=0x7f069c198a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.003+0000 7f06a2670640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c198a20 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7f0690006010 tx=0x7f0690003c10 comp rx=0 tx=0).stop 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 shutdown_connections 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f069c102ef0 0x7f069c19cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f06840a33b0 0x7f06840a5870 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f069c102570 0x7f069c198a20 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 --2- 172.21.15.37:0/2844764683 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f069c100e20 0x7f069c1984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 >> 172.21.15.37:0/2844764683 conn(0x7f069c0fc7d0 msgr2=0x7f069c06a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 shutdown_connections 2024-09-19T06:52:59.014 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:52:59.004+0000 7f06a2670640 1 -- 172.21.15.37:0/2844764683 wait complete. 2024-09-19T06:52:59.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 818252 7.21 MB/sec execute 446 sec latency 13.802 ms 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 -- 172.21.15.37:0/2116890360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 msgr2=0x7fda10100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 --2- 172.21.15.37:0/2116890360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10100e10 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8009930 tx=0x7fd9f802a420 comp rx=0 tx=0).stop 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 -- 172.21.15.37:0/2116890360 shutdown_connections 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 --2- 172.21.15.37:0/2116890360 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda10101350 0x7fda1010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 --2- 172.21.15.37:0/2116890360 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10100e10 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 --2- 172.21.15.37:0/2116890360 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda101069c0 0x7fda10106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 -- 172.21.15.37:0/2116890360 >> 172.21.15.37:0/2116890360 conn(0x7fda10077990 msgr2=0x7fda100fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 -- 172.21.15.37:0/2116890360 shutdown_connections 2024-09-19T06:52:59.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.389+0000 7fda14aac640 1 -- 172.21.15.37:0/2116890360 wait complete. 2024-09-19T06:52:59.392 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 Processor -- start 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 -- start start 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda10101350 0x7fda10197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda101069c0 0x7fda1019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fda10110dc0 con 0x7fda101009b0 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda10110c40 con 0x7fda101069c0 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda14aac640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda10110f40 con 0x7fda10101350 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43374/0 (socket says 172.21.15.37:43374) 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 -- 172.21.15.37:0/542333926 learned_addr learned my addr 172.21.15.37:0/542333926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda101069c0 0x7fda1019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda07fff640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda10101350 0x7fda10197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.393 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 -- 172.21.15.37:0/542333926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda10101350 msgr2=0x7fda10197790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda10101350 0x7fda10197790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 -- 172.21.15.37:0/542333926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda101069c0 msgr2=0x7fda1019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda101069c0 0x7fda1019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.390+0000 7fda0e59c640 1 -- 172.21.15.37:0/542333926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda1019c240 con 0x7fda101009b0 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.391+0000 7fda0e59c640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10197250 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fda0000ea60 tx=0x7fda0000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.391+0000 7fda077fe640 1 -- 172.21.15.37:0/542333926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0000cde0 con 0x7fda101009b0 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.391+0000 7fda077fe640 1 -- 172.21.15.37:0/542333926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda000035f0 con 0x7fda101009b0 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.391+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda1019c530 con 0x7fda101009b0 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.391+0000 7fda077fe640 1 -- 172.21.15.37:0/542333926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda00010670 con 0x7fda101009b0 2024-09-19T06:52:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.391+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda101a3dc0 con 0x7fda101009b0 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.393+0000 7fda077fe640 1 -- 172.21.15.37:0/542333926 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fda000037a0 con 0x7fda101009b0 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.393+0000 7fda1784c740 1 Processor -- start 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda1784c740 1 -- start start 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda1784c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd780e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda1784c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd82980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda1784c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd852b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda1784c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c9bccda2f0 con 0x55c9bcd78620 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda1784c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9bcc82270 con 0x55c9bcd77d00 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda1784c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9bccd62b0 con 0x55c9bcd82ec0 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd82980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.400 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd780e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd780e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57104/0 (socket says 172.21.15.37:57104) 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd82980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43388/0 (socket says 172.21.15.37:43388) 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0ed9d640 1 -- 172.21.15.37:0/1525937803 learned_addr learned my addr 172.21.15.37:0/1525937803 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda07fff640 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd852b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda077fe640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9fc0a34e0 0x7fd9fc0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0e59c640 1 -- 172.21.15.37:0/1525937803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 msgr2=0x55c9bcd852b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0e59c640 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd852b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0e59c640 1 -- 172.21.15.37:0/1525937803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 msgr2=0x55c9bcd780e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0e59c640 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd780e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0e59c640 1 -- 172.21.15.37:0/1525937803 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9bcc81340 con 0x55c9bcd78620 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.394+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.395+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9fc0a34e0 0x7fd9fc0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.401 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.395+0000 7fda077fe640 1 -- 172.21.15.37:0/542333926 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fda000cfff0 con 0x7fda101009b0 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.395+0000 7fda0e59c640 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd82980 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fda000cd710 tx=0x7fda00020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80f30337a0e6a45b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.395+0000 7fda04ff9640 1 -- 172.21.15.37:0/1525937803 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0009c070 con 0x55c9bcd78620 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.395+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9fc0a34e0 0x7fd9fc0a59a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda08009fd0 tx=0x7fda08004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.395+0000 7fda04ff9640 1 -- 172.21.15.37:0/1525937803 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0009c9a0 con 0x55c9bcd78620 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.395+0000 7fda04ff9640 1 -- 172.21.15.37:0/1525937803 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda0009bc00 con 0x55c9bcd78620 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 -- 172.21.15.37:0/1525937803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 msgr2=0x55c9bcd82980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd82980 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7fda000cd710 tx=0x7fda00020000 comp rx=0 tx=0).stop 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 -- 172.21.15.37:0/1525937803 shutdown_connections 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd852b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd82980 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 --2- 172.21.15.37:0/1525937803 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd780e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 -- 172.21.15.37:0/1525937803 >> 172.21.15.37:0/1525937803 conn(0x55c9bcd74270 msgr2=0x55c9bcd74650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.402 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 -- 172.21.15.37:0/1525937803 shutdown_connections 2024-09-19T06:52:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 -- 172.21.15.37:0/1525937803 wait complete. 2024-09-19T06:52:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 -- start start 2024-09-19T06:52:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.396+0000 7fda1784c740 1 Processor -- start 2024-09-19T06:52:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd77d00 0x55c9bcd7fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd802b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd807f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c9bcd85a90 con 0x55c9bcd77d00 2024-09-19T06:52:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c9bcc63290 con 0x55c9bcd82ec0 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c9bcd85c10 con 0x55c9bcd78620 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 10 client.0 _ll_register_callbacks cb 0x55c9bcc94140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd77d00 0x55c9bcd7fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd77d00 0x55c9bcd7fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43390/0 (socket says 172.21.15.37:43390) 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 -- 172.21.15.37:0/81033686 learned_addr learned my addr 172.21.15.37:0/81033686 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd807f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda07fff640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd802b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 -- 172.21.15.37:0/81033686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 msgr2=0x55c9bcd802b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd802b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 -- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd82ec0 msgr2=0x55c9bcd807f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd807f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 -- 172.21.15.37:0/81033686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9bc9ed210 con 0x55c9bcd77d00 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda07fff640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd802b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda0e59c640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd77d00 0x55c9bcd7fd70 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fda0001f000 tx=0x7fda000d0360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1238d497d9379048 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fd9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/81033686 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000cde60 con 0x55c9bcd77d00 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/81033686 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fda0009a5a0 con 0x55c9bcd77d00 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.397+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda0009c9a0 con 0x55c9bcd77d00 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/81033686 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fda0009bde0 con 0x55c9bcd77d00 2024-09-19T06:52:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 10 client.18612 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9bcab9620 con 0x55c9bcd77d00 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/81033686 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda0009c9a0 con 0x55c9bcd77d00 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/81033686 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fda000cd990 con 0x55c9bcd77d00 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 10 client.18612 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 10 client.18612 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 20 client.18612 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcd92c60 0x55c9bcd95050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 4 client.18612 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c9bcce7a20 con 0x55c9bcd92c60 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.398+0000 7fda07fff640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcd92c60 0x55c9bcd95050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.399+0000 7fda07fff640 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcd92c60 0x55c9bcd95050 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74c5804751bd6b31 server_cookie=690c9f385321622f in_seq=0 out_seq=0 2024-09-19T06:52:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.399+0000 7fd9f5ffb640 10 client.18612 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/81033686 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c9bcce7a20 con 0x55c9bcd92c60 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fd9f5ffb640 10 client.18612 handle_command_reply: tid=0 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 client.18612 shutdown 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd77d00 msgr2=0x55c9bcd7fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd77d00 0x55c9bcd7fd70 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7fda0001f000 tx=0x7fda000d0360 comp rx=0 tx=0).stop 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 shutdown_connections 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcd92c60 0x55c9bcd95050 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd807f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd802b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 --2- 172.21.15.37:0/81033686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd77d00 0x55c9bcd7fd70 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.411 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fd9f5ffb640 0 client.18612 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 >> 172.21.15.37:0/81033686 conn(0x55c9bcd74270 msgr2=0x55c9bcd711c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 shutdown_connections 2024-09-19T06:52:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 1 -- 172.21.15.37:0/81033686 wait complete. 2024-09-19T06:52:59.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.401+0000 7fda1784c740 20 client.18612 trim_cache size 0 max 16384 2024-09-19T06:52:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.417+0000 7fda1784c740 1 Processor -- start 2024-09-19T06:52:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.417+0000 7fda1784c740 1 -- start start 2024-09-19T06:52:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda1784c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd81ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda1784c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd9a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda1784c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd9ac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda1784c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fda101ae050 con 0x55c9bcd78620 2024-09-19T06:52:59.419 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda1784c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda101ab920 con 0x55c9bcd77d00 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda1784c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda101a9790 con 0x55c9bcd82ec0 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd9a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd9a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50362/0 (socket says 172.21.15.37:50362) 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 -- 172.21.15.37:0/1804330028 learned_addr learned my addr 172.21.15.37:0/1804330028 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda0e59c640 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd81ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd9ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 -- 172.21.15.37:0/1804330028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 msgr2=0x55c9bcd9ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd9ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 -- 172.21.15.37:0/1804330028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 msgr2=0x55c9bcd81ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd81ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 -- 172.21.15.37:0/1804330028 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9bcd740a0 con 0x55c9bcd78620 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fda07fff640 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd9a6e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8032fd0 tx=0x7fd9f80058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd8b2b23a18721f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.420 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fd9f7fff640 1 -- 172.21.15.37:0/1804330028 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8049070 con 0x55c9bcd78620 2024-09-19T06:52:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fd9f7fff640 1 -- 172.21.15.37:0/1804330028 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f8005520 con 0x55c9bcd78620 2024-09-19T06:52:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.418+0000 7fd9f7fff640 1 -- 172.21.15.37:0/1804330028 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f8035040 con 0x55c9bcd78620 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 -- 172.21.15.37:0/1804330028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 msgr2=0x55c9bcd9a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd9a6e0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8032fd0 tx=0x7fd9f80058b0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 -- 172.21.15.37:0/1804330028 shutdown_connections 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd9ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd78620 0x55c9bcd9a6e0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 --2- 172.21.15.37:0/1804330028 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd81ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 -- 172.21.15.37:0/1804330028 >> 172.21.15.37:0/1804330028 conn(0x55c9bcd74270 msgr2=0x55c9bcd815d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 -- 172.21.15.37:0/1804330028 shutdown_connections 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 -- 172.21.15.37:0/1804330028 wait complete. 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.419+0000 7fda1784c740 1 -- start start 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 1 Processor -- start 2024-09-19T06:52:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd86bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd87110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd87650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fda101a9de0 con 0x55c9bcd82ec0 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fda101ab250 con 0x55c9bcd77d00 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fda101aa4b0 con 0x55c9bcd78620 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda1784c740 10 client.0 _ll_register_callbacks cb 0x7fda08009a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd87650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd86bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd86bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:51010/0 (socket says 172.21.15.37:51010) 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd87650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50368/0 (socket says 172.21.15.37:50368) 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 -- 172.21.15.37:0/1903309085 learned_addr learned my addr 172.21.15.37:0/1903309085 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda07fff640 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd87110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 -- 172.21.15.37:0/1903309085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 msgr2=0x55c9bcd87110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd87110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 -- 172.21.15.37:0/1903309085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd82ec0 msgr2=0x55c9bcd87650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:52:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd87650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 -- 172.21.15.37:0/1903309085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9bcdfc6b0 con 0x55c9bcd77d00 2024-09-19T06:52:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0ed9d640 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd87650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:52:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fda0e59c640 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd86bd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda0000ea30 tx=0x7fda000854b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=809f0dd6921426d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:52:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.420+0000 7fd9f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/1903309085 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda000cde60 con 0x55c9bcd77d00 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9f8005520 con 0x55c9bcd77d00 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/1903309085 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7fda0009a5a0 con 0x55c9bcd77d00 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/1903309085 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fda000a0650 con 0x55c9bcd77d00 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 10 client.27932 fetch_fsmap learned FSMap version 39 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55c9bcdfd980 con 0x55c9bcd77d00 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/1903309085 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9f8005520 con 0x55c9bcd77d00 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/1903309085 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fda00097ff0 con 0x55c9bcd77d00 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 10 client.27932 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 10 client.27932 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 20 client.27932 populate_metadata read hostname 'smithi037' 2024-09-19T06:52:59.427 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcdfdc70 0x55c9bce00060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:52:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 4 client.27932 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:52:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.421+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c9bcce1d70 con 0x55c9bcdfdc70 2024-09-19T06:52:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.422+0000 7fda07fff640 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcdfdc70 0x55c9bce00060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:52:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.422+0000 7fd9f5ffb640 10 client.27932 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:59.428 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.422+0000 7fda07fff640 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcdfdc70 0x55c9bce00060 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b06e6f1502c69d3c server_cookie=58dffbe6cea76789 in_seq=0 out_seq=0 2024-09-19T06:52:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fd9f5ffb640 1 -- 172.21.15.37:0/1903309085 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c9bcce1d70 con 0x55c9bcdfdc70 2024-09-19T06:52:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fd9f5ffb640 10 client.27932 handle_command_reply: tid=0 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 client.27932 shutdown 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 msgr2=0x55c9bcd86bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd86bd0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fda0000ea30 tx=0x7fda000854b0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 shutdown_connections 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c9bcdfdc70 0x55c9bce00060 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c9bcd82ec0 0x55c9bcd87650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fd9f5ffb640 0 client.27932 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c9bcd78620 0x55c9bcd87110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 --2- 172.21.15.37:0/1903309085 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c9bcd77d00 0x55c9bcd86bd0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 >> 172.21.15.37:0/1903309085 conn(0x55c9bcd74270 msgr2=0x55c9bcd80530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 shutdown_connections 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 1 -- 172.21.15.37:0/1903309085 wait complete. 2024-09-19T06:52:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.562+0000 7fda1784c740 20 client.27932 trim_cache size 0 max 16384 2024-09-19T06:52:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:52:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:52:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:52:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9fc0a34e0 msgr2=0x7fd9fc0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9fc0a34e0 0x7fd9fc0a59a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fda08009fd0 tx=0x7fda08004a40 comp rx=0 tx=0).stop 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 msgr2=0x7fda10197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10197250 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7fda0000ea60 tx=0x7fda0000ef20 comp rx=0 tx=0).stop 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 shutdown_connections 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fda101069c0 0x7fda1019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fda10101350 0x7fda10197790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9fc0a34e0 0x7fd9fc0a59a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 --2- 172.21.15.37:0/542333926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fda101009b0 0x7fda10197250 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:52:59.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 >> 172.21.15.37:0/542333926 conn(0x7fda10077990 msgr2=0x7fda10103830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:52:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 shutdown_connections 2024-09-19T06:52:59.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:52:59.563+0000 7fda14aac640 1 -- 172.21.15.37:0/542333926 wait complete. 2024-09-19T06:52:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:59 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:52:59.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:52:59 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:59.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:59 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:52:59.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:52:59 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:59.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:59 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:52:59.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:52:59 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:52:59.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:53:00.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 820277 7.22 MB/sec execute 447 sec latency 86.661 ms 2024-09-19T06:53:00.289 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:00.666 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:00 smithi037 ceph-mon[31211]: pgmap v351: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T06:53:00.666 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:00 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:53:00.667 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:00 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:00.667 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:00 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:53:00.667 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:00 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 -- 172.21.15.37:0/522241365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd781027a0 msgr2=0x7efd781067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 --2- 172.21.15.37:0/522241365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd781027a0 0x7efd781067f0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7efd68009900 tx=0x7efd6802a410 comp rx=0 tx=0).stop 2024-09-19T06:53:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 -- 172.21.15.37:0/522241365 shutdown_connections 2024-09-19T06:53:00.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 --2- 172.21.15.37:0/522241365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd78106f20 0x7efd7810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 --2- 172.21.15.37:0/522241365 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd781027a0 0x7efd781067f0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 --2- 172.21.15.37:0/522241365 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd78101df0 0x7efd781021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 -- 172.21.15.37:0/522241365 >> 172.21.15.37:0/522241365 conn(0x7efd780fd8f0 msgr2=0x7efd780ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 -- 172.21.15.37:0/522241365 shutdown_connections 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.752+0000 7efd7f88c640 1 -- 172.21.15.37:0/522241365 wait complete. 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 Processor -- start 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 -- start start 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78101df0 0x7efd78198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd781027a0 0x7efd78198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd78106f20 0x7efd7819cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd78111f80 con 0x7efd78101df0 2024-09-19T06:53:00.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd78111e00 con 0x7efd78106f20 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.753+0000 7efd7f88c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd78112100 con 0x7efd781027a0 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78101df0 0x7efd78198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78101df0 0x7efd78198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50386/0 (socket says 172.21.15.37:50386) 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 -- 172.21.15.37:0/89184486 learned_addr learned my addr 172.21.15.37:0/89184486 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7de29640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd78106f20 0x7efd7819cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7ce27640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd781027a0 0x7efd78198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 -- 172.21.15.37:0/89184486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd781027a0 msgr2=0x7efd78198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd781027a0 0x7efd78198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 -- 172.21.15.37:0/89184486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd78106f20 msgr2=0x7efd7819cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd78106f20 0x7efd7819cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 -- 172.21.15.37:0/89184486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7819d4b0 con 0x7efd78101df0 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7de29640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd78106f20 0x7efd7819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:00.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7d628640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78101df0 0x7efd78198500 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7efd7400e9f0 tx=0x7efd7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd6e7fc640 1 -- 172.21.15.37:0/89184486 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7400ccc0 con 0x7efd78101df0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7819d7a0 con 0x7efd78101df0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd6e7fc640 1 -- 172.21.15.37:0/89184486 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7400ce60 con 0x7efd78101df0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd6e7fc640 1 -- 172.21.15.37:0/89184486 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74010690 con 0x7efd78101df0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.754+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd781a5080 con 0x7efd78101df0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.757+0000 7efd6e7fc640 1 -- 172.21.15.37:0/89184486 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7efd740046c0 con 0x7efd78101df0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.757+0000 7efd81c65740 1 Processor -- start 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.757+0000 7efd81c65740 1 -- start start 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd81c65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298054f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd81c65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd81c65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x562980560e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd81c65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5629804b52e0 con 0x56298054fca0 2024-09-19T06:53:00.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd81c65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56298045cce0 con 0x56298055ea90 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd81c65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5629804b12a0 con 0x56298054f380 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298054f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7d628640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298054f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:52324/0 (socket says 172.21.15.37:52324) 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 -- 172.21.15.37:0/3439387034 learned_addr learned my addr 172.21.15.37:0/3439387034 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7ce27640 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x562980560e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd6e7fc640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd640a33b0 0x7efd640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 -- 172.21.15.37:0/3439387034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 msgr2=0x562980560e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x562980560e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 -- 172.21.15.37:0/3439387034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 msgr2=0x56298055e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 -- 172.21.15.37:0/3439387034 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562980441440 con 0x56298054f380 2024-09-19T06:53:00.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd7de29640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd640a33b0 0x7efd640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.758+0000 7efd6e7fc640 1 -- 172.21.15.37:0/89184486 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7efd740d0170 con 0x7efd78101df0 2024-09-19T06:53:00.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.759+0000 7efd7de29640 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298054f760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd70011910 tx=0x7efd70011dd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f72f34cf9baf422a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.759+0000 7efd637fe640 1 -- 172.21.15.37:0/3439387034 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70017050 con 0x56298054f380 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.759+0000 7efd637fe640 1 -- 172.21.15.37:0/3439387034 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7001bb90 con 0x56298054f380 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.759+0000 7efd637fe640 1 -- 172.21.15.37:0/3439387034 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70026e50 con 0x56298054f380 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 -- 172.21.15.37:0/3439387034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 msgr2=0x56298054f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298054f760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7efd70011910 tx=0x7efd70011dd0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd7de29640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd640a33b0 0x7efd640a5870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd70023000 tx=0x7efd70024000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 -- 172.21.15.37:0/3439387034 shutdown_connections 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x562980560e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 --2- 172.21.15.37:0/3439387034 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298054f760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 -- 172.21.15.37:0/3439387034 >> 172.21.15.37:0/3439387034 conn(0x56298054ba30 msgr2=0x56298054be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 -- 172.21.15.37:0/3439387034 shutdown_connections 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 -- 172.21.15.37:0/3439387034 wait complete. 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 -- start start 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.760+0000 7efd81c65740 1 Processor -- start 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd81c65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054f380 0x56298055e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd81c65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054fca0 0x56298055b690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd81c65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298055ea90 0x56298055bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd81c65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562980561660 con 0x56298055ea90 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd81c65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562980470050 con 0x56298054f380 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd81c65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5629805617e0 con 0x56298054fca0 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd81c65740 10 client.0 _ll_register_callbacks cb 0x5629804b0160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298055ea90 0x56298055bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7d628640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054f380 0x56298055e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298055ea90 0x56298055bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50416/0 (socket says 172.21.15.37:50416) 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 -- 172.21.15.37:0/78755118 learned_addr learned my addr 172.21.15.37:0/78755118 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7ce27640 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054fca0 0x56298055b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 -- 172.21.15.37:0/78755118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054fca0 msgr2=0x56298055b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054fca0 0x56298055b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 -- 172.21.15.37:0/78755118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054f380 msgr2=0x56298055e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054f380 0x56298055e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 -- 172.21.15.37:0/78755118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629802263c0 con 0x56298055ea90 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7ce27640 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054fca0 0x56298055b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd7de29640 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298055ea90 0x56298055bbf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7efd7001bf70 tx=0x7efd7000fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6208ed93e828d3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.761+0000 7efd60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd60ff9640 1 -- 172.21.15.37:0/78755118 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70033020 con 0x56298055ea90 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd60ff9640 1 -- 172.21.15.37:0/78755118 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7efd70026440 con 0x56298055ea90 2024-09-19T06:53:00.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd7001bb90 con 0x56298055ea90 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd60ff9640 1 -- 172.21.15.37:0/78755118 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efd70017020 con 0x56298055ea90 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 10 client.18630 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56298056d070 con 0x56298055ea90 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd60ff9640 1 -- 172.21.15.37:0/78755118 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd7001bb90 con 0x56298055ea90 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd60ff9640 1 -- 172.21.15.37:0/78755118 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efd7002e070 con 0x56298055ea90 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 10 client.18630 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 10 client.18630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 20 client.18630 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56298056e4d0 0x5629805708c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 4 client.18630 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.762+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629804c2850 con 0x56298056e4d0 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.763+0000 7efd7d628640 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56298056e4d0 0x5629805708c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.763+0000 7efd60ff9640 10 client.18630 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.763+0000 7efd7d628640 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56298056e4d0 0x5629805708c0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd054a08986d74ce server_cookie=89d399a99ece806a in_seq=0 out_seq=0 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.763+0000 7efd60ff9640 1 -- 172.21.15.37:0/78755118 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5629804c2850 con 0x56298056e4d0 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.763+0000 7efd60ff9640 10 client.18630 handle_command_reply: tid=0 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 client.18630 shutdown 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298055ea90 msgr2=0x56298055bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298055ea90 0x56298055bbf0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7efd7001bf70 tx=0x7efd7000fba0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 shutdown_connections 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56298056e4d0 0x5629805708c0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298055ea90 0x56298055bbf0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd60ff9640 0 client.18630 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054fca0 0x56298055b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 --2- 172.21.15.37:0/78755118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054f380 0x56298055e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 >> 172.21.15.37:0/78755118 conn(0x56298054ba30 msgr2=0x562980551790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 shutdown_connections 2024-09-19T06:53:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 1 -- 172.21.15.37:0/78755118 wait complete. 2024-09-19T06:53:00.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.764+0000 7efd81c65740 20 client.18630 trim_cache size 0 max 16384 2024-09-19T06:53:00.778 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:00 smithi096 ceph-mon[35004]: pgmap v351: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T06:53:00.778 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:00 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:53:00.778 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:00 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:00.778 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:00 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:53:00.778 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:00 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.782+0000 7efd81c65740 1 Processor -- start 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.782+0000 7efd81c65740 1 -- start start 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd81c65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298056a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd81c65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd81c65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x5629805760f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd81c65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd781ae000 con 0x56298054fca0 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd81c65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd781ab8d0 con 0x56298055ea90 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd81c65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd781a9740 con 0x56298054f380 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7ce27640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7d628640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298056a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7de29640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x5629805760f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.784 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7ce27640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50428/0 (socket says 172.21.15.37:50428) 2024-09-19T06:53:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7d628640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298056a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:52338/0 (socket says 172.21.15.37:52338) 2024-09-19T06:53:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7ce27640 1 -- 172.21.15.37:0/2289246541 learned_addr learned my addr 172.21.15.37:0/2289246541 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7de29640 1 -- 172.21.15.37:0/2289246541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 msgr2=0x56298056a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7de29640 1 --2- 172.21.15.37:0/2289246541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298056a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7de29640 1 -- 172.21.15.37:0/2289246541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 msgr2=0x56298055a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.785 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7de29640 1 --2- 172.21.15.37:0/2289246541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7de29640 1 -- 172.21.15.37:0/2289246541 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562980570ea0 con 0x56298055ea90 2024-09-19T06:53:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.783+0000 7efd7de29640 1 --2- 172.21.15.37:0/2289246541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x5629805760f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd70026b40 tx=0x7efd70011dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0468d1e2e9e7a63 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd62ffd640 1 -- 172.21.15.37:0/2289246541 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd70033070 con 0x56298055ea90 2024-09-19T06:53:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd62ffd640 1 -- 172.21.15.37:0/2289246541 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd70011490 con 0x56298055ea90 2024-09-19T06:53:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd62ffd640 1 -- 172.21.15.37:0/2289246541 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7002b040 con 0x56298055ea90 2024-09-19T06:53:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd81c65740 1 -- 172.21.15.37:0/2289246541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 msgr2=0x5629805760f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.788 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd81c65740 1 --2- 172.21.15.37:0/2289246541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x5629805760f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efd70026b40 tx=0x7efd70011dd0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd81c65740 1 -- 172.21.15.37:0/2289246541 shutdown_connections 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd81c65740 1 --2- 172.21.15.37:0/2289246541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298055ea90 0x5629805760f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd81c65740 1 --2- 172.21.15.37:0/2289246541 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054fca0 0x56298055a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd81c65740 1 --2- 172.21.15.37:0/2289246541 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298054f380 0x56298056a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.784+0000 7efd81c65740 1 -- 172.21.15.37:0/2289246541 >> 172.21.15.37:0/2289246541 conn(0x56298054ba30 msgr2=0x562980559f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 -- 172.21.15.37:0/2289246541 shutdown_connections 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 -- 172.21.15.37:0/2289246541 wait complete. 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 -- start start 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 Processor -- start 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054f380 0x562980566fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054fca0 0x5629805622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298055ea90 0x562980562830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7efd781a9d90 con 0x56298054f380 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd781ab200 con 0x56298054fca0 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.785+0000 7efd81c65740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7efd781aa460 con 0x56298055ea90 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd81c65740 10 client.0 _ll_register_callbacks cb 0x7efd740986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7d628640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054f380 0x562980566fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7ce27640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054fca0 0x5629805622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7d628640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054f380 0x562980566fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50440/0 (socket says 172.21.15.37:50440) 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7d628640 1 -- 172.21.15.37:0/1374641064 learned_addr learned my addr 172.21.15.37:0/1374641064 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7de29640 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298055ea90 0x562980562830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7ce27640 1 -- 172.21.15.37:0/1374641064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298055ea90 msgr2=0x562980562830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7ce27640 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298055ea90 0x562980562830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7ce27640 1 -- 172.21.15.37:0/1374641064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054f380 msgr2=0x562980566fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7ce27640 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054f380 0x562980566fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7ce27640 1 -- 172.21.15.37:0/1374641064 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562980562e20 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd7ce27640 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054fca0 0x5629805622f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd68031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af3b20aee8d3c085 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd60ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd60ff9640 1 -- 172.21.15.37:0/1374641064 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd68035040 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd60ff9640 1 -- 172.21.15.37:0/1374641064 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8571+0+0 (secure 0 0 0) 0x7efd68004260 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.786+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd70011490 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd60ff9640 1 -- 172.21.15.37:0/1374641064 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7efd68004460 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 10 client.27950 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5629805d8ed0 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd60ff9640 1 -- 172.21.15.37:0/1374641064 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd70011490 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd60ff9640 1 -- 172.21.15.37:0/1374641064 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7efd6804e070 con 0x56298054fca0 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 10 client.27950 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 10 client.27950 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 20 client.27950 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:00.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5629805d9680 0x5629805dbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 4 client.27950 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:53:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629804bcd60 con 0x5629805d9680 2024-09-19T06:53:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.787+0000 7efd7d628640 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5629805d9680 0x5629805dbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.788+0000 7efd7d628640 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5629805d9680 0x5629805dbaa0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26686b1188f99241 server_cookie=226cf98a37326680 in_seq=0 out_seq=0 2024-09-19T06:53:00.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.788+0000 7efd60ff9640 10 client.27950 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd60ff9640 1 -- 172.21.15.37:0/1374641064 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5629804bcd60 con 0x5629805d9680 2024-09-19T06:53:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd60ff9640 10 client.27950 handle_command_reply: tid=0 2024-09-19T06:53:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 client.27950 shutdown 2024-09-19T06:53:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054fca0 msgr2=0x5629805622f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054fca0 0x5629805622f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efd68000c00 tx=0x7efd68031b10 comp rx=0 tx=0).stop 2024-09-19T06:53:00.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 shutdown_connections 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5629805d9680 0x5629805dbaa0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56298055ea90 0x562980562830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd60ff9640 0 client.27950 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56298054fca0 0x5629805622f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 --2- 172.21.15.37:0/1374641064 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56298054f380 0x562980566fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 >> 172.21.15.37:0/1374641064 conn(0x56298054ba30 msgr2=0x56298055a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 shutdown_connections 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 1 -- 172.21.15.37:0/1374641064 wait complete. 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.789+0000 7efd81c65740 20 client.27950 trim_cache size 0 max 16384 2024-09-19T06:53:00.794 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: "d98199d3-4b90-4ebe-b7ac-61b6aadeffe6": { 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "d98199d3-4b90-4ebe-b7ac-61b6aadeffe6", 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:53:00.795 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd640a33b0 msgr2=0x7efd640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd640a33b0 0x7efd640a5870 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7efd70023000 tx=0x7efd70024000 comp rx=0 tx=0).stop 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78101df0 msgr2=0x7efd78198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78101df0 0x7efd78198500 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7efd7400e9f0 tx=0x7efd7400edf0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 shutdown_connections 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd78106f20 0x7efd7819cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efd781027a0 0x7efd78198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7efd640a33b0 0x7efd640a5870 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 --2- 172.21.15.37:0/89184486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7efd78101df0 0x7efd78198500 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 >> 172.21.15.37:0/89184486 conn(0x7efd780fd8f0 msgr2=0x7efd780fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 shutdown_connections 2024-09-19T06:53:00.797 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:00.790+0000 7efd7f88c640 1 -- 172.21.15.37:0/89184486 wait complete. 2024-09-19T06:53:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:00 smithi151 ceph-mon[35236]: pgmap v351: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.7 MiB/s wr, 171 op/s 2024-09-19T06:53:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:00 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:53:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:00 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:00 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:53:00.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:00 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:01.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 821104 7.21 MB/sec execute 448 sec latency 64.926 ms 2024-09-19T06:53:01.517 INFO:tasks.cephfs.filesystem:scrub status for tag:d98199d3-4b90-4ebe-b7ac-61b6aadeffe6 - {'path': '/', 'tag': 'd98199d3-4b90-4ebe-b7ac-61b6aadeffe6', 'options': 'recursive,force'} 2024-09-19T06:53:02.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 823688 7.23 MB/sec execute 449 sec latency 37.930 ms 2024-09-19T06:53:02.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:02 smithi096 ceph-mon[35004]: pgmap v352: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-09-19T06:53:02.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:02 smithi037 ceph-mon[31211]: pgmap v352: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-09-19T06:53:02.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:02 smithi151 ceph-mon[35236]: pgmap v352: 97 pgs: 97 active+clean; 79 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 1.6 MiB/s wr, 147 op/s 2024-09-19T06:53:03.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 824115 7.22 MB/sec execute 450 sec latency 35.493 ms 2024-09-19T06:53:03.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:03 smithi096 ceph-mon[35004]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T06:53:03.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:03 smithi037 ceph-mon[31211]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T06:53:03.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:03 smithi151 ceph-mon[35236]: osdmap e97: 12 total, 12 up, 12 in 2024-09-19T06:53:04.230 INFO:tasks.workunit.client.0.smithi037.stdout: 1 825510 7.21 MB/sec execute 451 sec latency 24.812 ms 2024-09-19T06:53:04.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:04 smithi096 ceph-mon[35004]: pgmap v354: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:04.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:04 smithi037 ceph-mon[31211]: pgmap v354: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:04.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:04 smithi151 ceph-mon[35236]: pgmap v354: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:05.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 827645 7.23 MB/sec execute 452 sec latency 15.583 ms 2024-09-19T06:53:06.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 830415 7.24 MB/sec execute 453 sec latency 15.841 ms 2024-09-19T06:53:06.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:06 smithi096 ceph-mon[35004]: pgmap v355: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:06.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:06 smithi037 ceph-mon[31211]: pgmap v355: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:06.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:06 smithi151 ceph-mon[35236]: pgmap v355: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:07.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 831765 7.24 MB/sec execute 454 sec latency 20.050 ms 2024-09-19T06:53:07.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:07 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:53:07.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:07 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:53:07.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:07 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:53:07.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:07 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:53:07.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:07 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:53:07.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:07 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:53:08.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 833704 7.25 MB/sec execute 455 sec latency 366.076 ms 2024-09-19T06:53:08.763 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:08 smithi151 ceph-mon[35236]: pgmap v356: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:08.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:08 smithi096 ceph-mon[35004]: pgmap v356: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:08.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:08 smithi037 ceph-mon[31211]: pgmap v356: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 71 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 2.0 MiB/s wr, 315 op/s 2024-09-19T06:53:09.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 834694 7.25 MB/sec execute 456 sec latency 202.697 ms 2024-09-19T06:53:09.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:53:09.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:53:09.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:53:09.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:53:09.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:53:09.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:53:09.776 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:53:09.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:09.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:53:09.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:53:09.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:53:10.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 837124 7.26 MB/sec execute 457 sec latency 60.121 ms 2024-09-19T06:53:10.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:10 smithi096 ceph-mon[35004]: pgmap v357: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2024-09-19T06:53:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:10 smithi096 ceph-mon[35004]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T06:53:10.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:10 smithi037 ceph-mon[31211]: pgmap v357: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2024-09-19T06:53:10.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:10 smithi037 ceph-mon[31211]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T06:53:10.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:10 smithi151 ceph-mon[35236]: pgmap v357: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 2.0 MiB/s wr, 309 op/s 2024-09-19T06:53:10.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:10 smithi151 ceph-mon[35236]: osdmap e98: 12 total, 12 up, 12 in 2024-09-19T06:53:11.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 838644 7.26 MB/sec execute 458 sec latency 15.944 ms 2024-09-19T06:53:12.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 841448 7.28 MB/sec execute 459 sec latency 18.164 ms 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 -- 172.21.15.37:0/2863100150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101350 msgr2=0x7ffaac10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2863100150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101350 0x7ffaac10d880 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4009cd0 tx=0x7ffaa402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 -- 172.21.15.37:0/2863100150 shutdown_connections 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2863100150 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac101350 0x7ffaac10d880 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2863100150 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaac1009b0 0x7ffaac100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2863100150 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1069c0 0x7ffaac106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 -- 172.21.15.37:0/2863100150 >> 172.21.15.37:0/2863100150 conn(0x7ffaac077990 msgr2=0x7ffaac0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 -- 172.21.15.37:0/2863100150 shutdown_connections 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.659+0000 7ffab28dd640 1 -- 172.21.15.37:0/2863100150 wait complete. 2024-09-19T06:53:12.662 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 Processor -- start 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 -- start start 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1009b0 0x7ffaac198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaac101350 0x7ffaac1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac1069c0 0x7ffaac19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac110e00 con 0x7ffaac1069c0 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac110c80 con 0x7ffaac1009b0 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab28dd640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac110f80 con 0x7ffaac101350 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac1069c0 0x7ffaac19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffaabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1009b0 0x7ffaac198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaac101350 0x7ffaac1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffaabfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1009b0 0x7ffaac198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:43524/0 (socket says 172.21.15.37:43524) 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac1069c0 0x7ffaac19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46252/0 (socket says 172.21.15.37:46252) 2024-09-19T06:53:12.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffaabfff640 1 -- 172.21.15.37:0/2256107658 learned_addr learned my addr 172.21.15.37:0/2256107658 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.660+0000 7ffaab7fe640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaac101350 0x7ffaac1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:51440/0 (socket says 172.21.15.37:51440) 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaabfff640 1 -- 172.21.15.37:0/2256107658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaac101350 msgr2=0x7ffaac1988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaabfff640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaac101350 0x7ffaac1988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaabfff640 1 -- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac1069c0 msgr2=0x7ffaac19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaabfff640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac1069c0 0x7ffaac19cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaabfff640 1 -- 172.21.15.37:0/2256107658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaac19d360 con 0x7ffaac1009b0 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac1069c0 0x7ffaac19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaabfff640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1009b0 0x7ffaac198370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00e9f0 tx=0x7ffa9c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaa97fa640 1 -- 172.21.15.37:0/2256107658 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00cd80 con 0x7ffaac1009b0 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaac19d650 con 0x7ffaac1009b0 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaa97fa640 1 -- 172.21.15.37:0/2256107658 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa9c010910 con 0x7ffaac1009b0 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffaa97fa640 1 -- 172.21.15.37:0/2256107658 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c0188f0 con 0x7ffaac1009b0 2024-09-19T06:53:12.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.661+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaac1a4ee0 con 0x7ffaac1009b0 2024-09-19T06:53:12.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 Processor -- start 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 -- start start 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffaa97fa640 1 -- 172.21.15.37:0/2256107658 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa9c010ac0 con 0x7ffaac1009b0 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e3156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3161770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e31640a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5580e30bb2f0 con 0x5580e3156af0 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5580e3063270 con 0x5580e3161cb0 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab567c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5580e30b72b0 con 0x5580e3157410 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e3156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e3156ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46260/0 (socket says 172.21.15.37:46260) 2024-09-19T06:53:12.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 -- 172.21.15.37:0/2777792398 learned_addr learned my addr 172.21.15.37:0/2777792398 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e31640a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffaabfff640 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 -- 172.21.15.37:0/2777792398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 msgr2=0x5580e3161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 -- 172.21.15.37:0/2777792398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 msgr2=0x5580e31640a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e31640a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.664+0000 7ffab0e7a640 1 -- 172.21.15.37:0/2777792398 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580e3062340 con 0x5580e3156af0 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.665+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e3156ed0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4035000 tx=0x7ffaa4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6919a67e819e901e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.665+0000 7ffaa97fa640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa940a7870 0x7ffa940a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.665+0000 7ffaabfff640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa940a7870 0x7ffa940a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.665+0000 7ffa927fc640 1 -- 172.21.15.37:0/2777792398 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4048070 con 0x5580e3156af0 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.665+0000 7ffaa97fa640 1 -- 172.21.15.37:0/2256107658 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7ffa9c0d0090 con 0x7ffaac1009b0 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.665+0000 7ffa927fc640 1 -- 172.21.15.37:0/2777792398 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4036070 con 0x5580e3156af0 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.665+0000 7ffa927fc640 1 -- 172.21.15.37:0/2777792398 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4004810 con 0x5580e3156af0 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 -- 172.21.15.37:0/2777792398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 msgr2=0x5580e3156ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffaabfff640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa940a7870 0x7ffa940a9d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0cd860 tx=0x7ffa9c01f000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e3156ed0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4035000 tx=0x7ffaa4009a20 comp rx=0 tx=0).stop 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 -- 172.21.15.37:0/2777792398 shutdown_connections 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e31640a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3161770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 --2- 172.21.15.37:0/2777792398 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e3156ed0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 -- 172.21.15.37:0/2777792398 >> 172.21.15.37:0/2777792398 conn(0x5580e3153060 msgr2=0x5580e3153440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 -- 172.21.15.37:0/2777792398 shutdown_connections 2024-09-19T06:53:12.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 -- 172.21.15.37:0/2777792398 wait complete. 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 -- start start 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.666+0000 7ffab567c740 1 Processor -- start 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab567c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e315ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab567c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e315f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab567c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e315f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab567c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5580e3164880 con 0x5580e3161cb0 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab567c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5580e3044290 con 0x5580e3157410 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab567c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5580e3164a00 con 0x5580e3156af0 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab567c740 10 client.0 _ll_register_callbacks cb 0x5580e3075140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e315f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e315f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46264/0 (socket says 172.21.15.37:46264) 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 -- 172.21.15.37:0/1320461951 learned_addr learned my addr 172.21.15.37:0/1320461951 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:12.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e315f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e315ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 -- 172.21.15.37:0/1320461951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 msgr2=0x5580e315ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e315ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 -- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 msgr2=0x5580e315f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e315f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 -- 172.21.15.37:0/1320461951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580e2dce210 con 0x5580e3161cb0 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e315f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e315f6b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004470 tx=0x7ffaa4004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bd408e54c3202 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:12.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffa8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.667+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/1320461951 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4056060 con 0x5580e3161cb0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa4036070 con 0x5580e3161cb0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/1320461951 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7ffaa4003b70 con 0x5580e3161cb0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/1320461951 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffaa4048040 con 0x5580e3161cb0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 10 client.18648 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5580e2e9a620 con 0x5580e3161cb0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/1320461951 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4036070 con 0x5580e3161cb0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/1320461951 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffaa404e070 con 0x5580e3161cb0 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 10 client.18648 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 10 client.18648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 20 client.18648 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e3171b20 0x5580e3173f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.676 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 4 client.18648 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5580e30c8a20 con 0x5580e3171b20 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.668+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e3171b20 0x5580e3173f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.669+0000 7ffa8b7fe640 10 client.18648 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.669+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e3171b20 0x5580e3173f10 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=618d50ea0342cdf3 server_cookie=c2690c3ef3a4dd78 in_seq=0 out_seq=0 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.669+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/1320461951 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5580e30c8a20 con 0x5580e3171b20 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.669+0000 7ffa8b7fe640 10 client.18648 handle_command_reply: tid=0 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 client.18648 shutdown 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 msgr2=0x5580e315f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.677 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e315f6b0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004470 tx=0x7ffaa4004370 comp rx=0 tx=0).stop 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 shutdown_connections 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e3171b20 0x5580e3173f10 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e315f6b0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e315f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 --2- 172.21.15.37:0/1320461951 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e315ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffa8b7fe640 0 client.18648 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 >> 172.21.15.37:0/1320461951 conn(0x5580e3153060 msgr2=0x5580e3158d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 shutdown_connections 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 1 -- 172.21.15.37:0/1320461951 wait complete. 2024-09-19T06:53:12.678 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.670+0000 7ffab567c740 20 client.18648 trim_cache size 0 max 16384 2024-09-19T06:53:12.690 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 Processor -- start 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 -- start start 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e315f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e3176f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1adf20 con 0x5580e3156af0 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1ab7f0 con 0x5580e3161cb0 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab567c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1a9660 con 0x5580e3157410 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e315f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e315f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46274/0 (socket says 172.21.15.37:46274) 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.685+0000 7ffab0e7a640 1 -- 172.21.15.37:0/3430728281 learned_addr learned my addr 172.21.15.37:0/3430728281 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e3176f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.691 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffab0e7a640 1 -- 172.21.15.37:0/3430728281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 msgr2=0x5580e3160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3160ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffab0e7a640 1 -- 172.21.15.37:0/3430728281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 msgr2=0x5580e3176f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e3176f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffab0e7a640 1 -- 172.21.15.37:0/3430728281 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580e3157240 con 0x5580e3156af0 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e315f4e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004500 tx=0x7ffaa4009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5ee2bb9573b05de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffa91ffb640 1 -- 172.21.15.37:0/3430728281 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4056070 con 0x5580e3156af0 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffa91ffb640 1 -- 172.21.15.37:0/3430728281 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4036040 con 0x5580e3156af0 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.686+0000 7ffa91ffb640 1 -- 172.21.15.37:0/3430728281 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4003bd0 con 0x5580e3156af0 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 -- 172.21.15.37:0/3430728281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 msgr2=0x5580e315f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e315f4e0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7ffaa4004500 tx=0x7ffaa4009a20 comp rx=0 tx=0).stop 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 -- 172.21.15.37:0/3430728281 shutdown_connections 2024-09-19T06:53:12.692 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3161cb0 0x5580e3176f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3157410 0x5580e3160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 --2- 172.21.15.37:0/3430728281 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3156af0 0x5580e315f4e0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 -- 172.21.15.37:0/3430728281 >> 172.21.15.37:0/3430728281 conn(0x5580e3153060 msgr2=0x5580e3160490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 -- 172.21.15.37:0/3430728281 shutdown_connections 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 -- 172.21.15.37:0/3430728281 wait complete. 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 -- start start 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.687+0000 7ffab567c740 1 Processor -- start 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab567c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e3177020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab567c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e3177560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab567c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e3174460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab567c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1a9cb0 con 0x5580e3161cb0 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab567c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1ab120 con 0x5580e3157410 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab567c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffaac1aa380 con 0x5580e3156af0 2024-09-19T06:53:12.693 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab567c740 10 client.0 _ll_register_callbacks cb 0x7ffaa4046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e3174460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e3174460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46288/0 (socket says 172.21.15.37:46288) 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 -- 172.21.15.37:0/179905268 learned_addr learned my addr 172.21.15.37:0/179905268 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e3177560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e3177020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 -- 172.21.15.37:0/179905268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 msgr2=0x5580e3177020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e3177020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 -- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 msgr2=0x5580e3177560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e3177560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 -- 172.21.15.37:0/179905268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580e3174c60 con 0x5580e3161cb0 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e3177560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:12.694 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffab0e7a640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e3174460 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40044d0 tx=0x7ffaa4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1419fa331dd009 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffa8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.688+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/179905268 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffaa4069060 con 0x5580e3161cb0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/179905268 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7ffaa4031ce0 con 0x5580e3161cb0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffaa4036040 con 0x5580e3161cb0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/179905268 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7ffaa4056030 con 0x5580e3161cb0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 10 client.18654 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5580e31dbee0 con 0x5580e3161cb0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/179905268 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffaa4036040 con 0x5580e3161cb0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/179905268 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7ffaa404e030 con 0x5580e3161cb0 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 10 client.18654 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 10 client.18654 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 20 client.18654 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:12.695 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e31dc4f0 0x5580e31de8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 4 client.18654 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5580e30c2d70 con 0x5580e31dc4f0 2024-09-19T06:53:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.689+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e31dc4f0 0x5580e31de8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.690+0000 7ffaab7fe640 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e31dc4f0 0x5580e31de8e0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6637c0d193caa227 server_cookie=866f011d29aa7ed2 in_seq=0 out_seq=0 2024-09-19T06:53:12.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.690+0000 7ffa8b7fe640 10 client.18654 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:12.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:12 smithi096 ceph-mon[35004]: pgmap v359: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-09-19T06:53:12.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:12 smithi037 ceph-mon[31211]: pgmap v359: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.840+0000 7ffa8b7fe640 1 -- 172.21.15.37:0/179905268 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5580e30c2d70 con 0x5580e31dc4f0 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.840+0000 7ffa8b7fe640 10 client.18654 handle_command_reply: tid=0 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.840+0000 7ffab567c740 1 client.18654 shutdown 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 msgr2=0x5580e3174460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e3174460 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7ffaa40044d0 tx=0x7ffaa4000f40 comp rx=0 tx=0).stop 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 shutdown_connections 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5580e31dc4f0 0x5580e31de8e0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5580e3161cb0 0x5580e3174460 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5580e3157410 0x5580e3177560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 --2- 172.21.15.37:0/179905268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5580e3156af0 0x5580e3177020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 >> 172.21.15.37:0/179905268 conn(0x5580e3153060 msgr2=0x5580e316d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffa8b7fe640 0 client.18654 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 shutdown_connections 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 1 -- 172.21.15.37:0/179905268 wait complete. 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.841+0000 7ffab567c740 20 client.18654 trim_cache size 0 max 16384 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:53:12.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:53:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:53:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa940a7870 msgr2=0x7ffa940a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa940a7870 0x7ffa940a9d30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c0cd860 tx=0x7ffa9c01f000 comp rx=0 tx=0).stop 2024-09-19T06:53:12.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1009b0 msgr2=0x7ffaac198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1009b0 0x7ffaac198370 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7ffa9c00e9f0 tx=0x7ffa9c00eeb0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 shutdown_connections 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa940a7870 0x7ffa940a9d30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffaac1069c0 0x7ffaac19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffaac101350 0x7ffaac1988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 --2- 172.21.15.37:0/2256107658 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaac1009b0 0x7ffaac198370 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 >> 172.21.15.37:0/2256107658 conn(0x7ffaac077990 msgr2=0x7ffaac0699e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 shutdown_connections 2024-09-19T06:53:12.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:12.842+0000 7ffab28dd640 1 -- 172.21.15.37:0/2256107658 wait complete. 2024-09-19T06:53:12.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:12 smithi151 ceph-mon[35236]: pgmap v359: 97 pgs: 97 active+clean; 78 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 2.4 MiB/s wr, 372 op/s 2024-09-19T06:53:13.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 842448 7.27 MB/sec execute 460 sec latency 96.396 ms 2024-09-19T06:53:14.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 845090 7.29 MB/sec execute 461 sec latency 38.220 ms 2024-09-19T06:53:14.776 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:14 smithi096 ceph-mon[35004]: pgmap v360: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:14.777 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:14 smithi037 ceph-mon[31211]: pgmap v360: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:14.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:14 smithi151 ceph-mon[35236]: pgmap v360: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:15.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 846679 7.29 MB/sec execute 462 sec latency 27.013 ms 2024-09-19T06:53:16.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 848932 7.30 MB/sec execute 463 sec latency 15.481 ms 2024-09-19T06:53:16.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:16 smithi096 ceph-mon[35004]: pgmap v361: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:16.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:16 smithi037 ceph-mon[31211]: pgmap v361: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:16.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:16 smithi151 ceph-mon[35236]: pgmap v361: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:17.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 851345 7.31 MB/sec execute 464 sec latency 17.950 ms 2024-09-19T06:53:18.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 852911 7.31 MB/sec execute 465 sec latency 75.127 ms 2024-09-19T06:53:18.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:18 smithi096 ceph-mon[35004]: pgmap v362: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:18.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:18 smithi037 ceph-mon[31211]: pgmap v362: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:18.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:18 smithi151 ceph-mon[35236]: pgmap v362: 97 pgs: 97 active+clean; 89 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 163 op/s 2024-09-19T06:53:19.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 855467 7.33 MB/sec execute 466 sec latency 161.491 ms 2024-09-19T06:53:20.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 856590 7.32 MB/sec execute 467 sec latency 85.527 ms 2024-09-19T06:53:20.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:20 smithi096 ceph-mon[35004]: pgmap v363: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-09-19T06:53:20.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:20 smithi037 ceph-mon[31211]: pgmap v363: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-09-19T06:53:20.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:20 smithi151 ceph-mon[35236]: pgmap v363: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 2.2 MiB/s wr, 287 op/s 2024-09-19T06:53:21.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 859277 7.34 MB/sec execute 468 sec latency 14.758 ms 2024-09-19T06:53:22.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 860761 7.34 MB/sec execute 469 sec latency 51.786 ms 2024-09-19T06:53:22.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:22 smithi096 ceph-mon[35004]: pgmap v364: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 263 op/s 2024-09-19T06:53:22.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:22 smithi037 ceph-mon[31211]: pgmap v364: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 263 op/s 2024-09-19T06:53:22.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:22 smithi151 ceph-mon[35236]: pgmap v364: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 2.1 MiB/s wr, 263 op/s 2024-09-19T06:53:23.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 862973 7.36 MB/sec execute 470 sec latency 37.715 ms 2024-09-19T06:53:24.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 864723 7.35 MB/sec execute 471 sec latency 55.268 ms 2024-09-19T06:53:24.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:24 smithi096 ceph-mon[35004]: pgmap v365: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-09-19T06:53:24.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:24 smithi037 ceph-mon[31211]: pgmap v365: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-09-19T06:53:24.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:24 smithi151 ceph-mon[35236]: pgmap v365: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.8 MiB/s wr, 320 op/s 2024-09-19T06:53:25.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 866453 7.36 MB/sec execute 472 sec latency 99.015 ms 2024-09-19T06:53:25.938 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 -- 172.21.15.37:0/1907062906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e41009b0 msgr2=0x7fc8e4100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 --2- 172.21.15.37:0/1907062906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e41009b0 0x7fc8e4100e10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009930 tx=0x7fc8d802a420 comp rx=0 tx=0).stop 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 -- 172.21.15.37:0/1907062906 shutdown_connections 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 --2- 172.21.15.37:0/1907062906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e4101350 0x7fc8e410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 --2- 172.21.15.37:0/1907062906 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e41009b0 0x7fc8e4100e10 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 --2- 172.21.15.37:0/1907062906 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e41069c0 0x7fc8e4106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 -- 172.21.15.37:0/1907062906 >> 172.21.15.37:0/1907062906 conn(0x7fc8e4077990 msgr2=0x7fc8e40fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 -- 172.21.15.37:0/1907062906 shutdown_connections 2024-09-19T06:53:25.939 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 -- 172.21.15.37:0/1907062906 wait complete. 2024-09-19T06:53:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.937+0000 7fc8e9152640 1 Processor -- start 2024-09-19T06:53:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e9152640 1 -- start start 2024-09-19T06:53:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e9152640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e41009b0 0x7fc8e4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e9152640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e4101350 0x7fc8e4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e9152640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e41069c0 0x7fc8e419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e9152640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e4110dc0 con 0x7fc8e4101350 2024-09-19T06:53:25.941 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e9152640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e4110c40 con 0x7fc8e41009b0 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e9152640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e4110f40 con 0x7fc8e41069c0 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e4101350 0x7fc8e4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e4101350 0x7fc8e4198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43456/0 (socket says 172.21.15.37:43456) 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e41009b0 0x7fc8e4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 -- 172.21.15.37:0/478175510 learned_addr learned my addr 172.21.15.37:0/478175510 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e359e640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e41069c0 0x7fc8e419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 -- 172.21.15.37:0/478175510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e41069c0 msgr2=0x7fc8e419cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e41069c0 0x7fc8e419cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 -- 172.21.15.37:0/478175510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e41009b0 msgr2=0x7fc8e4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e41009b0 0x7fc8e4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.938+0000 7fc8e259c640 1 -- 172.21.15.37:0/478175510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8e419d330 con 0x7fc8e4101350 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.939+0000 7fc8e259c640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e4101350 0x7fc8e4198880 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009230 tx=0x7fc8d8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.939+0000 7fc8cbfff640 1 -- 172.21.15.37:0/478175510 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d803f030 con 0x7fc8e4101350 2024-09-19T06:53:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.939+0000 7fc8cbfff640 1 -- 172.21.15.37:0/478175510 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d8004440 con 0x7fc8e4101350 2024-09-19T06:53:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.939+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8e419d5c0 con 0x7fc8e4101350 2024-09-19T06:53:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.939+0000 7fc8cbfff640 1 -- 172.21.15.37:0/478175510 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8035070 con 0x7fc8e4101350 2024-09-19T06:53:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.939+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8e41a4eb0 con 0x7fc8e4101350 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.941+0000 7fc8cbfff640 1 -- 172.21.15.37:0/478175510 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc8d8042030 con 0x7fc8e4101350 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 Processor -- start 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 -- start start 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d707ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d712770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d7150a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b11d66c2f0 con 0x55b11d712cb0 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b11d614270 con 0x55b11d708410 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b11d6682b0 con 0x55b11d707af0 2024-09-19T06:53:25.950 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d712770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d7150a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d712770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:60400/0 (socket says 172.21.15.37:60400) 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2824403147 learned_addr learned my addr 172.21.15.37:0/2824403147 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8cbfff640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc8cc0a3460 0x7fc8cc0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e359e640 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d707ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2824403147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 msgr2=0x55b11d707ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d707ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e359e640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc8cc0a3460 0x7fc8cc0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2824403147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 msgr2=0x55b11d7150a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d7150a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2824403147 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b11d613340 con 0x55b11d708410 2024-09-19T06:53:25.951 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.942+0000 7fc8cbfff640 1 -- 172.21.15.37:0/478175510 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fc8d80f0090 con 0x7fc8e4101350 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.943+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d712770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009c70 tx=0x7fc8d4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19b272b53e4864fa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.943+0000 7fc8e359e640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc8cc0a3460 0x7fc8cc0a5920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005c40 tx=0x7fc8dc009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.943+0000 7fc8c97fa640 1 -- 172.21.15.37:0/2824403147 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4004230 con 0x55b11d708410 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.943+0000 7fc8c97fa640 1 -- 172.21.15.37:0/2824403147 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d40043d0 con 0x55b11d708410 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.943+0000 7fc8c97fa640 1 -- 172.21.15.37:0/2824403147 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40197c0 con 0x55b11d708410 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.943+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2824403147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 msgr2=0x55b11d712770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.943+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d712770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009c70 tx=0x7fc8d4011810 comp rx=0 tx=0).stop 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2824403147 shutdown_connections 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d7150a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d712770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2824403147 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d707ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2824403147 >> 172.21.15.37:0/2824403147 conn(0x55b11d704060 msgr2=0x55b11d704440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:25.952 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2824403147 shutdown_connections 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2824403147 wait complete. 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- start start 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 Processor -- start 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d70fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d710070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d7105b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b11d715880 con 0x55b11d707af0 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b11d5f5290 con 0x55b11d712cb0 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b11d715a00 con 0x55b11d708410 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8ebfc9740 10 client.0 _ll_register_callbacks cb 0x55b11d626140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d70fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.953 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d70fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43478/0 (socket says 172.21.15.37:43478) 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2929100832 learned_addr learned my addr 172.21.15.37:0/2929100832 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8e359e640 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d7105b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.944+0000 7fc8e259c640 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d710070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2929100832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 msgr2=0x55b11d710070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d710070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 msgr2=0x55b11d7105b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d7105b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/2929100832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b11d37f210 con 0x55b11d707af0 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d70fb30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009560 tx=0x7fc8d4005750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d75616e24c9bf258 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2929100832 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4027070 con 0x55b11d707af0 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2929100832 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fc8d4003800 con 0x55b11d707af0 2024-09-19T06:53:25.954 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d40043d0 con 0x55b11d707af0 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2929100832 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc8d4033050 con 0x55b11d707af0 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8ebfc9740 10 client.18666 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b11d44b620 con 0x55b11d707af0 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.945+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2929100832 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d40043d0 con 0x55b11d707af0 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2929100832 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc8d4018070 con 0x55b11d707af0 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8ebfc9740 10 client.18666 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8ebfc9740 10 client.18666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8ebfc9740 20 client.18666 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d722b30 0x55b11d724f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8ebfc9740 4 client.18666 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b11d679a20 con 0x55b11d722b30 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8e259c640 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d722b30 0x55b11d724f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8c6ff5640 10 client.18666 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:25.955 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.946+0000 7fc8e259c640 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d722b30 0x55b11d724f20 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e656e9ab1048a9 server_cookie=ff047dcd03b01bfa in_seq=0 out_seq=0 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2929100832 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b11d679a20 con 0x55b11d722b30 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8c6ff5640 10 client.18666 handle_command_reply: tid=0 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 client.18666 shutdown 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 msgr2=0x55b11d70fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d70fb30 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009560 tx=0x7fc8d4005750 comp rx=0 tx=0).stop 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 shutdown_connections 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d722b30 0x55b11d724f20 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d7105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d710070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8c6ff5640 0 client.18666 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2929100832 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d70fb30 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.947+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 >> 172.21.15.37:0/2929100832 conn(0x55b11d704060 msgr2=0x55b11d709d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:25.956 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.948+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 shutdown_connections 2024-09-19T06:53:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.948+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2929100832 wait complete. 2024-09-19T06:53:25.957 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.948+0000 7fc8ebfc9740 20 client.18666 trim_cache size 0 max 16384 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 Processor -- start 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 -- start start 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d711aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d7103b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d72a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41b0220 con 0x55b11d707af0 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41adaf0 con 0x55b11d712cb0 2024-09-19T06:53:25.971 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41ab960 con 0x55b11d708410 2024-09-19T06:53:25.972 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d711aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d711aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43488/0 (socket says 172.21.15.37:43488) 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/1322554955 learned_addr learned my addr 172.21.15.37:0/1322554955 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e359e640 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d72a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e259c640 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d7103b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/1322554955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 msgr2=0x55b11d7103b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d7103b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/1322554955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 msgr2=0x55b11d72a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d72a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.973 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.962+0000 7fc8e2d9d640 1 -- 172.21.15.37:0/1322554955 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b11d703e90 con 0x55b11d707af0 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.963+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d711aa0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4011850 tx=0x7fc8d40193e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827d8565f38dd13b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.963+0000 7fc8c8ff9640 1 -- 172.21.15.37:0/1322554955 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4015020 con 0x55b11d707af0 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.963+0000 7fc8c8ff9640 1 -- 172.21.15.37:0/1322554955 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d4003ce0 con 0x55b11d707af0 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.963+0000 7fc8c8ff9640 1 -- 172.21.15.37:0/1322554955 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4027030 con 0x55b11d707af0 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/1322554955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 msgr2=0x55b11d711aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d711aa0 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4011850 tx=0x7fc8d40193e0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/1322554955 shutdown_connections 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d712cb0 0x55b11d72a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d708410 0x55b11d7103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/1322554955 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d707af0 0x55b11d711aa0 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/1322554955 >> 172.21.15.37:0/1322554955 conn(0x55b11d704060 msgr2=0x55b11d711390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/1322554955 shutdown_connections 2024-09-19T06:53:25.974 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/1322554955 wait complete. 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- start start 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 Processor -- start 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d716400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d716940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d78b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41abfb0 con 0x55b11d712cb0 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41ad420 con 0x55b11d708410 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc8e41ac680 con 0x55b11d707af0 2024-09-19T06:53:25.975 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8ebfc9740 10 client.0 _ll_register_callbacks cb 0x7fc8d80b8460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8e359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d78b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8e359e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d78b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:43500/0 (socket says 172.21.15.37:43500) 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8e359e640 1 -- 172.21.15.37:0/2823808207 learned_addr learned my addr 172.21.15.37:0/2823808207 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8e259c640 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d716940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.964+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d716400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8e359e640 1 -- 172.21.15.37:0/2823808207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 msgr2=0x55b11d716400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8e359e640 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d716400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8e359e640 1 -- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 msgr2=0x55b11d716940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8e359e640 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d716940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8e359e640 1 -- 172.21.15.37:0/2823808207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b11d78bc90 con 0x55b11d712cb0 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8e359e640 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d78b650 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0043d0 tx=0x7fc8dc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cced4ed0abf0dc76 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:25.976 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8c6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2823808207 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc010c80 con 0x55b11d712cb0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2823808207 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fc8dc020480 con 0x55b11d712cb0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8d4003ce0 con 0x55b11d712cb0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2823808207 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fc8dc010e20 con 0x55b11d712cb0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8ebfc9740 10 client.18672 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55b11d78cfe0 con 0x55b11d712cb0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.965+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2823808207 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8d4003ce0 con 0x55b11d712cb0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2823808207 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fc8dc01d040 con 0x55b11d712cb0 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8ebfc9740 10 client.18672 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8ebfc9740 10 client.18672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8ebfc9740 20 client.18672 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d78d760 0x55b11d78fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8ebfc9740 4 client.18672 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:25.977 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b11d673d70 con 0x55b11d78d760 2024-09-19T06:53:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d78d760 0x55b11d78fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8c6ff5640 10 client.18672 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:25.978 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:25.966+0000 7fc8e2d9d640 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d78d760 0x55b11d78fb80 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2b105b47d1a6c48 server_cookie=5182e4958acfedec in_seq=0 out_seq=0 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.126+0000 7fc8c6ff5640 1 -- 172.21.15.37:0/2823808207 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b11d673d70 con 0x55b11d78d760 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.126+0000 7fc8c6ff5640 10 client.18672 handle_command_reply: tid=0 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.126+0000 7fc8ebfc9740 1 client.18672 shutdown 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 msgr2=0x55b11d78b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d78b650 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc0043d0 tx=0x7fc8dc004a40 comp rx=0 tx=0).stop 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 shutdown_connections 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b11d78d760 0x55b11d78fb80 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b11d712cb0 0x55b11d78b650 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b11d708410 0x55b11d716940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 --2- 172.21.15.37:0/2823808207 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b11d707af0 0x55b11d716400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 >> 172.21.15.37:0/2823808207 conn(0x55b11d704060 msgr2=0x55b11d7031c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:26.129 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8c6ff5640 0 client.18672 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 shutdown_connections 2024-09-19T06:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 1 -- 172.21.15.37:0/2823808207 wait complete. 2024-09-19T06:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.127+0000 7fc8ebfc9740 20 client.18672 trim_cache size 0 max 16384 2024-09-19T06:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:53:26.130 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc8cc0a3460 msgr2=0x7fc8cc0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc8cc0a3460 0x7fc8cc0a5920 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc005c40 tx=0x7fc8dc009500 comp rx=0 tx=0).stop 2024-09-19T06:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e4101350 msgr2=0x7fc8e4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:26.131 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e4101350 0x7fc8e4198880 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fc8d8009230 tx=0x7fc8d8004040 comp rx=0 tx=0).stop 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 shutdown_connections 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e41069c0 0x7fc8e419cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc8cc0a3460 0x7fc8cc0a5920 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc8e4101350 0x7fc8e4198880 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 --2- 172.21.15.37:0/478175510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc8e41009b0 0x7fc8e4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 >> 172.21.15.37:0/478175510 conn(0x7fc8e4077990 msgr2=0x7fc8e4069bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 shutdown_connections 2024-09-19T06:53:26.132 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:26.128+0000 7fc8e9152640 1 -- 172.21.15.37:0/478175510 wait complete. 2024-09-19T06:53:26.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 867961 7.36 MB/sec execute 473 sec latency 90.187 ms 2024-09-19T06:53:26.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:26 smithi096 ceph-mon[35004]: pgmap v366: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-19T06:53:26.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:26 smithi037 ceph-mon[31211]: pgmap v366: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-19T06:53:26.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:26 smithi151 ceph-mon[35236]: pgmap v366: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-19T06:53:27.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 870075 7.37 MB/sec execute 474 sec latency 95.046 ms 2024-09-19T06:53:28.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 872433 7.38 MB/sec execute 475 sec latency 19.617 ms 2024-09-19T06:53:28.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:28 smithi096 ceph-mon[35004]: pgmap v367: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-19T06:53:28.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:28 smithi037 ceph-mon[31211]: pgmap v367: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-19T06:53:28.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:28 smithi151 ceph-mon[35236]: pgmap v367: 97 pgs: 97 active+clean; 87 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 MiB/s rd, 1.9 MiB/s wr, 251 op/s 2024-09-19T06:53:29.231 INFO:tasks.workunit.client.0.smithi037.stdout: 1 873932 7.38 MB/sec execute 476 sec latency 69.659 ms 2024-09-19T06:53:30.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 876304 7.40 MB/sec execute 477 sec latency 45.914 ms 2024-09-19T06:53:30.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:30 smithi096 ceph-mon[35004]: pgmap v368: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.4 MiB/s wr, 428 op/s 2024-09-19T06:53:30.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:30 smithi037 ceph-mon[31211]: pgmap v368: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.4 MiB/s wr, 428 op/s 2024-09-19T06:53:30.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:30 smithi151 ceph-mon[35236]: pgmap v368: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 3.4 MiB/s wr, 428 op/s 2024-09-19T06:53:31.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 877801 7.39 MB/sec execute 478 sec latency 12.777 ms 2024-09-19T06:53:31.519 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:53:31.784 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:32.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 880555 7.42 MB/sec execute 479 sec latency 26.476 ms 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 -- 172.21.15.37:0/3894959702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 msgr2=0x7fd7781021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 --2- 172.21.15.37:0/3894959702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd7781021c0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd774009900 tx=0x7fd77402a410 comp rx=0 tx=0).stop 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 -- 172.21.15.37:0/3894959702 shutdown_connections 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 --2- 172.21.15.37:0/3894959702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd778106f10 0x7fd77810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 --2- 172.21.15.37:0/3894959702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd778102790 0x7fd7781067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 --2- 172.21.15.37:0/3894959702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd7781021c0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 -- 172.21.15.37:0/3894959702 >> 172.21.15.37:0/3894959702 conn(0x7fd7780fd8c0 msgr2=0x7fd7780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 -- 172.21.15.37:0/3894959702 shutdown_connections 2024-09-19T06:53:32.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.310+0000 7fd77feb4640 1 -- 172.21.15.37:0/3894959702 wait complete. 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 Processor -- start 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 -- start start 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd778198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd778102790 0x7fd778198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd778106f10 0x7fd77819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd778111f70 con 0x7fd778101de0 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd778111df0 con 0x7fd778106f10 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77feb4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781120f0 con 0x7fd778102790 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77dc50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd778198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77dc50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd778198430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37412/0 (socket says 172.21.15.37:37412) 2024-09-19T06:53:32.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77dc50640 1 -- 172.21.15.37:0/1071845048 learned_addr learned my addr 172.21.15.37:0/1071845048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77e451640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd778106f10 0x7fd77819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.311+0000 7fd77d44f640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd778102790 0x7fd778198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77dc50640 1 -- 172.21.15.37:0/1071845048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd778102790 msgr2=0x7fd778198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77dc50640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd778102790 0x7fd778198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77dc50640 1 -- 172.21.15.37:0/1071845048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd778106f10 msgr2=0x7fd77819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77dc50640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd778106f10 0x7fd77819cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77dc50640 1 -- 172.21.15.37:0/1071845048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77819d3e0 con 0x7fd778101de0 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77e451640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd778106f10 0x7fd77819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77d44f640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd778102790 0x7fd778198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77dc50640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd778198430 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd774006010 tx=0x7fd774004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd76effd640 1 -- 172.21.15.37:0/1071845048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77403f030 con 0x7fd778101de0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd76effd640 1 -- 172.21.15.37:0/1071845048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd774004440 con 0x7fd778101de0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77819d670 con 0x7fd778101de0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd76effd640 1 -- 172.21.15.37:0/1071845048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774035070 con 0x7fd778101de0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.312+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7781a4fb0 con 0x7fd778101de0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.314+0000 7fd76effd640 1 -- 172.21.15.37:0/1071845048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd774042030 con 0x7fd778101de0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 Processor -- start 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 -- start start 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fa5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562ca6f0b2e0 con 0x562ca6fa54a0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ca6eb2ce0 con 0x562ca6fa5dc0 2024-09-19T06:53:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd78228d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562ca6f072a0 con 0x562ca6fb0680 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd77e451640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fa5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd76effd640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7640a33b0 0x7fd7640a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd77e451640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fa5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37424/0 (socket says 172.21.15.37:37424) 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd77e451640 1 -- 172.21.15.37:0/3698748083 learned_addr learned my addr 172.21.15.37:0/3698748083 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd77dc50640 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd77d44f640 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.315+0000 7fd77e451640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7640a33b0 0x7fd7640a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd77e451640 1 -- 172.21.15.37:0/3698748083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 msgr2=0x562ca6fb2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd77e451640 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd77e451640 1 -- 172.21.15.37:0/3698748083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 msgr2=0x562ca6fb0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd77e451640 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd77e451640 1 -- 172.21.15.37:0/3698748083 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca6e97440 con 0x562ca6fa54a0 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd76effd640 1 -- 172.21.15.37:0/1071845048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fd7740f0060 con 0x7fd778101de0 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd77e451640 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fa5880 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd7700094d0 tx=0x7fd77000f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8789f3fec5f8918 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd763fff640 1 -- 172.21.15.37:0/3698748083 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770023030 con 0x562ca6fa54a0 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd763fff640 1 -- 172.21.15.37:0/3698748083 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd770013880 con 0x562ca6fa54a0 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd763fff640 1 -- 172.21.15.37:0/3698748083 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd77000eb80 con 0x562ca6fa54a0 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.316+0000 7fd77e451640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7640a33b0 0x7fd7640a5870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd770023000 tx=0x7fd770022000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 -- 172.21.15.37:0/3698748083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 msgr2=0x562ca6fa5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fa5880 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fd7700094d0 tx=0x7fd77000f7d0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 -- 172.21.15.37:0/3698748083 shutdown_connections 2024-09-19T06:53:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 --2- 172.21.15.37:0/3698748083 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fa5880 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 -- 172.21.15.37:0/3698748083 >> 172.21.15.37:0/3698748083 conn(0x562ca6fa1a30 msgr2=0x562ca6fa1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 -- 172.21.15.37:0/3698748083 shutdown_connections 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 -- 172.21.15.37:0/3698748083 wait complete. 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.317+0000 7fd78228d740 1 -- start start 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 1 Processor -- start 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa54a0 0x562ca6faff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa5dc0 0x562ca6fad220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fb0680 0x562ca6fad760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562ca6fb32a0 con 0x562ca6fb0680 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562ca6ec6050 con 0x562ca6fa54a0 2024-09-19T06:53:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562ca6fb3420 con 0x562ca6fa5dc0 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd78228d740 10 client.0 _ll_register_callbacks cb 0x562ca6f06160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fb0680 0x562ca6fad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77dc50640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa54a0 0x562ca6faff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fb0680 0x562ca6fad760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37440/0 (socket says 172.21.15.37:37440) 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 -- 172.21.15.37:0/3140192030 learned_addr learned my addr 172.21.15.37:0/3140192030 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77d44f640 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa5dc0 0x562ca6fad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 -- 172.21.15.37:0/3140192030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa5dc0 msgr2=0x562ca6fad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa5dc0 0x562ca6fad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 -- 172.21.15.37:0/3140192030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa54a0 msgr2=0x562ca6faff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa54a0 0x562ca6faff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.318+0000 7fd77e451640 1 -- 172.21.15.37:0/3140192030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca6c7c3c0 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd77e451640 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fb0680 0x562ca6fad760 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e7a0 tx=0x7fd77000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5543fd68f0555969 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd7617fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd7617fa640 1 -- 172.21.15.37:0/3140192030 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd770013a30 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd7617fa640 1 -- 172.21.15.37:0/3140192030 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fd77000ee20 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd770013880 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd7617fa640 1 -- 172.21.15.37:0/3140192030 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd7700043f0 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd78228d740 10 client.18684 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562ca6fbec10 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd7617fa640 1 -- 172.21.15.37:0/3140192030 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd770013880 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd7617fa640 1 -- 172.21.15.37:0/3140192030 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd77001f640 con 0x562ca6fb0680 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd78228d740 10 client.18684 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd78228d740 10 client.18684 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd78228d740 20 client.18684 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.319+0000 7fd78228d740 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca6fbffc0 0x562ca6fc23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.320+0000 7fd78228d740 4 client.18684 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.320+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562ca6f18850 con 0x562ca6fbffc0 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.320+0000 7fd77dc50640 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca6fbffc0 0x562ca6fc23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.320+0000 7fd77dc50640 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca6fbffc0 0x562ca6fc23b0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82f9c055299f6976 server_cookie=a04d13d2476ee252 in_seq=0 out_seq=0 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.320+0000 7fd7617fa640 10 client.18684 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd7617fa640 1 -- 172.21.15.37:0/3140192030 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562ca6f18850 con 0x562ca6fbffc0 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd7617fa640 10 client.18684 handle_command_reply: tid=0 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 client.18684 shutdown 2024-09-19T06:53:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fb0680 msgr2=0x562ca6fad760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fb0680 0x562ca6fad760 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fd77000e7a0 tx=0x7fd77000ec10 comp rx=0 tx=0).stop 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 shutdown_connections 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca6fbffc0 0x562ca6fc23b0 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fb0680 0x562ca6fad760 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa5dc0 0x562ca6fad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd7617fa640 0 client.18684 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 --2- 172.21.15.37:0/3140192030 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa54a0 0x562ca6faff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 >> 172.21.15.37:0/3140192030 conn(0x562ca6fa1a30 msgr2=0x562ca6fa0b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 shutdown_connections 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.321+0000 7fd78228d740 1 -- 172.21.15.37:0/3140192030 wait complete. 2024-09-19T06:53:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.322+0000 7fd78228d740 20 client.18684 trim_cache size 0 max 16384 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 Processor -- start 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 -- start start 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa54a0 0x562ca6fc7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa5dc0 0x562ca6fc8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fb0680 0x562ca6faa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781b0210 con 0x562ca6fa5dc0 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781adae0 con 0x562ca6fb0680 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd78228d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781ab950 con 0x562ca6fa54a0 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd77d44f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa5dc0 0x562ca6fc8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd77d44f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa5dc0 0x562ca6fc8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37442/0 (socket says 172.21.15.37:37442) 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd77d44f640 1 -- 172.21.15.37:0/2680801796 learned_addr learned my addr 172.21.15.37:0/2680801796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.339+0000 7fd77e451640 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fb0680 0x562ca6faa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77dc50640 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa54a0 0x562ca6fc7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77d44f640 1 -- 172.21.15.37:0/2680801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa54a0 msgr2=0x562ca6fc7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77d44f640 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa54a0 0x562ca6fc7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77d44f640 1 -- 172.21.15.37:0/2680801796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fb0680 msgr2=0x562ca6faa480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77d44f640 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fb0680 0x562ca6faa480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77d44f640 1 -- 172.21.15.37:0/2680801796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562ca6fc2720 con 0x562ca6fa5dc0 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77e451640 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fb0680 0x562ca6faa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd77d44f640 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa5dc0 0x562ca6fc8150 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd768009d50 tx=0x7fd768005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb7c341740ac0af server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd7637fe640 1 -- 172.21.15.37:0/2680801796 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd768005020 con 0x562ca6fa5dc0 2024-09-19T06:53:32.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd7637fe640 1 -- 172.21.15.37:0/2680801796 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7680056a0 con 0x562ca6fa5dc0 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.340+0000 7fd7637fe640 1 -- 172.21.15.37:0/2680801796 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76800f6a0 con 0x562ca6fa5dc0 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 -- 172.21.15.37:0/2680801796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa5dc0 msgr2=0x562ca6fc8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa5dc0 0x562ca6fc8150 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7fd768009d50 tx=0x7fd768005f90 comp rx=0 tx=0).stop 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 -- 172.21.15.37:0/2680801796 shutdown_connections 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fb0680 0x562ca6faa480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa5dc0 0x562ca6fc8150 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 --2- 172.21.15.37:0/2680801796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fa54a0 0x562ca6fc7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 -- 172.21.15.37:0/2680801796 >> 172.21.15.37:0/2680801796 conn(0x562ca6fa1a30 msgr2=0x562ca6faf2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 -- 172.21.15.37:0/2680801796 shutdown_connections 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 -- 172.21.15.37:0/2680801796 wait complete. 2024-09-19T06:53:32.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 -- start start 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.341+0000 7fd78228d740 1 Processor -- start 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fb8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb6230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781abfa0 con 0x562ca6fa54a0 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781ad410 con 0x562ca6fa5dc0 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd7781ac670 con 0x562ca6fb0680 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 10 client.0 _ll_register_callbacks cb 0x7fd7740daf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:32.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fb8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fb8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37446/0 (socket says 172.21.15.37:37446) 2024-09-19T06:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 -- 172.21.15.37:0/1113183558 learned_addr learned my addr 172.21.15.37:0/1113183558 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77d44f640 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77e451640 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 -- 172.21.15.37:0/1113183558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 msgr2=0x562ca6fb67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb67d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 -- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 msgr2=0x562ca6fb6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 -- 172.21.15.37:0/1113183558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562ca6fb7000 con 0x562ca6fa54a0 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd77dc50640 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fb8e90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd7740314b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8daa47c615f56e17 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd7617fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd7617fa640 1 -- 172.21.15.37:0/1113183558 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd774049040 con 0x562ca6fa54a0 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd7680056a0 con 0x562ca6fa54a0 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.342+0000 7fd7617fa640 1 -- 172.21.15.37:0/1113183558 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fd7740b8980 con 0x562ca6fa54a0 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd7617fa640 1 -- 172.21.15.37:0/1113183558 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fd7740c4f80 con 0x562ca6fa54a0 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 10 client.18690 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:32.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x562ca702acd0 con 0x562ca6fa54a0 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd7617fa640 1 -- 172.21.15.37:0/1113183558 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7680056a0 con 0x562ca6fa54a0 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd7617fa640 1 -- 172.21.15.37:0/1113183558 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fd7740d2040 con 0x562ca6fa54a0 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 10 client.18690 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 10 client.18690 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 20 client.18690 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca702b3f0 0x562ca702d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 4 client.18690 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562ca6fadaf0 con 0x562ca702b3f0 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.343+0000 7fd77d44f640 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca702b3f0 0x562ca702d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:32.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.344+0000 7fd7617fa640 10 client.18690 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.344+0000 7fd77d44f640 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca702b3f0 0x562ca702d7e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3677eb9b1c74984 server_cookie=d1c36deaa3c02cf0 in_seq=0 out_seq=0 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.344+0000 7fd7617fa640 1 -- 172.21.15.37:0/1113183558 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562ca6fadaf0 con 0x562ca702b3f0 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.344+0000 7fd7617fa640 10 client.18690 handle_command_reply: tid=0 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.344+0000 7fd78228d740 1 client.18690 shutdown 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.344+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 msgr2=0x562ca6fb8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.344+0000 7fd78228d740 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fb8e90 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7fd774000c00 tx=0x7fd7740314b0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 shutdown_connections 2024-09-19T06:53:32.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562ca702b3f0 0x562ca702d7e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562ca6fb0680 0x562ca6fb67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562ca6fa5dc0 0x562ca6fb6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 --2- 172.21.15.37:0/1113183558 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562ca6fa54a0 0x562ca6fb8e90 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd7617fa640 0 client.18690 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 >> 172.21.15.37:0/1113183558 conn(0x562ca6fa1a30 msgr2=0x562ca6fa0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 shutdown_connections 2024-09-19T06:53:32.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 1 -- 172.21.15.37:0/1113183558 wait complete. 2024-09-19T06:53:32.399 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:53:32.399 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:53:32.399 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:53:32.399 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:53:32.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd78228d740 20 client.18690 trim_cache size 0 max 16384 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7640a33b0 msgr2=0x7fd7640a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7640a33b0 0x7fd7640a5870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd770023000 tx=0x7fd770022000 comp rx=0 tx=0).stop 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 msgr2=0x7fd778198430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd778198430 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7fd774006010 tx=0x7fd774004040 comp rx=0 tx=0).stop 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 shutdown_connections 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd778106f10 0x7fd77819cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd778102790 0x7fd778198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd7640a33b0 0x7fd7640a5870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 --2- 172.21.15.37:0/1071845048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd778101de0 0x7fd778198430 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:32.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.345+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 >> 172.21.15.37:0/1071845048 conn(0x7fd7780fd8c0 msgr2=0x7fd7780fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:32.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.346+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 shutdown_connections 2024-09-19T06:53:32.401 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:32.346+0000 7fd77feb4640 1 -- 172.21.15.37:0/1071845048 wait complete. 2024-09-19T06:53:32.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:32 smithi096 ceph-mon[35004]: pgmap v369: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 258 op/s 2024-09-19T06:53:32.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:32 smithi037 ceph-mon[31211]: pgmap v369: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 258 op/s 2024-09-19T06:53:32.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:32 smithi151 ceph-mon[35236]: pgmap v369: 97 pgs: 97 active+clean; 78 MiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.5 MiB/s wr, 258 op/s 2024-09-19T06:53:33.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 882135 7.41 MB/sec execute 480 sec latency 18.610 ms 2024-09-19T06:53:33.379 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:53:33.380 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:53:33.644 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:34.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2860097038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8102790 msgr2=0x7f1da81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2860097038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8102790 0x7f1da81067e0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009900 tx=0x7f1d9002a410 comp rx=0 tx=0).stop 2024-09-19T06:53:34.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2860097038 shutdown_connections 2024-09-19T06:53:34.129 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2860097038 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da8106f10 0x7f1da810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2860097038 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8102790 0x7f1da81067e0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2860097038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da8101de0 0x7f1da81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2860097038 >> 172.21.15.37:0/2860097038 conn(0x7f1da80fd8c0 msgr2=0x7f1da80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2860097038 shutdown_connections 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.117+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2860097038 wait complete. 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 Processor -- start 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 -- start start 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8101de0 0x7f1da8198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da8102790 0x7f1da81988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da8106f10 0x7f1da819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8111f30 con 0x7f1da8101de0 2024-09-19T06:53:34.130 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1da8111db0 con 0x7f1da8102790 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1dad3e8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1da81120b0 con 0x7f1da8106f10 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1da6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8101de0 0x7f1da8198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1da6ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8101de0 0x7f1da8198380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37466/0 (socket says 172.21.15.37:37466) 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1da6ffd640 1 -- 172.21.15.37:0/2011370370 learned_addr learned my addr 172.21.15.37:0/2011370370 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.118+0000 7f1da67fc640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da8102790 0x7f1da81988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da77fe640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da8106f10 0x7f1da819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da6ffd640 1 -- 172.21.15.37:0/2011370370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da8106f10 msgr2=0x7f1da819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da6ffd640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da8106f10 0x7f1da819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da6ffd640 1 -- 172.21.15.37:0/2011370370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da8102790 msgr2=0x7f1da81988c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da6ffd640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da8102790 0x7f1da81988c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da6ffd640 1 -- 172.21.15.37:0/2011370370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1da819d330 con 0x7f1da8101de0 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da77fe640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da8106f10 0x7f1da819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:34.131 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1da6ffd640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8101de0 0x7f1da8198380 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c920 tx=0x7f1d9800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1d9ffff640 1 -- 172.21.15.37:0/2011370370 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98007c70 con 0x7f1da8101de0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1da819d620 con 0x7f1da8101de0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1d9ffff640 1 -- 172.21.15.37:0/2011370370 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1d98007e10 con 0x7f1da8101de0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1d9ffff640 1 -- 172.21.15.37:0/2011370370 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d98005260 con 0x7f1da8101de0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.119+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1da81a4f00 con 0x7f1da8101de0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.120+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1da81021c0 con 0x7f1da8101de0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.121+0000 7f1d9ffff640 1 -- 172.21.15.37:0/2011370370 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1d98005400 con 0x7f1da8101de0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.122+0000 7f1d9ffff640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1d940a33b0 0x7f1d940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.122+0000 7f1da67fc640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1d940a33b0 0x7f1d940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:34.132 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.122+0000 7f1d9ffff640 1 -- 172.21.15.37:0/2011370370 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f1d980cff00 con 0x7f1da8101de0 2024-09-19T06:53:34.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.123+0000 7f1da67fc640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1d940a33b0 0x7f1d940a5870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009230 tx=0x7f1d900058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:34.133 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.126+0000 7f1d9ffff640 1 -- 172.21.15.37:0/2011370370 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1d98014030 con 0x7f1da8101de0 2024-09-19T06:53:34.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 884139 7.42 MB/sec execute 481 sec latency 189.885 ms 2024-09-19T06:53:34.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.337+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1da8109b60 con 0x7f1da8101de0 2024-09-19T06:53:34.339 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.338+0000 7f1d9ffff640 1 -- 172.21.15.37:0/2011370370 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11903 (secure 0 0 0) 0x7f1d9809c850 con 0x7f1da8101de0 2024-09-19T06:53:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:53:34.340 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":39,"btime":"2024-09-19T06:50:41:875362+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:50:41.630153+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_26578":{"gid":26578,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2867779560","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2867779560},{"type":"v1","addr":"172.21.15.151:6839","nonce":2867779560}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:53:34.340 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 39 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1d940a33b0 msgr2=0x7f1d940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1d940a33b0 0x7f1d940a5870 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1d90009230 tx=0x7f1d900058d0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8101de0 msgr2=0x7f1da8198380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8101de0 0x7f1da8198380 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f1d9800c920 tx=0x7f1d9800cde0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 shutdown_connections 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1da8106f10 0x7f1da819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1da8102790 0x7f1da81988c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1d940a33b0 0x7f1d940a5870 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 --2- 172.21.15.37:0/2011370370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1da8101de0 0x7f1da8198380 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 >> 172.21.15.37:0/2011370370 conn(0x7f1da80fd8c0 msgr2=0x7f1da80fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:34.342 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 shutdown_connections 2024-09-19T06:53:34.343 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:34.340+0000 7f1dad3e8640 1 -- 172.21.15.37:0/2011370370 wait complete. 2024-09-19T06:53:34.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:34 smithi096 ceph-mon[35004]: pgmap v370: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 358 op/s 2024-09-19T06:53:34.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:34 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2011370370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:34.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:34 smithi037 ceph-mon[31211]: pgmap v370: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 358 op/s 2024-09-19T06:53:34.775 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:34 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2011370370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:34.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:34 smithi151 ceph-mon[35236]: pgmap v370: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 4.0 MiB/s wr, 358 op/s 2024-09-19T06:53:34.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:34 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2011370370' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:53:35.125 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:53:35.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 885709 7.42 MB/sec execute 482 sec latency 205.325 ms 2024-09-19T06:53:35.392 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:35.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1606667455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41082a0 msgr2=0x7f82e410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1606667455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41082a0 0x7f82e410ec40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f82dc009930 tx=0x7f82dc02a420 comp rx=0 tx=0).stop 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1606667455 shutdown_connections 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1606667455 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41082a0 0x7f82e410ec40 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1606667455 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82e41036e0 0x7f82e4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1606667455 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e4102d30 0x7f82e4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1606667455 >> 172.21.15.37:0/1606667455 conn(0x7f82e40fc7d0 msgr2=0x7f82e40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1606667455 shutdown_connections 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1606667455 wait complete. 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.858+0000 7f82eb3ad640 1 Processor -- start 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82eb3ad640 1 -- start start 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82eb3ad640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82e4102d30 0x7f82e4100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82eb3ad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41036e0 0x7f82e4100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82eb3ad640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e41082a0 0x7f82e4101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82eb3ad640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82e41120a0 con 0x7f82e41036e0 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82eb3ad640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4111f20 con 0x7f82e41082a0 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82eb3ad640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82e4112220 con 0x7f82e4102d30 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e8948640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41036e0 0x7f82e4100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e8948640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41036e0 0x7f82e4100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37488/0 (socket says 172.21.15.37:37488) 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e8948640 1 -- 172.21.15.37:0/1104701288 learned_addr learned my addr 172.21.15.37:0/1104701288 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e9149640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82e4102d30 0x7f82e4100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e994a640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e41082a0 0x7f82e4101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e9149640 1 -- 172.21.15.37:0/1104701288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e41082a0 msgr2=0x7f82e4101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e9149640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e41082a0 0x7f82e4101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e9149640 1 -- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41036e0 msgr2=0x7f82e4100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e9149640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41036e0 0x7f82e4100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.859+0000 7f82e9149640 1 -- 172.21.15.37:0/1104701288 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82e41a6a00 con 0x7f82e4102d30 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.860+0000 7f82e994a640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e41082a0 0x7f82e4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.860+0000 7f82e9149640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82e4102d30 0x7f82e4100990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f82d400eb00 tx=0x7f82d400c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.860+0000 7f82e27fc640 1 -- 172.21.15.37:0/1104701288 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4010430 con 0x7f82e4102d30 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.860+0000 7f82e27fc640 1 -- 172.21.15.37:0/1104701288 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d4010ab0 con 0x7f82e4102d30 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.860+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82e41a6cf0 con 0x7f82e4102d30 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.860+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82e41a71d0 con 0x7f82e4102d30 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.860+0000 7f82e27fc640 1 -- 172.21.15.37:0/1104701288 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4017bb0 con 0x7f82e4102d30 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.862+0000 7f82e27fc640 1 -- 172.21.15.37:0/1104701288 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f82d4021430 con 0x7f82e4102d30 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 Processor -- start 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 -- start start 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb21b40 0x55a0ccb20090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb20a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb2d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a0cca862f0 con 0x55a0ccb205d0 2024-09-19T06:53:35.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a0cca2dcf0 con 0x55a0ccb21b40 2024-09-19T06:53:35.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82ed786740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a0cca822b0 con 0x55a0ccb2b370 2024-09-19T06:53:35.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82e27fc640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f82cc0a33b0 0x7f82cc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82e9149640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb20a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82e994a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb21b40 0x55a0ccb20090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82e994a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb21b40 0x55a0ccb20090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38786/0 (socket says 172.21.15.37:38786) 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82e9149640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb20a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37490/0 (socket says 172.21.15.37:37490) 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82e994a640 1 -- 172.21.15.37:0/2388171820 learned_addr learned my addr 172.21.15.37:0/2388171820 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.863+0000 7f82e8948640 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb2d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e994a640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f82cc0a33b0 0x7f82cc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e27fc640 1 -- 172.21.15.37:0/1104701288 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f82d40d06b0 con 0x7f82e4102d30 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e9149640 1 -- 172.21.15.37:0/2388171820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 msgr2=0x55a0ccb2d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e9149640 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb2d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e9149640 1 -- 172.21.15.37:0/2388171820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb21b40 msgr2=0x55a0ccb20090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e9149640 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb21b40 0x55a0ccb20090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e9149640 1 -- 172.21.15.37:0/2388171820 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0cca123a0 con 0x55a0ccb205d0 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e9149640 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb20a10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f82d40cdd70 tx=0x7f82d400cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e6fdb5689c9274 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82e994a640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f82cc0a33b0 0x7f82cc0a5870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005c40 tx=0x7f82dc009260 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82cb7fe640 1 -- 172.21.15.37:0/2388171820 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d409c900 con 0x55a0ccb205d0 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82cb7fe640 1 -- 172.21.15.37:0/2388171820 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d406ae50 con 0x55a0ccb205d0 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.864+0000 7f82cb7fe640 1 -- 172.21.15.37:0/2388171820 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d4098f70 con 0x55a0ccb205d0 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 -- 172.21.15.37:0/2388171820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 msgr2=0x55a0ccb20a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb20a10 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f82d40cdd70 tx=0x7f82d400cf10 comp rx=0 tx=0).stop 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 -- 172.21.15.37:0/2388171820 shutdown_connections 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb2d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb20a10 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 --2- 172.21.15.37:0/2388171820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb21b40 0x55a0ccb20090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 -- 172.21.15.37:0/2388171820 >> 172.21.15.37:0/2388171820 conn(0x55a0ccae9310 msgr2=0x55a0ccb1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 -- 172.21.15.37:0/2388171820 shutdown_connections 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 -- 172.21.15.37:0/2388171820 wait complete. 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 -- start start 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.865+0000 7f82ed786740 1 Processor -- start 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82ed786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb2adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82ed786740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb28060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82ed786740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb2b370 0x55a0ccb287f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82ed786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a0ccb2e010 con 0x55a0ccb205d0 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82ed786740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a0cca41060 con 0x55a0ccb2b370 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82ed786740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a0ccb2e190 con 0x55a0ccb21b40 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82ed786740 10 client.0 _ll_register_callbacks cb 0x55a0cc7948c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:35.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb2adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb2adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37492/0 (socket says 172.21.15.37:37492) 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e994a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb2b370 0x55a0ccb287f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 -- 172.21.15.37:0/4192200926 learned_addr learned my addr 172.21.15.37:0/4192200926 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e8948640 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb28060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 -- 172.21.15.37:0/4192200926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 msgr2=0x55a0ccb28060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb28060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 -- 172.21.15.37:0/4192200926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb2b370 msgr2=0x55a0ccb287f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb2b370 0x55a0ccb287f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 -- 172.21.15.37:0/4192200926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0cc7f73c0 con 0x55a0ccb205d0 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82e9149640 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb2adb0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f82d4098d60 tx=0x7f82d40a5570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8010ef25af808f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.866+0000 7f82c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82c8ff9640 1 -- 172.21.15.37:0/4192200926 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d40b9020 con 0x55a0ccb205d0 2024-09-19T06:53:35.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d406ae50 con 0x55a0ccb205d0 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82c8ff9640 1 -- 172.21.15.37:0/4192200926 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f82d4087030 con 0x55a0ccb205d0 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82c8ff9640 1 -- 172.21.15.37:0/4192200926 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f82d40a0cb0 con 0x55a0ccb205d0 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 10 client.18708 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a0ccb39b30 con 0x55a0ccb205d0 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82c8ff9640 1 -- 172.21.15.37:0/4192200926 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d406ae50 con 0x55a0ccb205d0 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82c8ff9640 1 -- 172.21.15.37:0/4192200926 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f82d4062be0 con 0x55a0ccb205d0 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 10 client.18708 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 10 client.18708 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 20 client.18708 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccb3ae80 0x55a0ccb3d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 4 client.18708 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:35.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0cca93860 con 0x55a0ccb3ae80 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.867+0000 7f82e8948640 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccb3ae80 0x55a0ccb3d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.868+0000 7f82e8948640 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccb3ae80 0x55a0ccb3d270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cfd98f9d696481b8 server_cookie=9d62ebd99d4b2800 in_seq=0 out_seq=0 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.868+0000 7f82c8ff9640 10 client.18708 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.868+0000 7f82c8ff9640 1 -- 172.21.15.37:0/4192200926 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a0cca93860 con 0x55a0ccb3ae80 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.868+0000 7f82c8ff9640 10 client.18708 handle_command_reply: tid=0 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.868+0000 7f82ed786740 1 client.18708 shutdown 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 msgr2=0x55a0ccb2adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb2adb0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f82d4098d60 tx=0x7f82d40a5570 comp rx=0 tx=0).stop 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 shutdown_connections 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccb3ae80 0x55a0ccb3d270 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb2b370 0x55a0ccb287f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb28060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 --2- 172.21.15.37:0/4192200926 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb205d0 0x55a0ccb2adb0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82c8ff9640 0 client.18708 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:53:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 >> 172.21.15.37:0/4192200926 conn(0x55a0ccae9310 msgr2=0x55a0ccb22990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 shutdown_connections 2024-09-19T06:53:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 1 -- 172.21.15.37:0/4192200926 wait complete. 2024-09-19T06:53:35.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.869+0000 7f82ed786740 20 client.18708 trim_cache size 0 max 16384 2024-09-19T06:53:35.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.883+0000 7f82ed786740 1 Processor -- start 2024-09-19T06:53:35.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.883+0000 7f82ed786740 1 -- start start 2024-09-19T06:53:35.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82ed786740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb36b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82ed786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb21b40 0x55a0ccb252c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82ed786740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb25800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.887 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82ed786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82d00110d0 con 0x55a0ccb21b40 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82ed786740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000e9a0 con 0x55a0ccb205d0 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82ed786740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000c810 con 0x55a0ccb2b370 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e8948640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb21b40 0x55a0ccb252c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb36b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb36b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38804/0 (socket says 172.21.15.37:38804) 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e8948640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb21b40 0x55a0ccb252c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37494/0 (socket says 172.21.15.37:37494) 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 -- 172.21.15.37:0/707129401 learned_addr learned my addr 172.21.15.37:0/707129401 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e994a640 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb25800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 -- 172.21.15.37:0/707129401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 msgr2=0x55a0ccb25800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb25800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 -- 172.21.15.37:0/707129401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb21b40 msgr2=0x55a0ccb252c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb21b40 0x55a0ccb252c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.888 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 -- 172.21.15.37:0/707129401 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0ccb1d960 con 0x55a0ccb205d0 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.884+0000 7f82e9149640 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb36b60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82d409c600 tx=0x7f82d400cf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cd2dd2d5c22e8ee server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82caffd640 1 -- 172.21.15.37:0/707129401 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d40b9070 con 0x55a0ccb205d0 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82caffd640 1 -- 172.21.15.37:0/707129401 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d409c900 con 0x55a0ccb205d0 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82caffd640 1 -- 172.21.15.37:0/707129401 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d409bec0 con 0x55a0ccb205d0 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 -- 172.21.15.37:0/707129401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 msgr2=0x55a0ccb36b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb36b60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f82d409c600 tx=0x7f82d400cf10 comp rx=0 tx=0).stop 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 -- 172.21.15.37:0/707129401 shutdown_connections 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb2b370 0x55a0ccb25800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb21b40 0x55a0ccb252c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 --2- 172.21.15.37:0/707129401 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb36b60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 -- 172.21.15.37:0/707129401 >> 172.21.15.37:0/707129401 conn(0x55a0ccae9310 msgr2=0x55a0ccb36450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 -- 172.21.15.37:0/707129401 shutdown_connections 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.885+0000 7f82ed786740 1 -- 172.21.15.37:0/707129401 wait complete. 2024-09-19T06:53:35.889 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 -- start start 2024-09-19T06:53:35.890 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 Processor -- start 2024-09-19T06:53:35.892 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb3e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb3e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb2b370 0x55a0ccb3ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000ce60 con 0x55a0ccb2b370 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000e2d0 con 0x55a0ccb205d0 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f82d000d530 con 0x55a0ccb21b40 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82ed786740 10 client.0 _ll_register_callbacks cb 0x7f82dc009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82e994a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb2b370 0x55a0ccb3ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82e9149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb3e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.893 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82e994a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb2b370 0x55a0ccb3ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37510/0 (socket says 172.21.15.37:37510) 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82e8948640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb3e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.886+0000 7f82e9149640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb3e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38806/0 (socket says 172.21.15.37:38806) 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e994a640 1 -- 172.21.15.37:0/114496671 learned_addr learned my addr 172.21.15.37:0/114496671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e9149640 1 -- 172.21.15.37:0/114496671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 msgr2=0x55a0ccb3e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e9149640 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb3e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e9149640 1 -- 172.21.15.37:0/114496671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb2b370 msgr2=0x55a0ccb3ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e9149640 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb2b370 0x55a0ccb3ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e9149640 1 -- 172.21.15.37:0/114496671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0ccba4960 con 0x55a0ccb205d0 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e994a640 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb2b370 0x55a0ccb3ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e8948640 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb3e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82e9149640 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb3e1e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f82d4098d60 tx=0x7f82d409c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23fdec34d745cce4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82c8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:53:35.894 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82c8ff9640 1 -- 172.21.15.37:0/114496671 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82d40b9070 con 0x55a0ccb205d0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f82d409c900 con 0x55a0ccb205d0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82c8ff9640 1 -- 172.21.15.37:0/114496671 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f82d4062be0 con 0x55a0ccb205d0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82c8ff9640 1 -- 172.21.15.37:0/114496671 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f82d40a5570 con 0x55a0ccb205d0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82ed786740 10 client.28025 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.887+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55a0ccba4960 con 0x55a0ccb205d0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82c8ff9640 1 -- 172.21.15.37:0/114496671 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f82d409c900 con 0x55a0ccb205d0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82c8ff9640 1 -- 172.21.15.37:0/114496671 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f82d40980d0 con 0x55a0ccb205d0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82ed786740 10 client.28025 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82ed786740 10 client.28025 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82ed786740 20 client.28025 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82ed786740 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccba5ea0 0x55a0ccba8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82ed786740 4 client.28025 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a0cca8dd70 con 0x55a0ccba5ea0 2024-09-19T06:53:35.895 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82e8948640 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccba5ea0 0x55a0ccba8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82c8ff9640 10 client.28025 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.888+0000 7f82e8948640 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccba5ea0 0x55a0ccba8290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=12bffd4eaa1eb913 server_cookie=c195499759d2e1b4 in_seq=0 out_seq=0 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82c8ff9640 1 -- 172.21.15.37:0/114496671 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a0cca8dd70 con 0x55a0ccba5ea0 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82c8ff9640 10 client.28025 handle_command_reply: tid=0 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 client.28025 shutdown 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 msgr2=0x55a0ccb3e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb3e1e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f82d4098d60 tx=0x7f82d409c800 comp rx=0 tx=0).stop 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 shutdown_connections 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55a0ccba5ea0 0x55a0ccba8290 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a0ccb2b370 0x55a0ccb3ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a0ccb21b40 0x55a0ccb3e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.896 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82c8ff9640 0 client.28025 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:53:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 --2- 172.21.15.37:0/114496671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a0ccb205d0 0x55a0ccb3e1e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 >> 172.21.15.37:0/114496671 conn(0x55a0ccae9310 msgr2=0x55a0ccb36c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 shutdown_connections 2024-09-19T06:53:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 1 -- 172.21.15.37:0/114496671 wait complete. 2024-09-19T06:53:35.897 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.889+0000 7f82ed786740 20 client.28025 trim_cache size 0 max 16384 2024-09-19T06:53:35.897 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f82cc0a33b0 msgr2=0x7f82cc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f82cc0a33b0 0x7f82cc0a5870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f82dc005c40 tx=0x7f82dc009260 comp rx=0 tx=0).stop 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82e4102d30 msgr2=0x7f82e4100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82e4102d30 0x7f82e4100990 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f82d400eb00 tx=0x7f82d400c2d0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 shutdown_connections 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82e41082a0 0x7f82e4101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f82cc0a33b0 0x7f82cc0a5870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f82e41036e0 0x7f82e4100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 --2- 172.21.15.37:0/1104701288 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f82e4102d30 0x7f82e4100990 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:35.898 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 >> 172.21.15.37:0/1104701288 conn(0x7f82e40fc7d0 msgr2=0x7f82e4105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:35.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 shutdown_connections 2024-09-19T06:53:35.899 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:35.890+0000 7f82eb3ad640 1 -- 172.21.15.37:0/1104701288 wait complete. 2024-09-19T06:53:36.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 887792 7.44 MB/sec execute 483 sec latency 105.714 ms 2024-09-19T06:53:36.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:36 smithi096 ceph-mon[35004]: pgmap v371: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-09-19T06:53:36.774 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:36 smithi037 ceph-mon[31211]: pgmap v371: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-09-19T06:53:36.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:36 smithi151 ceph-mon[35236]: pgmap v371: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-09-19T06:53:36.868 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:53:37.143 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:37.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 889934 7.43 MB/sec execute 484 sec latency 17.891 ms 2024-09-19T06:53:37.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 -- 172.21.15.37:0/23106457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 msgr2=0x7fce8010bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 --2- 172.21.15.37:0/23106457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce8010bb50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fce74009900 tx=0x7fce7402a480 comp rx=0 tx=0).stop 2024-09-19T06:53:37.619 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 -- 172.21.15.37:0/23106457 shutdown_connections 2024-09-19T06:53:37.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 --2- 172.21.15.37:0/23106457 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce8010c170 0x7fce8010e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 --2- 172.21.15.37:0/23106457 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce8010bb50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 --2- 172.21.15.37:0/23106457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce80102d40 0x7fce80103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 -- 172.21.15.37:0/23106457 >> 172.21.15.37:0/23106457 conn(0x7fce800fc7a0 msgr2=0x7fce800febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 -- 172.21.15.37:0/23106457 shutdown_connections 2024-09-19T06:53:37.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.604+0000 7fce84da8640 1 -- 172.21.15.37:0/23106457 wait complete. 2024-09-19T06:53:37.620 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 Processor -- start 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 -- start start 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80102d40 0x7fce80198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce801989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce8010c170 0x7fce8019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce80111f70 con 0x7fce80103660 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce80111df0 con 0x7fce8010c170 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce84da8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce801120f0 con 0x7fce80102d40 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce801989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce801989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37526/0 (socket says 172.21.15.37:37526) 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce7dd9b640 1 -- 172.21.15.37:0/184303033 learned_addr learned my addr 172.21.15.37:0/184303033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce7ed9d640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce8010c170 0x7fce8019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.621 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.605+0000 7fce7e59c640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80102d40 0x7fce80198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce7dd9b640 1 -- 172.21.15.37:0/184303033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80102d40 msgr2=0x7fce80198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80102d40 0x7fce80198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce7dd9b640 1 -- 172.21.15.37:0/184303033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce8010c170 msgr2=0x7fce8019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce8010c170 0x7fce8019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce7dd9b640 1 -- 172.21.15.37:0/184303033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8019d440 con 0x7fce80103660 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce7ed9d640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce8010c170 0x7fce8019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce801989d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fce74006010 tx=0x7fce74004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce6f7fe640 1 -- 172.21.15.37:0/184303033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce7403f030 con 0x7fce80103660 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8019d6d0 con 0x7fce80103660 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce6f7fe640 1 -- 172.21.15.37:0/184303033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce74004440 con 0x7fce80103660 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce6f7fe640 1 -- 172.21.15.37:0/184303033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74035070 con 0x7fce80103660 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.606+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce801a5010 con 0x7fce80103660 2024-09-19T06:53:37.622 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.609+0000 7fce6f7fe640 1 -- 172.21.15.37:0/184303033 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fce74042030 con 0x7fce80103660 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.609+0000 7fce87181740 1 Processor -- start 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce87181740 1 -- start start 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce6f7fe640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce680a33b0 0x7fce680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce87181740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce87181740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd095ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce87181740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd0968760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce87181740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559bd08c12f0 con 0x559bd0966370 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce87181740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559bd0868cf0 con 0x559bd095cb40 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7ed9d640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce680a33b0 0x7fce680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.623 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce87181740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559bd08bd2b0 con 0x559bd095b5d0 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce6f7fe640 1 -- 172.21.15.37:0/184303033 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fce740f0100 con 0x7fce80103660 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38844/0 (socket says 172.21.15.37:38844) 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 -- 172.21.15.37:0/2385112371 learned_addr learned my addr 172.21.15.37:0/2385112371 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7ed9d640 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd0968760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd095ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 -- 172.21.15.37:0/2385112371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 msgr2=0x559bd095ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd095ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 -- 172.21.15.37:0/2385112371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 msgr2=0x559bd0968760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd0968760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.624 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.610+0000 7fce7e59c640 1 -- 172.21.15.37:0/2385112371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd084d3a0 con 0x559bd095cb40 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.611+0000 7fce7ed9d640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce680a33b0 0x7fce680a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fce7800a780 tx=0x7fce78014580 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.611+0000 7fce7e59c640 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095b090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce70009cf0 tx=0x7fce70011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a458ad86e42984 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.611+0000 7fce6cff9640 1 -- 172.21.15.37:0/2385112371 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70004230 con 0x559bd095cb40 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.611+0000 7fce6cff9640 1 -- 172.21.15.37:0/2385112371 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce700043d0 con 0x559bd095cb40 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.611+0000 7fce6cff9640 1 -- 172.21.15.37:0/2385112371 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce70019880 con 0x559bd095cb40 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 -- 172.21.15.37:0/2385112371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 msgr2=0x559bd095b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095b090 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fce70009cf0 tx=0x7fce70011810 comp rx=0 tx=0).stop 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 -- 172.21.15.37:0/2385112371 shutdown_connections 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd0968760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd095ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 --2- 172.21.15.37:0/2385112371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095b090 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 -- 172.21.15.37:0/2385112371 >> 172.21.15.37:0/2385112371 conn(0x559bd0924310 msgr2=0x559bd0957c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.625 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 -- 172.21.15.37:0/2385112371 shutdown_connections 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 -- 172.21.15.37:0/2385112371 wait complete. 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 -- start start 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 Processor -- start 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095b5d0 0x559bd0965cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd0962f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.612+0000 7fce87181740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd0966370 0x559bd09634a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce87181740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559bd0969070 con 0x559bd095cb40 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce87181740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559bd087c060 con 0x559bd095b5d0 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce87181740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559bd09691f0 con 0x559bd0966370 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd0962f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd0962f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37540/0 (socket says 172.21.15.37:37540) 2024-09-19T06:53:37.626 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095b5d0 0x559bd0965cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 -- 172.21.15.37:0/2254007395 learned_addr learned my addr 172.21.15.37:0/2254007395 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce87181740 10 client.0 _ll_register_callbacks cb 0x559bd05cf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7ed9d640 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd0966370 0x559bd09634a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 -- 172.21.15.37:0/2254007395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd0966370 msgr2=0x559bd09634a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd0966370 0x559bd09634a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 -- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095b5d0 msgr2=0x559bd0965cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095b5d0 0x559bd0965cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 -- 172.21.15.37:0/2254007395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd06323c0 con 0x559bd095cb40 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7e59c640 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095b5d0 0x559bd0965cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd0962f60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fce740bc100 tx=0x7fce74009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d194cc37b98ec92 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce65ffb640 1 -- 172.21.15.37:0/2254007395 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74049040 con 0x559bd095cb40 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce65ffb640 1 -- 172.21.15.37:0/2254007395 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fce740d52e0 con 0x559bd095cb40 2024-09-19T06:53:37.627 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce700043d0 con 0x559bd095cb40 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce65ffb640 1 -- 172.21.15.37:0/2254007395 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fce740c0700 con 0x559bd095cb40 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce87181740 10 client.18726 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.613+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bd0974a40 con 0x559bd095cb40 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce65ffb640 1 -- 172.21.15.37:0/2254007395 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce700043d0 con 0x559bd095cb40 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce65ffb640 1 -- 172.21.15.37:0/2254007395 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fce74031bd0 con 0x559bd095cb40 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce87181740 10 client.18726 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce87181740 10 client.18726 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce87181740 20 client.18726 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce87181740 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd0975f80 0x559bd09783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce87181740 4 client.18726 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559bd08ce860 con 0x559bd0975f80 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.614+0000 7fce7e59c640 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd0975f80 0x559bd09783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.628 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.615+0000 7fce7e59c640 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd0975f80 0x559bd09783d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f95d4e3bd4052344 server_cookie=4525312b84eac699 in_seq=0 out_seq=0 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.615+0000 7fce65ffb640 10 client.18726 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce65ffb640 1 -- 172.21.15.37:0/2254007395 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559bd08ce860 con 0x559bd0975f80 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce65ffb640 10 client.18726 handle_command_reply: tid=0 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 client.18726 shutdown 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 msgr2=0x559bd0962f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd0962f60 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fce740bc100 tx=0x7fce74009680 comp rx=0 tx=0).stop 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 shutdown_connections 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd0975f80 0x559bd09783d0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd0966370 0x559bd09634a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd0962f60 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce65ffb640 0 client.18726 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:53:37.629 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 --2- 172.21.15.37:0/2254007395 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095b5d0 0x559bd0965cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 >> 172.21.15.37:0/2254007395 conn(0x559bd0924310 msgr2=0x559bd095d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 shutdown_connections 2024-09-19T06:53:37.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 1 -- 172.21.15.37:0/2254007395 wait complete. 2024-09-19T06:53:37.630 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.616+0000 7fce87181740 20 client.18726 trim_cache size 0 max 16384 2024-09-19T06:53:37.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.629+0000 7fce87181740 1 Processor -- start 2024-09-19T06:53:37.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.629+0000 7fce87181740 1 -- start start 2024-09-19T06:53:37.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce87181740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0964ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce87181740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce87181740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd095e660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.637 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce87181740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce801b0240 con 0x559bd0966370 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce87181740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce801adb10 con 0x559bd095cb40 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce87181740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce801ab980 con 0x559bd095b5d0 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd095e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd095e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37546/0 (socket says 172.21.15.37:37546) 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7ed9d640 1 -- 172.21.15.37:0/1325650220 learned_addr learned my addr 172.21.15.37:0/1325650220 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7e59c640 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0964ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7dd9b640 1 -- 172.21.15.37:0/1325650220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 msgr2=0x559bd0964ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0964ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7dd9b640 1 -- 172.21.15.37:0/1325650220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 msgr2=0x559bd095e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd095e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7dd9b640 1 -- 172.21.15.37:0/1325650220 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559bd095eba0 con 0x559bd095cb40 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7e59c640 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0964ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7ed9d640 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd095e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095e120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fce740ba6b0 tx=0x7fce740bc300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f03145cca4f99b20 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.630+0000 7fce67fff640 1 -- 172.21.15.37:0/1325650220 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74049070 con 0x559bd095cb40 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce67fff640 1 -- 172.21.15.37:0/1325650220 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce740c4fc0 con 0x559bd095cb40 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce67fff640 1 -- 172.21.15.37:0/1325650220 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce740c4b30 con 0x559bd095cb40 2024-09-19T06:53:37.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 -- 172.21.15.37:0/1325650220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 msgr2=0x559bd095e120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095e120 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fce740ba6b0 tx=0x7fce740bc300 comp rx=0 tx=0).stop 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 -- 172.21.15.37:0/1325650220 shutdown_connections 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd0966370 0x559bd095e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd095cb40 0x559bd095e120 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 --2- 172.21.15.37:0/1325650220 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0964ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 -- 172.21.15.37:0/1325650220 >> 172.21.15.37:0/1325650220 conn(0x559bd0924310 msgr2=0x559bd09647c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 -- 172.21.15.37:0/1325650220 shutdown_connections 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.631+0000 7fce87181740 1 -- 172.21.15.37:0/1325650220 wait complete. 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 -- start start 2024-09-19T06:53:37.640 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 Processor -- start 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0969c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd096a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd0966370 0x559bd09d8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fce801abfd0 con 0x559bd095cb40 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fce801ad440 con 0x559bd0966370 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fce801ac6a0 con 0x559bd095b5d0 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce87181740 10 client.0 _ll_register_callbacks cb 0x7fce740bc1d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7ed9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd0966370 0x559bd09d8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd096a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7e59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0969c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd096a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37556/0 (socket says 172.21.15.37:37556) 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 -- 172.21.15.37:0/3346338459 learned_addr learned my addr 172.21.15.37:0/3346338459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:37.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 -- 172.21.15.37:0/3346338459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 msgr2=0x559bd0969c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0969c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 -- 172.21.15.37:0/3346338459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd0966370 msgr2=0x559bd09d8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:37.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd0966370 0x559bd09d8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7dd9b640 1 -- 172.21.15.37:0/3346338459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559bd09d9280 con 0x559bd095cb40 2024-09-19T06:53:37.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.632+0000 7fce7e59c640 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0969c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce7dd9b640 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd096a150 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fce7402a4c0 tx=0x7fce740d5460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b77042d778c3421 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce65ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce65ffb640 1 -- 172.21.15.37:0/3346338459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce74049030 con 0x559bd095cb40 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce65ffb640 1 -- 172.21.15.37:0/3346338459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fce740c4d30 con 0x559bd095cb40 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fce740c4fc0 con 0x559bd095cb40 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce65ffb640 1 -- 172.21.15.37:0/3346338459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fce740bb850 con 0x559bd095cb40 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 10 client.18732 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x559bd09e0610 con 0x559bd095cb40 2024-09-19T06:53:37.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce65ffb640 1 -- 172.21.15.37:0/3346338459 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fce740c4fc0 con 0x559bd095cb40 2024-09-19T06:53:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce65ffb640 1 -- 172.21.15.37:0/3346338459 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fce7403bab0 con 0x559bd095cb40 2024-09-19T06:53:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 10 client.18732 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 10 client.18732 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:53:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 20 client.18732 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd09e09f0 0x559bd09e2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 4 client.18732 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:37.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.633+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559bd08c8d70 con 0x559bd09e09f0 2024-09-19T06:53:37.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce7e59c640 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd09e09f0 0x559bd09e2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:37.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce65ffb640 10 client.18732 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:53:37.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce7e59c640 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd09e09f0 0x559bd09e2de0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=30634e8ecde89745 server_cookie=556defeff3939d7 in_seq=0 out_seq=0 2024-09-19T06:53:37.645 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce65ffb640 1 -- 172.21.15.37:0/3346338459 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559bd08c8d70 con 0x559bd09e09f0 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce65ffb640 10 client.18732 handle_command_reply: tid=0 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 client.18732 shutdown 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 msgr2=0x559bd096a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd096a150 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fce7402a4c0 tx=0x7fce740d5460 comp rx=0 tx=0).stop 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 shutdown_connections 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x559bd09e09f0 0x559bd09e2de0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559bd0966370 0x559bd09d8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x559bd095cb40 0x559bd096a150 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 --2- 172.21.15.37:0/3346338459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x559bd095b5d0 0x559bd0969c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 >> 172.21.15.37:0/3346338459 conn(0x559bd0924310 msgr2=0x559bd0964fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce65ffb640 0 client.18732 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:53:37.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 shutdown_connections 2024-09-19T06:53:37.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.634+0000 7fce87181740 1 -- 172.21.15.37:0/3346338459 wait complete. 2024-09-19T06:53:37.647 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce87181740 20 client.18732 trim_cache size 0 max 16384 2024-09-19T06:53:37.647 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:53:37.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce680a33b0 msgr2=0x7fce680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce680a33b0 0x7fce680a5870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fce7800a780 tx=0x7fce78014580 comp rx=0 tx=0).stop 2024-09-19T06:53:37.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 msgr2=0x7fce801989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:37.648 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce801989d0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7fce74006010 tx=0x7fce74004040 comp rx=0 tx=0).stop 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 shutdown_connections 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fce8010c170 0x7fce8019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fce680a33b0 0x7fce680a5870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fce80103660 0x7fce801989d0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 --2- 172.21.15.37:0/184303033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fce80102d40 0x7fce80198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 >> 172.21.15.37:0/184303033 conn(0x7fce800fc7a0 msgr2=0x7fce800fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 shutdown_connections 2024-09-19T06:53:37.649 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:37.635+0000 7fce84da8640 1 -- 172.21.15.37:0/184303033 wait complete. 2024-09-19T06:53:38.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 891531 7.44 MB/sec execute 485 sec latency 22.061 ms 2024-09-19T06:53:38.384 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:53:38.651 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:38.772 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:38 smithi037 ceph-mon[31211]: pgmap v372: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-09-19T06:53:38.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:38 smithi096 ceph-mon[35004]: pgmap v372: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-09-19T06:53:38.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:38 smithi151 ceph-mon[35236]: pgmap v372: 97 pgs: 97 active+clean; 84 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 3.0 MiB/s wr, 277 op/s 2024-09-19T06:53:39.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 -- 172.21.15.37:0/2680108287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 msgr2=0x7f7aec107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 --2- 172.21.15.37:0/2680108287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec107b90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4009930 tx=0x7f7ad402a420 comp rx=0 tx=0).stop 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 -- 172.21.15.37:0/2680108287 shutdown_connections 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 --2- 172.21.15.37:0/2680108287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aec1082a0 0x7f7aec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 --2- 172.21.15.37:0/2680108287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec107b90 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 --2- 172.21.15.37:0/2680108287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7aec102d30 0x7f7aec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 -- 172.21.15.37:0/2680108287 >> 172.21.15.37:0/2680108287 conn(0x7f7aec0fc7d0 msgr2=0x7f7aec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 -- 172.21.15.37:0/2680108287 shutdown_connections 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 -- 172.21.15.37:0/2680108287 wait complete. 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 Processor -- start 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.139+0000 7f7af19c0640 1 -- start start 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7af19c0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aec102d30 0x7f7aec100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7af19c0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7af19c0640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7aec1082a0 0x7f7aec101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7af19c0640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1120a0 con 0x7f7aec1036e0 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7af19c0640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec111f20 con 0x7f7aec102d30 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7af19c0640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec112220 con 0x7f7aec1082a0 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37570/0 (socket says 172.21.15.37:37570) 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 -- 172.21.15.37:0/868775331 learned_addr learned my addr 172.21.15.37:0/868775331 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7aeaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aec102d30 0x7f7aec100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7aec1082a0 0x7f7aec101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 -- 172.21.15.37:0/868775331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7aec1082a0 msgr2=0x7f7aec101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7aec1082a0 0x7f7aec101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 -- 172.21.15.37:0/868775331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aec102d30 msgr2=0x7f7aec100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aec102d30 0x7f7aec100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7ae27fc640 1 -- 172.21.15.37:0/868775331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aec1a6a80 con 0x7f7aec1036e0 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aec102d30 0x7f7aec100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.140+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7aec1082a0 0x7f7aec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.141+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec100ed0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005940 tx=0x7f7ad40040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.141+0000 7f7af09be640 1 -- 172.21.15.37:0/868775331 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad403f030 con 0x7f7aec1036e0 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.141+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aec1a6d10 con 0x7f7aec1036e0 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.141+0000 7f7af09be640 1 -- 172.21.15.37:0/868775331 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad40044c0 con 0x7f7aec1036e0 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.141+0000 7f7af09be640 1 -- 172.21.15.37:0/868775331 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4035070 con 0x7f7aec1036e0 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.141+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aec1a71e0 con 0x7f7aec1036e0 2024-09-19T06:53:39.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.144+0000 7f7af09be640 1 -- 172.21.15.37:0/868775331 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7ad4042030 con 0x7f7aec1036e0 2024-09-19T06:53:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.144+0000 7f7af3d99740 1 Processor -- start 2024-09-19T06:53:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af3d99740 1 -- start start 2024-09-19T06:53:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549322090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x562549322a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56254932d370 0x56254932f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af3d99740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5625492882f0 con 0x56254932d370 2024-09-19T06:53:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af09be640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ad80a33b0 0x7f7ad80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af3d99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56254922fcf0 con 0x5625493225d0 2024-09-19T06:53:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af3d99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5625492842b0 con 0x562549323b40 2024-09-19T06:53:39.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56254932d370 0x56254932f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ad80a33b0 0x7f7ad80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56254932d370 0x56254932f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37576/0 (socket says 172.21.15.37:37576) 2024-09-19T06:53:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 -- 172.21.15.37:0/1400447514 learned_addr learned my addr 172.21.15.37:0/1400447514 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x562549322a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549322090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7af09be640 1 -- 172.21.15.37:0/868775331 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f7ad40f0170 con 0x7f7aec1036e0 2024-09-19T06:53:39.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 -- 172.21.15.37:0/1400447514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 msgr2=0x562549322090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549322090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 -- 172.21.15.37:0/1400447514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 msgr2=0x562549322a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x562549322a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7ae27fc640 1 -- 172.21.15.37:0/1400447514 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5625492143a0 con 0x56254932d370 2024-09-19T06:53:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.145+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x562549322a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:39.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.146+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56254932d370 0x56254932f760 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40ed7a0 tx=0x7f7ad402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3041232d47d52873 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.146+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ad80a33b0 0x7f7ad80a5870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009270 tx=0x7f7ae4009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.146+0000 7f7ae1ffb640 1 -- 172.21.15.37:0/1400447514 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4049070 con 0x56254932d370 2024-09-19T06:53:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.146+0000 7f7ae1ffb640 1 -- 172.21.15.37:0/1400447514 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad40bcac0 con 0x56254932d370 2024-09-19T06:53:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.146+0000 7f7ae1ffb640 1 -- 172.21.15.37:0/1400447514 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40c5030 con 0x56254932d370 2024-09-19T06:53:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 -- 172.21.15.37:0/1400447514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56254932d370 msgr2=0x56254932f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56254932d370 0x56254932f760 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40ed7a0 tx=0x7f7ad402a420 comp rx=0 tx=0).stop 2024-09-19T06:53:39.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 -- 172.21.15.37:0/1400447514 shutdown_connections 2024-09-19T06:53:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56254932d370 0x56254932f760 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x562549322a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 --2- 172.21.15.37:0/1400447514 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549322090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 -- 172.21.15.37:0/1400447514 >> 172.21.15.37:0/1400447514 conn(0x5625492eb310 msgr2=0x56254931ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 -- 172.21.15.37:0/1400447514 shutdown_connections 2024-09-19T06:53:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 -- 172.21.15.37:0/1400447514 wait complete. 2024-09-19T06:53:39.163 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.147+0000 7f7af3d99740 1 -- start start 2024-09-19T06:53:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 1 Processor -- start 2024-09-19T06:53:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254932cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549329fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x56254932a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56254932fe30 con 0x562549323b40 2024-09-19T06:53:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562549243060 con 0x5625493225d0 2024-09-19T06:53:39.164 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56254932ffb0 con 0x56254932d370 2024-09-19T06:53:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7af3d99740 10 client.0 _ll_register_callbacks cb 0x562548f968c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549329fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549329fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37586/0 (socket says 172.21.15.37:37586) 2024-09-19T06:53:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 -- 172.21.15.37:0/596137145 learned_addr learned my addr 172.21.15.37:0/596137145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7aeaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254932cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.165 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x56254932a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 -- 172.21.15.37:0/596137145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 msgr2=0x56254932a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x56254932a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 -- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 msgr2=0x56254932cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254932cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.166 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7ae27fc640 1 -- 172.21.15.37:0/596137145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562548ff93c0 con 0x562549323b40 2024-09-19T06:53:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254932cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.148+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x56254932a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549329fb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40bc0c0 tx=0x7f7ad40c0870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e80ea8d494a9658 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7acaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7acaffd640 1 -- 172.21.15.37:0/596137145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4049050 con 0x562549323b40 2024-09-19T06:53:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7acaffd640 1 -- 172.21.15.37:0/596137145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f7ad40cec90 con 0x562549323b40 2024-09-19T06:53:39.167 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad40bcac0 con 0x562549323b40 2024-09-19T06:53:39.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7acaffd640 1 -- 172.21.15.37:0/596137145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7ad40c5030 con 0x562549323b40 2024-09-19T06:53:39.168 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7af3d99740 10 client.18744 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56254932fe30 con 0x562549323b40 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.149+0000 7f7acaffd640 1 -- 172.21.15.37:0/596137145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad40bcac0 con 0x562549323b40 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7acaffd640 1 -- 172.21.15.37:0/596137145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7ad40ceec0 con 0x562549323b40 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7af3d99740 10 client.18744 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7af3d99740 10 client.18744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7af3d99740 20 client.18744 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7af3d99740 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56254933c960 0x56254933ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7af3d99740 4 client.18744 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562549295860 con 0x56254933c960 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56254933c960 0x56254933ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7acaffd640 10 client.18744 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.169 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.150+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56254933c960 0x56254933ed80 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c4c2168831c7d29 server_cookie=d16cd40b39fa537c in_seq=0 out_seq=0 2024-09-19T06:53:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.151+0000 7f7acaffd640 1 -- 172.21.15.37:0/596137145 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562549295860 con 0x56254933c960 2024-09-19T06:53:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.151+0000 7f7acaffd640 10 client.18744 handle_command_reply: tid=0 2024-09-19T06:53:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.151+0000 7f7af3d99740 1 client.18744 shutdown 2024-09-19T06:53:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 msgr2=0x562549329fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549329fb0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40bc0c0 tx=0x7f7ad40c0870 comp rx=0 tx=0).stop 2024-09-19T06:53:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 shutdown_connections 2024-09-19T06:53:39.170 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56254933c960 0x56254933ed80 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x56254932a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549329fb0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 --2- 172.21.15.37:0/596137145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254932cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 >> 172.21.15.37:0/596137145 conn(0x5625492eb310 msgr2=0x562549324990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7acaffd640 0 client.18744 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 shutdown_connections 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 1 -- 172.21.15.37:0/596137145 wait complete. 2024-09-19T06:53:39.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.152+0000 7f7af3d99740 20 client.18744 trim_cache size 0 max 16384 2024-09-19T06:53:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.165+0000 7f7af3d99740 1 Processor -- start 2024-09-19T06:53:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.165+0000 7f7af3d99740 1 -- start start 2024-09-19T06:53:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254931ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.176 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549339000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x562549328e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7af3d99740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1b2410 con 0x562549323b40 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7af3d99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1afce0 con 0x5625493225d0 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7af3d99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1adb50 con 0x56254932d370 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549339000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549339000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37588/0 (socket says 172.21.15.37:37588) 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7aeaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254931ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 -- 172.21.15.37:0/2022237538 learned_addr learned my addr 172.21.15.37:0/2022237538 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x562549328e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 -- 172.21.15.37:0/2022237538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 msgr2=0x562549328e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x562549328e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.177 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 -- 172.21.15.37:0/2022237538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 msgr2=0x56254931ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254931ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 -- 172.21.15.37:0/2022237538 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562549325120 con 0x562549323b40 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.166+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549339000 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b8d90 tx=0x7f7ad402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a76006a28541216 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.167+0000 7f7ae17fa640 1 -- 172.21.15.37:0/2022237538 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40bcac0 con 0x562549323b40 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.167+0000 7f7ae17fa640 1 -- 172.21.15.37:0/2022237538 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad40d51f0 con 0x562549323b40 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.167+0000 7f7ae17fa640 1 -- 172.21.15.37:0/2022237538 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ad403bb70 con 0x562549323b40 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.167+0000 7f7af3d99740 1 -- 172.21.15.37:0/2022237538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 msgr2=0x562549339000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.167+0000 7f7af3d99740 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549339000 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7ad40b8d90 tx=0x7f7ad402a420 comp rx=0 tx=0).stop 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 -- 172.21.15.37:0/2022237538 shutdown_connections 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56254932d370 0x562549328e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.178 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562549323b40 0x562549339000 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 --2- 172.21.15.37:0/2022237538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5625493225d0 0x56254931ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 -- 172.21.15.37:0/2022237538 >> 172.21.15.37:0/2022237538 conn(0x5625492eb310 msgr2=0x562549328750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 -- 172.21.15.37:0/2022237538 shutdown_connections 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 -- 172.21.15.37:0/2022237538 wait complete. 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 -- start start 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.168+0000 7f7af3d99740 1 Processor -- start 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625493225d0 0x562549335a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549330110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7af3d99740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56254932d370 0x562549330850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7af3d99740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1ae1a0 con 0x5625493225d0 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7af3d99740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1af610 con 0x56254932d370 2024-09-19T06:53:39.179 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7af3d99740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7aec1ae870 con 0x562549323b40 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7af3d99740 10 client.0 _ll_register_callbacks cb 0x7f7adc00bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625493225d0 0x562549335a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625493225d0 0x562549335a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37602/0 (socket says 172.21.15.37:37602) 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 -- 172.21.15.37:0/308641094 learned_addr learned my addr 172.21.15.37:0/308641094 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeb7fe640 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56254932d370 0x562549330850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549330110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 -- 172.21.15.37:0/308641094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 msgr2=0x562549330110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.180 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549330110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 -- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56254932d370 msgr2=0x562549330850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56254932d370 0x562549330850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 -- 172.21.15.37:0/308641094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5625493a55f0 con 0x5625493225d0 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7aeaffd640 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625493225d0 0x562549335a20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x56254933f5d0 tx=0x7f7adc012b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e146073bf77d78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.169+0000 7f7acaffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7acaffd640 1 -- 172.21.15.37:0/308641094 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7adc01ec30 con 0x5625493225d0 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7acaffd640 1 -- 172.21.15.37:0/308641094 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f7adc01edd0 con 0x5625493225d0 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ad40d51f0 con 0x5625493225d0 2024-09-19T06:53:39.181 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7acaffd640 1 -- 172.21.15.37:0/308641094 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f7adc032050 con 0x5625493225d0 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 10 client.18750 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5625493a6770 con 0x5625493225d0 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7acaffd640 1 -- 172.21.15.37:0/308641094 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ad40d51f0 con 0x5625493225d0 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7acaffd640 1 -- 172.21.15.37:0/308641094 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f7adc0047a0 con 0x5625493225d0 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 10 client.18750 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 10 client.18750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 20 client.18750 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5625493a6e90 0x5625493a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 4 client.18750 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.170+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56254928fd70 con 0x5625493a6e90 2024-09-19T06:53:39.182 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.171+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5625493a6e90 0x5625493a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.171+0000 7f7ae27fc640 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5625493a6e90 0x5625493a9280 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20673d5a323d6e8d server_cookie=d968a4fae4af910d in_seq=0 out_seq=0 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.171+0000 7f7acaffd640 10 client.18750 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7acaffd640 1 -- 172.21.15.37:0/308641094 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56254928fd70 con 0x5625493a6e90 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7acaffd640 10 client.18750 handle_command_reply: tid=0 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 client.18750 shutdown 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625493225d0 msgr2=0x562549335a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625493225d0 0x562549335a20 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x56254933f5d0 tx=0x7f7adc012b80 comp rx=0 tx=0).stop 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 shutdown_connections 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5625493a6e90 0x5625493a9280 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56254932d370 0x562549330850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562549323b40 0x562549330110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.183 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 --2- 172.21.15.37:0/308641094 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5625493225d0 0x562549335a20 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7acaffd640 0 client.18750 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 >> 172.21.15.37:0/308641094 conn(0x5625492eb310 msgr2=0x562549328f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 shutdown_connections 2024-09-19T06:53:39.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 1 -- 172.21.15.37:0/308641094 wait complete. 2024-09-19T06:53:39.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.173+0000 7f7af3d99740 20 client.18750 trim_cache size 0 max 16384 2024-09-19T06:53:39.184 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:53:39.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ad80a33b0 msgr2=0x7f7ad80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ad80a33b0 0x7f7ad80a5870 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009270 tx=0x7f7ae4009500 comp rx=0 tx=0).stop 2024-09-19T06:53:39.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 msgr2=0x7f7aec100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec100ed0 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f7ad4005940 tx=0x7f7ad40040c0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 shutdown_connections 2024-09-19T06:53:39.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7aec1082a0 0x7f7aec101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7ad80a33b0 0x7f7ad80a5870 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7aec1036e0 0x7f7aec100ed0 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 --2- 172.21.15.37:0/868775331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aec102d30 0x7f7aec100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 >> 172.21.15.37:0/868775331 conn(0x7f7aec0fc7d0 msgr2=0x7f7aec105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 shutdown_connections 2024-09-19T06:53:39.186 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:39.174+0000 7f7af19c0640 1 -- 172.21.15.37:0/868775331 wait complete. 2024-09-19T06:53:39.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 894099 7.46 MB/sec execute 486 sec latency 15.820 ms 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 -- 172.21.15.37:0/1353270990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498101ad0 msgr2=0x7f5498101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 --2- 172.21.15.37:0/1353270990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498101ad0 0x7f5498101f30 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f548c009930 tx=0x7f548c02a420 comp rx=0 tx=0).stop 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 -- 172.21.15.37:0/1353270990 shutdown_connections 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 --2- 172.21.15.37:0/1353270990 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5498102470 0x7f549810e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 --2- 172.21.15.37:0/1353270990 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498101ad0 0x7f5498101f30 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 --2- 172.21.15.37:0/1353270990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5498107b70 0x7f5498107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 -- 172.21.15.37:0/1353270990 >> 172.21.15.37:0/1353270990 conn(0x7f54980fd7d0 msgr2=0x7f54980ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 -- 172.21.15.37:0/1353270990 shutdown_connections 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 -- 172.21.15.37:0/1353270990 wait complete. 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 Processor -- start 2024-09-19T06:53:39.237 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.235+0000 7f549f465640 1 -- start start 2024-09-19T06:53:39.246 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549f465640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5498101ad0 0x7f54980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549f465640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498102470 0x7f5498074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549f465640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5498107b70 0x7f5498074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549f465640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5498111f30 con 0x7f5498102470 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549f465640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5498111db0 con 0x7f5498107b70 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549f465640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54981120b0 con 0x7f5498101ad0 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498102470 0x7f5498074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498102470 0x7f5498074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37628/0 (socket says 172.21.15.37:37628) 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 -- 172.21.15.37:0/1895175055 learned_addr learned my addr 172.21.15.37:0/1895175055 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549da02640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5498107b70 0x7f5498074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.247 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549d201640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5498101ad0 0x7f54980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 -- 172.21.15.37:0/1895175055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5498101ad0 msgr2=0x7f54980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5498101ad0 0x7f54980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 -- 172.21.15.37:0/1895175055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5498107b70 msgr2=0x7f5498074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5498107b70 0x7f5498074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 -- 172.21.15.37:0/1895175055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54981a6850 con 0x7f5498102470 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549d201640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5498101ad0 0x7f54980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549ca00640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498102470 0x7f5498074a30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f548c02a940 tx=0x7f548c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f548a7fc640 1 -- 172.21.15.37:0/1895175055 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c03f070 con 0x7f5498102470 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f548a7fc640 1 -- 172.21.15.37:0/1895175055 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f548c031c90 con 0x7f5498102470 2024-09-19T06:53:39.248 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.236+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54981a6ae0 con 0x7f5498102470 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.237+0000 7f548a7fc640 1 -- 172.21.15.37:0/1895175055 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c035030 con 0x7f5498102470 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.237+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54981a6fd0 con 0x7f5498102470 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.239+0000 7f548a7fc640 1 -- 172.21.15.37:0/1895175055 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f548c042070 con 0x7f5498102470 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.239+0000 7f54a2205740 1 Processor -- start 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.239+0000 7f54a2205740 1 -- start start 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f54a2205740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56e99ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f54a2205740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f54a2205740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ea70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f54a2205740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563b56dfe2f0 con 0x563b56e99af0 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f54a2205740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b56da6270 con 0x563b56e9a410 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f54a2205740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563b56dfa2b0 con 0x563b56ea4cb0 2024-09-19T06:53:39.249 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56e99ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56e99ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37644/0 (socket says 172.21.15.37:37644) 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 -- 172.21.15.37:0/1898799488 learned_addr learned my addr 172.21.15.37:0/1898799488 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f548a7fc640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f54840a3460 0x7f54840a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549d201640 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549ca00640 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ea70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f54840a3460 0x7f54840a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 -- 172.21.15.37:0/1898799488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 msgr2=0x563b56ea70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ea70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 -- 172.21.15.37:0/1898799488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 msgr2=0x563b56ea4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.250 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f548a7fc640 1 -- 172.21.15.37:0/1895175055 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f548c0f0440 con 0x7f5498102470 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 -- 172.21.15.37:0/1898799488 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b56da5340 con 0x563b56e99af0 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549d201640 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.240+0000 7f549da02640 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56e99ed0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f5490013b20 tx=0x7f549000f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a884569b6e95557f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54837fe640 1 -- 172.21.15.37:0/1898799488 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490022020 con 0x563b56e99af0 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54837fe640 1 -- 172.21.15.37:0/1898799488 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5490010430 con 0x563b56e99af0 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54837fe640 1 -- 172.21.15.37:0/1898799488 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5490013560 con 0x563b56e99af0 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f549da02640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f54840a3460 0x7f54840a5920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5490023000 tx=0x7f5490021000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54a2205740 1 -- 172.21.15.37:0/1898799488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 msgr2=0x563b56e99ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54a2205740 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56e99ed0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f5490013b20 tx=0x7f549000f3c0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54a2205740 1 -- 172.21.15.37:0/1898799488 shutdown_connections 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54a2205740 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ea70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.251 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54a2205740 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.241+0000 7f54a2205740 1 --2- 172.21.15.37:0/1898799488 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56e99ed0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 -- 172.21.15.37:0/1898799488 >> 172.21.15.37:0/1898799488 conn(0x563b56e96060 msgr2=0x563b56e96440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 -- 172.21.15.37:0/1898799488 shutdown_connections 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 -- 172.21.15.37:0/1898799488 wait complete. 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 -- start start 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 Processor -- start 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e99af0 0x563b56ea1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e9a410 0x563b56ea20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.242+0000 7f54a2205740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ea2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f54a2205740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563b56ea7880 con 0x563b56e9a410 2024-09-19T06:53:39.252 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f54a2205740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563b56d87290 con 0x563b56e99af0 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f54a2205740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563b56ea7a00 con 0x563b56ea4cb0 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f54a2205740 10 client.0 _ll_register_callbacks cb 0x563b56db8140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549d201640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e99af0 0x563b56ea1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549ca00640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e9a410 0x563b56ea20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549d201640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e99af0 0x563b56ea1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:38960/0 (socket says 172.21.15.37:38960) 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549d201640 1 -- 172.21.15.37:0/3059730273 learned_addr learned my addr 172.21.15.37:0/3059730273 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549ca00640 1 -- 172.21.15.37:0/3059730273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 msgr2=0x563b56ea2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549ca00640 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ea2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549ca00640 1 -- 172.21.15.37:0/3059730273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e99af0 msgr2=0x563b56ea1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549ca00640 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e99af0 0x563b56ea1b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549ca00640 1 -- 172.21.15.37:0/3059730273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b56b11210 con 0x563b56e9a410 2024-09-19T06:53:39.253 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f549ca00640 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e9a410 0x563b56ea20d0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f548c0edb30 tx=0x7f548c08b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9a63ed05c3eaf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f5480ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f5480ff9640 1 -- 172.21.15.37:0/3059730273 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f548c049040 con 0x563b56e9a410 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f5480ff9640 1 -- 172.21.15.37:0/3059730273 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f548c0b8d60 con 0x563b56e9a410 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5490010430 con 0x563b56e9a410 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f5480ff9640 1 -- 172.21.15.37:0/3059730273 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f548c087540 con 0x563b56e9a410 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.243+0000 7f54a2205740 10 client.18762 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563b56bdd620 con 0x563b56e9a410 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f5480ff9640 1 -- 172.21.15.37:0/3059730273 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5490010430 con 0x563b56e9a410 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f5480ff9640 1 -- 172.21.15.37:0/3059730273 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f548c08b0a0 con 0x563b56e9a410 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f54a2205740 10 client.18762 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f54a2205740 10 client.18762 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:39.254 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f54a2205740 20 client.18762 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f54a2205740 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56eb4b90 0x563b56eb6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f54a2205740 4 client.18762 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b56e0ba20 con 0x563b56eb4b90 2024-09-19T06:53:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f549d201640 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56eb4b90 0x563b56eb6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f5480ff9640 10 client.18762 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.255 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.244+0000 7f549d201640 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56eb4b90 0x563b56eb6fa0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ee0536937898782 server_cookie=efd651eaffe2b714 in_seq=0 out_seq=0 2024-09-19T06:53:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f5480ff9640 1 -- 172.21.15.37:0/3059730273 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563b56e0ba20 con 0x563b56eb4b90 2024-09-19T06:53:39.256 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f5480ff9640 10 client.18762 handle_command_reply: tid=0 2024-09-19T06:53:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 client.18762 shutdown 2024-09-19T06:53:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e9a410 msgr2=0x563b56ea20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e9a410 0x563b56ea20d0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f548c0edb30 tx=0x7f548c08b3e0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 shutdown_connections 2024-09-19T06:53:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56eb4b90 0x563b56eb6fa0 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.257 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ea2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e9a410 0x563b56ea20d0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f5480ff9640 0 client.18762 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 --2- 172.21.15.37:0/3059730273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e99af0 0x563b56ea1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.245+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 >> 172.21.15.37:0/3059730273 conn(0x563b56e96060 msgr2=0x563b56e9bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.246+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 shutdown_connections 2024-09-19T06:53:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.246+0000 7f54a2205740 1 -- 172.21.15.37:0/3059730273 wait complete. 2024-09-19T06:53:39.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.246+0000 7f54a2205740 20 client.18762 trim_cache size 0 max 16384 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 Processor -- start 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 -- start start 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ebc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ebcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54981b21c0 con 0x563b56e99af0 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54981afa90 con 0x563b56e9a410 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f54a2205740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54981ad900 con 0x563b56ea4cb0 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.260+0000 7f549d201640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.264 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37668/0 (socket says 172.21.15.37:37668) 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 -- 172.21.15.37:0/3731554966 learned_addr learned my addr 172.21.15.37:0/3731554966 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549ca00640 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ebc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549da02640 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ebcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 -- 172.21.15.37:0/3731554966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 msgr2=0x563b56ebcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ebcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 -- 172.21.15.37:0/3731554966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 msgr2=0x563b56ebc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ebc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 -- 172.21.15.37:0/3731554966 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b56eb75d0 con 0x563b56e99af0 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f549d201640 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea2440 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5494005f00 tx=0x7f549400b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a393793dc039fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f5482ffd640 1 -- 172.21.15.37:0/3731554966 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004030 con 0x563b56e99af0 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f5482ffd640 1 -- 172.21.15.37:0/3731554966 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494005510 con 0x563b56e99af0 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.261+0000 7f5482ffd640 1 -- 172.21.15.37:0/3731554966 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004ce0 con 0x563b56e99af0 2024-09-19T06:53:39.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 -- 172.21.15.37:0/3731554966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 msgr2=0x563b56ea2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea2440 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f5494005f00 tx=0x7f549400b550 comp rx=0 tx=0).stop 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 -- 172.21.15.37:0/3731554966 shutdown_connections 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56ebcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ebc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 --2- 172.21.15.37:0/3731554966 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea2440 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 -- 172.21.15.37:0/3731554966 >> 172.21.15.37:0/3731554966 conn(0x563b56e96060 msgr2=0x563b56ea3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 -- 172.21.15.37:0/3731554966 shutdown_connections 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 -- 172.21.15.37:0/3731554966 wait complete. 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 -- start start 2024-09-19T06:53:39.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.262+0000 7f54a2205740 1 Processor -- start 2024-09-19T06:53:39.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea9660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56f1e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f54981adf50 con 0x563b56e99af0 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f54981af3c0 con 0x563b56e9a410 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f54981ae620 con 0x563b56ea4cb0 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 10 client.0 _ll_register_callbacks cb 0x7f548c031490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:37682/0 (socket says 172.21.15.37:37682) 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 -- 172.21.15.37:0/179677823 learned_addr learned my addr 172.21.15.37:0/179677823 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:39.270 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549ca00640 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549da02640 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56f1e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 -- 172.21.15.37:0/179677823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 msgr2=0x563b56f1e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56f1e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 -- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 msgr2=0x563b56ea9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea9660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 -- 172.21.15.37:0/179677823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b56e9c940 con 0x563b56e99af0 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f549d201640 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea9120 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f549400bc70 tx=0x7f549400bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=295b1e6814e6afc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f5480ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:39.271 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f5480ff9640 1 -- 172.21.15.37:0/179677823 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5494004440 con 0x563b56e99af0 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f5480ff9640 1 -- 172.21.15.37:0/179677823 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f54940035f0 con 0x563b56e99af0 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.263+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5494005510 con 0x563b56e99af0 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f5480ff9640 1 -- 172.21.15.37:0/179677823 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f5494036050 con 0x563b56e99af0 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 10 client.18768 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x563b56f1fe90 con 0x563b56e99af0 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f5480ff9640 1 -- 172.21.15.37:0/179677823 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5494005510 con 0x563b56e99af0 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f5480ff9640 1 -- 172.21.15.37:0/179677823 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f549401d940 con 0x563b56e99af0 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 10 client.18768 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 10 client.18768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 20 client.18768 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56f20540 0x563b56f22960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:39.272 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 4 client.18768 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b56e05d70 con 0x563b56f20540 2024-09-19T06:53:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.264+0000 7f549ca00640 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56f20540 0x563b56f22960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.265+0000 7f5480ff9640 10 client.18768 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.265+0000 7f549ca00640 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56f20540 0x563b56f22960 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22b27258d818fdf8 server_cookie=46437fc6d9e6d18c in_seq=0 out_seq=0 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.420+0000 7f5480ff9640 1 -- 172.21.15.37:0/179677823 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b56e05d70 con 0x563b56f20540 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.420+0000 7f5480ff9640 10 client.18768 handle_command_reply: tid=0 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.420+0000 7f54a2205740 1 client.18768 shutdown 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 msgr2=0x563b56ea9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea9120 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f549400bc70 tx=0x7f549400bca0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 shutdown_connections 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563b56f20540 0x563b56f22960 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563b56ea4cb0 0x563b56f1e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563b56e9a410 0x563b56ea9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.423 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 --2- 172.21.15.37:0/179677823 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563b56e99af0 0x563b56ea9120 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f5480ff9640 0 client.18768 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 >> 172.21.15.37:0/179677823 conn(0x563b56e96060 msgr2=0x563b56e949a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 shutdown_connections 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 1 -- 172.21.15.37:0/179677823 wait complete. 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f54a2205740 20 client.18768 trim_cache size 0 max 16384 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:53:39.424 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:53:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f54840a3460 msgr2=0x7f54840a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f549f465640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f54840a3460 0x7f54840a5920 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f5490023000 tx=0x7f5490021000 comp rx=0 tx=0).stop 2024-09-19T06:53:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498102470 msgr2=0x7f5498074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:39.425 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f549f465640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498102470 0x7f5498074a30 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f548c02a940 tx=0x7f548c003b90 comp rx=0 tx=0).stop 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 shutdown_connections 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.421+0000 7f549f465640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5498107b70 0x7f5498074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.422+0000 7f549f465640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f54840a3460 0x7f54840a5920 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.422+0000 7f549f465640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5498102470 0x7f5498074a30 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.422+0000 7f549f465640 1 --2- 172.21.15.37:0/1895175055 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5498101ad0 0x7f54980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.422+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 >> 172.21.15.37:0/1895175055 conn(0x7f54980fd7d0 msgr2=0x7f54980fdea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.422+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 shutdown_connections 2024-09-19T06:53:39.426 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:39.422+0000 7f549f465640 1 -- 172.21.15.37:0/1895175055 wait complete. 2024-09-19T06:53:40.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 895214 7.45 MB/sec execute 487 sec latency 80.663 ms 2024-09-19T06:53:40.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:40 smithi151 ceph-mon[35236]: pgmap v373: 97 pgs: 97 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 365 op/s 2024-09-19T06:53:40.888 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:53:40.888 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:53:40.911 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:40 smithi037 ceph-mon[31211]: pgmap v373: 97 pgs: 97 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 365 op/s 2024-09-19T06:53:41.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:40 smithi096 ceph-mon[35004]: pgmap v373: 97 pgs: 97 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 4.2 MiB/s wr, 365 op/s 2024-09-19T06:53:41.167 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:41.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 897995 7.47 MB/sec execute 488 sec latency 14.993 ms 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 -- 172.21.15.37:0/2878467979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c106f10 msgr2=0x7f339c10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 --2- 172.21.15.37:0/2878467979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c106f10 0x7f339c10ea90 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f3394009ca0 tx=0x7f339402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 -- 172.21.15.37:0/2878467979 shutdown_connections 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 --2- 172.21.15.37:0/2878467979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c106f10 0x7f339c10ea90 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 --2- 172.21.15.37:0/2878467979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f339c102790 0x7f339c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 --2- 172.21.15.37:0/2878467979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339c101de0 0x7f339c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 -- 172.21.15.37:0/2878467979 >> 172.21.15.37:0/2878467979 conn(0x7f339c0fd8c0 msgr2=0x7f339c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 -- 172.21.15.37:0/2878467979 shutdown_connections 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.714+0000 7f33a4770640 1 -- 172.21.15.37:0/2878467979 wait complete. 2024-09-19T06:53:41.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.715+0000 7f33a4770640 1 Processor -- start 2024-09-19T06:53:41.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.715+0000 7f33a4770640 1 -- start start 2024-09-19T06:53:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.715+0000 7f33a4770640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c101de0 0x7f339c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.715+0000 7f33a4770640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339c102790 0x7f339c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.715+0000 7f33a4770640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f339c106f10 0x7f339c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.715+0000 7f33a4770640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f339c111f70 con 0x7f339c101de0 2024-09-19T06:53:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a4770640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f339c111df0 con 0x7f339c102790 2024-09-19T06:53:41.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a4770640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1120f0 con 0x7f339c106f10 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c101de0 0x7f339c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a2d0d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f339c106f10 0x7f339c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a1d0b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339c102790 0x7f339c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c101de0 0x7f339c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35576/0 (socket says 172.21.15.37:35576) 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 -- 172.21.15.37:0/834775425 learned_addr learned my addr 172.21.15.37:0/834775425 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 -- 172.21.15.37:0/834775425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f339c106f10 msgr2=0x7f339c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f339c106f10 0x7f339c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 -- 172.21.15.37:0/834775425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339c102790 msgr2=0x7f339c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339c102790 0x7f339c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 -- 172.21.15.37:0/834775425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c1a68b0 con 0x7f339c101de0 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a1d0b640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339c102790 0x7f339c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:41.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33a250c640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c101de0 0x7f339c0744f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f339800e9f0 tx=0x7f339800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.716+0000 7f33937fe640 1 -- 172.21.15.37:0/834775425 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339800cd80 con 0x7f339c101de0 2024-09-19T06:53:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.717+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c1a6ba0 con 0x7f339c101de0 2024-09-19T06:53:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.717+0000 7f33937fe640 1 -- 172.21.15.37:0/834775425 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3398010910 con 0x7f339c101de0 2024-09-19T06:53:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.717+0000 7f33937fe640 1 -- 172.21.15.37:0/834775425 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33980188f0 con 0x7f339c101de0 2024-09-19T06:53:41.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.717+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c1a70e0 con 0x7f339c101de0 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.719+0000 7f33937fe640 1 -- 172.21.15.37:0/834775425 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3398010ac0 con 0x7f339c101de0 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.719+0000 7f33a6b49740 1 Processor -- start 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a6b49740 1 -- start start 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f228b60 0x56375f228f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f233800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f233d40 0x56375f236130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a6b49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56375f18e280 con 0x56375f229480 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a6b49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56375f135b10 con 0x56375f228b60 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a6b49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56375f18a240 con 0x56375f233d40 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33937fe640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f33880a33b0 0x7f33880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a250c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f233800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a250c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f233800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35588/0 (socket says 172.21.15.37:35588) 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a2d0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f228b60 0x56375f228f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a250c640 1 -- 172.21.15.37:0/693239432 learned_addr learned my addr 172.21.15.37:0/693239432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a1d0b640 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f233d40 0x56375f236130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.720+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f33880a33b0 0x7f33880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33937fe640 1 -- 172.21.15.37:0/834775425 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f33980d0160 con 0x7f339c101de0 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33a250c640 1 -- 172.21.15.37:0/693239432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f233d40 msgr2=0x56375f236130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33a250c640 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f233d40 0x56375f236130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33a250c640 1 -- 172.21.15.37:0/693239432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f228b60 msgr2=0x56375f228f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33a250c640 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f228b60 0x56375f228f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33a250c640 1 -- 172.21.15.37:0/693239432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375f1421f0 con 0x56375f229480 2024-09-19T06:53:41.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33a250c640 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f233800 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3398098e30 tx=0x7f339809c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bca4ce04dd1db06 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f33880a33b0 0x7f33880a5870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3394004af0 tx=0x7f3394004a80 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f3390ff9640 1 -- 172.21.15.37:0/693239432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33980a5020 con 0x56375f229480 2024-09-19T06:53:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f3390ff9640 1 -- 172.21.15.37:0/693239432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339809c7f0 con 0x56375f229480 2024-09-19T06:53:41.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.721+0000 7f3390ff9640 1 -- 172.21.15.37:0/693239432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f339809cb60 con 0x56375f229480 2024-09-19T06:53:41.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 -- 172.21.15.37:0/693239432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 msgr2=0x56375f233800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f233800 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f3398098e30 tx=0x7f339809c6c0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 -- 172.21.15.37:0/693239432 shutdown_connections 2024-09-19T06:53:41.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f233d40 0x56375f236130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f233800 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 --2- 172.21.15.37:0/693239432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f228b60 0x56375f228f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 -- 172.21.15.37:0/693239432 >> 172.21.15.37:0/693239432 conn(0x56375f2250f0 msgr2=0x56375f2254d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 -- 172.21.15.37:0/693239432 shutdown_connections 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 -- 172.21.15.37:0/693239432 wait complete. 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 -- start start 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.722+0000 7f33a6b49740 1 Processor -- start 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f228b60 0x56375f230c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f231160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2316a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a6b49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56375f236910 con 0x56375f229480 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a6b49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56375f148f10 con 0x56375f233d40 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a6b49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56375f236a90 con 0x56375f228b60 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a6b49740 10 client.0 _ll_register_callbacks cb 0x56375f111a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:41.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a1d0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f231160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a1d0b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f231160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35602/0 (socket says 172.21.15.37:35602) 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a1d0b640 1 -- 172.21.15.37:0/3953581599 learned_addr learned my addr 172.21.15.37:0/3953581599 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2316a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a250c640 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f228b60 0x56375f230c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a2d0d640 1 -- 172.21.15.37:0/3953581599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f228b60 msgr2=0x56375f230c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f228b60 0x56375f230c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a2d0d640 1 -- 172.21.15.37:0/3953581599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 msgr2=0x56375f231160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f231160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a2d0d640 1 -- 172.21.15.37:0/3953581599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375eeff6a0 con 0x56375f233d40 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2316a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3394006010 tx=0x7f33940095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11151cda3dde79d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.723+0000 7f3385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f3385ffb640 1 -- 172.21.15.37:0/3953581599 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33940309d0 con 0x56375f233d40 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f3385ffb640 1 -- 172.21.15.37:0/3953581599 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f3394030b70 con 0x56375f233d40 2024-09-19T06:53:41.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f339809c7f0 con 0x56375f233d40 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f3385ffb640 1 -- 172.21.15.37:0/3953581599 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f339402feb0 con 0x56375f233d40 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 10 client.28091 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56375f242570 con 0x56375f233d40 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f3385ffb640 1 -- 172.21.15.37:0/3953581599 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f339809c7f0 con 0x56375f233d40 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f3385ffb640 1 -- 172.21.15.37:0/3953581599 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f339405c070 con 0x56375f233d40 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 10 client.28091 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 10 client.28091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 20 client.28091 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2439c0 0x56375f245e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 4 client.28091 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.724+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56375f19b7f0 con 0x56375f2439c0 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.725+0000 7f33a250c640 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2439c0 0x56375f245e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.725+0000 7f33a250c640 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2439c0 0x56375f245e20 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe5567b338fb8c3 server_cookie=667cfa2a9425c128 in_seq=0 out_seq=0 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.725+0000 7f3385ffb640 10 client.28091 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.725+0000 7f3385ffb640 1 -- 172.21.15.37:0/3953581599 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56375f19b7f0 con 0x56375f2439c0 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.725+0000 7f3385ffb640 10 client.28091 handle_command_reply: tid=0 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 client.28091 shutdown 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 msgr2=0x56375f2316a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2316a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f3394006010 tx=0x7f33940095c0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 shutdown_connections 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2439c0 0x56375f245e20 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2316a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f229480 0x56375f231160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f3385ffb640 0 client.28091 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3953581599 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f228b60 0x56375f230c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 >> 172.21.15.37:0/3953581599 conn(0x56375f2250f0 msgr2=0x56375f22ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:41.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 shutdown_connections 2024-09-19T06:53:41.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 1 -- 172.21.15.37:0/3953581599 wait complete. 2024-09-19T06:53:41.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.726+0000 7f33a6b49740 20 client.28091 trim_cache size 0 max 16384 2024-09-19T06:53:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.743+0000 7f33a6b49740 1 Processor -- start 2024-09-19T06:53:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a6b49740 1 -- start start 2024-09-19T06:53:41.746 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f22b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f232a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f24b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a6b49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1b2400 con 0x56375f228b60 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a6b49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1afcd0 con 0x56375f233d40 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a6b49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1adb40 con 0x56375f229480 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f22b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f22b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35618/0 (socket says 172.21.15.37:35618) 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 -- 172.21.15.37:0/3538005954 learned_addr learned my addr 172.21.15.37:0/3538005954 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f24b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a1d0b640 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f232a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.747 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 -- 172.21.15.37:0/3538005954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 msgr2=0x56375f232a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f232a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 -- 172.21.15.37:0/3538005954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 msgr2=0x56375f24b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f24b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 -- 172.21.15.37:0/3538005954 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56375f224f20 con 0x56375f228b60 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.744+0000 7f33a250c640 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f22b5c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f339809cac0 tx=0x7f339809c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846431210a1c69e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f3387fff640 1 -- 172.21.15.37:0/3538005954 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33980a5020 con 0x56375f228b60 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f3387fff640 1 -- 172.21.15.37:0/3538005954 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33980ae070 con 0x56375f228b60 2024-09-19T06:53:41.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f3387fff640 1 -- 172.21.15.37:0/3538005954 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33980cdaa0 con 0x56375f228b60 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f33a6b49740 1 -- 172.21.15.37:0/3538005954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 msgr2=0x56375f22b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f22b5c0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f339809cac0 tx=0x7f339809c2b0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f33a6b49740 1 -- 172.21.15.37:0/3538005954 shutdown_connections 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f24b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f232a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3538005954 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f22b5c0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.745+0000 7f33a6b49740 1 -- 172.21.15.37:0/3538005954 >> 172.21.15.37:0/3538005954 conn(0x56375f2250f0 msgr2=0x56375f232370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 -- 172.21.15.37:0/3538005954 shutdown_connections 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 -- 172.21.15.37:0/3538005954 wait complete. 2024-09-19T06:53:41.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 -- start start 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 Processor -- start 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f237de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f238320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2ad700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1ae190 con 0x56375f228b60 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1af600 con 0x56375f233d40 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.746+0000 7f33a6b49740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1ae860 con 0x56375f229480 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a6b49740 10 client.0 _ll_register_callbacks cb 0x7f3394009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:41.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a1d0b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f238320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2ad700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2ad700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:33128/0 (socket says 172.21.15.37:33128) 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 -- 172.21.15.37:0/3148288506 learned_addr learned my addr 172.21.15.37:0/3148288506 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a250c640 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f237de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 -- 172.21.15.37:0/3148288506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 msgr2=0x56375f238320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f238320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 -- 172.21.15.37:0/3148288506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 msgr2=0x56375f237de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f237de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 -- 172.21.15.37:0/3148288506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56375f225d80 con 0x56375f233d40 2024-09-19T06:53:41.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a2d0d640 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2ad700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f339405eea0 tx=0x7f339404be40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a0a430f4749b3f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f3385ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f3385ffb640 1 -- 172.21.15.37:0/3148288506 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394063060 con 0x56375f233d40 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33980ae070 con 0x56375f233d40 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.747+0000 7f3385ffb640 1 -- 172.21.15.37:0/3148288506 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f33940309d0 con 0x56375f233d40 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f3385ffb640 1 -- 172.21.15.37:0/3148288506 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f3394031800 con 0x56375f233d40 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 10 client.28097 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x56375f2aeb80 con 0x56375f233d40 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f3385ffb640 1 -- 172.21.15.37:0/3148288506 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f33980ae070 con 0x56375f233d40 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f3385ffb640 1 -- 172.21.15.37:0/3148288506 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f339405c040 con 0x56375f233d40 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 10 client.28097 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 10 client.28097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 20 client.28097 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:41.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2af320 0x56375f2b1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 4 client.28097 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:53:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56375f195d00 con 0x56375f2af320 2024-09-19T06:53:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.748+0000 7f33a250c640 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2af320 0x56375f2b1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.749+0000 7f33a250c640 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2af320 0x56375f2b1740 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3a9c14c4cfecb18 server_cookie=1ba218d9ef931b03 in_seq=0 out_seq=0 2024-09-19T06:53:41.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.749+0000 7f3385ffb640 10 client.28097 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f3385ffb640 1 -- 172.21.15.37:0/3148288506 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56375f195d00 con 0x56375f2af320 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f3385ffb640 10 client.28097 handle_command_reply: tid=0 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 client.28097 shutdown 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 msgr2=0x56375f2ad700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2ad700 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f339405eea0 tx=0x7f339404be40 comp rx=0 tx=0).stop 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 shutdown_connections 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56375f2af320 0x56375f2b1740 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56375f233d40 0x56375f2ad700 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56375f229480 0x56375f238320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 --2- 172.21.15.37:0/3148288506 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56375f228b60 0x56375f237de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f3385ffb640 0 client.28097 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 >> 172.21.15.37:0/3148288506 conn(0x56375f2250f0 msgr2=0x56375f224270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 shutdown_connections 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 1 -- 172.21.15.37:0/3148288506 wait complete. 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.750+0000 7f33a6b49740 20 client.28097 trim_cache size 0 max 16384 2024-09-19T06:53:41.757 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:53:41.758 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:53:41.758 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "cc7e60a2-6a64-4545-892b-9c56d837811c", 2024-09-19T06:53:41.758 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:53:41.758 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:53:41.758 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f33880a33b0 msgr2=0x7f33880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f33880a33b0 0x7f33880a5870 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f3394004af0 tx=0x7f3394004a80 comp rx=0 tx=0).stop 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c101de0 msgr2=0x7f339c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c101de0 0x7f339c0744f0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f339800e9f0 tx=0x7f339800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 shutdown_connections 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f339c106f10 0x7f339c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f339c102790 0x7f339c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f33880a33b0 0x7f33880a5870 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 --2- 172.21.15.37:0/834775425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f339c101de0 0x7f339c0744f0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 >> 172.21.15.37:0/834775425 conn(0x7f339c0fd8c0 msgr2=0x7f339c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 shutdown_connections 2024-09-19T06:53:41.759 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:41.751+0000 7f33a4770640 1 -- 172.21.15.37:0/834775425 wait complete. 2024-09-19T06:53:41.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:41 smithi151 ceph-mon[35236]: pgmap v374: 97 pgs: 97 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-09-19T06:53:42.023 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:41 smithi096 ceph-mon[35004]: pgmap v374: 97 pgs: 97 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-09-19T06:53:42.024 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:41 smithi037 ceph-mon[31211]: pgmap v374: 97 pgs: 97 active+clean; 92 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 2.6 MiB/s wr, 187 op/s 2024-09-19T06:53:42.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 899431 7.47 MB/sec execute 489 sec latency 55.280 ms 2024-09-19T06:53:42.622 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:42 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:53:42.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:42 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:42.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:42 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:53:42.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:42 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:53:42.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:42 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:42.623 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:42 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:42.624 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:53:42.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:42 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:53:42.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:42 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:42.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:42 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:53:42.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:42 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:53:42.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:42 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:42.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:42 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:42.934 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:53:43.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:42 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:53:43.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:42 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:43.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:42 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:53:43.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:42 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:53:43.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:42 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:43.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:42 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:53:43.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 901866 7.48 MB/sec execute 490 sec latency 55.120 ms 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2318728091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 msgr2=0x7f9e9810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2318728091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e9810eaa0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009ca0 tx=0x7f9e9002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2318728091 shutdown_connections 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2318728091 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e9810eaa0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2318728091 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e981027a0 0x7f9e981067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2318728091 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101df0 0x7f9e981021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2318728091 >> 172.21.15.37:0/2318728091 conn(0x7f9e980fd8f0 msgr2=0x7f9e980ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2318728091 shutdown_connections 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.383+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2318728091 wait complete. 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 Processor -- start 2024-09-19T06:53:43.405 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 -- start start 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101df0 0x7f9e980744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e981027a0 0x7f9e98074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e98074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98111f80 con 0x7f9e98106f20 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98111e00 con 0x7f9e98101df0 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.384+0000 7f9e9f69a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e98112100 con 0x7f9e981027a0 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e98074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e98074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35642/0 (socket says 172.21.15.37:35642) 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2037209272 learned_addr learned my addr 172.21.15.37:0/2037209272 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9d436640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101df0 0x7f9e980744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e981027a0 0x7f9e98074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2037209272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e981027a0 msgr2=0x7f9e98074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e981027a0 0x7f9e98074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.406 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2037209272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101df0 msgr2=0x7f9e980744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101df0 0x7f9e980744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2037209272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e981a6910 con 0x7f9e98106f20 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e98074f70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9e90031a20 tx=0x7f9e90031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e8e7fc640 1 -- 172.21.15.37:0/2037209272 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e90003c30 con 0x7f9e98106f20 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e981a6ba0 con 0x7f9e98106f20 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e8e7fc640 1 -- 172.21.15.37:0/2037209272 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90003dd0 con 0x7f9e98106f20 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e8e7fc640 1 -- 172.21.15.37:0/2037209272 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9002fc40 con 0x7f9e98106f20 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.385+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e981a7130 con 0x7f9e98106f20 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.388+0000 7f9e8e7fc640 1 -- 172.21.15.37:0/2037209272 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9e90004050 con 0x7f9e98106f20 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 Processor -- start 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 -- start start 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa064a0 0x55d4dfa06880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9e8e7fc640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9e840a33b0 0x7f9e840a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.407 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa11140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa13a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d4df96c2e0 con 0x55d4dfa11680 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d4df913ce0 con 0x55d4dfa06dc0 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d4df9682a0 con 0x55d4dfa064a0 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9e9d436640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9e840a33b0 0x7f9e840a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa13a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9e9cc35640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa11140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa13a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35656/0 (socket says 172.21.15.37:35656) 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9e9cc35640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa11140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:33144/0 (socket says 172.21.15.37:33144) 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.389+0000 7f9e9dc37640 1 -- 172.21.15.37:0/1742013834 learned_addr learned my addr 172.21.15.37:0/1742013834 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa064a0 0x55d4dfa06880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.408 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e8e7fc640 1 -- 172.21.15.37:0/2037209272 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f9e900f0e50 con 0x7f9e98106f20 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9cc35640 1 -- 172.21.15.37:0/1742013834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa064a0 msgr2=0x55d4dfa06880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa064a0 0x55d4dfa06880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9cc35640 1 -- 172.21.15.37:0/1742013834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 msgr2=0x55d4dfa13a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa13a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9cc35640 1 -- 172.21.15.37:0/1742013834 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4df8f8440 con 0x55d4dfa06dc0 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa13a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa11140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e981087f0 tx=0x7f9e8800a5e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93518d31d0d5c91c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e9d436640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9e840a33b0 0x7f9e840a5870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009bd0 tx=0x7f9e94005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e837fe640 1 -- 172.21.15.37:0/1742013834 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88004020 con 0x55d4dfa06dc0 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e837fe640 1 -- 172.21.15.37:0/1742013834 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e88005500 con 0x55d4dfa06dc0 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.390+0000 7f9e837fe640 1 -- 172.21.15.37:0/1742013834 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e88004d20 con 0x55d4dfa06dc0 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 -- 172.21.15.37:0/1742013834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa06dc0 msgr2=0x55d4dfa11140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa11140 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9e981087f0 tx=0x7f9e8800a5e0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 -- 172.21.15.37:0/1742013834 shutdown_connections 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa13a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa11140 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/1742013834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa064a0 0x55d4dfa06880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 -- 172.21.15.37:0/1742013834 >> 172.21.15.37:0/1742013834 conn(0x55d4dfa02a30 msgr2=0x55d4dfa02e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 -- 172.21.15.37:0/1742013834 shutdown_connections 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.391+0000 7f9ea1a73740 1 -- 172.21.15.37:0/1742013834 wait complete. 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 -- start start 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 Processor -- start 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa11050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa0e300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa0e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d4dfa14250 con 0x55d4dfa11680 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d4df927050 con 0x55d4dfa064a0 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d4dfa143d0 con 0x55d4dfa06dc0 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9ea1a73740 10 client.0 _ll_register_callbacks cb 0x55d4df967160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.392+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa0e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9d436640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa11050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa0e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35660/0 (socket says 172.21.15.37:35660) 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9d436640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa11050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:33156/0 (socket says 172.21.15.37:33156) 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 -- 172.21.15.37:0/4042536145 learned_addr learned my addr 172.21.15.37:0/4042536145 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa0e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 -- 172.21.15.37:0/4042536145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 msgr2=0x55d4dfa0e300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa0e300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 -- 172.21.15.37:0/4042536145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 msgr2=0x55d4dfa11050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.411 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa11050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 -- 172.21.15.37:0/4042536145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4df6dd3c0 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa0e8f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b96a0 tx=0x7f9e900bce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ac3e47c5aa3a5e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e80ff9640 1 -- 172.21.15.37:0/4042536145 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900c15f0 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e80ff9640 1 -- 172.21.15.37:0/4042536145 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f9e90004730 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e88005500 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9e80ff9640 1 -- 172.21.15.37:0/4042536145 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9e900c5e20 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9ea1a73740 10 client.18798 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.393+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d4dfa14250 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9e80ff9640 1 -- 172.21.15.37:0/4042536145 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e88005500 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9e80ff9640 1 -- 172.21.15.37:0/4042536145 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9e900ee790 con 0x55d4dfa11680 2024-09-19T06:53:43.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9ea1a73740 10 client.18798 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9ea1a73740 10 client.18798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9ea1a73740 20 client.18798 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa21160 0x55d4dfa23570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9ea1a73740 4 client.18798 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4df979850 con 0x55d4dfa21160 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9e9d436640 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa21160 0x55d4dfa23570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9e9d436640 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa21160 0x55d4dfa23570 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b010a53de9613fb6 server_cookie=de2c98e38b24524b in_seq=0 out_seq=0 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.394+0000 7f9e80ff9640 10 client.18798 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.395+0000 7f9e80ff9640 1 -- 172.21.15.37:0/4042536145 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d4df979850 con 0x55d4dfa21160 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.395+0000 7f9e80ff9640 10 client.18798 handle_command_reply: tid=0 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.395+0000 7f9ea1a73740 1 client.18798 shutdown 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 msgr2=0x55d4dfa0e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa0e8f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b96a0 tx=0x7f9e900bce40 comp rx=0 tx=0).stop 2024-09-19T06:53:43.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 shutdown_connections 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa21160 0x55d4dfa23570 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa0e8f0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa0e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/4042536145 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa11050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 >> 172.21.15.37:0/4042536145 conn(0x55d4dfa02a30 msgr2=0x55d4dfa08710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9e80ff9640 0 client.18798 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 shutdown_connections 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 1 -- 172.21.15.37:0/4042536145 wait complete. 2024-09-19T06:53:43.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.396+0000 7f9ea1a73740 20 client.18798 trim_cache size 0 max 16384 2024-09-19T06:53:43.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 Processor -- start 2024-09-19T06:53:43.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 -- start start 2024-09-19T06:53:43.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa10270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa28e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa29390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981b2430 con 0x55d4dfa11680 2024-09-19T06:53:43.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981afd00 con 0x55d4dfa064a0 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.415+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981adb70 con 0x55d4dfa06dc0 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa29390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa29390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35672/0 (socket says 172.21.15.37:35672) 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2182131189 learned_addr learned my addr 172.21.15.37:0/2182131189 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9d436640 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa10270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa28e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2182131189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 msgr2=0x55d4dfa28e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa28e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2182131189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 msgr2=0x55d4dfa10270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa10270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 -- 172.21.15.37:0/2182131189 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4dfa113c0 con 0x55d4dfa11680 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa29390 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9e90004020 tx=0x7f9e9002f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597acc56b410f888 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:43.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e82ffd640 1 -- 172.21.15.37:0/2182131189 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900c15f0 con 0x55d4dfa11680 2024-09-19T06:53:43.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e82ffd640 1 -- 172.21.15.37:0/2182131189 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90036070 con 0x55d4dfa11680 2024-09-19T06:53:43.420 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.416+0000 7f9e82ffd640 1 -- 172.21.15.37:0/2182131189 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e900ee790 con 0x55d4dfa11680 2024-09-19T06:53:43.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 -- 172.21.15.37:0/2182131189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 msgr2=0x55d4dfa29390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa29390 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f9e90004020 tx=0x7f9e9002f610 comp rx=0 tx=0).stop 2024-09-19T06:53:43.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 -- 172.21.15.37:0/2182131189 shutdown_connections 2024-09-19T06:53:43.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa11680 0x55d4dfa29390 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa28e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/2182131189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa10270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 -- 172.21.15.37:0/2182131189 >> 172.21.15.37:0/2182131189 conn(0x55d4dfa02a30 msgr2=0x55d4dfa0fb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 -- 172.21.15.37:0/2182131189 shutdown_connections 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.417+0000 7f9ea1a73740 1 -- 172.21.15.37:0/2182131189 wait complete. 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 -- start start 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 Processor -- start 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa19f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa172b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa11680 0x55d4dfa17a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981ae1c0 con 0x55d4dfa06dc0 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981af630 con 0x55d4dfa064a0 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9e981ae890 con 0x55d4dfa11680 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9ea1a73740 10 client.0 _ll_register_callbacks cb 0x7f9e900ee5c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:43.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9e9cc35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa172b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9e9d436640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa19f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9e9cc35640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa172b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:35678/0 (socket says 172.21.15.37:35678) 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9e9cc35640 1 -- 172.21.15.37:0/3441437072 learned_addr learned my addr 172.21.15.37:0/3441437072 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.418+0000 7f9e9dc37640 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa11680 0x55d4dfa17a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e9d436640 1 -- 172.21.15.37:0/3441437072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa11680 msgr2=0x55d4dfa17a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e9d436640 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa11680 0x55d4dfa17a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e9d436640 1 -- 172.21.15.37:0/3441437072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa06dc0 msgr2=0x55d4dfa172b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e9d436640 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa172b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e9d436640 1 -- 172.21.15.37:0/3441437072 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4dfa8ae30 con 0x55d4dfa064a0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e9d436640 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa19f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b9650 tx=0x7f9e94004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c532c32e7a9091b5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e80ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e80ff9640 1 -- 172.21.15.37:0/3441437072 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e940293f0 con 0x55d4dfa064a0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e80ff9640 1 -- 172.21.15.37:0/3441437072 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7f9e94029590 con 0x55d4dfa064a0 2024-09-19T06:53:43.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e90036070 con 0x55d4dfa064a0 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9e80ff9640 1 -- 172.21.15.37:0/3441437072 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9e940297d0 con 0x55d4dfa064a0 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9ea1a73740 10 client.28115 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.419+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d4dfa8c000 con 0x55d4dfa064a0 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9e80ff9640 1 -- 172.21.15.37:0/3441437072 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9e90036070 con 0x55d4dfa064a0 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9e80ff9640 1 -- 172.21.15.37:0/3441437072 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7f9e94038c90 con 0x55d4dfa064a0 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9ea1a73740 10 client.28115 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9ea1a73740 10 client.28115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9ea1a73740 20 client.28115 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa8c780 0x55d4dfa8eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9ea1a73740 4 client.28115 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4df973d60 con 0x55d4dfa8c780 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa8c780 0x55d4dfa8eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9e9cc35640 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa8c780 0x55d4dfa8eba0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c355087579ebeb51 server_cookie=20f96bf72a593f85 in_seq=0 out_seq=0 2024-09-19T06:53:43.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.420+0000 7f9e80ff9640 10 client.28115 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9e80ff9640 1 -- 172.21.15.37:0/3441437072 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d4df973d60 con 0x55d4dfa8c780 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9e80ff9640 10 client.28115 handle_command_reply: tid=0 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 client.28115 shutdown 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 msgr2=0x55d4dfa19f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa19f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b9650 tx=0x7f9e94004730 comp rx=0 tx=0).stop 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 shutdown_connections 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55d4dfa8c780 0x55d4dfa8eba0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d4dfa11680 0x55d4dfa17a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d4dfa06dc0 0x55d4dfa172b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9e80ff9640 0 client.28115 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 --2- 172.21.15.37:0/3441437072 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d4dfa064a0 0x55d4dfa19f60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 >> 172.21.15.37:0/3441437072 conn(0x55d4dfa02a30 msgr2=0x55d4dfa0c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 shutdown_connections 2024-09-19T06:53:43.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 1 -- 172.21.15.37:0/3441437072 wait complete. 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.421+0000 7f9ea1a73740 20 client.28115 trim_cache size 0 max 16384 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stdout: "cc7e60a2-6a64-4545-892b-9c56d837811c": { 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "cc7e60a2-6a64-4545-892b-9c56d837811c", 2024-09-19T06:53:43.428 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:53:43.429 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:53:43.429 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:53:43.429 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:53:43.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9e840a33b0 msgr2=0x7f9e840a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9e840a33b0 0x7f9e840a5870 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009bd0 tx=0x7f9e94005f90 comp rx=0 tx=0).stop 2024-09-19T06:53:43.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 msgr2=0x7f9e98074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:43.429 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e98074f70 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f9e90031a20 tx=0x7f9e90031a50 comp rx=0 tx=0).stop 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 shutdown_connections 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9e840a33b0 0x7f9e840a5870 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9e98106f20 0x7f9e98074f70 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9e981027a0 0x7f9e98074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 --2- 172.21.15.37:0/2037209272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9e98101df0 0x7f9e980744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 >> 172.21.15.37:0/2037209272 conn(0x7f9e980fd8f0 msgr2=0x7f9e980fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 shutdown_connections 2024-09-19T06:53:43.430 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:53:43.422+0000 7f9e9f69a640 1 -- 172.21.15.37:0/2037209272 wait complete. 2024-09-19T06:53:43.762 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:43 smithi037 ceph-mon[31211]: pgmap v375: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-09-19T06:53:43.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:43 smithi151 ceph-mon[35236]: pgmap v375: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-09-19T06:53:44.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:43 smithi096 ceph-mon[35004]: pgmap v375: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 370 op/s 2024-09-19T06:53:44.172 INFO:tasks.cephfs.filesystem:scrub status for tag:cc7e60a2-6a64-4545-892b-9c56d837811c - {'path': '/', 'tag': 'cc7e60a2-6a64-4545-892b-9c56d837811c', 'options': 'recursive,force'} 2024-09-19T06:53:44.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 903523 7.48 MB/sec execute 491 sec latency 41.343 ms 2024-09-19T06:53:45.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 905515 7.49 MB/sec execute 492 sec latency 55.891 ms 2024-09-19T06:53:46.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:45 smithi151 ceph-mon[35236]: pgmap v376: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-19T06:53:46.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 907769 7.49 MB/sec execute 493 sec latency 19.552 ms 2024-09-19T06:53:46.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:45 smithi096 ceph-mon[35004]: pgmap v376: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-19T06:53:46.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:45 smithi037 ceph-mon[31211]: pgmap v376: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-19T06:53:47.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 909536 7.50 MB/sec execute 494 sec latency 18.273 ms 2024-09-19T06:53:48.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:47 smithi151 ceph-mon[35236]: pgmap v377: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-19T06:53:48.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 912389 7.52 MB/sec execute 495 sec latency 24.316 ms 2024-09-19T06:53:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:47 smithi096 ceph-mon[35004]: pgmap v377: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-19T06:53:48.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:47 smithi037 ceph-mon[31211]: pgmap v377: 97 pgs: 97 active+clean; 79 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 2.7 MiB/s wr, 271 op/s 2024-09-19T06:53:49.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 913874 7.52 MB/sec execute 496 sec latency 16.687 ms 2024-09-19T06:53:50.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:49 smithi151 ceph-mon[35236]: pgmap v378: 97 pgs: 97 active+clean; 86 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 369 op/s 2024-09-19T06:53:50.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 916015 7.53 MB/sec execute 497 sec latency 47.902 ms 2024-09-19T06:53:50.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:49 smithi096 ceph-mon[35004]: pgmap v378: 97 pgs: 97 active+clean; 86 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 369 op/s 2024-09-19T06:53:50.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:49 smithi037 ceph-mon[31211]: pgmap v378: 97 pgs: 97 active+clean; 86 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 4.2 MiB/s wr, 369 op/s 2024-09-19T06:53:51.232 INFO:tasks.workunit.client.0.smithi037.stdout: 1 917037 7.54 MB/sec execute 498 sec latency 64.881 ms 2024-09-19T06:53:52.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:51 smithi151 ceph-mon[35236]: pgmap v379: 97 pgs: 97 active+clean; 86 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-09-19T06:53:52.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 919711 7.57 MB/sec execute 499 sec latency 13.832 ms 2024-09-19T06:53:52.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:51 smithi096 ceph-mon[35004]: pgmap v379: 97 pgs: 97 active+clean; 86 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-09-19T06:53:52.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:51 smithi037 ceph-mon[31211]: pgmap v379: 97 pgs: 97 active+clean; 86 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.0 MiB/s wr, 281 op/s 2024-09-19T06:53:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 -- 172.21.15.37:0/61566425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141069c0 msgr2=0x7fa914106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 --2- 172.21.15.37:0/61566425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141069c0 0x7fa914106da0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fa910009900 tx=0x7fa91002a410 comp rx=0 tx=0).stop 2024-09-19T06:53:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 -- 172.21.15.37:0/61566425 shutdown_connections 2024-09-19T06:53:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 --2- 172.21.15.37:0/61566425 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa914101350 0x7fa91410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 --2- 172.21.15.37:0/61566425 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9141009b0 0x7fa914100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.517 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 --2- 172.21.15.37:0/61566425 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141069c0 0x7fa914106da0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 -- 172.21.15.37:0/61566425 >> 172.21.15.37:0/61566425 conn(0x7fa914077990 msgr2=0x7fa9140fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 -- 172.21.15.37:0/61566425 shutdown_connections 2024-09-19T06:53:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.515+0000 7fa91c12a640 1 -- 172.21.15.37:0/61566425 wait complete. 2024-09-19T06:53:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 Processor -- start 2024-09-19T06:53:52.518 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 -- start start 2024-09-19T06:53:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141009b0 0x7fa914198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.519 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa914101350 0x7fa914198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9141069c0 0x7fa91419cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa914111e90 con 0x7fa9141009b0 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa914111d10 con 0x7fa9141069c0 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91c12a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa914112010 con 0x7fa914101350 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141009b0 0x7fa914198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141009b0 0x7fa914198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39760/0 (socket says 172.21.15.37:39760) 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 -- 172.21.15.37:0/1332502313 learned_addr learned my addr 172.21.15.37:0/1332502313 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa9196c5640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa914101350 0x7fa914198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9141069c0 0x7fa91419cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 -- 172.21.15.37:0/1332502313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa914101350 msgr2=0x7fa914198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa914101350 0x7fa914198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 -- 172.21.15.37:0/1332502313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9141069c0 msgr2=0x7fa91419cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9141069c0 0x7fa91419cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.516+0000 7fa919ec6640 1 -- 172.21.15.37:0/1332502313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa91419d2f0 con 0x7fa9141009b0 2024-09-19T06:53:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.517+0000 7fa919ec6640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141009b0 0x7fa914198300 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa9100042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.517+0000 7fa90affd640 1 -- 172.21.15.37:0/1332502313 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91003f030 con 0x7fa9141009b0 2024-09-19T06:53:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.517+0000 7fa90affd640 1 -- 172.21.15.37:0/1332502313 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa910003ce0 con 0x7fa9141009b0 2024-09-19T06:53:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.517+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa91419d580 con 0x7fa9141009b0 2024-09-19T06:53:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.517+0000 7fa90affd640 1 -- 172.21.15.37:0/1332502313 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910035070 con 0x7fa9141009b0 2024-09-19T06:53:52.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.517+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9141a4e70 con 0x7fa9141009b0 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.519+0000 7fa90affd640 1 -- 172.21.15.37:0/1332502313 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa910042030 con 0x7fa9141009b0 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 Processor -- start 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 -- start start 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fd2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fe1b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fe44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557ea7f372f0 con 0x557ea7fe20c0 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ea7edf270 con 0x557ea7fd32b0 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91eec3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ea7f332b0 con 0x557ea7fd2990 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa9196c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fe44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa9196c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fe44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39774/0 (socket says 172.21.15.37:39774) 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa9196c5640 1 -- 172.21.15.37:0/3019899935 learned_addr learned my addr 172.21.15.37:0/3019899935 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa90affd640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9000a34e0 0x7fa9000a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fd2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa919ec6640 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fe1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.520+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9000a34e0 0x7fa9000a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa90affd640 1 -- 172.21.15.37:0/1332502313 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fa9100f0140 con 0x7fa9141009b0 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa9196c5640 1 -- 172.21.15.37:0/3019899935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 msgr2=0x557ea7fd2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa9196c5640 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fd2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa9196c5640 1 -- 172.21.15.37:0/3019899935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 msgr2=0x557ea7fe1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa9196c5640 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fe1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa9196c5640 1 -- 172.21.15.37:0/3019899935 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea7ede340 con 0x557ea7fe20c0 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9000a34e0 0x7fa9000a59a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009fd0 tx=0x7fa90c009600 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa9196c5640 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fe44b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa904009c90 tx=0x7fa904011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c88c8765e204d54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa8fffff640 1 -- 172.21.15.37:0/3019899935 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90401e070 con 0x557ea7fe20c0 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa8fffff640 1 -- 172.21.15.37:0/3019899935 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa904019b10 con 0x557ea7fe20c0 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.521+0000 7fa8fffff640 1 -- 172.21.15.37:0/3019899935 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90400ea00 con 0x557ea7fe20c0 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 -- 172.21.15.37:0/3019899935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 msgr2=0x557ea7fe44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fe44b0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fa904009c90 tx=0x7fa904011810 comp rx=0 tx=0).stop 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 -- 172.21.15.37:0/3019899935 shutdown_connections 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fe44b0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fe1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 --2- 172.21.15.37:0/3019899935 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fd2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 -- 172.21.15.37:0/3019899935 >> 172.21.15.37:0/3019899935 conn(0x557ea7fcf060 msgr2=0x557ea7fcf440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 -- 172.21.15.37:0/3019899935 shutdown_connections 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 -- 172.21.15.37:0/3019899935 wait complete. 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 -- start start 2024-09-19T06:53:52.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.522+0000 7fa91eec3740 1 Processor -- start 2024-09-19T06:53:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd2990 0x557ea7fdf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fdf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fe20c0 0x557ea7fdfac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91eec3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557ea7fe4cf0 con 0x557ea7fd2990 2024-09-19T06:53:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91eec3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557ea7ec0290 con 0x557ea7fd32b0 2024-09-19T06:53:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91eec3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557ea7fe4e70 con 0x557ea7fe20c0 2024-09-19T06:53:52.531 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91eec3740 10 client.0 _ll_register_callbacks cb 0x557ea7ef1140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd2990 0x557ea7fdf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd2990 0x557ea7fdf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39790/0 (socket says 172.21.15.37:39790) 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 -- 172.21.15.37:0/510956266 learned_addr learned my addr 172.21.15.37:0/510956266 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa9196c5640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fdf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fe20c0 0x557ea7fdfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 -- 172.21.15.37:0/510956266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fe20c0 msgr2=0x557ea7fdfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fe20c0 0x557ea7fdfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 -- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 msgr2=0x557ea7fdf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fdf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 -- 172.21.15.37:0/510956266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea7c4a210 con 0x557ea7fd2990 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fe20c0 0x557ea7fdfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa919ec6640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd2990 0x557ea7fdf040 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa91008b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64523aed14249dd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.532 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.523+0000 7fa8fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/510956266 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa910049040 con 0x557ea7fd2990 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/510956266 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fa91008e020 con 0x557ea7fd2990 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa904019b10 con 0x557ea7fd2990 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/510956266 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa9100b8040 con 0x557ea7fd2990 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 10 client.18816 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ea7d16620 con 0x557ea7fd2990 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/510956266 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa904019b10 con 0x557ea7fd2990 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/510956266 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa9100bc860 con 0x557ea7fd2990 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 10 client.18816 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 10 client.18816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 20 client.18816 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea7ff20f0 0x557ea7ff4560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.533 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 4 client.18816 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ea7f44a20 con 0x557ea7ff20f0 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.524+0000 7fa9196c5640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea7ff20f0 0x557ea7ff4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.525+0000 7fa9196c5640 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea7ff20f0 0x557ea7ff4560 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c2d644a3f7336b8 server_cookie=4f29874f223fe1a2 in_seq=0 out_seq=0 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.525+0000 7fa8fd7fa640 10 client.18816 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/510956266 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557ea7f44a20 con 0x557ea7ff20f0 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa8fd7fa640 10 client.18816 handle_command_reply: tid=0 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 client.18816 shutdown 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd2990 msgr2=0x557ea7fdf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd2990 0x557ea7fdf040 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fa910000c00 tx=0x7fa91008b3b0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 shutdown_connections 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea7ff20f0 0x557ea7ff4560 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fe20c0 0x557ea7fdfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.534 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd32b0 0x557ea7fdf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 --2- 172.21.15.37:0/510956266 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd2990 0x557ea7fdf040 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa8fd7fa640 0 client.18816 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 >> 172.21.15.37:0/510956266 conn(0x557ea7fcf060 msgr2=0x557ea7fd4dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.526+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 shutdown_connections 2024-09-19T06:53:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.527+0000 7fa91eec3740 1 -- 172.21.15.37:0/510956266 wait complete. 2024-09-19T06:53:52.535 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.527+0000 7fa91eec3740 20 client.18816 trim_cache size 0 max 16384 2024-09-19T06:53:52.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 Processor -- start 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 -- start start 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd2990 0x557ea7ff9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd32b0 0x557ea7ffa350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fdc280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141adec0 con 0x557ea7fe20c0 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141ab790 con 0x557ea7fd2990 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91eec3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141a9620 con 0x557ea7fd32b0 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91a6c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fdc280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91a6c7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fdc280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39806/0 (socket says 172.21.15.37:39806) 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa91a6c7640 1 -- 172.21.15.37:0/27821390 learned_addr learned my addr 172.21.15.37:0/27821390 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa9196c5640 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd32b0 0x557ea7ffa350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.541+0000 7fa919ec6640 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd2990 0x557ea7ff9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa91a6c7640 1 -- 172.21.15.37:0/27821390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd32b0 msgr2=0x557ea7ffa350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd32b0 0x557ea7ffa350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa91a6c7640 1 -- 172.21.15.37:0/27821390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd2990 msgr2=0x557ea7ff9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd2990 0x557ea7ff9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa91a6c7640 1 -- 172.21.15.37:0/27821390 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ea7fe1bd0 con 0x557ea7fe20c0 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fdc280 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa90c012a50 tx=0x7fa90c00c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cfb96639eb1117a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa8ff7fe640 1 -- 172.21.15.37:0/27821390 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c00ae50 con 0x557ea7fe20c0 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa8ff7fe640 1 -- 172.21.15.37:0/27821390 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c01e970 con 0x557ea7fe20c0 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.542+0000 7fa8ff7fe640 1 -- 172.21.15.37:0/27821390 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa90c0053e0 con 0x557ea7fe20c0 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 -- 172.21.15.37:0/27821390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 msgr2=0x557ea7fdc280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fdc280 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fa90c012a50 tx=0x7fa90c00c000 comp rx=0 tx=0).stop 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 -- 172.21.15.37:0/27821390 shutdown_connections 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fe20c0 0x557ea7fdc280 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd32b0 0x557ea7ffa350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 --2- 172.21.15.37:0/27821390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fd2990 0x557ea7ff9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 -- 172.21.15.37:0/27821390 >> 172.21.15.37:0/27821390 conn(0x557ea7fcf060 msgr2=0x557ea7fe10c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 -- 172.21.15.37:0/27821390 shutdown_connections 2024-09-19T06:53:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 -- 172.21.15.37:0/27821390 wait complete. 2024-09-19T06:53:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 -- start start 2024-09-19T06:53:52.547 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.543+0000 7fa91eec3740 1 Processor -- start 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fe5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd32b0 0x557ea7fe6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fe20c0 0x557ea7fe6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141a9c70 con 0x557ea7fd32b0 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141ab0c0 con 0x557ea7fe20c0 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9141aa320 con 0x557ea7fd2990 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 10 client.0 _ll_register_callbacks cb 0x7fa91003b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd32b0 0x557ea7fe6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd32b0 0x557ea7fe6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:39808/0 (socket says 172.21.15.37:39808) 2024-09-19T06:53:52.550 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 -- 172.21.15.37:0/1190598818 learned_addr learned my addr 172.21.15.37:0/1190598818 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91a6c7640 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fe20c0 0x557ea7fe6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa919ec6640 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fe5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 -- 172.21.15.37:0/1190598818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 msgr2=0x557ea7fe5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fe5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 -- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fe20c0 msgr2=0x557ea7fe6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fe20c0 0x557ea7fe6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 -- 172.21.15.37:0/1190598818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ea805be30 con 0x557ea7fd32b0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa9196c5640 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd32b0 0x557ea7fe6420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x557ea7fd0150 tx=0x7fa90400e560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86cb14cae2915bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa8fd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/1190598818 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa904035060 con 0x557ea7fd32b0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/1190598818 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8571+0+0 (secure 0 0 0) 0x7fa9040199e0 con 0x557ea7fd32b0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.544+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa90c01e970 con 0x557ea7fd32b0 2024-09-19T06:53:52.551 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/1190598818 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fa90401e040 con 0x557ea7fd32b0 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 10 client.18822 fetch_fsmap learned FSMap version 39 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x557ea805d120 con 0x557ea7fd32b0 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/1190598818 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa90c01e970 con 0x557ea7fd32b0 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/1190598818 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 39) ==== 8346+0+0 (secure 0 0 0) 0x7fa904003780 con 0x557ea7fd32b0 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 10 client.18822 fetch_fsmap finished waiting for FSMap version 39 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 10 client.18822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 20 client.18822 populate_metadata read hostname 'smithi037' 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea805d730 0x557ea805fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 4 client.18822 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:53:52.552 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ea7f3ed70 con 0x557ea805d730 2024-09-19T06:53:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.545+0000 7fa919ec6640 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea805d730 0x557ea805fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:53:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.546+0000 7fa919ec6640 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea805d730 0x557ea805fb50 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b8dc5e64341ef1c server_cookie=d7d5dd64d4201c14 in_seq=0 out_seq=0 2024-09-19T06:53:52.553 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.546+0000 7fa8fd7fa640 10 client.18822 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.739+0000 7fa8fd7fa640 1 -- 172.21.15.37:0/1190598818 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ea7f3ed70 con 0x557ea805d730 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.739+0000 7fa8fd7fa640 10 client.18822 handle_command_reply: tid=0 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.739+0000 7fa91eec3740 1 client.18822 shutdown 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.739+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd32b0 msgr2=0x557ea7fe6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.739+0000 7fa91eec3740 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd32b0 0x557ea7fe6420 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x557ea7fd0150 tx=0x7fa90400e560 comp rx=0 tx=0).stop 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 shutdown_connections 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557ea805d730 0x557ea805fb50 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557ea7fe20c0 0x557ea7fe6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa8fd7fa640 0 client.18822 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:53:52.742 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557ea7fd32b0 0x557ea7fe6420 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 --2- 172.21.15.37:0/1190598818 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557ea7fd2990 0x557ea7fe5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 >> 172.21.15.37:0/1190598818 conn(0x557ea7fcf060 msgr2=0x557ea7fe08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 shutdown_connections 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 1 -- 172.21.15.37:0/1190598818 wait complete. 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.740+0000 7fa91eec3740 20 client.18822 trim_cache size 0 max 16384 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:53:52.743 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:53:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9000a34e0 msgr2=0x7fa9000a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9000a34e0 0x7fa9000a59a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa90c009fd0 tx=0x7fa90c009600 comp rx=0 tx=0).stop 2024-09-19T06:53:52.744 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141009b0 msgr2=0x7fa914198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141009b0 0x7fa914198300 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fa910006010 tx=0x7fa9100042f0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 shutdown_connections 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9141069c0 0x7fa91419cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa914101350 0x7fa914198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9000a34e0 0x7fa9000a59a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 --2- 172.21.15.37:0/1332502313 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9141009b0 0x7fa914198300 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 >> 172.21.15.37:0/1332502313 conn(0x7fa914077990 msgr2=0x7fa9141037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 shutdown_connections 2024-09-19T06:53:52.745 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:53:52.741+0000 7fa91c12a640 1 -- 172.21.15.37:0/1332502313 wait complete. 2024-09-19T06:53:53.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:52 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:53:53.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:52 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:53:53.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 920980 7.57 MB/sec execute 500 sec latency 141.751 ms 2024-09-19T06:53:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:52 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:53:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:52 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:53:53.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:52 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:53:53.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:52 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:53:53.836 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:53:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setuser ceph since I am not root 2024-09-19T06:53:53.836 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:53:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setgroup ceph since I am not root 2024-09-19T06:53:53.836 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:53:53 smithi151 ceph-mds[66270]: -- 172.21.15.151:0/3260171867 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559539923c20 con 0x559539901400 2024-09-19T06:53:53.836 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:53:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: starting mds.i at 2024-09-19T06:53:54.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:53 smithi151 ceph-mon[35236]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 469 op/s 2024-09-19T06:53:54.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:53 smithi151 ceph-mon[35236]: daemon mds.i restarted 2024-09-19T06:53:54.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 923907 7.59 MB/sec execute 501 sec latency 13.524 ms 2024-09-19T06:53:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:53 smithi096 ceph-mon[35004]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 469 op/s 2024-09-19T06:53:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:53 smithi096 ceph-mon[35004]: daemon mds.i restarted 2024-09-19T06:53:54.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:53 smithi037 ceph-mon[31211]: pgmap v380: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 4.6 MiB/s wr, 469 op/s 2024-09-19T06:53:54.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:53 smithi037 ceph-mon[31211]: daemon mds.i restarted 2024-09-19T06:53:55.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:54 smithi151 ceph-mon[35236]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T06:53:55.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:54 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6838/1999039793,v1:172.21.15.151:6839/1999039793] up:boot 2024-09-19T06:53:55.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:54 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:53:55.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:54 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:53:55.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:54 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:53:55.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 925399 7.59 MB/sec execute 502 sec latency 60.399 ms 2024-09-19T06:53:55.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:54 smithi096 ceph-mon[35004]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T06:53:55.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:54 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6838/1999039793,v1:172.21.15.151:6839/1999039793] up:boot 2024-09-19T06:53:55.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:54 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:53:55.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:54 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:53:55.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:54 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:53:55.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:54 smithi037 ceph-mon[31211]: osdmap e99: 12 total, 12 up, 12 in 2024-09-19T06:53:55.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:54 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6838/1999039793,v1:172.21.15.151:6839/1999039793] up:boot 2024-09-19T06:53:55.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:54 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:53:55.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:54 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:53:55.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:54 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:53:56.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:55 smithi151 ceph-mon[35236]: pgmap v382: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-09-19T06:53:56.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 927901 7.60 MB/sec execute 503 sec latency 12.925 ms 2024-09-19T06:53:56.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:55 smithi096 ceph-mon[35004]: pgmap v382: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-09-19T06:53:56.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:55 smithi037 ceph-mon[31211]: pgmap v382: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-09-19T06:53:57.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 930882 7.62 MB/sec execute 504 sec latency 12.626 ms 2024-09-19T06:53:58.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:57 smithi151 ceph-mon[35236]: pgmap v383: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-09-19T06:53:58.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 932862 7.62 MB/sec execute 505 sec latency 14.827 ms 2024-09-19T06:53:58.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:57 smithi096 ceph-mon[35004]: pgmap v383: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-09-19T06:53:58.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:57 smithi037 ceph-mon[31211]: pgmap v383: 97 pgs: 97 active+clean; 89 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.7 MiB/s wr, 343 op/s 2024-09-19T06:53:59.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 935153 7.63 MB/sec execute 506 sec latency 15.531 ms 2024-09-19T06:54:00.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:53:59 smithi151 ceph-mon[35236]: pgmap v384: 97 pgs: 97 active+clean; 80 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-19T06:54:00.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 937978 7.65 MB/sec execute 507 sec latency 56.238 ms 2024-09-19T06:54:00.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:53:59 smithi096 ceph-mon[35004]: pgmap v384: 97 pgs: 97 active+clean; 80 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-19T06:54:00.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:53:59 smithi037 ceph-mon[31211]: pgmap v384: 97 pgs: 97 active+clean; 80 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-19T06:54:01.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 938820 7.64 MB/sec execute 508 sec latency 29.939 ms 2024-09-19T06:54:02.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:01 smithi151 ceph-mon[35236]: pgmap v385: 97 pgs: 97 active+clean; 80 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-19T06:54:02.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 941180 7.65 MB/sec execute 509 sec latency 30.574 ms 2024-09-19T06:54:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:01 smithi096 ceph-mon[35004]: pgmap v385: 97 pgs: 97 active+clean; 80 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-19T06:54:02.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:01 smithi037 ceph-mon[31211]: pgmap v385: 97 pgs: 97 active+clean; 80 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 3.7 MiB/s wr, 357 op/s 2024-09-19T06:54:03.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 941855 7.65 MB/sec execute 510 sec latency 33.002 ms 2024-09-19T06:54:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:02 smithi096 ceph-mon[35004]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T06:54:03.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:02 smithi037 ceph-mon[31211]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T06:54:03.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:02 smithi151 ceph-mon[35236]: osdmap e100: 12 total, 12 up, 12 in 2024-09-19T06:54:04.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 944004 7.65 MB/sec execute 511 sec latency 37.889 ms 2024-09-19T06:54:04.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:03 smithi096 ceph-mon[35004]: pgmap v387: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2024-09-19T06:54:04.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:03 smithi037 ceph-mon[31211]: pgmap v387: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2024-09-19T06:54:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:03 smithi151 ceph-mon[35236]: pgmap v387: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 71 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.7 MiB/s wr, 294 op/s 2024-09-19T06:54:05.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 945842 7.65 MB/sec execute 512 sec latency 57.477 ms 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 -- 172.21.15.37:0/1453938748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa610101370 msgr2=0x7fa61010d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1453938748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa610101370 0x7fa61010d8a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fa608009cd0 tx=0x7fa60802a8e0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 -- 172.21.15.37:0/1453938748 shutdown_connections 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1453938748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa610101370 0x7fa61010d8a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1453938748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6101009d0 0x7fa610100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1453938748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6101069e0 0x7fa610106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 -- 172.21.15.37:0/1453938748 >> 172.21.15.37:0/1453938748 conn(0x7fa6100fc6f0 msgr2=0x7fa6100feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.838 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 -- 172.21.15.37:0/1453938748 shutdown_connections 2024-09-19T06:54:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.836+0000 7fa617c8b640 1 -- 172.21.15.37:0/1453938748 wait complete. 2024-09-19T06:54:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 Processor -- start 2024-09-19T06:54:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 -- start start 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6101009d0 0x7fa6101025a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa610101370 0x7fa610102ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6101069e0 0x7fa610103020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110e70 con 0x7fa6101009d0 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110cf0 con 0x7fa610101370 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa617c8b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa610110ff0 con 0x7fa6101069e0 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6101009d0 0x7fa6101025a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6101009d0 0x7fa6101025a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41892/0 (socket says 172.21.15.37:41892) 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 -- 172.21.15.37:0/1193804878 learned_addr learned my addr 172.21.15.37:0/1193804878 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615226640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa610101370 0x7fa610102ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa616228640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6101069e0 0x7fa610103020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 -- 172.21.15.37:0/1193804878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6101069e0 msgr2=0x7fa610103020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6101069e0 0x7fa610103020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 -- 172.21.15.37:0/1193804878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa610101370 msgr2=0x7fa610102ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa610101370 0x7fa610102ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.837+0000 7fa615a27640 1 -- 172.21.15.37:0/1193804878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6101a6830 con 0x7fa6101009d0 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.838+0000 7fa615a27640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6101009d0 0x7fa6101025a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00d770 tx=0x7fa60c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.838+0000 7fa602ffd640 1 -- 172.21.15.37:0/1193804878 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c017070 con 0x7fa6101009d0 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.838+0000 7fa602ffd640 1 -- 172.21.15.37:0/1193804878 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa60c005500 con 0x7fa6101009d0 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.838+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6101a6b20 con 0x7fa6101009d0 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.838+0000 7fa602ffd640 1 -- 172.21.15.37:0/1193804878 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c004e00 con 0x7fa6101009d0 2024-09-19T06:54:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.838+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6101a7070 con 0x7fa6101009d0 2024-09-19T06:54:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.840+0000 7fa602ffd640 1 -- 172.21.15.37:0/1193804878 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa60c004fa0 con 0x7fa6101009d0 2024-09-19T06:54:05.849 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 Processor -- start 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 -- start start 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf436af0 0x55a1bf436ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf441770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf441cb0 0x55a1bf4440a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1bf39b2f0 con 0x55a1bf436af0 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1bf343270 con 0x55a1bf437410 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa61aa23740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a1bf3972b0 con 0x55a1bf441cb0 2024-09-19T06:54:05.850 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa616228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf436af0 0x55a1bf436ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa615a27640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf441770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa616228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf436af0 0x55a1bf436ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41900/0 (socket says 172.21.15.37:41900) 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa616228640 1 -- 172.21.15.37:0/666925900 learned_addr learned my addr 172.21.15.37:0/666925900 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.841+0000 7fa615226640 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf441cb0 0x55a1bf4440a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa602ffd640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5fc0a34e0 0x7fa5fc0a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa615a27640 1 -- 172.21.15.37:0/666925900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf441cb0 msgr2=0x55a1bf4440a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa615a27640 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf441cb0 0x55a1bf4440a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.851 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa615a27640 1 -- 172.21.15.37:0/666925900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf436af0 msgr2=0x55a1bf436ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa615a27640 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf436af0 0x55a1bf436ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa615a27640 1 -- 172.21.15.37:0/666925900 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bf342340 con 0x55a1bf437410 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa615226640 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf441cb0 0x55a1bf4440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa616228640 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf436af0 0x55a1bf436ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa616228640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5fc0a34e0 0x7fa5fc0a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa602ffd640 1 -- 172.21.15.37:0/1193804878 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8615+0+0 (secure 0 0 0) 0x7fa60c0cf740 con 0x7fa6101009d0 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa615a27640 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf441770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0ccdd0 tx=0x7fa60c090df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9317e91bc16cbe4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa5fbfff640 1 -- 172.21.15.37:0/666925900 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c09b4a0 con 0x55a1bf437410 2024-09-19T06:54:05.852 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa616228640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5fc0a34e0 0x7fa5fc0a59a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa608009a60 tx=0x7fa60803f040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa5fbfff640 1 -- 172.21.15.37:0/666925900 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa60c0046c0 con 0x55a1bf437410 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.842+0000 7fa5fbfff640 1 -- 172.21.15.37:0/666925900 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c09b2c0 con 0x55a1bf437410 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 -- 172.21.15.37:0/666925900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 msgr2=0x55a1bf441770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf441770 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0ccdd0 tx=0x7fa60c090df0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 -- 172.21.15.37:0/666925900 shutdown_connections 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf441cb0 0x55a1bf4440a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf441770 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 --2- 172.21.15.37:0/666925900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf436af0 0x55a1bf436ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 -- 172.21.15.37:0/666925900 >> 172.21.15.37:0/666925900 conn(0x55a1bf433060 msgr2=0x55a1bf433440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.853 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 -- 172.21.15.37:0/666925900 shutdown_connections 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 -- 172.21.15.37:0/666925900 wait complete. 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.843+0000 7fa61aa23740 1 -- start start 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 1 Processor -- start 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf43eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf437410 0x55a1bf43f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf441cb0 0x55a1bf43f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a1bf444820 con 0x55a1bf437410 2024-09-19T06:54:05.854 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a1bf324290 con 0x55a1bf441cb0 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a1bf4449a0 con 0x55a1bf436af0 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa61aa23740 10 client.0 _ll_register_callbacks cb 0x55a1bf355140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa615226640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf437410 0x55a1bf43f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa615226640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf437410 0x55a1bf43f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41910/0 (socket says 172.21.15.37:41910) 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa615226640 1 -- 172.21.15.37:0/2373014971 learned_addr learned my addr 172.21.15.37:0/2373014971 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa616228640 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf441cb0 0x55a1bf43f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.844+0000 7fa615a27640 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf43eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa615226640 1 -- 172.21.15.37:0/2373014971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 msgr2=0x55a1bf43eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa615226640 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf43eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa615226640 1 -- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf441cb0 msgr2=0x55a1bf43f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa615226640 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf441cb0 0x55a1bf43f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.855 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa615226640 1 -- 172.21.15.37:0/2373014971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bf0ae210 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa615226640 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf437410 0x55a1bf43f070 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fa604012700 tx=0x7fa604012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d005844af8351b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa5f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa5f97fa640 1 -- 172.21.15.37:0/2373014971 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604004050 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa5f97fa640 1 -- 172.21.15.37:0/2373014971 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8615+0+0 (secure 0 0 0) 0x7fa6040041f0 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a1bf3a8a20 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa5f97fa640 1 -- 172.21.15.37:0/2373014971 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa6040043f0 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa61aa23740 10 client.18840 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a1bf17a620 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa5f97fa640 1 -- 172.21.15.37:0/2373014971 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a1bf3a8a20 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa5f97fa640 1 -- 172.21.15.37:0/2373014971 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa604003bf0 con 0x55a1bf437410 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa61aa23740 10 client.18840 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:05.856 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.845+0000 7fa61aa23740 10 client.18840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.846+0000 7fa61aa23740 20 client.18840 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.846+0000 7fa61aa23740 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf451c50 0x55a1bf454040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.846+0000 7fa61aa23740 4 client.18840 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.846+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1bf3a2d70 con 0x55a1bf451c50 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.846+0000 7fa615a27640 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf451c50 0x55a1bf454040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.846+0000 7fa615a27640 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf451c50 0x55a1bf454040 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8310cc8b7b88d4cf server_cookie=b828a8bd8a1c895e in_seq=0 out_seq=0 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.846+0000 7fa5f97fa640 10 client.18840 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa5f97fa640 1 -- 172.21.15.37:0/2373014971 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a1bf3a2d70 con 0x55a1bf451c50 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa5f97fa640 10 client.18840 handle_command_reply: tid=0 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 client.18840 shutdown 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf437410 msgr2=0x55a1bf43f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf437410 0x55a1bf43f070 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fa604012700 tx=0x7fa604012bc0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.857 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 shutdown_connections 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf451c50 0x55a1bf454040 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf441cb0 0x55a1bf43f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf437410 0x55a1bf43f070 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 --2- 172.21.15.37:0/2373014971 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf43eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa5f97fa640 0 client.18840 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.847+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 >> 172.21.15.37:0/2373014971 conn(0x55a1bf433060 msgr2=0x55a1bf438dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.848+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 shutdown_connections 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.848+0000 7fa61aa23740 1 -- 172.21.15.37:0/2373014971 wait complete. 2024-09-19T06:54:05.858 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.848+0000 7fa61aa23740 20 client.18840 trim_cache size 0 max 16384 2024-09-19T06:54:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 Processor -- start 2024-09-19T06:54:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 -- start start 2024-09-19T06:54:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf440aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf43f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf459900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101b2400 con 0x55a1bf441cb0 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101afcd0 con 0x55a1bf437410 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.862+0000 7fa61aa23740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101adb40 con 0x55a1bf436af0 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf459900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa615226640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf43f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf459900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41924/0 (socket says 172.21.15.37:41924) 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 -- 172.21.15.37:0/434780611 learned_addr learned my addr 172.21.15.37:0/434780611 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa615a27640 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf440aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 -- 172.21.15.37:0/434780611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 msgr2=0x55a1bf440aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf440aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 -- 172.21.15.37:0/434780611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 msgr2=0x55a1bf43f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf43f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 -- 172.21.15.37:0/434780611 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1bf4543b0 con 0x55a1bf441cb0 2024-09-19T06:54:05.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa616228640 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf459900 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fa608035000 tx=0x7fa608041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6b1eccf5b9f6b12 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa5fb7fe640 1 -- 172.21.15.37:0/434780611 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608004040 con 0x55a1bf441cb0 2024-09-19T06:54:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa5fb7fe640 1 -- 172.21.15.37:0/434780611 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa608036040 con 0x55a1bf441cb0 2024-09-19T06:54:05.867 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.863+0000 7fa5fb7fe640 1 -- 172.21.15.37:0/434780611 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608030be0 con 0x55a1bf441cb0 2024-09-19T06:54:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 -- 172.21.15.37:0/434780611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 msgr2=0x55a1bf459900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf459900 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fa608035000 tx=0x7fa608041000 comp rx=0 tx=0).stop 2024-09-19T06:54:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 -- 172.21.15.37:0/434780611 shutdown_connections 2024-09-19T06:54:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf459900 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.870 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf437410 0x55a1bf43f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 --2- 172.21.15.37:0/434780611 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf436af0 0x55a1bf440aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 -- 172.21.15.37:0/434780611 >> 172.21.15.37:0/434780611 conn(0x55a1bf433060 msgr2=0x55a1bf440390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 -- 172.21.15.37:0/434780611 shutdown_connections 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 -- 172.21.15.37:0/434780611 wait complete. 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.864+0000 7fa61aa23740 1 -- start start 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 1 Processor -- start 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf436af0 0x55a1bf44a1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf437410 0x55a1bf43b660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf43bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ae190 con 0x55a1bf441cb0 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101af600 con 0x55a1bf436af0 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa6101ae860 con 0x55a1bf437410 2024-09-19T06:54:05.871 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa61aa23740 10 client.0 _ll_register_callbacks cb 0x7fa604009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf43bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf43bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41932/0 (socket says 172.21.15.37:41932) 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa615a27640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf436af0 0x55a1bf44a1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 -- 172.21.15.37:0/1891464515 learned_addr learned my addr 172.21.15.37:0/1891464515 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa615226640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf437410 0x55a1bf43b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 -- 172.21.15.37:0/1891464515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf437410 msgr2=0x55a1bf43b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf437410 0x55a1bf43b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 -- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf436af0 msgr2=0x55a1bf44a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf436af0 0x55a1bf44a1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa616228640 1 -- 172.21.15.37:0/1891464515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1bf43c1e0 con 0x55a1bf441cb0 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa615226640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf437410 0x55a1bf43b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:05.872 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.865+0000 7fa615a27640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf436af0 0x55a1bf44a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa616228640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf43bba0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa608053040 tx=0x7fa608030de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a2d4a193e28037 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa5f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa5f97fa640 1 -- 172.21.15.37:0/1891464515 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608003df0 con 0x55a1bf441cb0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa5f97fa640 1 -- 172.21.15.37:0/1891464515 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8615+0+0 (secure 0 0 0) 0x7fa608003f90 con 0x55a1bf441cb0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa608036040 con 0x55a1bf441cb0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa5f97fa640 1 -- 172.21.15.37:0/1891464515 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fa608004190 con 0x55a1bf441cb0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa61aa23740 10 client.18846 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55a1bf4bc710 con 0x55a1bf441cb0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa5f97fa640 1 -- 172.21.15.37:0/1891464515 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa608036040 con 0x55a1bf441cb0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa5f97fa640 1 -- 172.21.15.37:0/1891464515 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fa608030070 con 0x55a1bf441cb0 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa61aa23740 10 client.18846 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.866+0000 7fa61aa23740 10 client.18846 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:05.873 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.867+0000 7fa61aa23740 20 client.18846 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.867+0000 7fa61aa23740 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf4bce00 0x55a1bf4bf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.867+0000 7fa61aa23740 4 client.18846 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.867+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a1bf31f900 con 0x55a1bf4bce00 2024-09-19T06:54:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.867+0000 7fa615a27640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf4bce00 0x55a1bf4bf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.867+0000 7fa5f97fa640 10 client.18846 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:05.874 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:05.867+0000 7fa615a27640 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf4bce00 0x55a1bf4bf220 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0fbee5293cae4ac server_cookie=69524f64906d14d4 in_seq=0 out_seq=0 2024-09-19T06:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa5f97fa640 1 -- 172.21.15.37:0/1891464515 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a1bf31f900 con 0x55a1bf4bce00 2024-09-19T06:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa5f97fa640 10 client.18846 handle_command_reply: tid=0 2024-09-19T06:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 client.18846 shutdown 2024-09-19T06:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 msgr2=0x55a1bf43bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf43bba0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fa608053040 tx=0x7fa608030de0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 shutdown_connections 2024-09-19T06:54:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55a1bf4bce00 0x55a1bf4bf220 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a1bf441cb0 0x55a1bf43bba0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a1bf437410 0x55a1bf43b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 --2- 172.21.15.37:0/1891464515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a1bf436af0 0x55a1bf44a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 >> 172.21.15.37:0/1891464515 conn(0x55a1bf433060 msgr2=0x55a1bf4321c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa5f97fa640 0 client.18846 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 shutdown_connections 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 1 -- 172.21.15.37:0/1891464515 wait complete. 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.056+0000 7fa61aa23740 20 client.18846 trim_cache size 0 max 16384 2024-09-19T06:54:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5fc0a34e0 msgr2=0x7fa5fc0a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5fc0a34e0 0x7fa5fc0a59a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa608009a60 tx=0x7fa60803f040 comp rx=0 tx=0).stop 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6101009d0 msgr2=0x7fa6101025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6101009d0 0x7fa6101025a0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fa60c00d770 tx=0x7fa60c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 shutdown_connections 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa6101069e0 0x7fa610103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa610101370 0x7fa610102ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa5fc0a34e0 0x7fa5fc0a59a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 --2- 172.21.15.37:0/1193804878 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa6101009d0 0x7fa6101025a0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 >> 172.21.15.37:0/1193804878 conn(0x7fa6100fc6f0 msgr2=0x7fa6101046e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 shutdown_connections 2024-09-19T06:54:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:06.057+0000 7fa617c8b640 1 -- 172.21.15.37:0/1193804878 wait complete. 2024-09-19T06:54:06.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 948087 7.66 MB/sec execute 513 sec latency 97.366 ms 2024-09-19T06:54:06.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:05 smithi096 ceph-mon[35004]: pgmap v388: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-19T06:54:06.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:05 smithi037 ceph-mon[31211]: pgmap v388: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-19T06:54:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:05 smithi151 ceph-mon[35236]: pgmap v388: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-19T06:54:07.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 950140 7.67 MB/sec execute 514 sec latency 12.480 ms 2024-09-19T06:54:08.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:07 smithi151 ceph-mon[35236]: pgmap v389: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-19T06:54:08.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 952641 7.68 MB/sec execute 515 sec latency 13.521 ms 2024-09-19T06:54:08.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:07 smithi096 ceph-mon[35004]: pgmap v389: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-19T06:54:08.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:07 smithi037 ceph-mon[31211]: pgmap v389: 97 pgs: 20 active+clean+snaptrim, 36 active+clean+snaptrim_wait, 41 active+clean; 73 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.6 MiB/s wr, 279 op/s 2024-09-19T06:54:09.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 955114 7.69 MB/sec execute 516 sec latency 45.833 ms 2024-09-19T06:54:10.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 957154 7.70 MB/sec execute 517 sec latency 17.941 ms 2024-09-19T06:54:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:09 smithi096 ceph-mon[35004]: pgmap v390: 97 pgs: 97 active+clean; 57 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-09-19T06:54:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:54:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:09 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:10.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:09 smithi037 ceph-mon[31211]: pgmap v390: 97 pgs: 97 active+clean; 57 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-09-19T06:54:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:54:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:09 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:09 smithi151 ceph-mon[35236]: pgmap v390: 97 pgs: 97 active+clean; 57 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.6 MiB/s wr, 382 op/s 2024-09-19T06:54:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:54:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:10.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:09 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:11.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 959446 7.71 MB/sec execute 518 sec latency 103.598 ms 2024-09-19T06:54:11.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:10 smithi096 ceph-mon[35004]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T06:54:11.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:10 smithi037 ceph-mon[31211]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T06:54:11.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:10 smithi151 ceph-mon[35236]: osdmap e101: 12 total, 12 up, 12 in 2024-09-19T06:54:12.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 961621 7.71 MB/sec execute 519 sec latency 55.006 ms 2024-09-19T06:54:12.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:11 smithi096 ceph-mon[35004]: pgmap v392: 97 pgs: 97 active+clean; 57 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 429 op/s 2024-09-19T06:54:12.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:11 smithi037 ceph-mon[31211]: pgmap v392: 97 pgs: 97 active+clean; 57 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 429 op/s 2024-09-19T06:54:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:11 smithi151 ceph-mon[35236]: pgmap v392: 97 pgs: 97 active+clean; 57 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.1 MiB/s wr, 429 op/s 2024-09-19T06:54:13.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 963641 7.72 MB/sec execute 520 sec latency 13.729 ms 2024-09-19T06:54:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:13 smithi096 ceph-mon[35004]: pgmap v393: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 364 op/s 2024-09-19T06:54:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:54:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:54:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:13 smithi037 ceph-mon[31211]: pgmap v393: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 364 op/s 2024-09-19T06:54:14.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:54:14.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:54:14.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:13 smithi151 ceph-mon[35236]: pgmap v393: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.2 MiB/s wr, 364 op/s 2024-09-19T06:54:14.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:54:14.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:54:14.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:54:14.173 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:54:14.233 INFO:tasks.workunit.client.0.smithi037.stdout: 1 966509 7.74 MB/sec execute 521 sec latency 28.695 ms 2024-09-19T06:54:14.450 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:14.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 -- 172.21.15.37:0/3953318210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102570 msgr2=0x7facc41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 --2- 172.21.15.37:0/3953318210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102570 0x7facc41029b0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7facb8009900 tx=0x7facb802a480 comp rx=0 tx=0).stop 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 -- 172.21.15.37:0/3953318210 shutdown_connections 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 --2- 172.21.15.37:0/3953318210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc4102ef0 0x7facc410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 --2- 172.21.15.37:0/3953318210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102570 0x7facc41029b0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 --2- 172.21.15.37:0/3953318210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facc4100e20 0x7facc4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 -- 172.21.15.37:0/3953318210 >> 172.21.15.37:0/3953318210 conn(0x7facc40fc7d0 msgr2=0x7facc40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 -- 172.21.15.37:0/3953318210 shutdown_connections 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.937+0000 7facc99b3640 1 -- 172.21.15.37:0/3953318210 wait complete. 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 Processor -- start 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 -- start start 2024-09-19T06:54:14.946 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc4100e20 0x7facc41984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facc4102570 0x7facc4198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102ef0 0x7facc419cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7facc4110f80 con 0x7facc4102ef0 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7facc4110e00 con 0x7facc4102570 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc99b3640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7facc4111100 con 0x7facc4100e20 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102ef0 0x7facc419cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc4100e20 0x7facc41984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facc4102570 0x7facc4198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102ef0 0x7facc419cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44948/0 (socket says 172.21.15.37:44948) 2024-09-19T06:54:14.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc2ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc4100e20 0x7facc41984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:53700/0 (socket says 172.21.15.37:53700) 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.938+0000 7facc37fe640 1 -- 172.21.15.37:0/3205641416 learned_addr learned my addr 172.21.15.37:0/3205641416 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc37fe640 1 -- 172.21.15.37:0/3205641416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc4100e20 msgr2=0x7facc41984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc37fe640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc4100e20 0x7facc41984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc37fe640 1 -- 172.21.15.37:0/3205641416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facc4102570 msgr2=0x7facc4198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc37fe640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facc4102570 0x7facc4198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc37fe640 1 -- 172.21.15.37:0/3205641416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facc419d490 con 0x7facc4102ef0 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc27fc640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facc4102570 0x7facc4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc37fe640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102ef0 0x7facc419cdb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7facbc00eaa0 tx=0x7facbc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc89b1640 1 -- 172.21.15.37:0/3205641416 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc00cdb0 con 0x7facc4102ef0 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc89b1640 1 -- 172.21.15.37:0/3205641416 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facbc005020 con 0x7facc4102ef0 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facc419d780 con 0x7facc4102ef0 2024-09-19T06:54:14.948 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc89b1640 1 -- 172.21.15.37:0/3205641416 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facbc0116b0 con 0x7facc4102ef0 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.939+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facc41a5060 con 0x7facc4102ef0 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7facc89b1640 1 -- 172.21.15.37:0/3205641416 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7facbc005560 con 0x7facc4102ef0 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 Processor -- start 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 -- start start 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fe2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612fed140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612fefa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557612f482e0 con 0x557612fed680 2024-09-19T06:54:14.949 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557612eefce0 con 0x557612fe24a0 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7faccbd8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557612f442a0 con 0x557612fe2dc0 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7facc27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fe2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7facc89b1640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7facac0a3430 0x7facac0a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7facc27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fe2880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57302/0 (socket says 172.21.15.37:57302) 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.942+0000 7facc27fc640 1 -- 172.21.15.37:0/327837851 learned_addr learned my addr 172.21.15.37:0/327837851 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc2ffd640 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612fed140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc37fe640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7facac0a3430 0x7facac0a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc27fc640 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612fefa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc89b1640 1 -- 172.21.15.37:0/3205641416 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7facbc0cfe70 con 0x7facc4102ef0 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc2ffd640 1 -- 172.21.15.37:0/327837851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 msgr2=0x557612fe2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc2ffd640 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fe2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.950 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc2ffd640 1 -- 172.21.15.37:0/327837851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 msgr2=0x557612fefa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc2ffd640 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612fefa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc2ffd640 1 -- 172.21.15.37:0/327837851 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557612ed4440 con 0x557612fe2dc0 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc2ffd640 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612fed140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7facb4009c20 tx=0x7facb4011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f13e3cde9540e24 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7facc37fe640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7facac0a3430 0x7facac0a58f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7facbc09c160 tx=0x7facbc0bc040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.943+0000 7faca9ffb640 1 -- 172.21.15.37:0/327837851 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb4004050 con 0x557612fe2dc0 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faca9ffb640 1 -- 172.21.15.37:0/327837851 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb4005500 con 0x557612fe2dc0 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faca9ffb640 1 -- 172.21.15.37:0/327837851 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb40035f0 con 0x557612fe2dc0 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 -- 172.21.15.37:0/327837851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 msgr2=0x557612fed140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.951 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612fed140 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7facb4009c20 tx=0x7facb4011810 comp rx=0 tx=0).stop 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 -- 172.21.15.37:0/327837851 shutdown_connections 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612fefa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612fed140 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 --2- 172.21.15.37:0/327837851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fe2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 -- 172.21.15.37:0/327837851 >> 172.21.15.37:0/327837851 conn(0x557612fdea30 msgr2=0x557612fdee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 -- 172.21.15.37:0/327837851 shutdown_connections 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.944+0000 7faccbd8c740 1 -- 172.21.15.37:0/327837851 wait complete. 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 -- start start 2024-09-19T06:54:14.952 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 Processor -- start 2024-09-19T06:54:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fe2dc0 0x557612fea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fed680 0x557612fea710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x557612ff0250 con 0x557612fe2dc0 2024-09-19T06:54:14.955 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x557612f03050 con 0x557612fe24a0 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x557612ff03d0 con 0x557612fed680 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7faccbd8c740 10 client.0 _ll_register_callbacks cb 0x557612f43160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7facc27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7facc2ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fe2dc0 0x557612fea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7facc27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fecf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57306/0 (socket says 172.21.15.37:57306) 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.945+0000 7facc27fc640 1 -- 172.21.15.37:0/1119637098 learned_addr learned my addr 172.21.15.37:0/1119637098 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7facc27fc640 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fed680 0x557612fea710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7facc27fc640 1 -- 172.21.15.37:0/1119637098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fed680 msgr2=0x557612fea710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7facc27fc640 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fed680 0x557612fea710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7facc27fc640 1 -- 172.21.15.37:0/1119637098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fe2dc0 msgr2=0x557612fea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7facc27fc640 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fe2dc0 0x557612fea1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.956 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7facc27fc640 1 -- 172.21.15.37:0/1119637098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557612cb93c0 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7facc27fc640 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fecf20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7facb8009900 tx=0x7facb8005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7fa7312af47f029d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7faca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7faca2ffd640 1 -- 172.21.15.37:0/1119637098 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8035040 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb4005500 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7faca2ffd640 1 -- 172.21.15.37:0/1119637098 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7facb8005020 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7faca2ffd640 1 -- 172.21.15.37:0/1119637098 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7facb8003e70 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7faccbd8c740 10 client.28169 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.946+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557612ffbbc0 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faca2ffd640 1 -- 172.21.15.37:0/1119637098 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb4005500 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faca2ffd640 1 -- 172.21.15.37:0/1119637098 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7facb8031540 con 0x557612fe24a0 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faccbd8c740 10 client.28169 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faccbd8c740 10 client.28169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:14.957 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faccbd8c740 20 client.28169 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557612ffcfd0 0x557612fff3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faccbd8c740 4 client.28169 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557612f55850 con 0x557612ffcfd0 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7facc2ffd640 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557612ffcfd0 0x557612fff3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7facc2ffd640 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557612ffcfd0 0x557612fff3c0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=383fafdb5966e4f2 server_cookie=c6545fbd0164bf8f in_seq=0 out_seq=0 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.947+0000 7faca2ffd640 10 client.28169 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faca2ffd640 1 -- 172.21.15.37:0/1119637098 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557612f55850 con 0x557612ffcfd0 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faca2ffd640 10 client.28169 handle_command_reply: tid=0 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 client.28169 shutdown 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 msgr2=0x557612fecf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fecf20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7facb8009900 tx=0x7facb8005dd0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 shutdown_connections 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557612ffcfd0 0x557612fff3c0 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.958 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fed680 0x557612fea710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fe2dc0 0x557612fea1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1119637098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fecf20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 >> 172.21.15.37:0/1119637098 conn(0x557612fdea30 msgr2=0x557612fe4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.948+0000 7faca2ffd640 0 client.28169 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.949+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 shutdown_connections 2024-09-19T06:54:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.949+0000 7faccbd8c740 1 -- 172.21.15.37:0/1119637098 wait complete. 2024-09-19T06:54:14.959 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.949+0000 7faccbd8c740 20 client.28169 trim_cache size 0 max 16384 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 Processor -- start 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 -- start start 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fdda60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557613004c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x5576130051b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7facc41b0270 con 0x557612fed680 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7facc41adb40 con 0x557612fe24a0 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7faccbd8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7facc41ab9b0 con 0x557612fe2dc0 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x5576130051b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x5576130051b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44960/0 (socket says 172.21.15.37:44960) 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 -- 172.21.15.37:0/3574682702 learned_addr learned my addr 172.21.15.37:0/3574682702 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fdda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc2ffd640 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557613004c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 -- 172.21.15.37:0/3574682702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 msgr2=0x557613004c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557613004c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 -- 172.21.15.37:0/3574682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 msgr2=0x557612fdda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fdda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc27fc640 1 -- 172.21.15.37:0/3574682702 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557612fffa20 con 0x557612fed680 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.967+0000 7facc2ffd640 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557613004c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:14.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.968+0000 7facc27fc640 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x5576130051b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7facb8003880 tx=0x7facb8005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b094f81f48b1e9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.968+0000 7faca97fa640 1 -- 172.21.15.37:0/3574682702 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8000e60 con 0x557612fed680 2024-09-19T06:54:14.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.968+0000 7faca97fa640 1 -- 172.21.15.37:0/3574682702 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8005850 con 0x557612fed680 2024-09-19T06:54:14.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.968+0000 7faca97fa640 1 -- 172.21.15.37:0/3574682702 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8035070 con 0x557612fed680 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 -- 172.21.15.37:0/3574682702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 msgr2=0x5576130051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x5576130051b0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7facb8003880 tx=0x7facb8005fd0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 -- 172.21.15.37:0/3574682702 shutdown_connections 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x5576130051b0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557613004c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 --2- 172.21.15.37:0/3574682702 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612fdda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 -- 172.21.15.37:0/3574682702 >> 172.21.15.37:0/3574682702 conn(0x557612fdea30 msgr2=0x557612fdd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 -- 172.21.15.37:0/3574682702 shutdown_connections 2024-09-19T06:54:14.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 -- 172.21.15.37:0/3574682702 wait complete. 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 -- start start 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.969+0000 7faccbd8c740 1 Processor -- start 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612ff5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612ff2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faccbd8c740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612ff34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faccbd8c740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7facc41ac000 con 0x557612fed680 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faccbd8c740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7facc41ad470 con 0x557612fe24a0 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faccbd8c740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7facc41ac6d0 con 0x557612fe2dc0 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faccbd8c740 10 client.0 _ll_register_callbacks cb 0x7facb8009640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612ff5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612ff5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57326/0 (socket says 172.21.15.37:57326) 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 -- 172.21.15.37:0/1122932048 learned_addr learned my addr 172.21.15.37:0/1122932048 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:14.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612ff34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc2ffd640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612ff2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 -- 172.21.15.37:0/1122932048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 msgr2=0x557612ff2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612ff2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 -- 172.21.15.37:0/1122932048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 msgr2=0x557612ff5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612ff5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 -- 172.21.15.37:0/1122932048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557612ff3b80 con 0x557612fed680 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc2ffd640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612ff2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7facc27fc640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612ff34e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7facb8031b00 tx=0x7facb8005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba776160fd8c5bf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faca2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.970+0000 7faca2ffd640 1 -- 172.21.15.37:0/1122932048 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facb8035070 con 0x557612fed680 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faca2ffd640 1 -- 172.21.15.37:0/1122932048 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7facb803b660 con 0x557612fed680 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facb8005850 con 0x557612fed680 2024-09-19T06:54:14.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faca2ffd640 1 -- 172.21.15.37:0/1122932048 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7facb8057040 con 0x557612fed680 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 10 client.18864 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x557613067af0 con 0x557612fed680 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faca2ffd640 1 -- 172.21.15.37:0/1122932048 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7facb8005850 con 0x557612fed680 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faca2ffd640 1 -- 172.21.15.37:0/1122932048 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7facb8067070 con 0x557612fed680 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 10 client.18864 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 10 client.18864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 20 client.18864 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557613068100 0x55761306a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 4 client.18864 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557612feaaa0 con 0x557613068100 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.971+0000 7facc2ffd640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557613068100 0x55761306a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:14.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.972+0000 7faca2ffd640 10 client.18864 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.972+0000 7facc2ffd640 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557613068100 0x55761306a4f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b03a1be6ad15c615 server_cookie=576563b425558fec in_seq=0 out_seq=0 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.972+0000 7faca2ffd640 1 -- 172.21.15.37:0/1122932048 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x557612feaaa0 con 0x557613068100 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.972+0000 7faca2ffd640 10 client.18864 handle_command_reply: tid=0 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.972+0000 7faccbd8c740 1 client.18864 shutdown 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 msgr2=0x557612ff34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612ff34e0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7facb8031b00 tx=0x7facb8005780 comp rx=0 tx=0).stop 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 shutdown_connections 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x557613068100 0x55761306a4f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x557612fed680 0x557612ff34e0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x557612fe2dc0 0x557612ff2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 --2- 172.21.15.37:0/1122932048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x557612fe24a0 0x557612ff5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 >> 172.21.15.37:0/1122932048 conn(0x557612fdea30 msgr2=0x557612fdd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faca2ffd640 0 client.18864 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 shutdown_connections 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 1 -- 172.21.15.37:0/1122932048 wait complete. 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7faccbd8c740 20 client.18864 trim_cache size 0 max 16384 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:54:14.980 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:54:14.981 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:54:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7facac0a3430 msgr2=0x7facac0a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7facc99b3640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7facac0a3430 0x7facac0a58f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7facbc09c160 tx=0x7facbc0bc040 comp rx=0 tx=0).stop 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102ef0 msgr2=0x7facc419cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.973+0000 7facc99b3640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102ef0 0x7facc419cdb0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7facbc00eaa0 tx=0x7facbc00ef60 comp rx=0 tx=0).stop 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 shutdown_connections 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7facac0a3430 0x7facac0a58f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7facc4102ef0 0x7facc419cdb0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7facc4102570 0x7facc4198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 --2- 172.21.15.37:0/3205641416 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7facc4100e20 0x7facc41984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 >> 172.21.15.37:0/3205641416 conn(0x7facc40fc7d0 msgr2=0x7facc406a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 shutdown_connections 2024-09-19T06:54:14.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:14.974+0000 7facc99b3640 1 -- 172.21.15.37:0/3205641416 wait complete. 2024-09-19T06:54:15.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 968756 7.74 MB/sec execute 522 sec latency 24.758 ms 2024-09-19T06:54:15.868 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:15 smithi037 ceph-mon[31211]: pgmap v394: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 347 op/s 2024-09-19T06:54:15.869 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:54:15.869 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:54:16.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:15 smithi151 ceph-mon[35236]: pgmap v394: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 347 op/s 2024-09-19T06:54:16.154 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:16.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 970517 7.75 MB/sec execute 523 sec latency 57.462 ms 2024-09-19T06:54:16.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:15 smithi096 ceph-mon[35004]: pgmap v394: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 347 op/s 2024-09-19T06:54:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 -- 172.21.15.37:0/293002232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 msgr2=0x7f28b8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:16.653 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 --2- 172.21.15.37:0/293002232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b8103170 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009930 tx=0x7f28a802a420 comp rx=0 tx=0).stop 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 -- 172.21.15.37:0/293002232 shutdown_connections 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 --2- 172.21.15.37:0/293002232 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b8108290 0x7f28b810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 --2- 172.21.15.37:0/293002232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8103740 0x7f28b8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 --2- 172.21.15.37:0/293002232 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b8103170 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 -- 172.21.15.37:0/293002232 >> 172.21.15.37:0/293002232 conn(0x7f28b80fc7d0 msgr2=0x7f28b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 -- 172.21.15.37:0/293002232 shutdown_connections 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.639+0000 7f28be908640 1 -- 172.21.15.37:0/293002232 wait complete. 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 Processor -- start 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 -- start start 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b819a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:16.654 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8103740 0x7f28b819ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b8108290 0x7f28b819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8112090 con 0x7f28b8102d90 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8111f10 con 0x7f28b8103740 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28be908640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8112210 con 0x7f28b8108290 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b819a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28affff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8103740 0x7f28b819ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28b7fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b819a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44988/0 (socket says 172.21.15.37:44988) 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28b7fff640 1 -- 172.21.15.37:0/340630750 learned_addr learned my addr 172.21.15.37:0/340630750 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28bcea5640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b8108290 0x7f28b819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28b7fff640 1 -- 172.21.15.37:0/340630750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b8108290 msgr2=0x7f28b819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28b7fff640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b8108290 0x7f28b819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28b7fff640 1 -- 172.21.15.37:0/340630750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8103740 msgr2=0x7f28b819ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:16.655 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.640+0000 7f28b7fff640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8103740 0x7f28b819ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.641+0000 7f28b7fff640 1 -- 172.21.15.37:0/340630750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b819f6f0 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.641+0000 7f28b7fff640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b819a740 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009230 tx=0x7f28a8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.641+0000 7f28b5ffb640 1 -- 172.21.15.37:0/340630750 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a803f030 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.641+0000 7f28b5ffb640 1 -- 172.21.15.37:0/340630750 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f28a8003ad0 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.641+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b819f980 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.641+0000 7f28b5ffb640 1 -- 172.21.15.37:0/340630750 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a8035070 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.641+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b81a72c0 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.642+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b806a7f0 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.643+0000 7f28b5ffb640 1 -- 172.21.15.37:0/340630750 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f28a8042030 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.644+0000 7f28b5ffb640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f28a40a33b0 0x7f28a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.644+0000 7f28affff640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f28a40a33b0 0x7f28a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.644+0000 7f28b5ffb640 1 -- 172.21.15.37:0/340630750 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f28a80f0790 con 0x7f28b8102d90 2024-09-19T06:54:16.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.645+0000 7f28affff640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f28a40a33b0 0x7f28a40a5870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28b80fe560 tx=0x7f28a0005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:16.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.648+0000 7f28b5ffb640 1 -- 172.21.15.37:0/340630750 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f28a80bca90 con 0x7f28b8102d90 2024-09-19T06:54:16.874 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.873+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f28b8103170 con 0x7f28b8102d90 2024-09-19T06:54:16.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.874+0000 7f28b5ffb640 1 -- 172.21.15.37:0/340630750 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11903 (secure 0 0 0) 0x7f28a80c0d90 con 0x7f28b8102d90 2024-09-19T06:54:16.875 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:54:16.875 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":41,"btime":"2024-09-19T06:53:53:843716+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:53:53.843702+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27811":{"gid":27811,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/1999039793","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1999039793},{"type":"v1","addr":"172.21.15.151:6839","nonce":1999039793}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:54:16.876 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 41 2024-09-19T06:54:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f28a40a33b0 msgr2=0x7f28a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f28a40a33b0 0x7f28a40a5870 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f28b80fe560 tx=0x7f28a0005f90 comp rx=0 tx=0).stop 2024-09-19T06:54:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 msgr2=0x7f28b819a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:16.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b819a740 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f28a8009230 tx=0x7f28a8004270 comp rx=0 tx=0).stop 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 shutdown_connections 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f28b8108290 0x7f28b819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28b8103740 0x7f28b819ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f28a40a33b0 0x7f28a40a5870 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 --2- 172.21.15.37:0/340630750 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f28b8102d90 0x7f28b819a740 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 >> 172.21.15.37:0/340630750 conn(0x7f28b80fc7d0 msgr2=0x7f28b8100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 shutdown_connections 2024-09-19T06:54:16.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:16.877+0000 7f28be908640 1 -- 172.21.15.37:0/340630750 wait complete. 2024-09-19T06:54:17.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 973414 7.77 MB/sec execute 524 sec latency 20.567 ms 2024-09-19T06:54:17.630 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:54:17.900 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:18.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:17 smithi037 ceph-mon[31211]: pgmap v395: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 347 op/s 2024-09-19T06:54:18.018 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:17 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/340630750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:18.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:17 smithi151 ceph-mon[35236]: pgmap v395: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 347 op/s 2024-09-19T06:54:18.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:17 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/340630750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:18.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 974964 7.77 MB/sec execute 525 sec latency 93.287 ms 2024-09-19T06:54:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:17 smithi096 ceph-mon[35004]: pgmap v395: 97 pgs: 97 active+clean; 67 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.0 MiB/s wr, 347 op/s 2024-09-19T06:54:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:17 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/340630750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 -- 172.21.15.37:0/1821821688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 msgr2=0x7fb8a01029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 --2- 172.21.15.37:0/1821821688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a01029b0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fb894009900 tx=0x7fb89402a480 comp rx=0 tx=0).stop 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 -- 172.21.15.37:0/1821821688 shutdown_connections 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 --2- 172.21.15.37:0/1821821688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8a0102ef0 0x7fb8a010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 --2- 172.21.15.37:0/1821821688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a01029b0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 --2- 172.21.15.37:0/1821821688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8a0100e20 0x7fb8a0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 -- 172.21.15.37:0/1821821688 >> 172.21.15.37:0/1821821688 conn(0x7fb8a00fc7d0 msgr2=0x7fb8a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 -- 172.21.15.37:0/1821821688 shutdown_connections 2024-09-19T06:54:18.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 -- 172.21.15.37:0/1821821688 wait complete. 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 Processor -- start 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.359+0000 7fb8a6390640 1 -- start start 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb8a6390640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8a0100e20 0x7fb8a01970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb8a6390640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a0197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb8a6390640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8a0102ef0 0x7fb8a0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb8a6390640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0110f80 con 0x7fb8a0102570 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb8a6390640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0110e00 con 0x7fb8a0100e20 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb8a6390640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a0111100 con 0x7fb8a0102ef0 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a0197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a0197600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45012/0 (socket says 172.21.15.37:45012) 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 -- 172.21.15.37:0/2216049918 learned_addr learned my addr 172.21.15.37:0/2216049918 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:18.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89ffff640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8a0100e20 0x7fb8a01970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8a0102ef0 0x7fb8a0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 -- 172.21.15.37:0/2216049918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8a0102ef0 msgr2=0x7fb8a0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8a0102ef0 0x7fb8a0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 -- 172.21.15.37:0/2216049918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8a0100e20 msgr2=0x7fb8a01970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8a0100e20 0x7fb8a01970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 -- 172.21.15.37:0/2216049918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8a01a6930 con 0x7fb8a0102570 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89ffff640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8a0100e20 0x7fb8a01970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.360+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a0197600 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb894006010 tx=0x7fb894003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.361+0000 7fb89d7fa640 1 -- 172.21.15.37:0/2216049918 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb89403f070 con 0x7fb8a0102570 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.361+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8a01a6bc0 con 0x7fb8a0102570 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.361+0000 7fb89d7fa640 1 -- 172.21.15.37:0/2216049918 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb894031c30 con 0x7fb8a0102570 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.361+0000 7fb89d7fa640 1 -- 172.21.15.37:0/2216049918 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb894035030 con 0x7fb8a0102570 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.361+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8a01a70a0 con 0x7fb8a0102570 2024-09-19T06:54:18.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.363+0000 7fb89d7fa640 1 -- 172.21.15.37:0/2216049918 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb894042070 con 0x7fb8a0102570 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.363+0000 7fb8a8769740 1 Processor -- start 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.363+0000 7fb8a8769740 1 -- start start 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e151b40 0x56055e150090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e150a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e15b370 0x56055e15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a8769740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56055e0b62f0 con 0x56055e151b40 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a8769740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56055e05dcf0 con 0x56055e1505d0 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a8769740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56055e0b22b0 con 0x56055e15b370 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e151b40 0x56055e150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e151b40 0x56055e150090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45016/0 (socket says 172.21.15.37:45016) 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 -- 172.21.15.37:0/3084169481 learned_addr learned my addr 172.21.15.37:0/3084169481 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb89ffff640 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb89d7fa640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8880a33b0 0x7fb8880a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 -- 172.21.15.37:0/3084169481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e15b370 msgr2=0x56055e15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e15b370 0x56055e15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 -- 172.21.15.37:0/3084169481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 msgr2=0x56055e150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 -- 172.21.15.37:0/3084169481 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055e0423a0 con 0x56055e151b40 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb89d7fa640 1 -- 172.21.15.37:0/2216049918 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fb8940f0290 con 0x7fb8a0102570 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e151b40 0x56055e150090 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb898007f40 tx=0x7fb89800f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f91b1654df201 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.364+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8880a33b0 0x7fb8880a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8867fc640 1 -- 172.21.15.37:0/3084169481 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898010b20 con 0x56055e151b40 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8867fc640 1 -- 172.21.15.37:0/3084169481 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89801adb0 con 0x56055e151b40 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8867fc640 1 -- 172.21.15.37:0/3084169481 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898019b70 con 0x56055e151b40 2024-09-19T06:54:18.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a8769740 1 -- 172.21.15.37:0/3084169481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e151b40 msgr2=0x56055e150090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a8769740 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e151b40 0x56055e150090 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fb898007f40 tx=0x7fb89800f850 comp rx=0 tx=0).stop 2024-09-19T06:54:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8880a33b0 0x7fb8880a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb89801e000 tx=0x7fb89801f000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a8769740 1 -- 172.21.15.37:0/3084169481 shutdown_connections 2024-09-19T06:54:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a8769740 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e15b370 0x56055e15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a8769740 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a8769740 1 --2- 172.21.15.37:0/3084169481 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e151b40 0x56055e150090 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.365+0000 7fb8a8769740 1 -- 172.21.15.37:0/3084169481 >> 172.21.15.37:0/3084169481 conn(0x56055e119310 msgr2=0x56055e14cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 -- 172.21.15.37:0/3084169481 shutdown_connections 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 -- 172.21.15.37:0/3084169481 wait complete. 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 -- start start 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 Processor -- start 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e1505d0 0x56055e15ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e151b40 0x56055e158010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e158600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56055e15e070 con 0x56055e15b370 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56055e071060 con 0x56055e151b40 2024-09-19T06:54:18.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56055e15e1f0 con 0x56055e1505d0 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.366+0000 7fb8a8769740 10 client.0 _ll_register_callbacks cb 0x56055ddc48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45028/0 (socket says 172.21.15.37:45028) 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 -- 172.21.15.37:0/2947790933 learned_addr learned my addr 172.21.15.37:0/2947790933 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e151b40 0x56055e158010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb89ffff640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e1505d0 0x56055e15ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 -- 172.21.15.37:0/2947790933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e1505d0 msgr2=0x56055e15ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e1505d0 0x56055e15ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 -- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e151b40 msgr2=0x56055e158010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e151b40 0x56055e158010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 -- 172.21.15.37:0/2947790933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055de273c0 con 0x56055e15b370 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e151b40 0x56055e158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e158600 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb898010d10 tx=0x7fb898004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb6e64112b20af02 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb87f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb87f7fe640 1 -- 172.21.15.37:0/2947790933 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898034060 con 0x56055e15b370 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb87f7fe640 1 -- 172.21.15.37:0/2947790933 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fb89802c070 con 0x56055e15b370 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb89801adb0 con 0x56055e15b370 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb87f7fe640 1 -- 172.21.15.37:0/2947790933 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb898019970 con 0x56055e15b370 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a8769740 10 client.18882 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.367+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56055e169b40 con 0x56055e15b370 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb87f7fe640 1 -- 172.21.15.37:0/2947790933 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb89801adb0 con 0x56055e15b370 2024-09-19T06:54:18.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb87f7fe640 1 -- 172.21.15.37:0/2947790933 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb898004030 con 0x56055e15b370 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb8a8769740 10 client.18882 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb8a8769740 10 client.18882 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb8a8769740 20 client.18882 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e16af40 0x56055e16d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb8a8769740 4 client.18882 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56055e0c3860 con 0x56055e16af40 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb89ffff640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e16af40 0x56055e16d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb87f7fe640 10 client.18882 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.368+0000 7fb89ffff640 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e16af40 0x56055e16d330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61fe62017fb0c5b2 server_cookie=d5c9f543a4e5dbcb in_seq=0 out_seq=0 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb87f7fe640 1 -- 172.21.15.37:0/2947790933 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56055e0c3860 con 0x56055e16af40 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb87f7fe640 10 client.18882 handle_command_reply: tid=0 2024-09-19T06:54:18.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 client.18882 shutdown 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 msgr2=0x56055e158600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e158600 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fb898010d10 tx=0x7fb898004a00 comp rx=0 tx=0).stop 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 shutdown_connections 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e16af40 0x56055e16d330 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e158600 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e151b40 0x56055e158010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2947790933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e1505d0 0x56055e15ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb87f7fe640 0 client.18882 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 >> 172.21.15.37:0/2947790933 conn(0x56055e119310 msgr2=0x56055e152930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:18.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 shutdown_connections 2024-09-19T06:54:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.369+0000 7fb8a8769740 1 -- 172.21.15.37:0/2947790933 wait complete. 2024-09-19T06:54:18.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.370+0000 7fb8a8769740 20 client.18882 trim_cache size 0 max 16384 2024-09-19T06:54:18.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.382+0000 7fb8a8769740 1 Processor -- start 2024-09-19T06:54:18.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.382+0000 7fb8a8769740 1 -- start start 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e1568e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e170350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e170890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a8769740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01b2330 con 0x56055e15b370 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a8769740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01afc00 con 0x56055e1505d0 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a8769740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01ada50 con 0x56055e151b40 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e170890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e170890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45040/0 (socket says 172.21.15.37:45040) 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 -- 172.21.15.37:0/2538007770 learned_addr learned my addr 172.21.15.37:0/2538007770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e170350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb89ffff640 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e1568e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 -- 172.21.15.37:0/2538007770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 msgr2=0x56055e170350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e170350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 -- 172.21.15.37:0/2538007770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 msgr2=0x56055e1568e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e1568e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 -- 172.21.15.37:0/2538007770 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56055e153120 con 0x56055e15b370 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb89ffff640 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e170350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.383+0000 7fb8a492d640 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e170890 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb8980044d0 tx=0x7fb898007d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b03af3811bba1bb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb885ffb640 1 -- 172.21.15.37:0/2538007770 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898034070 con 0x56055e15b370 2024-09-19T06:54:18.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb885ffb640 1 -- 172.21.15.37:0/2538007770 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb898019b10 con 0x56055e15b370 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb885ffb640 1 -- 172.21.15.37:0/2538007770 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898003f90 con 0x56055e15b370 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 -- 172.21.15.37:0/2538007770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 msgr2=0x56055e170890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e170890 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fb8980044d0 tx=0x7fb898007d90 comp rx=0 tx=0).stop 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 -- 172.21.15.37:0/2538007770 shutdown_connections 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e170890 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e170350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 --2- 172.21.15.37:0/2538007770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e1568e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 -- 172.21.15.37:0/2538007770 >> 172.21.15.37:0/2538007770 conn(0x56055e119310 msgr2=0x56055e15a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 -- 172.21.15.37:0/2538007770 shutdown_connections 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.384+0000 7fb8a8769740 1 -- 172.21.15.37:0/2538007770 wait complete. 2024-09-19T06:54:18.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 -- start start 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 Processor -- start 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e170a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e16ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e16e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01ae0c0 con 0x56055e15b370 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01af530 con 0x56055e1505d0 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8a01ae790 con 0x56055e151b40 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a8769740 10 client.0 _ll_register_callbacks cb 0x7fb89801d020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e16e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a492d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e16e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45048/0 (socket says 172.21.15.37:45048) 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb8a492d640 1 -- 172.21.15.37:0/1539926002 learned_addr learned my addr 172.21.15.37:0/1539926002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:18.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb89ffff640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e170a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.385+0000 7fb89f7fe640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e16ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a492d640 1 -- 172.21.15.37:0/1539926002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 msgr2=0x56055e16ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a492d640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e16ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a492d640 1 -- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 msgr2=0x56055e170a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a492d640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e170a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a492d640 1 -- 172.21.15.37:0/1539926002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56055e16eb10 con 0x56055e15b370 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb89ffff640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e170a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a492d640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e16e310 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb898019ce0 tx=0x7fb89801adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e11c063c6a75e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb87f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb87f7fe640 1 -- 172.21.15.37:0/1539926002 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb898034030 con 0x56055e15b370 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb87f7fe640 1 -- 172.21.15.37:0/1539926002 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fb898020040 con 0x56055e15b370 2024-09-19T06:54:18.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb898019b10 con 0x56055e15b370 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb87f7fe640 1 -- 172.21.15.37:0/1539926002 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb898052050 con 0x56055e15b370 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a8769740 10 client.18888 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56055e1d57c0 con 0x56055e15b370 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.386+0000 7fb87f7fe640 1 -- 172.21.15.37:0/1539926002 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb898019b10 con 0x56055e15b370 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb87f7fe640 1 -- 172.21.15.37:0/1539926002 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb898026480 con 0x56055e15b370 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb8a8769740 10 client.18888 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb8a8769740 10 client.18888 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb8a8769740 20 client.18888 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb8a8769740 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e1d5d70 0x56055e1d8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb8a8769740 4 client.18888 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56055e0bdd70 con 0x56055e1d5d70 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb89ffff640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e1d5d70 0x56055e1d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:18.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb89ffff640 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e1d5d70 0x56055e1d8160 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bfc0dff6205eb2a3 server_cookie=15ac366f63a3229d in_seq=0 out_seq=0 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb87f7fe640 10 client.18888 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb87f7fe640 1 -- 172.21.15.37:0/1539926002 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56055e0bdd70 con 0x56055e1d5d70 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb87f7fe640 10 client.18888 handle_command_reply: tid=0 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.387+0000 7fb8a8769740 1 client.18888 shutdown 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 msgr2=0x56055e16e310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e16e310 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fb898019ce0 tx=0x7fb89801adf0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 shutdown_connections 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x56055e1d5d70 0x56055e1d8160 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56055e15b370 0x56055e16e310 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56055e151b40 0x56055e16ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 --2- 172.21.15.37:0/1539926002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56055e1505d0 0x56055e170a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 >> 172.21.15.37:0/1539926002 conn(0x56055e119310 msgr2=0x56055e14b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:18.396 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb87f7fe640 0 client.18888 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:54:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 shutdown_connections 2024-09-19T06:54:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 1 -- 172.21.15.37:0/1539926002 wait complete. 2024-09-19T06:54:18.397 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a8769740 20 client.18888 trim_cache size 0 max 16384 2024-09-19T06:54:18.397 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:54:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8880a33b0 msgr2=0x7fb8880a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a6390640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8880a33b0 0x7fb8880a5870 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb89801e000 tx=0x7fb89801f000 comp rx=0 tx=0).stop 2024-09-19T06:54:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 msgr2=0x7fb8a0197600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.388+0000 7fb8a6390640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a0197600 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fb894006010 tx=0x7fb894003c10 comp rx=0 tx=0).stop 2024-09-19T06:54:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 shutdown_connections 2024-09-19T06:54:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb8a0102ef0 0x7fb8a0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8880a33b0 0x7fb8880a5870 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb8a0102570 0x7fb8a0197600 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 --2- 172.21.15.37:0/2216049918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb8a0100e20 0x7fb8a01970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 >> 172.21.15.37:0/2216049918 conn(0x7fb8a00fc7d0 msgr2=0x7fb8a0105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 shutdown_connections 2024-09-19T06:54:18.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:18.389+0000 7fb8a6390640 1 -- 172.21.15.37:0/2216049918 wait complete. 2024-09-19T06:54:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 -- 172.21.15.37:0/1902426123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 msgr2=0x7f0d0810baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 --2- 172.21.15.37:0/1902426123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 0x7f0d0810baa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc009860 tx=0x7f0cfc02a3d0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 -- 172.21.15.37:0/1902426123 shutdown_connections 2024-09-19T06:54:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 --2- 172.21.15.37:0/1902426123 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 0x7f0d0810e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 --2- 172.21.15.37:0/1902426123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 0x7f0d0810baa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 --2- 172.21.15.37:0/1902426123 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d08102ca0 0x7f0d08103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.154 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 -- 172.21.15.37:0/1902426123 >> 172.21.15.37:0/1902426123 conn(0x7f0d080fc6f0 msgr2=0x7f0d080feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 -- 172.21.15.37:0/1902426123 shutdown_connections 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.151+0000 7f0d0d366640 1 -- 172.21.15.37:0/1902426123 wait complete. 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 Processor -- start 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 -- start start 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d08102ca0 0x7f0d08198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 0x7f0d08198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 0x7f0d0819cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08111ee0 con 0x7f0d0810c0c0 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08111d60 con 0x7f0d081035c0 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d0d366640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08112060 con 0x7f0d08102ca0 2024-09-19T06:54:19.155 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 0x7f0d0819cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 0x7f0d08198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 0x7f0d0819cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45068/0 (socket says 172.21.15.37:45068) 2024-09-19T06:54:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 -- 172.21.15.37:0/2909511675 learned_addr learned my addr 172.21.15.37:0/2909511675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d08102ca0 0x7f0d08198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 -- 172.21.15.37:0/2909511675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d08102ca0 msgr2=0x7f0d08198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d08102ca0 0x7f0d08198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.156 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 -- 172.21.15.37:0/2909511675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 msgr2=0x7f0d08198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 0x7f0d08198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.152+0000 7f0d077fe640 1 -- 172.21.15.37:0/2909511675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0819d320 con 0x7f0d0810c0c0 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.153+0000 7f0d077fe640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 0x7f0d0819cbf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000efc0 tx=0x7f0d0000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.153+0000 7f0ceffff640 1 -- 172.21.15.37:0/2909511675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00003970 con 0x7f0d0810c0c0 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.153+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0819d610 con 0x7f0d0810c0c0 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.153+0000 7f0ceffff640 1 -- 172.21.15.37:0/2909511675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d00004020 con 0x7f0d0810c0c0 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.153+0000 7f0ceffff640 1 -- 172.21.15.37:0/2909511675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0000b9f0 con 0x7f0d0810c0c0 2024-09-19T06:54:19.157 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.153+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d081a4ea0 con 0x7f0d0810c0c0 2024-09-19T06:54:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.155+0000 7f0ceffff640 1 -- 172.21.15.37:0/2909511675 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0d0001d020 con 0x7f0d0810c0c0 2024-09-19T06:54:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.155+0000 7f0d101dc740 1 Processor -- start 2024-09-19T06:54:19.167 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.155+0000 7f0d101dc740 1 -- start start 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe452f9d0 0x556fe452fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe45414b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d101dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556fe44942f0 con 0x556fe45302f0 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d101dc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556fe443c270 con 0x556fe452f9d0 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d101dc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556fe44902b0 con 0x556fe453f0c0 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe452f9d0 0x556fe452fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe452f9d0 0x556fe452fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57448/0 (socket says 172.21.15.37:57448) 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 -- 172.21.15.37:0/1011726711 learned_addr learned my addr 172.21.15.37:0/1011726711 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:19.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe45414b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0ceffff640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0cf00a3460 0x7f0cf00a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 -- 172.21.15.37:0/1011726711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 msgr2=0x556fe45414b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe45414b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 -- 172.21.15.37:0/1011726711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 msgr2=0x556fe453eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 -- 172.21.15.37:0/1011726711 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe443b340 con 0x556fe452f9d0 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d077fe640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0cf00a3460 0x7f0cf00a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe452f9d0 0x556fe452fdb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02f4b0 tx=0x7f0cfc005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=967398e0a4a9858 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.156+0000 7f0ceffff640 1 -- 172.21.15.37:0/2909511675 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f0d000cfb60 con 0x7f0d0810c0c0 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0ced7fa640 1 -- 172.21.15.37:0/1011726711 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc004050 con 0x556fe452f9d0 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0ced7fa640 1 -- 172.21.15.37:0/1011726711 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cfc030880 con 0x556fe452f9d0 2024-09-19T06:54:19.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0ced7fa640 1 -- 172.21.15.37:0/1011726711 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc005500 con 0x556fe452f9d0 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 -- 172.21.15.37:0/1011726711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe452f9d0 msgr2=0x556fe452fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d077fe640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0cf00a3460 0x7f0cf00a5920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d000cd880 tx=0x7f0d000bc040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe452f9d0 0x556fe452fdb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02f4b0 tx=0x7f0cfc005a90 comp rx=0 tx=0).stop 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 -- 172.21.15.37:0/1011726711 shutdown_connections 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe45414b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1011726711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe452f9d0 0x556fe452fdb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 -- 172.21.15.37:0/1011726711 >> 172.21.15.37:0/1011726711 conn(0x556fe452c060 msgr2=0x556fe452c440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 -- 172.21.15.37:0/1011726711 shutdown_connections 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.157+0000 7f0d101dc740 1 -- 172.21.15.37:0/1011726711 wait complete. 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 -- start start 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 Processor -- start 2024-09-19T06:54:19.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe453caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x556fe4541c30 con 0x556fe45302f0 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556fe441d290 con 0x556fe453f0c0 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x556fe4541db0 con 0x556fe452f9d0 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d101dc740 10 client.0 _ll_register_callbacks cb 0x556fe444e140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45086/0 (socket says 172.21.15.37:45086) 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d06ffd640 1 -- 172.21.15.37:0/1991287777 learned_addr learned my addr 172.21.15.37:0/1991287777 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe453caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.158+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d06ffd640 1 -- 172.21.15.37:0/1991287777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 msgr2=0x556fe453c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d06ffd640 1 -- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 msgr2=0x556fe453caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe453caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d06ffd640 1 -- 172.21.15.37:0/1991287777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe41a7210 con 0x556fe45302f0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453c5b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8012740 tx=0x7f0cf8012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ace166a1d0af4f5a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0ceaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0ceaff5640 1 -- 172.21.15.37:0/1991287777 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8004050 con 0x556fe45302f0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0ceaff5640 1 -- 172.21.15.37:0/1991287777 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f0cf80041f0 con 0x556fe45302f0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cfc030880 con 0x556fe45302f0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0ceaff5640 1 -- 172.21.15.37:0/1991287777 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0cf80043f0 con 0x556fe45302f0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d101dc740 10 client.18900 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556fe4273620 con 0x556fe45302f0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0ceaff5640 1 -- 172.21.15.37:0/1991287777 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cfc030880 con 0x556fe45302f0 2024-09-19T06:54:19.172 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.159+0000 7f0ceaff5640 1 -- 172.21.15.37:0/1991287777 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0cf8003c40 con 0x556fe45302f0 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d101dc740 10 client.18900 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d101dc740 10 client.18900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d101dc740 20 client.18900 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe454f160 0x556fe4551550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d101dc740 4 client.18900 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe44a1a20 con 0x556fe454f160 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe454f160 0x556fe4551550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0d067fc640 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe454f160 0x556fe4551550 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74bcbf10548b3fe server_cookie=6d6f17717b1d93ce in_seq=0 out_seq=0 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.160+0000 7f0ceaff5640 10 client.18900 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0ceaff5640 1 -- 172.21.15.37:0/1991287777 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556fe44a1a20 con 0x556fe454f160 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0ceaff5640 10 client.18900 handle_command_reply: tid=0 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 client.18900 shutdown 2024-09-19T06:54:19.173 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 msgr2=0x556fe453c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453c5b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8012740 tx=0x7f0cf8012c00 comp rx=0 tx=0).stop 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 shutdown_connections 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe454f160 0x556fe4551550 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe453caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe453c5b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0ceaff5640 0 client.18900 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 --2- 172.21.15.37:0/1991287777 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 >> 172.21.15.37:0/1991287777 conn(0x556fe452c060 msgr2=0x556fe4531dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 shutdown_connections 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 1 -- 172.21.15.37:0/1991287777 wait complete. 2024-09-19T06:54:19.174 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.162+0000 7f0d101dc740 20 client.18900 trim_cache size 0 max 16384 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 Processor -- start 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 -- start start 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe452b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe4556e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081adea0 con 0x556fe45302f0 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081ab770 con 0x556fe453f0c0 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.177+0000 7f0d101dc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a95e0 con 0x556fe452f9d0 2024-09-19T06:54:19.179 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe452b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe452b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45098/0 (socket says 172.21.15.37:45098) 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 -- 172.21.15.37:0/4271423675 learned_addr learned my addr 172.21.15.37:0/4271423675 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d067fc640 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe4556e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 -- 172.21.15.37:0/4271423675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 msgr2=0x556fe453c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 -- 172.21.15.37:0/4271423675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 msgr2=0x556fe4556e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe4556e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 -- 172.21.15.37:0/4271423675 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe4530120 con 0x556fe45302f0 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe452b0b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80106a0 tx=0x7f0cf80109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37401e31c7fd0c6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0cecff9640 1 -- 172.21.15.37:0/4271423675 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8028040 con 0x556fe45302f0 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0cecff9640 1 -- 172.21.15.37:0/4271423675 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8012e50 con 0x556fe45302f0 2024-09-19T06:54:19.180 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.178+0000 7f0cecff9640 1 -- 172.21.15.37:0/4271423675 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8005300 con 0x556fe45302f0 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 -- 172.21.15.37:0/4271423675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 msgr2=0x556fe452b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe452b0b0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f0cf80106a0 tx=0x7f0cf80109e0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 -- 172.21.15.37:0/4271423675 shutdown_connections 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe453f0c0 0x556fe4556e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe45302f0 0x556fe452b0b0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 --2- 172.21.15.37:0/4271423675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe452f9d0 0x556fe453c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 -- 172.21.15.37:0/4271423675 >> 172.21.15.37:0/4271423675 conn(0x556fe452c060 msgr2=0x556fe452a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 -- 172.21.15.37:0/4271423675 shutdown_connections 2024-09-19T06:54:19.184 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 -- 172.21.15.37:0/4271423675 wait complete. 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 -- start start 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.179+0000 7f0d101dc740 1 Processor -- start 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe452f9d0 0x556fe4539190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe45302f0 0x556fe45396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe4539c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081a9c30 con 0x556fe452f9d0 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081ab0a0 con 0x556fe45302f0 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0d081aa300 con 0x556fe453f0c0 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 10 client.0 _ll_register_callbacks cb 0x7f0cf8009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe452f9d0 0x556fe4539190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe45302f0 0x556fe45396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe452f9d0 0x556fe4539190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:45110/0 (socket says 172.21.15.37:45110) 2024-09-19T06:54:19.185 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe45302f0 0x556fe45396d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:57486/0 (socket says 172.21.15.37:57486) 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 -- 172.21.15.37:0/2389298957 learned_addr learned my addr 172.21.15.37:0/2389298957 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe4539c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 -- 172.21.15.37:0/2389298957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 msgr2=0x556fe4539c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe4539c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 -- 172.21.15.37:0/2389298957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe45302f0 msgr2=0x556fe45396d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe45302f0 0x556fe45396d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 -- 172.21.15.37:0/2389298957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe45b8f40 con 0x556fe452f9d0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d067fc640 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe452f9d0 0x556fe4539190 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02a8f0 tx=0x7f0cfc005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40a81c9ab770abc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0ceaff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0ceaff5640 1 -- 172.21.15.37:0/2389298957 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc005410 con 0x556fe452f9d0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0ceaff5640 1 -- 172.21.15.37:0/2389298957 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f0cfc0055b0 con 0x556fe452f9d0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.180+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cf8012e50 con 0x556fe452f9d0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0ceaff5640 1 -- 172.21.15.37:0/2389298957 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc004020 con 0x556fe452f9d0 2024-09-19T06:54:19.186 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 10 client.18906 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x556fe45ba230 con 0x556fe452f9d0 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0ceaff5640 1 -- 172.21.15.37:0/2389298957 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf8012e50 con 0x556fe452f9d0 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0ceaff5640 1 -- 172.21.15.37:0/2389298957 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f0cfc0035f0 con 0x556fe452f9d0 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 10 client.18906 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 10 client.18906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 20 client.18906 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe45ba510 0x556fe45bc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 4 client.18906 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556fe449bd70 con 0x556fe45ba510 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.181+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe45ba510 0x556fe45bc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.182+0000 7f0d06ffd640 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe45ba510 0x556fe45bc900 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591a01e1907c51ad server_cookie=bd892b807aec2440 in_seq=0 out_seq=0 2024-09-19T06:54:19.187 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.182+0000 7f0ceaff5640 10 client.18906 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:19.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 977315 7.78 MB/sec execute 526 sec latency 58.110 ms 2024-09-19T06:54:19.306 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:54:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0ceaff5640 1 -- 172.21.15.37:0/2389298957 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556fe449bd70 con 0x556fe45ba510 2024-09-19T06:54:19.412 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0ceaff5640 10 client.18906 handle_command_reply: tid=0 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 client.18906 shutdown 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe452f9d0 msgr2=0x556fe4539190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe452f9d0 0x556fe4539190 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc02a8f0 tx=0x7f0cfc005a90 comp rx=0 tx=0).stop 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 shutdown_connections 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x556fe45ba510 0x556fe45bc900 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x556fe453f0c0 0x556fe4539c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556fe45302f0 0x556fe45396d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 --2- 172.21.15.37:0/2389298957 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x556fe452f9d0 0x556fe4539190 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 >> 172.21.15.37:0/2389298957 conn(0x556fe452c060 msgr2=0x556fe452b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.411+0000 7f0ceaff5640 0 client.18906 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 shutdown_connections 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d101dc740 1 -- 172.21.15.37:0/2389298957 wait complete. 2024-09-19T06:54:19.413 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d101dc740 20 client.18906 trim_cache size 0 max 16384 2024-09-19T06:54:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:54:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:54:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:54:19.414 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0cf00a3460 msgr2=0x7f0cf00a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0cf00a3460 0x7f0cf00a5920 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0d000cd880 tx=0x7f0d000bc040 comp rx=0 tx=0).stop 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 msgr2=0x7f0d0819cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 0x7f0d0819cbf0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f0d0000efc0 tx=0x7f0d0000c800 comp rx=0 tx=0).stop 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 shutdown_connections 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0cf00a3460 0x7f0cf00a5920 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0d0810c0c0 0x7f0d0819cbf0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0d081035c0 0x7f0d08198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 --2- 172.21.15.37:0/2909511675 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0d08102ca0 0x7f0d08198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.412+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 >> 172.21.15.37:0/2909511675 conn(0x7f0d080fc6f0 msgr2=0x7f0d080fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.413+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 shutdown_connections 2024-09-19T06:54:19.415 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:19.413+0000 7f0d0d366640 1 -- 172.21.15.37:0/2909511675 wait complete. 2024-09-19T06:54:19.619 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:20.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:19 smithi037 ceph-mon[31211]: pgmap v396: 97 pgs: 97 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2024-09-19T06:54:20.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:19 smithi151 ceph-mon[35236]: pgmap v396: 97 pgs: 97 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2024-09-19T06:54:20.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 978995 7.77 MB/sec execute 527 sec latency 134.613 ms 2024-09-19T06:54:20.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.236+0000 7fb88e081640 1 -- 172.21.15.37:0/1027057838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888103740 msgr2=0x7fb888107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.236+0000 7fb88e081640 1 --2- 172.21.15.37:0/1027057838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888103740 0x7fb888107b80 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fb870009930 tx=0x7fb87002a420 comp rx=0 tx=0).stop 2024-09-19T06:54:20.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 -- 172.21.15.37:0/1027057838 shutdown_connections 2024-09-19T06:54:20.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 --2- 172.21.15.37:0/1027057838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb888108290 0x7fb88810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 --2- 172.21.15.37:0/1027057838 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888103740 0x7fb888107b80 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 --2- 172.21.15.37:0/1027057838 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb888102d90 0x7fb888103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 -- 172.21.15.37:0/1027057838 >> 172.21.15.37:0/1027057838 conn(0x7fb8880fc7d0 msgr2=0x7fb8880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:20.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 -- 172.21.15.37:0/1027057838 shutdown_connections 2024-09-19T06:54:20.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 -- 172.21.15.37:0/1027057838 wait complete. 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 Processor -- start 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.237+0000 7fb88e081640 1 -- start start 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb88e081640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb888102d90 0x7fb888198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb88e081640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb888103740 0x7fb888198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb88e081640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888108290 0x7fb88819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb88e081640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb888112090 con 0x7fb888108290 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb88e081640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb888111f10 con 0x7fb888102d90 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb88e081640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb888112210 con 0x7fb888103740 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb887fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888108290 0x7fb88819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb887fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888108290 0x7fb88819cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55982/0 (socket says 172.21.15.37:55982) 2024-09-19T06:54:20.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb887fff640 1 -- 172.21.15.37:0/3901757952 learned_addr learned my addr 172.21.15.37:0/3901757952 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:20.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.238+0000 7fb87effd640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb888103740 0x7fb888198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb8877fe640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb888102d90 0x7fb888198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb887fff640 1 -- 172.21.15.37:0/3901757952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb888103740 msgr2=0x7fb888198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb887fff640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb888103740 0x7fb888198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb887fff640 1 -- 172.21.15.37:0/3901757952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb888102d90 msgr2=0x7fb888198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb887fff640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb888102d90 0x7fb888198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb887fff640 1 -- 172.21.15.37:0/3901757952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb88819d5c0 con 0x7fb888108290 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb887fff640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888108290 0x7fb88819cee0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb88000eb10 tx=0x7fb88000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb8857fa640 1 -- 172.21.15.37:0/3901757952 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880004b20 con 0x7fb888108290 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb8857fa640 1 -- 172.21.15.37:0/3901757952 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb880004cc0 con 0x7fb888108290 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.239+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb88819d8b0 con 0x7fb888108290 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.240+0000 7fb8857fa640 1 -- 172.21.15.37:0/3901757952 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb880010760 con 0x7fb888108290 2024-09-19T06:54:20.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.240+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8881a5190 con 0x7fb888108290 2024-09-19T06:54:20.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.242+0000 7fb8857fa640 1 -- 172.21.15.37:0/3901757952 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb880005020 con 0x7fb888108290 2024-09-19T06:54:20.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.242+0000 7fb89045a740 1 Processor -- start 2024-09-19T06:54:20.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.242+0000 7fb89045a740 1 -- start start 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb89045a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb9b40 0x55904adb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb89045a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb85d0 0x55904adb8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb89045a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 0x55904adc5760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb89045a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55904ad1e2f0 con 0x55904adb85d0 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb89045a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55904acc5cf0 con 0x55904adb9b40 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb89045a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55904ad1a2b0 con 0x55904adc3370 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb8857fa640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8740a3430 0x7fb8740a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb85d0 0x55904adb8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb85d0 0x55904adb8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:55988/0 (socket says 172.21.15.37:55988) 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb8877fe640 1 -- 172.21.15.37:0/2524077389 learned_addr learned my addr 172.21.15.37:0/2524077389 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb87effd640 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb9b40 0x55904adb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb87effd640 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 0x55904adc5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb887fff640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8740a3430 0x7fb8740a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb8857fa640 1 -- 172.21.15.37:0/3901757952 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fb8800d0290 con 0x7fb888108290 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb87effd640 1 -- 172.21.15.37:0/2524077389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 msgr2=0x55904adc5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb87effd640 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 0x55904adc5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb87effd640 1 -- 172.21.15.37:0/2524077389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb85d0 msgr2=0x55904adb8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb87effd640 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb85d0 0x55904adb8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.243+0000 7fb87effd640 1 -- 172.21.15.37:0/2524077389 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904acaa3a0 con 0x55904adb9b40 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.244+0000 7fb87effd640 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb9b40 0x55904adb8090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb870009230 tx=0x7fb870005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8464aa1c4827d370 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.244+0000 7fb887fff640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8740a3430 0x7fb8740a58f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb8800cdfb0 tx=0x7fb88006f370 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.244+0000 7fb87dffb640 1 -- 172.21.15.37:0/2524077389 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870035070 con 0x55904adb9b40 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.244+0000 7fb87dffb640 1 -- 172.21.15.37:0/2524077389 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb870005020 con 0x55904adb9b40 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.244+0000 7fb87dffb640 1 -- 172.21.15.37:0/2524077389 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb870004a70 con 0x55904adb9b40 2024-09-19T06:54:20.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.244+0000 7fb89045a740 1 -- 172.21.15.37:0/2524077389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb9b40 msgr2=0x55904adb8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.244+0000 7fb89045a740 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb9b40 0x55904adb8090 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb870009230 tx=0x7fb870005fd0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 -- 172.21.15.37:0/2524077389 shutdown_connections 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 0x55904adc5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb85d0 0x55904adb8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 --2- 172.21.15.37:0/2524077389 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb9b40 0x55904adb8090 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 -- 172.21.15.37:0/2524077389 >> 172.21.15.37:0/2524077389 conn(0x55904ad81310 msgr2=0x55904adb4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 -- 172.21.15.37:0/2524077389 shutdown_connections 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 -- 172.21.15.37:0/2524077389 wait complete. 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 -- start start 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 Processor -- start 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 0x55904adc2e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.245+0000 7fb89045a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904adc0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb89045a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55904adc6010 con 0x55904adb9b40 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb89045a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55904acd9060 con 0x55904adc3370 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb89045a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55904adc6190 con 0x55904adb85d0 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb89045a740 10 client.0 _ll_register_callbacks cb 0x55904aa2c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc00e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56000/0 (socket says 172.21.15.37:56000) 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 -- 172.21.15.37:0/3159605507 learned_addr learned my addr 172.21.15.37:0/3159605507 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb87effd640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 0x55904adc2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb87effd640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904adc0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 -- 172.21.15.37:0/3159605507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 msgr2=0x55904adc2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 0x55904adc2e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 -- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 msgr2=0x55904adc0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904adc0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 -- 172.21.15.37:0/3159605507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904aa8f3c0 con 0x55904adb9b40 2024-09-19T06:54:20.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb87effd640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904adc0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb8877fe640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc00e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb87800bf20 tx=0x7fb8780128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3290908c97c26547 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb866ffd640 1 -- 172.21.15.37:0/3159605507 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878005220 con 0x55904adb9b40 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb866ffd640 1 -- 172.21.15.37:0/3159605507 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fb8780053c0 con 0x55904adb9b40 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.246+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb870005020 con 0x55904adb9b40 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb866ffd640 1 -- 172.21.15.37:0/3159605507 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb8780055c0 con 0x55904adb9b40 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 10 client.18918 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55904add1be0 con 0x55904adb9b40 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb866ffd640 1 -- 172.21.15.37:0/3159605507 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb870005020 con 0x55904adb9b40 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb866ffd640 1 -- 172.21.15.37:0/3159605507 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb878019030 con 0x55904adb9b40 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 10 client.18918 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 10 client.18918 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:54:20.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 20 client.18918 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:20.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904add3140 0x55904add5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 4 client.18918 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:20.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55904ad2b860 con 0x55904add3140 2024-09-19T06:54:20.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.247+0000 7fb87effd640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904add3140 0x55904add5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb87effd640 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904add3140 0x55904add5560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b33dd7cff552972c server_cookie=612401388109efb8 in_seq=0 out_seq=0 2024-09-19T06:54:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb866ffd640 10 client.18918 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:54:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb866ffd640 1 -- 172.21.15.37:0/3159605507 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55904ad2b860 con 0x55904add3140 2024-09-19T06:54:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb866ffd640 10 client.18918 handle_command_reply: tid=0 2024-09-19T06:54:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 client.18918 shutdown 2024-09-19T06:54:20.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 msgr2=0x55904adc00e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc00e0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fb87800bf20 tx=0x7fb8780128e0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 shutdown_connections 2024-09-19T06:54:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904add3140 0x55904add5560 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904adc0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc00e0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.248+0000 7fb89045a740 1 --2- 172.21.15.37:0/3159605507 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 0x55904adc2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.249+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 >> 172.21.15.37:0/3159605507 conn(0x55904ad81310 msgr2=0x55904adba9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:20.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.249+0000 7fb866ffd640 0 client.18918 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:54:20.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.249+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 shutdown_connections 2024-09-19T06:54:20.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.249+0000 7fb89045a740 1 -- 172.21.15.37:0/3159605507 wait complete. 2024-09-19T06:54:20.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.249+0000 7fb89045a740 20 client.18918 trim_cache size 0 max 16384 2024-09-19T06:54:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.270+0000 7fb89045a740 1 Processor -- start 2024-09-19T06:54:20.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.270+0000 7fb89045a740 1 -- start start 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.270+0000 7fb89045a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 0x55904adbf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.270+0000 7fb89045a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adbfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb89045a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904addae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb89045a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881adf80 con 0x55904adb9b40 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb89045a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881ab850 con 0x55904adc3370 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb89045a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881a96c0 con 0x55904adb85d0 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adbfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adbfee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56010/0 (socket says 172.21.15.37:56010) 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 -- 172.21.15.37:0/612371587 learned_addr learned my addr 172.21.15.37:0/612371587 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 -- 172.21.15.37:0/612371587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 msgr2=0x55904adbf9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 --2- 172.21.15.37:0/612371587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 0x55904adbf9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 -- 172.21.15.37:0/612371587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 msgr2=0x55904addae10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 --2- 172.21.15.37:0/612371587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904addae10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 -- 172.21.15.37:0/612371587 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55904adc0be0 con 0x55904adb9b40 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.271+0000 7fb8877fe640 1 --2- 172.21.15.37:0/612371587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adbfee0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb878010f90 tx=0x7fb87800bbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846cc25a6ee6d10a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.272+0000 7fb87d7fa640 1 -- 172.21.15.37:0/612371587 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87802e020 con 0x55904adb9b40 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.272+0000 7fb87d7fa640 1 -- 172.21.15.37:0/612371587 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb878010960 con 0x55904adb9b40 2024-09-19T06:54:20.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.272+0000 7fb87d7fa640 1 -- 172.21.15.37:0/612371587 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb87802a3f0 con 0x55904adb9b40 2024-09-19T06:54:20.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:19 smithi096 ceph-mon[35004]: pgmap v396: 97 pgs: 97 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 228 op/s 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 -- 172.21.15.37:0/612371587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 msgr2=0x55904adbfee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 --2- 172.21.15.37:0/612371587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adbfee0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fb878010f90 tx=0x7fb87800bbc0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 -- 172.21.15.37:0/612371587 shutdown_connections 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 --2- 172.21.15.37:0/612371587 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adc3370 0x55904addae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 --2- 172.21.15.37:0/612371587 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adbfee0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 --2- 172.21.15.37:0/612371587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adb85d0 0x55904adbf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 -- 172.21.15.37:0/612371587 >> 172.21.15.37:0/612371587 conn(0x55904ad81310 msgr2=0x55904adbb310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 -- 172.21.15.37:0/612371587 shutdown_connections 2024-09-19T06:54:20.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 -- 172.21.15.37:0/612371587 wait complete. 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.273+0000 7fb89045a740 1 -- start start 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.274+0000 7fb89045a740 1 Processor -- start 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.274+0000 7fb89045a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb85d0 0x55904adc63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.274+0000 7fb89045a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.274+0000 7fb89045a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 0x55904adc6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.274+0000 7fb89045a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881a9d10 con 0x55904adb9b40 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.274+0000 7fb89045a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881ab180 con 0x55904adb85d0 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.274+0000 7fb89045a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb8881aa3e0 con 0x55904adc3370 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb89045a740 10 client.0 _ll_register_callbacks cb 0x7fb87800b840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56018/0 (socket says 172.21.15.37:56018) 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 -- 172.21.15.37:0/3741272146 learned_addr learned my addr 172.21.15.37:0/3741272146 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:20.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 -- 172.21.15.37:0/3741272146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 msgr2=0x55904adc6e60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 0x55904adc6e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 -- 172.21.15.37:0/3741272146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb85d0 msgr2=0x55904adc63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb85d0 0x55904adc63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 -- 172.21.15.37:0/3741272146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55904ae445e0 con 0x55904adb9b40 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.275+0000 7fb8877fe640 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc6920 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb878010fe0 tx=0x7fb87801ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ba2ec38fc0da0e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.276+0000 7fb866ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.276+0000 7fb866ffd640 1 -- 172.21.15.37:0/3741272146 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb878010b10 con 0x55904adb9b40 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.276+0000 7fb866ffd640 1 -- 172.21.15.37:0/3741272146 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fb878033630 con 0x55904adb9b40 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.276+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb878010960 con 0x55904adb9b40 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.276+0000 7fb866ffd640 1 -- 172.21.15.37:0/3741272146 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fb878043050 con 0x55904adb9b40 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.276+0000 7fb89045a740 10 client.18924 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:20.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55904ae45890 con 0x55904adb9b40 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb866ffd640 1 -- 172.21.15.37:0/3741272146 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb878033d10 con 0x55904adb9b40 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb866ffd640 1 -- 172.21.15.37:0/3741272146 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fb878019070 con 0x55904adb9b40 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb89045a740 10 client.18924 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb89045a740 10 client.18924 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb89045a740 20 client.18924 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb89045a740 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904ae45ee0 0x55904ae482d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb89045a740 4 client.18924 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.277+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7fb878010960 con 0x55904ae45ee0 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.278+0000 7fb87effd640 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904ae45ee0 0x55904ae482d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:20.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.278+0000 7fb866ffd640 10 client.18924 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.278+0000 7fb87effd640 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904ae45ee0 0x55904ae482d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c69f908d111e5fd server_cookie=92ec1491c6c70a74 in_seq=0 out_seq=0 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.279+0000 7fb866ffd640 1 -- 172.21.15.37:0/3741272146 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7fb878010960 con 0x55904ae45ee0 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.279+0000 7fb866ffd640 10 client.18924 handle_command_reply: tid=0 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.280+0000 7fb89045a740 1 client.18924 shutdown 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.280+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 msgr2=0x55904adc6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.280+0000 7fb89045a740 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc6920 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fb878010fe0 tx=0x7fb87801ac30 comp rx=0 tx=0).stop 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 shutdown_connections 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55904ae45ee0 0x55904ae482d0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb866ffd640 0 client.18924 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55904adc3370 0x55904adc6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55904adb9b40 0x55904adc6920 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 --2- 172.21.15.37:0/3741272146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55904adb85d0 0x55904adc63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 >> 172.21.15.37:0/3741272146 conn(0x55904ad81310 msgr2=0x55904adb34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:20.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 shutdown_connections 2024-09-19T06:54:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.281+0000 7fb89045a740 1 -- 172.21.15.37:0/3741272146 wait complete. 2024-09-19T06:54:20.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.283+0000 7fb89045a740 20 client.18924 trim_cache size 0 max 16384 2024-09-19T06:54:20.284 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:54:20.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.284+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8740a3430 msgr2=0x7fb8740a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.284+0000 7fb88e081640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8740a3430 0x7fb8740a58f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb8800cdfb0 tx=0x7fb88006f370 comp rx=0 tx=0).stop 2024-09-19T06:54:20.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.284+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888108290 msgr2=0x7fb88819cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:20.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888108290 0x7fb88819cee0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fb88000eb10 tx=0x7fb88000c2d0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 shutdown_connections 2024-09-19T06:54:20.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb8740a3430 0x7fb8740a58f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb888108290 0x7fb88819cee0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb888103740 0x7fb888198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 --2- 172.21.15.37:0/3901757952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb888102d90 0x7fb888198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:20.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 >> 172.21.15.37:0/3901757952 conn(0x7fb8880fc7d0 msgr2=0x7fb888105e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:20.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 shutdown_connections 2024-09-19T06:54:20.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:20.285+0000 7fb88e081640 1 -- 172.21.15.37:0/3901757952 wait complete. 2024-09-19T06:54:20.973 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:54:21.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 981000 7.78 MB/sec execute 528 sec latency 40.089 ms 2024-09-19T06:54:21.235 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:21.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.714+0000 7f01c5beb640 1 -- 172.21.15.37:0/255629314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0100e20 msgr2=0x7f01c0101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.714+0000 7f01c5beb640 1 --2- 172.21.15.37:0/255629314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0100e20 0x7f01c0101200 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f01b0009900 tx=0x7f01b002a480 comp rx=0 tx=0).stop 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.714+0000 7f01c5beb640 1 -- 172.21.15.37:0/255629314 shutdown_connections 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.714+0000 7f01c5beb640 1 --2- 172.21.15.37:0/255629314 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0102ef0 0x7f01c010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.714+0000 7f01c5beb640 1 --2- 172.21.15.37:0/255629314 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c0102570 0x7f01c01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.714+0000 7f01c5beb640 1 --2- 172.21.15.37:0/255629314 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0100e20 0x7f01c0101200 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.714+0000 7f01c5beb640 1 -- 172.21.15.37:0/255629314 >> 172.21.15.37:0/255629314 conn(0x7f01c00fc7d0 msgr2=0x7f01c00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.715+0000 7f01c5beb640 1 -- 172.21.15.37:0/255629314 shutdown_connections 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.715+0000 7f01c5beb640 1 -- 172.21.15.37:0/255629314 wait complete. 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.715+0000 7f01c5beb640 1 Processor -- start 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01c5beb640 1 -- start start 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01c5beb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0100e20 0x7f01c0199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01c5beb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c0102570 0x7f01c0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01c5beb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0102ef0 0x7f01c01a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01c5beb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0110f80 con 0x7f01c0102ef0 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01c5beb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0110e00 con 0x7f01c0100e20 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01c5beb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0111100 con 0x7f01c0102570 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0102ef0 0x7f01c01a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0100e20 0x7f01c0199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0102ef0 0x7f01c01a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56048/0 (socket says 172.21.15.37:56048) 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 -- 172.21.15.37:0/788163913 learned_addr learned my addr 172.21.15.37:0/788163913 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0100e20 0x7f01c0199310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39354/0 (socket says 172.21.15.37:39354) 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01beffd640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c0102570 0x7f01c0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 -- 172.21.15.37:0/788163913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c0102570 msgr2=0x7f01c0199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c0102570 0x7f01c0199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 -- 172.21.15.37:0/788163913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0100e20 msgr2=0x7f01c0199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0100e20 0x7f01c0199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bffff640 1 -- 172.21.15.37:0/788163913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c01a48d0 con 0x7f01c0102ef0 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.716+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0100e20 0x7f01c0199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.717+0000 7f01bffff640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0102ef0 0x7f01c01a4390 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f01b800efd0 tx=0x7f01b800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.717+0000 7f01bcff9640 1 -- 172.21.15.37:0/788163913 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8003b70 con 0x7f01c0102ef0 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.717+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c0197100 con 0x7f01c0102ef0 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.717+0000 7f01bcff9640 1 -- 172.21.15.37:0/788163913 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b8003d10 con 0x7f01c0102ef0 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.717+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c0197640 con 0x7f01c0102ef0 2024-09-19T06:54:21.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.717+0000 7f01bcff9640 1 -- 172.21.15.37:0/788163913 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80109f0 con 0x7f01c0102ef0 2024-09-19T06:54:21.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.719+0000 7f01bcff9640 1 -- 172.21.15.37:0/788163913 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f01b8004020 con 0x7f01c0102ef0 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.719+0000 7f01c7fc4740 1 Processor -- start 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.719+0000 7f01c7fc4740 1 -- start start 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6aff1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6affe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee6af572f0 con 0x55ee6aff15d0 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee6aefecf0 con 0x55ee6aff2b40 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee6af532b0 con 0x55ee6affc370 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6aff1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39370/0 (socket says 172.21.15.37:39370) 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 -- 172.21.15.37:0/1484997504 learned_addr learned my addr 172.21.15.37:0/1484997504 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:21.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01bcff9640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f01a80a33b0 0x7f01a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6affe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 -- 172.21.15.37:0/1484997504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 msgr2=0x55ee6affe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6affe760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 -- 172.21.15.37:0/1484997504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 msgr2=0x55ee6aff1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6aff1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01beffd640 1 -- 172.21.15.37:0/1484997504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6aee33a0 con 0x55ee6aff2b40 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01bcff9640 1 -- 172.21.15.37:0/788163913 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f01b80d01b0 con 0x7f01c0102ef0 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.720+0000 7f01bffff640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f01a80a33b0 0x7f01a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.721+0000 7f01beffd640 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff1090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01b4012510 tx=0x7f01b400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f4a4729ede6755 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.721+0000 7f01a5ffb640 1 -- 172.21.15.37:0/1484997504 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4004030 con 0x55ee6aff2b40 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.721+0000 7f01a5ffb640 1 -- 172.21.15.37:0/1484997504 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b40041d0 con 0x55ee6aff2b40 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.721+0000 7f01a5ffb640 1 -- 172.21.15.37:0/1484997504 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b40106c0 con 0x55ee6aff2b40 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1484997504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 msgr2=0x55ee6aff1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01bffff640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f01a80a33b0 0x7f01a80a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f01b80a4b40 tx=0x7f01b80bc040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff1090 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f01b4012510 tx=0x7f01b400b870 comp rx=0 tx=0).stop 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1484997504 shutdown_connections 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6affe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6aff1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1484997504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff1090 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1484997504 >> 172.21.15.37:0/1484997504 conn(0x55ee6afba310 msgr2=0x55ee6afedc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1484997504 shutdown_connections 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1484997504 wait complete. 2024-09-19T06:54:21.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 -- start start 2024-09-19T06:54:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.722+0000 7f01c7fc4740 1 Processor -- start 2024-09-19T06:54:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6affbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6aff2b40 0x55ee6aff9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6affc370 0x55ee6aff9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55ee6afff070 con 0x55ee6affc370 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ee6af12060 con 0x55ee6aff15d0 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55ee6afff1f0 con 0x55ee6aff2b40 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01c7fc4740 10 client.0 _ll_register_callbacks cb 0x55ee6ac658c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6affc370 0x55ee6aff9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6affc370 0x55ee6aff9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56062/0 (socket says 172.21.15.37:56062) 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 -- 172.21.15.37:0/337330168 learned_addr learned my addr 172.21.15.37:0/337330168 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6affbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6aff2b40 0x55ee6aff9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 -- 172.21.15.37:0/337330168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6aff2b40 msgr2=0x55ee6aff9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6aff2b40 0x55ee6aff9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 -- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 msgr2=0x55ee6affbd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6affbd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 -- 172.21.15.37:0/337330168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6acc83c0 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f01beffd640 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6affc370 0x55ee6aff9600 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f01b400beb0 tx=0x7f01b4003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34beefd973ee5ac6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f019effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.723+0000 7f019effd640 1 -- 172.21.15.37:0/337330168 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b40044e0 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f019effd640 1 -- 172.21.15.37:0/337330168 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f01b40035f0 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b40041d0 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f019effd640 1 -- 172.21.15.37:0/337330168 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f01b40037f0 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 10 client.18936 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ee6b00aaf0 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f019effd640 1 -- 172.21.15.37:0/337330168 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b40041d0 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f019effd640 1 -- 172.21.15.37:0/337330168 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f01b401d070 con 0x55ee6affc370 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 10 client.18936 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 10 client.18936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 20 client.18936 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:21.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b00e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 4 client.18936 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ee6af64860 con 0x55ee6b00be40 2024-09-19T06:54:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.724+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b00e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.725+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b00e230 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adfddf0f3489e6ff server_cookie=2d714401988b1ed8 in_seq=0 out_seq=0 2024-09-19T06:54:21.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.725+0000 7f019effd640 10 client.18936 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f019effd640 1 -- 172.21.15.37:0/337330168 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ee6af64860 con 0x55ee6b00be40 2024-09-19T06:54:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f019effd640 10 client.18936 handle_command_reply: tid=0 2024-09-19T06:54:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 client.18936 shutdown 2024-09-19T06:54:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6affc370 msgr2=0x55ee6aff9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6affc370 0x55ee6aff9600 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f01b400beb0 tx=0x7f01b4003cc0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 shutdown_connections 2024-09-19T06:54:21.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b00e230 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6affc370 0x55ee6aff9600 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f019effd640 0 client.18936 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6aff2b40 0x55ee6aff9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/337330168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6affbd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.726+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 >> 172.21.15.37:0/337330168 conn(0x55ee6afba310 msgr2=0x55ee6aff3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.727+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 shutdown_connections 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.727+0000 7f01c7fc4740 1 -- 172.21.15.37:0/337330168 wait complete. 2024-09-19T06:54:21.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.727+0000 7f01c7fc4740 20 client.18936 trim_cache size 0 max 16384 2024-09-19T06:54:21.742 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.741+0000 7f01c7fc4740 1 Processor -- start 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.741+0000 7f01c7fc4740 1 -- start start 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.741+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6b008310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6aff4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01b24f0 con 0x55ee6aff15d0 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01afdc0 con 0x55ee6aff2b40 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01adc30 con 0x55ee6affc370 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6b008310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6b008310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56068/0 (socket says 172.21.15.37:56068) 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 -- 172.21.15.37:0/3793602984 learned_addr learned my addr 172.21.15.37:0/3793602984 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6aff4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.743 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 -- 172.21.15.37:0/3793602984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 msgr2=0x55ee6aff4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6aff4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 -- 172.21.15.37:0/3793602984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 msgr2=0x55ee6aff4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 -- 172.21.15.37:0/3793602984 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ee6aff7560 con 0x55ee6aff15d0 2024-09-19T06:54:21.744 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01beffd640 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6b008310 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f01b4000c00 tx=0x7f01b40124a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e170b43fd6e74c4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:21.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01a57fa640 1 -- 172.21.15.37:0/3793602984 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4012e60 con 0x55ee6aff15d0 2024-09-19T06:54:21.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01a57fa640 1 -- 172.21.15.37:0/3793602984 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b4005850 con 0x55ee6aff15d0 2024-09-19T06:54:21.748 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.742+0000 7f01a57fa640 1 -- 172.21.15.37:0/3793602984 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b4023040 con 0x55ee6aff15d0 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 -- 172.21.15.37:0/3793602984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 msgr2=0x55ee6b008310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6b008310 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f01b4000c00 tx=0x7f01b40124a0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 -- 172.21.15.37:0/3793602984 shutdown_connections 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6aff4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff2b40 0x55ee6aff4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/3793602984 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff15d0 0x55ee6b008310 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 -- 172.21.15.37:0/3793602984 >> 172.21.15.37:0/3793602984 conn(0x55ee6afba310 msgr2=0x55ee6aff8100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 -- 172.21.15.37:0/3793602984 shutdown_connections 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.743+0000 7f01c7fc4740 1 -- 172.21.15.37:0/3793602984 wait complete. 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 -- start start 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 Processor -- start 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6b06ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff2b40 0x55ee6b06f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.749 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6afffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01ae280 con 0x55ee6aff2b40 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01af6f0 con 0x55ee6aff15d0 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01ae950 con 0x55ee6affc370 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01c7fc4740 10 client.0 _ll_register_callbacks cb 0x7f01b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff2b40 0x55ee6b06f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01bf7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff2b40 0x55ee6b06f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56078/0 (socket says 172.21.15.37:56078) 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01bf7fe640 1 -- 172.21.15.37:0/1441562071 learned_addr learned my addr 172.21.15.37:0/1441562071 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01beffd640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6b06ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.744+0000 7f01beffd640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6afffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01bf7fe640 1 -- 172.21.15.37:0/1441562071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 msgr2=0x55ee6afffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6afffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01bf7fe640 1 -- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 msgr2=0x55ee6b06ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6b06ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.750 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01bf7fe640 1 -- 172.21.15.37:0/1441562071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ee6b0000f0 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01beffd640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6b06ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01bf7fe640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff2b40 0x55ee6b06f1a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f01b00314e0 tx=0x7f01b0003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca00106e6cab64a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f019effd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f019effd640 1 -- 172.21.15.37:0/1441562071 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b004c070 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f019effd640 1 -- 172.21.15.37:0/1441562071 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f01b0003dc0 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01b4005850 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f019effd640 1 -- 172.21.15.37:0/1441562071 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f01b0035040 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01c7fc4740 10 client.18942 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.745+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55ee6b0764c0 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f019effd640 1 -- 172.21.15.37:0/1441562071 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01b4005850 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f019effd640 1 -- 172.21.15.37:0/1441562071 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f01b0031770 con 0x55ee6aff2b40 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f01c7fc4740 10 client.18942 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:21.751 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f01c7fc4740 10 client.18942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f01c7fc4740 20 client.18942 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b078cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f01c7fc4740 4 client.18942 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ee6af5ed70 con 0x55ee6b00be40 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f01beffd640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b078cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.746+0000 7f019effd640 10 client.18942 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01beffd640 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b078cb0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c07d9995ea205f server_cookie=907a8d6a79ccc007 in_seq=0 out_seq=0 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f019effd640 1 -- 172.21.15.37:0/1441562071 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ee6af5ed70 con 0x55ee6b00be40 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f019effd640 10 client.18942 handle_command_reply: tid=0 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 client.18942 shutdown 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff2b40 msgr2=0x55ee6b06f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff2b40 0x55ee6b06f1a0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f01b00314e0 tx=0x7f01b0003780 comp rx=0 tx=0).stop 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 shutdown_connections 2024-09-19T06:54:21.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55ee6b00be40 0x55ee6b078cb0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55ee6affc370 0x55ee6afffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55ee6aff2b40 0x55ee6b06f1a0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 --2- 172.21.15.37:0/1441562071 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ee6aff15d0 0x55ee6b06ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 >> 172.21.15.37:0/1441562071 conn(0x55ee6afba310 msgr2=0x55ee6affa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f019effd640 0 client.18942 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 shutdown_connections 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 1 -- 172.21.15.37:0/1441562071 wait complete. 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.747+0000 7f01c7fc4740 20 client.18942 trim_cache size 0 max 16384 2024-09-19T06:54:21.753 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:54:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f01a80a33b0 msgr2=0x7f01a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f01a80a33b0 0x7f01a80a5870 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f01b80a4b40 tx=0x7f01b80bc040 comp rx=0 tx=0).stop 2024-09-19T06:54:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0102ef0 msgr2=0x7f01c01a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:21.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0102ef0 0x7f01c01a4390 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f01b800efd0 tx=0x7f01b800c6a0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 shutdown_connections 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f01a80a33b0 0x7f01a80a5870 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f01c0102ef0 0x7f01c01a4390 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f01c0102570 0x7f01c0199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 --2- 172.21.15.37:0/788163913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f01c0100e20 0x7f01c0199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 >> 172.21.15.37:0/788163913 conn(0x7f01c00fc7d0 msgr2=0x7f01c00fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 shutdown_connections 2024-09-19T06:54:21.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:21.748+0000 7f01c5beb640 1 -- 172.21.15.37:0/788163913 wait complete. 2024-09-19T06:54:22.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:21 smithi037 ceph-mon[31211]: pgmap v397: 97 pgs: 97 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 209 op/s 2024-09-19T06:54:22.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:21 smithi151 ceph-mon[35236]: pgmap v397: 97 pgs: 97 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 209 op/s 2024-09-19T06:54:22.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 983325 7.79 MB/sec execute 529 sec latency 29.472 ms 2024-09-19T06:54:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:21 smithi096 ceph-mon[35004]: pgmap v397: 97 pgs: 97 active+clean; 74 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.6 MiB/s wr, 209 op/s 2024-09-19T06:54:23.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 985288 7.79 MB/sec execute 530 sec latency 13.711 ms 2024-09-19T06:54:23.560 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:54:23.560 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:54:23.822 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:23.836 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:23 smithi037 ceph-mon[31211]: pgmap v398: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 410 op/s 2024-09-19T06:54:24.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:23 smithi151 ceph-mon[35236]: pgmap v398: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 410 op/s 2024-09-19T06:54:24.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 987873 7.81 MB/sec execute 531 sec latency 48.033 ms 2024-09-19T06:54:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 -- 172.21.15.37:0/501034300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 msgr2=0x7f4a54100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 --2- 172.21.15.37:0/501034300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54100ee0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009930 tx=0x7f4a3c02a420 comp rx=0 tx=0).stop 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 -- 172.21.15.37:0/501034300 shutdown_connections 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 --2- 172.21.15.37:0/501034300 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a54101420 0x7f4a5410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 --2- 172.21.15.37:0/501034300 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54100ee0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 --2- 172.21.15.37:0/501034300 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a54106a80 0x7f4a54106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 -- 172.21.15.37:0/501034300 >> 172.21.15.37:0/501034300 conn(0x7f4a540fc7a0 msgr2=0x7f4a540febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 -- 172.21.15.37:0/501034300 shutdown_connections 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.239+0000 7f4a599cb640 1 -- 172.21.15.37:0/501034300 wait complete. 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 Processor -- start 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 -- start start 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a54101420 0x7f4a54198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a54106a80 0x7f4a5419cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54110e40 con 0x7f4a54100a80 2024-09-19T06:54:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54110cc0 con 0x7f4a54106a80 2024-09-19T06:54:24.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a599cb640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54110fc0 con 0x7f4a54101420 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56090/0 (socket says 172.21.15.37:56090) 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a52ffd640 1 -- 172.21.15.37:0/2370367600 learned_addr learned my addr 172.21.15.37:0/2370367600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a54101420 0x7f4a54198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.240+0000 7f4a537fe640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a54106a80 0x7f4a5419cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a52ffd640 1 -- 172.21.15.37:0/2370367600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a54101420 msgr2=0x7f4a54198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a54101420 0x7f4a54198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a52ffd640 1 -- 172.21.15.37:0/2370367600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a54106a80 msgr2=0x7f4a5419cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a54106a80 0x7f4a5419cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a52ffd640 1 -- 172.21.15.37:0/2370367600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a5419d400 con 0x7f4a54100a80 2024-09-19T06:54:24.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54198450 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400c890 tx=0x7f4a4400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:24.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a589c9640 1 -- 172.21.15.37:0/2370367600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44004220 con 0x7f4a54100a80 2024-09-19T06:54:24.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a589c9640 1 -- 172.21.15.37:0/2370367600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a440043c0 con 0x7f4a54100a80 2024-09-19T06:54:24.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a5419d6f0 con 0x7f4a54100a80 2024-09-19T06:54:24.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a589c9640 1 -- 172.21.15.37:0/2370367600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440049e0 con 0x7f4a54100a80 2024-09-19T06:54:24.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.241+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a54102580 con 0x7f4a54100a80 2024-09-19T06:54:24.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.243+0000 7f4a589c9640 1 -- 172.21.15.37:0/2370367600 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4a440079b0 con 0x7f4a54100a80 2024-09-19T06:54:24.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.244+0000 7f4a589c9640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4a400a3430 0x7f4a400a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.244+0000 7f4a5bda4740 1 Processor -- start 2024-09-19T06:54:24.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.244+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4a400a3430 0x7f4a400a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.244+0000 7f4a5bda4740 1 -- start start 2024-09-19T06:54:24.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.244+0000 7f4a589c9640 1 -- 172.21.15.37:0/2370367600 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f4a440cfc40 con 0x7f4a54100a80 2024-09-19T06:54:24.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec4b60 0x56201aec4f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecf800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201aed2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56201ae2a280 con 0x56201aecfd40 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56201add1b10 con 0x56201aec5480 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56201ae26240 con 0x56201aec4b60 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201aed2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201aed2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56092/0 (socket says 172.21.15.37:56092) 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 -- 172.21.15.37:0/1204392299 learned_addr learned my addr 172.21.15.37:0/1204392299 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec4b60 0x56201aec4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4a400a3430 0x7f4a400a58f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009900 tx=0x7f4a3c03c040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecf800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 -- 172.21.15.37:0/1204392299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec4b60 msgr2=0x56201aec4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec4b60 0x56201aec4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 -- 172.21.15.37:0/1204392299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 msgr2=0x56201aecf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecf800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a537fe640 1 -- 172.21.15.37:0/1204392299 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201adde1f0 con 0x56201aecfd40 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.245+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.246+0000 7f4a537fe640 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201aed2130 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009fd0 tx=0x7f4a4c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f80b1f496665d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.246+0000 7f4a49ffb640 1 -- 172.21.15.37:0/1204392299 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c004d20 con 0x56201aecfd40 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.246+0000 7f4a49ffb640 1 -- 172.21.15.37:0/1204392299 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a4c004050 con 0x56201aecfd40 2024-09-19T06:54:24.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.246+0000 7f4a49ffb640 1 -- 172.21.15.37:0/1204392299 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4c005240 con 0x56201aecfd40 2024-09-19T06:54:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1204392299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 msgr2=0x56201aed2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201aed2130 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f4a4c009fd0 tx=0x7f4a4c0118d0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1204392299 shutdown_connections 2024-09-19T06:54:24.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201aed2130 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecf800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1204392299 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec4b60 0x56201aec4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1204392299 >> 172.21.15.37:0/1204392299 conn(0x56201aec10f0 msgr2=0x56201aec14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1204392299 shutdown_connections 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1204392299 wait complete. 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.247+0000 7f4a5bda4740 1 -- start start 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 1 Processor -- start 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec4b60 0x56201aeccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aecd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56201aed2910 con 0x56201aec4b60 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56201ade4f10 con 0x56201aec5480 2024-09-19T06:54:24.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56201aed2a90 con 0x56201aecfd40 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a5bda4740 10 client.0 _ll_register_callbacks cb 0x56201adada40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.248+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39422/0 (socket says 172.21.15.37:39422) 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 -- 172.21.15.37:0/1195369474 learned_addr learned my addr 172.21.15.37:0/1195369474 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec4b60 0x56201aeccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a537fe640 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aecd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 -- 172.21.15.37:0/1195369474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 msgr2=0x56201aecd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aecd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 -- 172.21.15.37:0/1195369474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec4b60 msgr2=0x56201aeccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec4b60 0x56201aeccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 -- 172.21.15.37:0/1195369474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201ab9b6a0 con 0x56201aec5480 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecd0f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c02a940 tx=0x7f4a3c02a420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d1159cf07f9a4c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:24.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.249+0000 7f4a32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:54:24.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a32ffd640 1 -- 172.21.15.37:0/1195369474 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c057060 con 0x56201aec5480 2024-09-19T06:54:24.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a32ffd640 1 -- 172.21.15.37:0/1195369474 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f4a3c03ac00 con 0x56201aec5480 2024-09-19T06:54:24.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a4c004050 con 0x56201aec5480 2024-09-19T06:54:24.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a32ffd640 1 -- 172.21.15.37:0/1195369474 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4a3c035050 con 0x56201aec5480 2024-09-19T06:54:24.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 10 client.28253 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:24.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56201aede410 con 0x56201aec5480 2024-09-19T06:54:24.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a32ffd640 1 -- 172.21.15.37:0/1195369474 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a3c03ae00 con 0x56201aec5480 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a32ffd640 1 -- 172.21.15.37:0/1195369474 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4a3c046810 con 0x56201aec5480 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 10 client.28253 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 10 client.28253 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 20 client.28253 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201aedfd10 0x56201aee2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 4 client.28253 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.250+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a4c004050 con 0x56201aedfd10 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.251+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201aedfd10 0x56201aee2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.251+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201aedfd10 0x56201aee2130 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1223a5dcfc2e5d24 server_cookie=905cad06a756b700 in_seq=0 out_seq=0 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.251+0000 7f4a32ffd640 10 client.28253 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.252+0000 7f4a32ffd640 1 -- 172.21.15.37:0/1195369474 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f4a4c004050 con 0x56201aedfd10 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.252+0000 7f4a32ffd640 10 client.28253 handle_command_reply: tid=0 2024-09-19T06:54:24.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.252+0000 7f4a5bda4740 1 client.28253 shutdown 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.252+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 msgr2=0x56201aecd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecd0f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c02a940 tx=0x7f4a3c02a420 comp rx=0 tx=0).stop 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 shutdown_connections 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201aedfd10 0x56201aee2130 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aecd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec5480 0x56201aecd0f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1195369474 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec4b60 0x56201aeccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a32ffd640 0 client.28253 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 >> 172.21.15.37:0/1195369474 conn(0x56201aec10f0 msgr2=0x56201aec0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 shutdown_connections 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1195369474 wait complete. 2024-09-19T06:54:24.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.253+0000 7f4a5bda4740 20 client.28253 trim_cache size 0 max 16384 2024-09-19T06:54:24.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:23 smithi096 ceph-mon[35004]: pgmap v398: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.9 MiB/s wr, 410 op/s 2024-09-19T06:54:24.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.277+0000 7f4a5bda4740 1 Processor -- start 2024-09-19T06:54:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.277+0000 7f4a5bda4740 1 -- start start 2024-09-19T06:54:24.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.277+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aec75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.277+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec5480 0x56201aecea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aee7a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec5480 0x56201aecea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aec75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aec75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39430/0 (socket says 172.21.15.37:39430) 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a4bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec5480 0x56201aecea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56124/0 (socket says 172.21.15.37:56124) 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 -- 172.21.15.37:0/1596502684 learned_addr learned my addr 172.21.15.37:0/1596502684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ae120 con 0x56201aec5480 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ab9f0 con 0x56201aec4b60 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541a9860 con 0x56201aecfd40 2024-09-19T06:54:24.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a537fe640 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aee7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 -- 172.21.15.37:0/1596502684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 msgr2=0x56201aee7a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aee7a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 -- 172.21.15.37:0/1596502684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec5480 msgr2=0x56201aecea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec5480 0x56201aecea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.278+0000 7f4a52ffd640 1 -- 172.21.15.37:0/1596502684 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56201aecf480 con 0x56201aec4b60 2024-09-19T06:54:24.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.279+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aec75c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a44098970 tx=0x7f4a4400a5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6841f8fde57da69c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:24.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.279+0000 7f4a497fa640 1 -- 172.21.15.37:0/1596502684 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4401d040 con 0x56201aec4b60 2024-09-19T06:54:24.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.279+0000 7f4a497fa640 1 -- 172.21.15.37:0/1596502684 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a440b4030 con 0x56201aec4b60 2024-09-19T06:54:24.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.279+0000 7f4a497fa640 1 -- 172.21.15.37:0/1596502684 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440987a0 con 0x56201aec4b60 2024-09-19T06:54:24.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 msgr2=0x56201aec75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aec75c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4a44098970 tx=0x7f4a4400a5b0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 shutdown_connections 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aecfd40 0x56201aee7a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aec5480 0x56201aecea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/1596502684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aec75c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 >> 172.21.15.37:0/1596502684 conn(0x56201aec10f0 msgr2=0x56201aece300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.280+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 shutdown_connections 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.281+0000 7f4a5bda4740 1 -- 172.21.15.37:0/1596502684 wait complete. 2024-09-19T06:54:24.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.281+0000 7f4a5bda4740 1 -- start start 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.281+0000 7f4a5bda4740 1 Processor -- start 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aed3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec5480 0x56201aed4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a5bda4740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201af49a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541a9eb0 con 0x56201aecfd40 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541ab320 con 0x56201aec4b60 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a5bda4740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4a541aa580 con 0x56201aec5480 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a5bda4740 10 client.0 _ll_register_callbacks cb 0x7f4a3c031790 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a537fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201af49a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aed3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a52ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aed3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39442/0 (socket says 172.21.15.37:39442) 2024-09-19T06:54:24.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.282+0000 7f4a52ffd640 1 -- 172.21.15.37:0/897314618 learned_addr learned my addr 172.21.15.37:0/897314618 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:24.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.283+0000 7f4a52ffd640 1 -- 172.21.15.37:0/897314618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec5480 msgr2=0x56201aed4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.283+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec5480 0x56201aed4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.283+0000 7f4a52ffd640 1 -- 172.21.15.37:0/897314618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 msgr2=0x56201af49a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:24.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.283+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201af49a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.283+0000 7f4a52ffd640 1 -- 172.21.15.37:0/897314618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56201aed4be0 con 0x56201aec4b60 2024-09-19T06:54:24.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a52ffd640 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aed3dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4a44005fd0 tx=0x7f4a440a4bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9c6bf92277b51777 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a32ffd640 1 -- 172.21.15.37:0/897314618 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4407e070 con 0x56201aec4b60 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a32ffd640 1 -- 172.21.15.37:0/897314618 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f4a44007ca0 con 0x56201aec4b60 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a440b4030 con 0x56201aec4b60 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a32ffd640 1 -- 172.21.15.37:0/897314618 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f4a4401d040 con 0x56201aec4b60 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a5bda4740 10 client.28259 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.284+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x56201af4b0a0 con 0x56201aec4b60 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a32ffd640 1 -- 172.21.15.37:0/897314618 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a440b4030 con 0x56201aec4b60 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a32ffd640 1 -- 172.21.15.37:0/897314618 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f4a4407ac10 con 0x56201aec4b60 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a5bda4740 10 client.28259 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:24.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a5bda4740 10 client.28259 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a5bda4740 20 client.28259 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201af4b6b0 0x56201af4daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a5bda4740 4 client.28259 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:54:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.285+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56201ae377f0 con 0x56201af4b6b0 2024-09-19T06:54:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.286+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201af4b6b0 0x56201af4daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.288+0000 7f4a4bfff640 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201af4b6b0 0x56201af4daa0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13e45a869a55546f server_cookie=a295536e68de1bc1 in_seq=0 out_seq=0 2024-09-19T06:54:24.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.288+0000 7f4a32ffd640 10 client.28259 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:24.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a32ffd640 1 -- 172.21.15.37:0/897314618 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56201ae377f0 con 0x56201af4b6b0 2024-09-19T06:54:24.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a32ffd640 10 client.28259 handle_command_reply: tid=0 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 client.28259 shutdown 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 msgr2=0x56201aed3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aed3dc0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4a44005fd0 tx=0x7f4a440a4bd0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 shutdown_connections 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56201af4b6b0 0x56201af4daa0 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56201aecfd40 0x56201af49a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56201aec5480 0x56201aed4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.294 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 --2- 172.21.15.37:0/897314618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56201aec4b60 0x56201aed3dc0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 >> 172.21.15.37:0/897314618 conn(0x56201aec10f0 msgr2=0x56201aecd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.293+0000 7f4a32ffd640 0 client.28259 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.294+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 shutdown_connections 2024-09-19T06:54:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.294+0000 7f4a5bda4740 1 -- 172.21.15.37:0/897314618 wait complete. 2024-09-19T06:54:24.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.294+0000 7f4a5bda4740 20 client.28259 trim_cache size 0 max 16384 2024-09-19T06:54:24.295 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:54:24.296 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:54:24.296 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "bc958c3a-dca4-474b-a610-a92739621f1e", 2024-09-19T06:54:24.296 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:54:24.296 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4a400a3430 msgr2=0x7f4a400a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4a400a3430 0x7f4a400a58f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c009900 tx=0x7f4a3c03c040 comp rx=0 tx=0).stop 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 msgr2=0x7f4a54198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54198450 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400c890 tx=0x7f4a4400cd50 comp rx=0 tx=0).stop 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 shutdown_connections 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4a54106a80 0x7f4a5419cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a54101420 0x7f4a54198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4a400a3430 0x7f4a400a58f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 --2- 172.21.15.37:0/2370367600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4a54100a80 0x7f4a54198450 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 >> 172.21.15.37:0/2370367600 conn(0x7f4a540fc7a0 msgr2=0x7f4a54104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 shutdown_connections 2024-09-19T06:54:24.297 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:24.295+0000 7f4a599cb640 1 -- 172.21.15.37:0/2370367600 wait complete. 2024-09-19T06:54:24.774 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:54:24 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: 2024-09-19T06:54:24.317+0000 7fa3b1df3640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000240 [...14,head] /volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4/client.0/tmp/clients/client0/~dmtmp/SEED/ auth{1=1,2=1} v209765 ap=2 RANDEPHEMERALPIN f(v8 m2024-09-19T06:53:51.424609+0000 3=3+0) n(v43 rc2024-09-19T06:53:51.424609+0000 4=3+1) old_inodes=5 (inest sync->mix g=1,2) caps={24845=pAsLsXsFs/pAsLsXsFsx@67} mcw={2=AsLsXsFs} | dirtyscattered=0 request=0 lock=0 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 randepin 0x55e4dae24100]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000240:[<0x10000000009/SEED v3709>,<0x10000000008/~dmtmp v65>,<0x10000000007/client0 v49>,<0x10000000006/clients v19>,<0x10000000005/tmp v19>,<0x100000001fb/client.0 v20>,<0x100000001fa/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4 v36>,<0x10000000001/sv_1 v59>,<0x10000000000/qa v49>,<0x1/volumes v48>]//[]","memoryvalue":"(2)0x10000000240:[<0x10000000009/SEED v209765>,<0x10000000008/~dmtmp v1260>,<0x10000000007/client0 v1204>,<0x10000000006/clients v1103>,<0x10000000005/tmp v1092>,<0x100000001fb/client.0 v1077>,<0x100000001fa/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4 v1093>,<0x10000000001/sv_1 v1095>,<0x10000000000/qa v1080>,<0x1/volumes v1043>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2024-09-19T06:53:51.416610+0000 2=2+0)","ondisk_value.rstat":"n(v0 rc2024-09-19T06:53:51.424609+0000 4=3+1)","memory_value.dirstat":"f(v8 m2024-09-19T06:53:51.424609+0000 3=3+0)","memory_value.rstat":"n(v43 rc2024-09-19T06:53:51.424609+0000 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2024-09-19T06:54:24.969 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:54:25.231 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:25.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 990120 7.81 MB/sec execute 532 sec latency 21.120 ms 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.748+0000 7f7041896640 1 -- 172.21.15.37:0/1317061405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 msgr2=0x7f703c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.748+0000 7f7041896640 1 --2- 172.21.15.37:0/1317061405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c101200 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f702c009890 tx=0x7f702c02a3f0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 -- 172.21.15.37:0/1317061405 shutdown_connections 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 --2- 172.21.15.37:0/1317061405 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f703c102ef0 0x7f703c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 --2- 172.21.15.37:0/1317061405 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703c102570 0x7f703c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 --2- 172.21.15.37:0/1317061405 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c101200 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 -- 172.21.15.37:0/1317061405 >> 172.21.15.37:0/1317061405 conn(0x7f703c0fc7d0 msgr2=0x7f703c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 -- 172.21.15.37:0/1317061405 shutdown_connections 2024-09-19T06:54:25.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 -- 172.21.15.37:0/1317061405 wait complete. 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 Processor -- start 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.749+0000 7f7041896640 1 -- start start 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f7041896640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c1a0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f7041896640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703c102570 0x7f703c1a0980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f7041896640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f703c102ef0 0x7f703c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f7041896640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f703c10fe80 con 0x7f703c102ef0 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f7041896640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f703c10fd00 con 0x7f703c100e20 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f7041896640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f703c110000 con 0x7f703c102570 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c1a0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f703c102ef0 0x7f703c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c1a0440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39462/0 (socket says 172.21.15.37:39462) 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 -- 172.21.15.37:0/1497211938 learned_addr learned my addr 172.21.15.37:0/1497211938 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:25.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703a7fc640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703c102570 0x7f703c1a0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 -- 172.21.15.37:0/1497211938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703c102570 msgr2=0x7f703c1a0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703c102570 0x7f703c1a0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 -- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f703c102ef0 msgr2=0x7f703c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f703c102ef0 0x7f703c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 -- 172.21.15.37:0/1497211938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f703c19ad30 con 0x7f703c100e20 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f703c102ef0 0x7f703c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.750+0000 7f703affd640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c1a0440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f702c009860 tx=0x7f702c030460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.751+0000 7f7040894640 1 -- 172.21.15.37:0/1497211938 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c03f020 con 0x7f703c100e20 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.751+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f703c19afc0 con 0x7f703c100e20 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.751+0000 7f7040894640 1 -- 172.21.15.37:0/1497211938 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702c004340 con 0x7f703c100e20 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.751+0000 7f7040894640 1 -- 172.21.15.37:0/1497211938 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c037040 con 0x7f703c100e20 2024-09-19T06:54:25.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.751+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f703c1a7310 con 0x7f703c100e20 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.753+0000 7f7040894640 1 -- 172.21.15.37:0/1497211938 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f702c042020 con 0x7f703c100e20 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.753+0000 7f7043c6f740 1 Processor -- start 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.753+0000 7f7043c6f740 1 -- start start 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.753+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad772880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad77d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f7043c6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558cad6d82e0 con 0x558cad77d680 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f7043c6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558cad67fce0 con 0x558cad7724a0 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f7043c6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558cad6d42a0 con 0x558cad772dc0 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad772880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad772880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39472/0 (socket says 172.21.15.37:39472) 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703a7fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.773 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703b7fe640 1 -- 172.21.15.37:0/2861622540 learned_addr learned my addr 172.21.15.37:0/2861622540 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703affd640 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad77d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703a7fc640 1 -- 172.21.15.37:0/2861622540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 msgr2=0x558cad77d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703a7fc640 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad77d140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703a7fc640 1 -- 172.21.15.37:0/2861622540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 msgr2=0x558cad772880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703a7fc640 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad772880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.774 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703a7fc640 1 -- 172.21.15.37:0/2861622540 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad664440 con 0x558cad77d680 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f703a7fc640 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77fa70 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f703c19b740 tx=0x7f7024005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c174dc2cb875c14a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.754+0000 7f7040894640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70280a33b0 0x7f70280a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7031ffb640 1 -- 172.21.15.37:0/2861622540 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024009d90 con 0x558cad77d680 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70280a33b0 0x7f70280a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7031ffb640 1 -- 172.21.15.37:0/2861622540 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702401c070 con 0x558cad77d680 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7031ffb640 1 -- 172.21.15.37:0/2861622540 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70240166f0 con 0x558cad77d680 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7040894640 1 -- 172.21.15.37:0/1497211938 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f702c0efbb0 con 0x7f703c100e20 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7043c6f740 1 -- 172.21.15.37:0/2861622540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 msgr2=0x558cad77fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70280a33b0 0x7f70280a5870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7034005c40 tx=0x7f70340095c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7043c6f740 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77fa70 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f703c19b740 tx=0x7f7024005f90 comp rx=0 tx=0).stop 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7043c6f740 1 -- 172.21.15.37:0/2861622540 shutdown_connections 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7043c6f740 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77fa70 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.775 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7043c6f740 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad77d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7043c6f740 1 --2- 172.21.15.37:0/2861622540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad772880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.755+0000 7f7043c6f740 1 -- 172.21.15.37:0/2861622540 >> 172.21.15.37:0/2861622540 conn(0x558cad76ea30 msgr2=0x558cad76ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 -- 172.21.15.37:0/2861622540 shutdown_connections 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 -- 172.21.15.37:0/2861622540 wait complete. 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 -- start start 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 Processor -- start 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad7724a0 0x558cad77cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad77a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x558cad780250 con 0x558cad77d680 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558cad693050 con 0x558cad772dc0 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.756+0000 7f7043c6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x558cad7803d0 con 0x558cad7724a0 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f7043c6f740 10 client.0 _ll_register_callbacks cb 0x558cad6d3160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:25.776 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad77a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad77a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:39476/0 (socket says 172.21.15.37:39476) 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 -- 172.21.15.37:0/3921431244 learned_addr learned my addr 172.21.15.37:0/3921431244 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56184/0 (socket says 172.21.15.37:56184) 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703affd640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad7724a0 0x558cad77cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 -- 172.21.15.37:0/3921431244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad7724a0 msgr2=0x558cad77cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad7724a0 0x558cad77cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 -- 172.21.15.37:0/3921431244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 msgr2=0x558cad77a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 -- 172.21.15.37:0/3921431244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cad4493c0 con 0x558cad772dc0 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703affd640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad7724a0 0x558cad77cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:25.777 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703b7fe640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f703a7fc640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad77a220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7024000c00 tx=0x7f70240176b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a4c64ba51a0fa73 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f701affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f701affd640 1 -- 172.21.15.37:0/3921431244 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024004050 con 0x558cad772dc0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702401c070 con 0x558cad772dc0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.757+0000 7f701affd640 1 -- 172.21.15.37:0/3921431244 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f70240041f0 con 0x558cad772dc0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f701affd640 1 -- 172.21.15.37:0/3921431244 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f70240043f0 con 0x558cad772dc0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 10 client.28271 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558cad78bc10 con 0x558cad772dc0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f701affd640 1 -- 172.21.15.37:0/3921431244 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702401c070 con 0x558cad772dc0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f701affd640 1 -- 172.21.15.37:0/3921431244 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f7024004920 con 0x558cad772dc0 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 10 client.28271 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 10 client.28271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 20 client.28271 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:25.778 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad78d020 0x558cad78f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 4 client.28271 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558cad6e5850 con 0x558cad78d020 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.758+0000 7f703affd640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad78d020 0x558cad78f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.759+0000 7f703affd640 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad78d020 0x558cad78f410 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91452e80650d2d server_cookie=85fb9f3d7d3ad541 in_seq=0 out_seq=0 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.759+0000 7f701affd640 10 client.28271 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.760+0000 7f701affd640 1 -- 172.21.15.37:0/3921431244 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x558cad6e5850 con 0x558cad78d020 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.760+0000 7f701affd640 10 client.28271 handle_command_reply: tid=0 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.760+0000 7f7043c6f740 1 client.28271 shutdown 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 msgr2=0x558cad77a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad77a220 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7024000c00 tx=0x7f70240176b0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 shutdown_connections 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad78d020 0x558cad78f410 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad77a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.779 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad77a220 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f701affd640 0 client.28271 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 --2- 172.21.15.37:0/3921431244 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad7724a0 0x558cad77cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 >> 172.21.15.37:0/3921431244 conn(0x558cad76ea30 msgr2=0x558cad774710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 shutdown_connections 2024-09-19T06:54:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 1 -- 172.21.15.37:0/3921431244 wait complete. 2024-09-19T06:54:25.780 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.761+0000 7f7043c6f740 20 client.28271 trim_cache size 0 max 16384 2024-09-19T06:54:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 Processor -- start 2024-09-19T06:54:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 -- start start 2024-09-19T06:54:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad7724a0 0x558cad7894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad794ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad77d680 0x558cad795000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.789 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1b2470 con 0x558cad7724a0 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1afd40 con 0x558cad772dc0 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f7043c6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1adbb0 con 0x558cad77d680 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f703affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad7724a0 0x558cad7894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f703affd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad7724a0 0x558cad7894b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56190/0 (socket says 172.21.15.37:56190) 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.778+0000 7f703affd640 1 -- 172.21.15.37:0/1070745664 learned_addr learned my addr 172.21.15.37:0/1070745664 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703a7fc640 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad794ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad77d680 0x558cad795000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703affd640 1 -- 172.21.15.37:0/1070745664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad77d680 msgr2=0x558cad795000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703affd640 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad77d680 0x558cad795000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703affd640 1 -- 172.21.15.37:0/1070745664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 msgr2=0x558cad794ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703affd640 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad794ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703affd640 1 -- 172.21.15.37:0/1070745664 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558cad76e860 con 0x558cad7724a0 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703a7fc640 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad794ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:25.790 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad77d680 0x558cad795000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f703affd640 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad7724a0 0x558cad7894b0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f702c0ed8d0 tx=0x7f702c005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6a53991f8415ac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f70317fa640 1 -- 172.21.15.37:0/1070745664 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c0bbab0 con 0x558cad7724a0 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f70317fa640 1 -- 172.21.15.37:0/1070745664 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702c0bc670 con 0x558cad7724a0 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.779+0000 7f70317fa640 1 -- 172.21.15.37:0/1070745664 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f702c0bb8d0 con 0x558cad7724a0 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 -- 172.21.15.37:0/1070745664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad7724a0 msgr2=0x558cad7894b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad7724a0 0x558cad7894b0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f702c0ed8d0 tx=0x7f702c005f90 comp rx=0 tx=0).stop 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 -- 172.21.15.37:0/1070745664 shutdown_connections 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad77d680 0x558cad795000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad772dc0 0x558cad794ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1070745664 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad7724a0 0x558cad7894b0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 -- 172.21.15.37:0/1070745664 >> 172.21.15.37:0/1070745664 conn(0x558cad76ea30 msgr2=0x558cad775900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 -- 172.21.15.37:0/1070745664 shutdown_connections 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 -- 172.21.15.37:0/1070745664 wait complete. 2024-09-19T06:54:25.791 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.780+0000 7f7043c6f740 1 -- start start 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 1 Processor -- start 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad785a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad776be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad777120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1ae200 con 0x558cad77d680 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1af670 con 0x558cad7724a0 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f703c1ae8d0 con 0x558cad772dc0 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f7043c6f740 10 client.0 _ll_register_callbacks cb 0x7f7034009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad777120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703affd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad785a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad777120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56202/0 (socket says 172.21.15.37:56202) 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703b7fe640 1 -- 172.21.15.37:0/1985228688 learned_addr learned my addr 172.21.15.37:0/1985228688 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:25.792 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703a7fc640 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad776be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703b7fe640 1 -- 172.21.15.37:0/1985228688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 msgr2=0x558cad776be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad776be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703b7fe640 1 -- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 msgr2=0x558cad785a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.781+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad785a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f703b7fe640 1 -- 172.21.15.37:0/1985228688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558cad76f6c0 con 0x558cad77d680 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f703affd640 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad785a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f703b7fe640 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad777120 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f703400bf10 tx=0x7f7034004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6baf220d3f0e3575 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f701affd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f701affd640 1 -- 172.21.15.37:0/1985228688 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70340196e0 con 0x558cad77d680 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f701affd640 1 -- 172.21.15.37:0/1985228688 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7f7034019880 con 0x558cad77d680 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f702c0bc670 con 0x558cad77d680 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f701affd640 1 -- 172.21.15.37:0/1985228688 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f7034035050 con 0x558cad77d680 2024-09-19T06:54:25.793 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f7043c6f740 10 client.18978 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x558cad7f7530 con 0x558cad77d680 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.782+0000 7f701affd640 1 -- 172.21.15.37:0/1985228688 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f702c0bc670 con 0x558cad77d680 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f701affd640 1 -- 172.21.15.37:0/1985228688 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7f703400b040 con 0x558cad77d680 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f7043c6f740 10 client.18978 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f7043c6f740 10 client.18978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f7043c6f740 20 client.18978 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad7f7d00 0x558cad7fa120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f7043c6f740 4 client.18978 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558cad6dfd60 con 0x558cad7f7d00 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f703affd640 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad7f7d00 0x558cad7fa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f703affd640 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad7f7d00 0x558cad7fa120 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2173c209e7f6ae2 server_cookie=bdfda42fb658d5f3 in_seq=0 out_seq=0 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.783+0000 7f701affd640 10 client.18978 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f701affd640 1 -- 172.21.15.37:0/1985228688 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x558cad6dfd60 con 0x558cad7f7d00 2024-09-19T06:54:25.794 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f701affd640 10 client.18978 handle_command_reply: tid=0 2024-09-19T06:54:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 client.18978 shutdown 2024-09-19T06:54:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 msgr2=0x558cad777120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad777120 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f703400bf10 tx=0x7f7034004a40 comp rx=0 tx=0).stop 2024-09-19T06:54:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 shutdown_connections 2024-09-19T06:54:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x558cad7f7d00 0x558cad7fa120 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.795 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x558cad77d680 0x558cad777120 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x558cad772dc0 0x558cad776be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 --2- 172.21.15.37:0/1985228688 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558cad7724a0 0x558cad785a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 >> 172.21.15.37:0/1985228688 conn(0x558cad76ea30 msgr2=0x558cad76d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f701affd640 0 client.18978 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 shutdown_connections 2024-09-19T06:54:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 1 -- 172.21.15.37:0/1985228688 wait complete. 2024-09-19T06:54:25.796 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.784+0000 7f7043c6f740 20 client.18978 trim_cache size 0 max 16384 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "bc958c3a-dca4-474b-a610-a92739621f1e": { 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "bc958c3a-dca4-474b-a610-a92739621f1e", 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:54:25.797 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:54:25.798 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:54:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70280a33b0 msgr2=0x7f70280a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70280a33b0 0x7f70280a5870 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7034005c40 tx=0x7f70340095c0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 msgr2=0x7f703c1a0440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c1a0440 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f702c009860 tx=0x7f702c030460 comp rx=0 tx=0).stop 2024-09-19T06:54:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 shutdown_connections 2024-09-19T06:54:25.798 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f70280a33b0 0x7f70280a5870 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f703c102ef0 0x7f703c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f703c102570 0x7f703c1a0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 --2- 172.21.15.37:0/1497211938 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f703c100e20 0x7f703c1a0440 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 >> 172.21.15.37:0/1497211938 conn(0x7f703c0fc7d0 msgr2=0x7f703c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 shutdown_connections 2024-09-19T06:54:25.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:25.785+0000 7f7041896640 1 -- 172.21.15.37:0/1497211938 wait complete. 2024-09-19T06:54:26.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:54:26.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:54:26.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:54:26.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.b log and `damage ls` output for details 2024-09-19T06:54:26.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.026 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:25 smithi037 ceph-mon[31211]: pgmap v399: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.b log and `damage ls` output for details 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:25 smithi151 ceph-mon[35236]: pgmap v399: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-09-19T06:54:26.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 991556 7.82 MB/sec execute 533 sec latency 67.416 ms 2024-09-19T06:54:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:54:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:54:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:54:26.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.b log and `damage ls` output for details 2024-09-19T06:54:26.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:54:26.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:25 smithi096 ceph-mon[35004]: pgmap v399: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-09-19T06:54:26.495 INFO:tasks.cephfs.filesystem:scrub status for tag:bc958c3a-dca4-474b-a610-a92739621f1e - {'path': '/', 'tag': 'bc958c3a-dca4-474b-a610-a92739621f1e', 'options': 'recursive,force'} 2024-09-19T06:54:27.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 993667 7.82 MB/sec execute 534 sec latency 17.873 ms 2024-09-19T06:54:28.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:27 smithi151 ceph-mon[35236]: pgmap v400: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-09-19T06:54:28.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 995332 7.83 MB/sec execute 535 sec latency 26.410 ms 2024-09-19T06:54:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:27 smithi096 ceph-mon[35004]: pgmap v400: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-09-19T06:54:28.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:27 smithi037 ceph-mon[31211]: pgmap v400: 97 pgs: 97 active+clean; 61 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.1 MiB/s wr, 315 op/s 2024-09-19T06:54:29.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 997968 7.84 MB/sec execute 536 sec latency 26.001 ms 2024-09-19T06:54:30.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:29 smithi151 ceph-mon[35236]: pgmap v401: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 418 op/s 2024-09-19T06:54:30.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 999236 7.83 MB/sec execute 537 sec latency 15.993 ms 2024-09-19T06:54:30.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:29 smithi096 ceph-mon[35004]: pgmap v401: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 418 op/s 2024-09-19T06:54:30.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:29 smithi037 ceph-mon[31211]: pgmap v401: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.1 MiB/s wr, 418 op/s 2024-09-19T06:54:31.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1001875 7.85 MB/sec execute 538 sec latency 43.614 ms 2024-09-19T06:54:32.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:31 smithi151 ceph-mon[35236]: pgmap v402: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-09-19T06:54:32.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1003290 7.85 MB/sec execute 539 sec latency 32.423 ms 2024-09-19T06:54:32.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:31 smithi096 ceph-mon[35004]: pgmap v402: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-09-19T06:54:32.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:31 smithi037 ceph-mon[31211]: pgmap v402: 97 pgs: 97 active+clean; 86 MiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.6 MiB/s wr, 322 op/s 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 -- 172.21.15.37:0/1036612630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c810c0d0 msgr2=0x7fe4c810e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 --2- 172.21.15.37:0/1036612630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c810c0d0 0x7fe4c810e570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009be0 tx=0x7fe4c002a6c0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 -- 172.21.15.37:0/1036612630 shutdown_connections 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 --2- 172.21.15.37:0/1036612630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c810c0d0 0x7fe4c810e570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 --2- 172.21.15.37:0/1036612630 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c81035c0 0x7fe4c810baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 --2- 172.21.15.37:0/1036612630 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c8102ca0 0x7fe4c8103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 -- 172.21.15.37:0/1036612630 >> 172.21.15.37:0/1036612630 conn(0x7fe4c80fc6b0 msgr2=0x7fe4c80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 -- 172.21.15.37:0/1036612630 shutdown_connections 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.510+0000 7fe4cd559640 1 -- 172.21.15.37:0/1036612630 wait complete. 2024-09-19T06:54:32.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 Processor -- start 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 -- start start 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c8102ca0 0x7fe4c81008f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c81035c0 0x7fe4c8100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c810c0d0 0x7fe4c8101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8111e80 con 0x7fe4c810c0d0 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8111d00 con 0x7fe4c81035c0 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4cd559640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c8112000 con 0x7fe4c8102ca0 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c8102ca0 0x7fe4c81008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c810c0d0 0x7fe4c8101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c81035c0 0x7fe4c8100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c81035c0 0x7fe4c8100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:50520/0 (socket says 172.21.15.37:50520) 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4c77fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c810c0d0 0x7fe4c8101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60552/0 (socket says 172.21.15.37:60552) 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.511+0000 7fe4bffff640 1 -- 172.21.15.37:0/3659561796 learned_addr learned my addr 172.21.15.37:0/3659561796 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:32.514 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c77fe640 1 -- 172.21.15.37:0/3659561796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c8102ca0 msgr2=0x7fe4c81008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c77fe640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c8102ca0 0x7fe4c81008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c77fe640 1 -- 172.21.15.37:0/3659561796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c81035c0 msgr2=0x7fe4c8100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c77fe640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c81035c0 0x7fe4c8100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c77fe640 1 -- 172.21.15.37:0/3659561796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4c81a6790 con 0x7fe4c810c0d0 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4bffff640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c81035c0 0x7fe4c8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c77fe640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c810c0d0 0x7fe4c8101370 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0034040 tx=0x7fe4c00305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c4ff9640 1 -- 172.21.15.37:0/3659561796 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0041020 con 0x7fe4c810c0d0 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4c81a6a20 con 0x7fe4c810c0d0 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c4ff9640 1 -- 172.21.15.37:0/3659561796 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4c0037040 con 0x7fe4c810c0d0 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4c4ff9640 1 -- 172.21.15.37:0/3659561796 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0036070 con 0x7fe4c810c0d0 2024-09-19T06:54:32.515 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.512+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4c81a6f10 con 0x7fe4c810c0d0 2024-09-19T06:54:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.514+0000 7fe4c4ff9640 1 -- 172.21.15.37:0/3659561796 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe4c0044020 con 0x7fe4c810c0d0 2024-09-19T06:54:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 Processor -- start 2024-09-19T06:54:32.520 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 -- start start 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f7eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7f410 0x55b9b3f89770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f8c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3ee32f0 con 0x55b9b3f89cb0 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3e8b270 con 0x55b9b3f7f410 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3edf2b0 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7f410 0x55b9b3f89770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7f410 0x55b9b3f89770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:50532/0 (socket says 172.21.15.37:50532) 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/3397685002 learned_addr learned my addr 172.21.15.37:0/3397685002 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4bffff640 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f8c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.515+0000 7fe4bffff640 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f7eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/3397685002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 msgr2=0x55b9b3f7eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.521 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f7eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/3397685002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f89cb0 msgr2=0x55b9b3f8c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f8c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/3397685002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b3e8a340 con 0x55b9b3f7f410 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c4ff9640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe4b40a3460 0x7fe4b40a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4bffff640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe4b40a3460 0x7fe4b40a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7f410 0x55b9b3f89770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009d20 tx=0x7fe4b8011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5063ac3d3df459b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4c4ff9640 1 -- 172.21.15.37:0/3659561796 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fe4c00f06d0 con 0x7fe4c810c0d0 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4bd7fa640 1 -- 172.21.15.37:0/3397685002 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b801e070 con 0x55b9b3f7f410 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4bd7fa640 1 -- 172.21.15.37:0/3397685002 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b8019b10 con 0x55b9b3f7f410 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.516+0000 7fe4bd7fa640 1 -- 172.21.15.37:0/3397685002 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b800eb90 con 0x55b9b3f7f410 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 -- 172.21.15.37:0/3397685002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7f410 msgr2=0x55b9b3f89770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4bffff640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe4b40a3460 0x7fe4b40a5920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55b9b3f7c3e0 tx=0x7fe4b0009840 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:32.522 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7f410 0x55b9b3f89770 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8009d20 tx=0x7fe4b8011810 comp rx=0 tx=0).stop 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 -- 172.21.15.37:0/3397685002 shutdown_connections 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f8c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7f410 0x55b9b3f89770 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/3397685002 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f7eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 -- 172.21.15.37:0/3397685002 >> 172.21.15.37:0/3397685002 conn(0x55b9b3f7b060 msgr2=0x55b9b3f7b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 -- 172.21.15.37:0/3397685002 shutdown_connections 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 -- 172.21.15.37:0/3397685002 wait complete. 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 -- start start 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.517+0000 7fe4d03d2740 1 Processor -- start 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f86ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f870e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f87620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3f8c880 con 0x55b9b3f7f410 2024-09-19T06:54:32.523 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3e6c290 con 0x55b9b3f89cb0 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55b9b3f8ca00 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4d03d2740 10 client.0 _ll_register_callbacks cb 0x55b9b3e9d140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f870e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f86ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f870e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60574/0 (socket says 172.21.15.37:60574) 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 -- 172.21.15.37:0/1306957565 learned_addr learned my addr 172.21.15.37:0/1306957565 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4c77fe640 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f87620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 -- 172.21.15.37:0/1306957565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 msgr2=0x55b9b3f86ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f86ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 -- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f89cb0 msgr2=0x55b9b3f87620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f87620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 -- 172.21.15.37:0/1306957565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b3bf6210 con 0x55b9b3f7f410 2024-09-19T06:54:32.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4bffff640 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f870e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0012010 tx=0x7fe4b0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b44c38511effe41 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.518+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1306957565 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b0023630 con 0x55b9b3f7f410 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b8019b10 con 0x55b9b3f7f410 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1306957565 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fe4b00237d0 con 0x55b9b3f7f410 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1306957565 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe4b00239d0 con 0x55b9b3f7f410 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 10 client.18990 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b9b3cc2620 con 0x55b9b3f7f410 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1306957565 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b8019b10 con 0x55b9b3f7f410 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1306957565 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe4b0033ca0 con 0x55b9b3f7f410 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 10 client.18990 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 10 client.18990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 20 client.18990 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b3f99d20 0x55b9b3f9c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 4 client.18990 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b9b3ef0a20 con 0x55b9b3f99d20 2024-09-19T06:54:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.519+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b3f99d20 0x55b9b3f9c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.520+0000 7fe4ae7fc640 10 client.18990 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:32.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.520+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b3f99d20 0x55b9b3f9c110 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3bc6029b9abbda3 server_cookie=e70dfc64723621b1 in_seq=0 out_seq=0 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1306957565 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55b9b3ef0a20 con 0x55b9b3f99d20 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4ae7fc640 10 client.18990 handle_command_reply: tid=0 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 client.18990 shutdown 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 msgr2=0x55b9b3f870e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f870e0 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0012010 tx=0x7fe4b0004730 comp rx=0 tx=0).stop 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 shutdown_connections 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b3f99d20 0x55b9b3f9c110 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f87620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4ae7fc640 0 client.18990 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f870e0 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1306957565 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f86ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 >> 172.21.15.37:0/1306957565 conn(0x55b9b3f7b060 msgr2=0x55b9b3f80d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 shutdown_connections 2024-09-19T06:54:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1306957565 wait complete. 2024-09-19T06:54:32.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.521+0000 7fe4d03d2740 20 client.18990 trim_cache size 0 max 16384 2024-09-19T06:54:32.537 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.535+0000 7fe4d03d2740 1 Processor -- start 2024-09-19T06:54:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.535+0000 7fe4d03d2740 1 -- start start 2024-09-19T06:54:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f96240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f87450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3fa1a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81b22d0 con 0x55b9b3f7f410 2024-09-19T06:54:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81afba0 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.538 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81ada10 con 0x55b9b3f89cb0 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f87450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f96240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f87450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60578/0 (socket says 172.21.15.37:60578) 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4bffff640 1 -- 172.21.15.37:0/1471890744 learned_addr learned my addr 172.21.15.37:0/1471890744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4c77fe640 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3fa1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/1471890744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 msgr2=0x55b9b3fa1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3fa1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/1471890744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 msgr2=0x55b9b3f87450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f87450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/1471890744 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9b3f85b30 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.536+0000 7fe4bffff640 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f87450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.537+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f96240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8011d10 tx=0x7fe4b80051f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3fcdc41ebd036e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.537+0000 7fe4bcff9640 1 -- 172.21.15.37:0/1471890744 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8033070 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.539 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.537+0000 7fe4bcff9640 1 -- 172.21.15.37:0/1471890744 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b801f040 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.540 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.537+0000 7fe4bcff9640 1 -- 172.21.15.37:0/1471890744 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b801e040 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.537+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1471890744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 msgr2=0x55b9b3f96240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.537+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f96240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8011d10 tx=0x7fe4b80051f0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1471890744 shutdown_connections 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3fa1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f87450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1471890744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f96240 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1471890744 >> 172.21.15.37:0/1471890744 conn(0x55b9b3f7b060 msgr2=0x55b9b3f88c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1471890744 shutdown_connections 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1471890744 wait complete. 2024-09-19T06:54:32.543 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- start start 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 Processor -- start 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f83540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f83a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81ae060 con 0x55b9b3f7f410 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81af4d0 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe4c81ae730 con 0x55b9b3f89cb0 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4d03d2740 10 client.0 _ll_register_callbacks cb 0x7fe4b0009db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f83540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4c6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4bffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f83540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60588/0 (socket says 172.21.15.37:60588) 2024-09-19T06:54:32.544 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4bffff640 1 -- 172.21.15.37:0/1265278484 learned_addr learned my addr 172.21.15.37:0/1265278484 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.538+0000 7fe4c77fe640 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f83a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/1265278484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 msgr2=0x55b9b3f83a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f83a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/1265278484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 msgr2=0x55b9b3f83540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f83540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4c6ffd640 1 -- 172.21.15.37:0/1265278484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9b3f83fc0 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4bffff640 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4c6ffd640 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f920a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8011850 tx=0x7fe4b8009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a2ca65c8a0a1acc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4ae7fc640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1265278484 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4b801e030 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe4b801f040 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1265278484 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8571+0+0 (secure 0 0 0) 0x7fe4b800ea60 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.545 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1265278484 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7fe4b8033070 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4d03d2740 10 client.28295 fetch_fsmap learned FSMap version 41 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55b9b4004680 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1265278484 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe4b801f040 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1265278484 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 41) ==== 8346+0+0 (secure 0 0 0) 0x7fe4b803e070 con 0x55b9b3f7eaf0 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.539+0000 7fe4d03d2740 10 client.28295 fetch_fsmap finished waiting for FSMap version 41 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4d03d2740 10 client.28295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4d03d2740 20 client.28295 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b4004af0 0x55b9b4006ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4d03d2740 4 client.28295 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b9b3eead70 con 0x55b9b4004af0 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4bffff640 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b4004af0 0x55b9b4006ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4ae7fc640 10 client.28295 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:32.546 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.540+0000 7fe4bffff640 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b4004af0 0x55b9b4006ee0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1efed7295dff47d server_cookie=dece39256ede2a5f in_seq=0 out_seq=0 2024-09-19T06:54:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4ae7fc640 1 -- 172.21.15.37:0/1265278484 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b9b3eead70 con 0x55b9b4004af0 2024-09-19T06:54:32.724 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4ae7fc640 10 client.28295 handle_command_reply: tid=0 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 client.28295 shutdown 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 msgr2=0x55b9b3f920a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f920a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8011850 tx=0x7fe4b8009c20 comp rx=0 tx=0).stop 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 shutdown_connections 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55b9b4004af0 0x55b9b4006ee0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55b9b3f89cb0 0x55b9b3f83a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55b9b3f7f410 0x55b9b3f83540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 --2- 172.21.15.37:0/1265278484 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b9b3f7eaf0 0x55b9b3f920a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 >> 172.21.15.37:0/1265278484 conn(0x55b9b3f7b060 msgr2=0x55b9b3f88400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4ae7fc640 0 client.28295 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 shutdown_connections 2024-09-19T06:54:32.725 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.723+0000 7fe4d03d2740 1 -- 172.21.15.37:0/1265278484 wait complete. 2024-09-19T06:54:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4d03d2740 20 client.28295 trim_cache size 0 max 16384 2024-09-19T06:54:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:54:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:54:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:54:32.726 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe4b40a3460 msgr2=0x7fe4b40a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe4b40a3460 0x7fe4b40a5920 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x55b9b3f7c3e0 tx=0x7fe4b0009840 comp rx=0 tx=0).stop 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c810c0d0 msgr2=0x7fe4c8101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c810c0d0 0x7fe4c8101370 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0034040 tx=0x7fe4c00305d0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 shutdown_connections 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe4b40a3460 0x7fe4b40a5920 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe4c810c0d0 0x7fe4c8101370 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe4c81035c0 0x7fe4c8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 --2- 172.21.15.37:0/3659561796 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe4c8102ca0 0x7fe4c81008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:32.727 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.724+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 >> 172.21.15.37:0/3659561796 conn(0x7fe4c80fc6b0 msgr2=0x7fe4c80fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.725+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 shutdown_connections 2024-09-19T06:54:32.728 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:32.725+0000 7fe4cd559640 1 -- 172.21.15.37:0/3659561796 wait complete. 2024-09-19T06:54:33.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:32 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:54:33.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:32 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:54:33.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1005494 7.86 MB/sec execute 540 sec latency 71.226 ms 2024-09-19T06:54:33.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:32 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:54:33.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:32 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:54:33.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:32 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:54:33.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:32 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:54:33.612 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:54:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setuser ceph since I am not root 2024-09-19T06:54:33.612 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:54:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setgroup ceph since I am not root 2024-09-19T06:54:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:54:33 smithi151 ceph-mds[66270]: -- 172.21.15.151:0/3412481835 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f07ba67860 con 0x55f07ba45400 2024-09-19T06:54:33.613 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:54:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: starting mds.i at 2024-09-19T06:54:34.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:33 smithi151 ceph-mon[35236]: pgmap v403: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 500 op/s 2024-09-19T06:54:34.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:33 smithi151 ceph-mon[35236]: daemon mds.i restarted 2024-09-19T06:54:34.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1007073 7.85 MB/sec execute 541 sec latency 16.559 ms 2024-09-19T06:54:34.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:33 smithi096 ceph-mon[35004]: pgmap v403: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 500 op/s 2024-09-19T06:54:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:33 smithi096 ceph-mon[35004]: daemon mds.i restarted 2024-09-19T06:54:34.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:33 smithi037 ceph-mon[31211]: pgmap v403: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.0 MiB/s wr, 500 op/s 2024-09-19T06:54:34.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:33 smithi037 ceph-mon[31211]: daemon mds.i restarted 2024-09-19T06:54:35.112 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:34 smithi151 ceph-mon[35236]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T06:54:35.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:34 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6838/1806889964,v1:172.21.15.151:6839/1806889964] up:boot 2024-09-19T06:54:35.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:34 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:54:35.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:34 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:54:35.113 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:34 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:54:35.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1009267 7.86 MB/sec execute 542 sec latency 19.703 ms 2024-09-19T06:54:35.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:34 smithi096 ceph-mon[35004]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T06:54:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:34 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6838/1806889964,v1:172.21.15.151:6839/1806889964] up:boot 2024-09-19T06:54:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:34 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:54:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:34 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:54:35.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:34 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:54:35.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:34 smithi037 ceph-mon[31211]: osdmap e102: 12 total, 12 up, 12 in 2024-09-19T06:54:35.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:34 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6838/1806889964,v1:172.21.15.151:6839/1806889964] up:boot 2024-09-19T06:54:35.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:34 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:54:35.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:34 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:54:35.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:34 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:54:36.234 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1011394 7.86 MB/sec execute 543 sec latency 49.567 ms 2024-09-19T06:54:36.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:35 smithi096 ceph-mon[35004]: pgmap v405: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-19T06:54:36.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:35 smithi037 ceph-mon[31211]: pgmap v405: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-19T06:54:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:35 smithi151 ceph-mon[35236]: pgmap v405: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-19T06:54:37.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1013027 7.87 MB/sec execute 544 sec latency 28.419 ms 2024-09-19T06:54:38.164 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:37 smithi151 ceph-mon[35236]: pgmap v406: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-19T06:54:38.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1015866 7.88 MB/sec execute 545 sec latency 20.402 ms 2024-09-19T06:54:38.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:37 smithi096 ceph-mon[35004]: pgmap v406: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-19T06:54:38.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:37 smithi037 ceph-mon[31211]: pgmap v406: 97 pgs: 97 active+clean; 76 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.3 MiB/s wr, 336 op/s 2024-09-19T06:54:39.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1017237 7.88 MB/sec execute 546 sec latency 19.979 ms 2024-09-19T06:54:40.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1019732 7.90 MB/sec execute 547 sec latency 17.502 ms 2024-09-19T06:54:40.260 INFO:tasks.quiescer.fs.[cephfs]:Total duration (44.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-19T06:54:40.260 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T06:54:40.261 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:54:40.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:39 smithi096 ceph-mon[35004]: pgmap v407: 97 pgs: 97 active+clean; 79 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-19T06:54:40.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:39 smithi037 ceph-mon[31211]: pgmap v407: 97 pgs: 97 active+clean; 79 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-19T06:54:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:39 smithi151 ceph-mon[35236]: pgmap v407: 97 pgs: 97 active+clean; 79 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-19T06:54:40.549 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:41.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.041+0000 7f7598b2a640 1 -- 172.21.15.37:0/3154699626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 msgr2=0x7f7594101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:41.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.041+0000 7f7598b2a640 1 --2- 172.21.15.37:0/3154699626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f7594101200 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f7584009900 tx=0x7f758402a480 comp rx=0 tx=0).stop 2024-09-19T06:54:41.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.041+0000 7f7598b2a640 1 -- 172.21.15.37:0/3154699626 shutdown_connections 2024-09-19T06:54:41.171 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.041+0000 7f7598b2a640 1 --2- 172.21.15.37:0/3154699626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594102ef0 0x7f759410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.042+0000 7f7598b2a640 1 --2- 172.21.15.37:0/3154699626 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7594102570 0x7f75941029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.042+0000 7f7598b2a640 1 --2- 172.21.15.37:0/3154699626 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f7594101200 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.042+0000 7f7598b2a640 1 -- 172.21.15.37:0/3154699626 >> 172.21.15.37:0/3154699626 conn(0x7f75940fc7d0 msgr2=0x7f75940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.042+0000 7f7598b2a640 1 -- 172.21.15.37:0/3154699626 shutdown_connections 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.042+0000 7f7598b2a640 1 -- 172.21.15.37:0/3154699626 wait complete. 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.042+0000 7f7598b2a640 1 Processor -- start 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7598b2a640 1 -- start start 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7598b2a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f75941984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7598b2a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594102570 0x7f7594198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7598b2a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7594102ef0 0x7f759419cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7598b2a640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f75941120a0 con 0x7f7594100e20 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7598b2a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7594111f20 con 0x7f7594102570 2024-09-19T06:54:41.172 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7598b2a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7594112220 con 0x7f7594102ef0 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f75941984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f75941984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46526/0 (socket says 172.21.15.37:46526) 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 -- 172.21.15.37:0/92435354 learned_addr learned my addr 172.21.15.37:0/92435354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f758bfff640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594102570 0x7f7594198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f7592d9d640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7594102ef0 0x7f759419cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 -- 172.21.15.37:0/92435354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7594102ef0 msgr2=0x7f759419cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7594102ef0 0x7f759419cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 -- 172.21.15.37:0/92435354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594102570 msgr2=0x7f7594198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594102570 0x7f7594198a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f759259c640 1 -- 172.21.15.37:0/92435354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f759419d470 con 0x7f7594100e20 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.043+0000 7f758bfff640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594102570 0x7f7594198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.044+0000 7f759259c640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f75941984c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7584006010 tx=0x7f7584003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.044+0000 7f758b7fe640 1 -- 172.21.15.37:0/92435354 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758403f070 con 0x7f7594100e20 2024-09-19T06:54:41.173 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.044+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f759419d700 con 0x7f7594100e20 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.044+0000 7f758b7fe640 1 -- 172.21.15.37:0/92435354 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7584031c30 con 0x7f7594100e20 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.044+0000 7f758b7fe640 1 -- 172.21.15.37:0/92435354 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584035030 con 0x7f7594100e20 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.044+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75941a5040 con 0x7f7594100e20 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.045+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7594101200 con 0x7f7594100e20 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.048+0000 7f758b7fe640 1 -- 172.21.15.37:0/92435354 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7584042070 con 0x7f7594100e20 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.048+0000 7f758b7fe640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75800a33b0 0x7f75800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.048+0000 7f758b7fe640 1 -- 172.21.15.37:0/92435354 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f75840f0840 con 0x7f7594100e20 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.048+0000 7f758bfff640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75800a33b0 0x7f75800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.049+0000 7f758bfff640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75800a33b0 0x7f75800a5870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75940ffea0 tx=0x7f757c00b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:41.174 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.051+0000 7f758b7fe640 1 -- 172.21.15.37:0/92435354 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f75840bcbd0 con 0x7f7594100e20 2024-09-19T06:54:41.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1021257 7.89 MB/sec execute 548 sec latency 58.113 ms 2024-09-19T06:54:41.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.279+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7594109090 con 0x7f7594100e20 2024-09-19T06:54:41.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.280+0000 7f758b7fe640 1 -- 172.21.15.37:0/92435354 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11903 (secure 0 0 0) 0x7f75840c0ed0 con 0x7f7594100e20 2024-09-19T06:54:41.282 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:54:41.282 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-19T06:54:33:862495+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:54:33.862475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27958":{"gid":27958,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/1806889964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1806889964},{"type":"v1","addr":"172.21.15.151:6839","nonce":1806889964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:54:41.282 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-19T06:54:41.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.282+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75800a33b0 msgr2=0x7f75800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:41.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.282+0000 7f7598b2a640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75800a33b0 0x7f75800a5870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f75940ffea0 tx=0x7f757c00b390 comp rx=0 tx=0).stop 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 msgr2=0x7f75941984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f75941984c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f7584006010 tx=0x7f7584003c10 comp rx=0 tx=0).stop 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 shutdown_connections 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7594102ef0 0x7f759419cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7594102570 0x7f7594198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75800a33b0 0x7f75800a5870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 --2- 172.21.15.37:0/92435354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7594100e20 0x7f75941984c0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 >> 172.21.15.37:0/92435354 conn(0x7f75940fc7d0 msgr2=0x7f75940fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 shutdown_connections 2024-09-19T06:54:41.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:41.283+0000 7f7598b2a640 1 -- 172.21.15.37:0/92435354 wait complete. 2024-09-19T06:54:42.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1023368 7.90 MB/sec execute 549 sec latency 26.118 ms 2024-09-19T06:54:42.252 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:41 smithi037 ceph-mon[31211]: pgmap v408: 97 pgs: 97 active+clean; 79 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-19T06:54:42.252 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:41 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/92435354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:42.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:41 smithi096 ceph-mon[35004]: pgmap v408: 97 pgs: 97 active+clean; 79 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-19T06:54:42.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:41 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/92435354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:42.279 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T06:54:42.279 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T06:54:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:41 smithi151 ceph-mon[35236]: pgmap v408: 97 pgs: 97 active+clean; 79 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.3 MiB/s wr, 325 op/s 2024-09-19T06:54:42.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:41 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/92435354' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:54:42.588 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 -- 172.21.15.37:0/3499263749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 msgr2=0x7f71fc10ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.066 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 --2- 172.21.15.37:0/3499263749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 0x7f71fc10ea90 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f71f4009ca0 tx=0x7f71f402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 -- 172.21.15.37:0/3499263749 shutdown_connections 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 --2- 172.21.15.37:0/3499263749 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 0x7f71fc10ea90 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 --2- 172.21.15.37:0/3499263749 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 0x7f71fc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 --2- 172.21.15.37:0/3499263749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71fc101de0 0x7f71fc1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 -- 172.21.15.37:0/3499263749 >> 172.21.15.37:0/3499263749 conn(0x7f71fc0fd8c0 msgr2=0x7f71fc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.058+0000 7f720419d640 1 -- 172.21.15.37:0/3499263749 shutdown_connections 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.059+0000 7f720419d640 1 -- 172.21.15.37:0/3499263749 wait complete. 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.059+0000 7f720419d640 1 Processor -- start 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.059+0000 7f720419d640 1 -- start start 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.059+0000 7f720419d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71fc101de0 0x7f71fc19a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.059+0000 7f720419d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 0x7f71fc19ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.059+0000 7f720419d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 0x7f71fc19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.067 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f720419d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc111f70 con 0x7f71fc106f10 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f720419d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc111df0 con 0x7f71fc101de0 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f720419d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1120f0 con 0x7f71fc102790 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f720273a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 0x7f71fc19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201738640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 0x7f71fc19ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201f39640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71fc101de0 0x7f71fc19a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201738640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 0x7f71fc19ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:46122/0 (socket says 172.21.15.37:46122) 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201738640 1 -- 172.21.15.37:0/2443994190 learned_addr learned my addr 172.21.15.37:0/2443994190 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201f39640 1 -- 172.21.15.37:0/2443994190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 msgr2=0x7f71fc19ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201f39640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 0x7f71fc19ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201f39640 1 -- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 msgr2=0x7f71fc19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201f39640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 0x7f71fc19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201f39640 1 -- 172.21.15.37:0/2443994190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc19f5e0 con 0x7f71fc101de0 2024-09-19T06:54:43.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201738640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 0x7f71fc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.060+0000 7f7201f39640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71fc101de0 0x7f71fc19a630 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e9f0 tx=0x7f71f800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.061+0000 7f71f2ffd640 1 -- 172.21.15.37:0/2443994190 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f800ccc0 con 0x7f71fc101de0 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.061+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc19f8d0 con 0x7f71fc101de0 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.061+0000 7f71f2ffd640 1 -- 172.21.15.37:0/2443994190 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f8010910 con 0x7f71fc101de0 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.061+0000 7f71f2ffd640 1 -- 172.21.15.37:0/2443994190 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8018960 con 0x7f71fc101de0 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.061+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc1a71b0 con 0x7f71fc101de0 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.063+0000 7f7206576740 1 Processor -- start 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.063+0000 7f71f2ffd640 1 -- 172.21.15.37:0/2443994190 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f71f8010ac0 con 0x7f71fc101de0 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7206576740 1 -- start start 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7206576740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d4760 0x5572b82d4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7206576740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d5080 0x5572b82df400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7206576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7206576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5572b823a2a0 con 0x5572b82df940 2024-09-19T06:54:43.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7206576740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572b81e1b10 con 0x5572b82d4760 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7206576740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572b8236260 con 0x5572b82d5080 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f720273a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d4760 0x5572b82d4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f720273a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d4760 0x5572b82d4b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58130/0 (socket says 172.21.15.37:58130) 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7201738640 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f720273a640 1 -- 172.21.15.37:0/1589266613 learned_addr learned my addr 172.21.15.37:0/1589266613 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f7201f39640 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d5080 0x5572b82df400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.064+0000 7f71f2ffd640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f71e80a33b0 0x7f71e80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f7201738640 1 -- 172.21.15.37:0/1589266613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d5080 msgr2=0x5572b82df400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f7201738640 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d5080 0x5572b82df400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f7201738640 1 -- 172.21.15.37:0/1589266613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d4760 msgr2=0x5572b82d4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f7201738640 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d4760 0x5572b82d4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f7201738640 1 -- 172.21.15.37:0/1589266613 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b7fabb90 con 0x5572b82df940 2024-09-19T06:54:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f7201f39640 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d5080 0x5572b82df400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f71f2ffd640 1 -- 172.21.15.37:0/2443994190 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f71f80d00b0 con 0x7f71fc101de0 2024-09-19T06:54:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f720273a640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f71e80a33b0 0x7f71e80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.071 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f7201738640 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e1d30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0114c0 tx=0x7f71ec011980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ec47e2b516aa34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f71e7fff640 1 -- 172.21.15.37:0/1589266613 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec00f680 con 0x5572b82df940 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f71e7fff640 1 -- 172.21.15.37:0/1589266613 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ec00fd00 con 0x5572b82df940 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.065+0000 7f71e7fff640 1 -- 172.21.15.37:0/1589266613 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec019df0 con 0x5572b82df940 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 -- 172.21.15.37:0/1589266613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 msgr2=0x5572b82e1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f720273a640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f71e80a33b0 0x7f71e80a5870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f71f402f540 tx=0x7f71f402a8c0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e1d30 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f71ec0114c0 tx=0x7f71ec011980 comp rx=0 tx=0).stop 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 -- 172.21.15.37:0/1589266613 shutdown_connections 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e1d30 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d5080 0x5572b82df400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 --2- 172.21.15.37:0/1589266613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d4760 0x5572b82d4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 -- 172.21.15.37:0/1589266613 >> 172.21.15.37:0/1589266613 conn(0x5572b82d0cf0 msgr2=0x5572b82d10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 -- 172.21.15.37:0/1589266613 shutdown_connections 2024-09-19T06:54:43.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 -- 172.21.15.37:0/1589266613 wait complete. 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.066+0000 7f7206576740 1 -- start start 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 1 Processor -- start 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82dc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82dccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82dd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5572b82e2510 con 0x5572b82df940 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5572b81c26e0 con 0x5572b82d5080 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5572b82e2690 con 0x5572b82d4760 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7206576740 10 client.0 _ll_register_callbacks cb 0x5572b81e6a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7201738640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82dccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f720273a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82dd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7201738640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82dccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58140/0 (socket says 172.21.15.37:58140) 2024-09-19T06:54:43.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7201738640 1 -- 172.21.15.37:0/3136751502 learned_addr learned my addr 172.21.15.37:0/3136751502 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.067+0000 7f7201f39640 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82dc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7201738640 1 -- 172.21.15.37:0/3136751502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 msgr2=0x5572b82dc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7201738640 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82dc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7201738640 1 -- 172.21.15.37:0/3136751502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 msgr2=0x5572b82dd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7201738640 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82dd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7201738640 1 -- 172.21.15.37:0/3136751502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b82eb980 con 0x5572b82d5080 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7201f39640 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82dc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7201738640 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82dccd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f71ec019de0 tx=0x7f71ec019e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a93a8197db3cf8b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f71e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f71e57fa640 1 -- 172.21.15.37:0/3136751502 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec028030 con 0x5572b82d5080 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f71e57fa640 1 -- 172.21.15.37:0/3136751502 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f71ec016040 con 0x5572b82d5080 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71ec00fd00 con 0x5572b82d5080 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f71e57fa640 1 -- 172.21.15.37:0/3136751502 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f71ec032050 con 0x5572b82d5080 2024-09-19T06:54:43.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7206576740 10 client.28319 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5572b82ee2f0 con 0x5572b82d5080 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.068+0000 7f71e57fa640 1 -- 172.21.15.37:0/3136751502 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71ec00fd00 con 0x5572b82d5080 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f71e57fa640 1 -- 172.21.15.37:0/3136751502 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f71ec004730 con 0x5572b82d5080 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7206576740 10 client.28319 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7206576740 10 client.28319 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7206576740 20 client.28319 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7206576740 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b82ef750 0x5572b82f1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7206576740 4 client.28319 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572b82479d0 con 0x5572b82ef750 2024-09-19T06:54:43.078 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7201f39640 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b82ef750 0x5572b82f1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f71e57fa640 10 client.28319 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.069+0000 7f7201f39640 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b82ef750 0x5572b82f1b40 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=482b725d1815f682 server_cookie=20f2e59b51b554e6 in_seq=0 out_seq=0 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f71e57fa640 1 -- 172.21.15.37:0/3136751502 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5572b82479d0 con 0x5572b82ef750 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f71e57fa640 10 client.28319 handle_command_reply: tid=0 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 client.28319 shutdown 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 msgr2=0x5572b82dccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82dccd0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f71ec019de0 tx=0x7f71ec019e10 comp rx=0 tx=0).stop 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 shutdown_connections 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b82ef750 0x5572b82f1b40 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82dd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82dccd0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 --2- 172.21.15.37:0/3136751502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82dc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f71e57fa640 0 client.28319 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:43.079 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 >> 172.21.15.37:0/3136751502 conn(0x5572b82d0cf0 msgr2=0x5572b82cfe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 shutdown_connections 2024-09-19T06:54:43.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.070+0000 7f7206576740 1 -- 172.21.15.37:0/3136751502 wait complete. 2024-09-19T06:54:43.080 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.071+0000 7f7206576740 20 client.28319 trim_cache size 0 max 16384 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 Processor -- start 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 -- start start 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82d4760 0x5572b82f7200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82df940 0x5572b82d9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1b2390 con 0x5572b82d4760 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1afc60 con 0x5572b82d5080 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7206576740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1adab0 con 0x5572b82df940 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7201f39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82d4760 0x5572b82f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7201f39640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82d4760 0x5572b82f7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46564/0 (socket says 172.21.15.37:46564) 2024-09-19T06:54:43.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7201f39640 1 -- 172.21.15.37:0/3776159279 learned_addr learned my addr 172.21.15.37:0/3776159279 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.088+0000 7f7201738640 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f720273a640 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82df940 0x5572b82d9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f7201f39640 1 -- 172.21.15.37:0/3776159279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82df940 msgr2=0x5572b82d9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f7201f39640 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82df940 0x5572b82d9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f7201f39640 1 -- 172.21.15.37:0/3776159279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 msgr2=0x5572b82f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f7201f39640 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f7201f39640 1 -- 172.21.15.37:0/3776159279 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572b82cc260 con 0x5572b82d4760 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f720273a640 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82df940 0x5572b82d9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f7201738640 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f7201f39640 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82d4760 0x5572b82f7200 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e9c0 tx=0x7f71f80bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142bc0fe38685b69 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f71e77fe640 1 -- 172.21.15.37:0/3776159279 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80a5000 con 0x5572b82d4760 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f71e77fe640 1 -- 172.21.15.37:0/3776159279 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f809c440 con 0x5572b82d4760 2024-09-19T06:54:43.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.089+0000 7f71e77fe640 1 -- 172.21.15.37:0/3776159279 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f80d2310 con 0x5572b82d4760 2024-09-19T06:54:43.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 -- 172.21.15.37:0/3776159279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82d4760 msgr2=0x5572b82f7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82d4760 0x5572b82f7200 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e9c0 tx=0x7f71f80bc040 comp rx=0 tx=0).stop 2024-09-19T06:54:43.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 -- 172.21.15.37:0/3776159279 shutdown_connections 2024-09-19T06:54:43.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82df940 0x5572b82d9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 --2- 172.21.15.37:0/3776159279 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82d4760 0x5572b82f7200 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 -- 172.21.15.37:0/3776159279 >> 172.21.15.37:0/3776159279 conn(0x5572b82d0cf0 msgr2=0x5572b82de810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 -- 172.21.15.37:0/3776159279 shutdown_connections 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 -- 172.21.15.37:0/3776159279 wait complete. 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 -- start start 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.090+0000 7f7206576740 1 Processor -- start 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7206576740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7206576740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82e7e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7206576740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7206576740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1ae120 con 0x5572b82df940 2024-09-19T06:54:43.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7206576740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1af590 con 0x5572b82d5080 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7206576740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc1ae7f0 con 0x5572b82d4760 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7206576740 10 client.0 _ll_register_callbacks cb 0x7f71ec009eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82e7e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82e7e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:58164/0 (socket says 172.21.15.37:58164) 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 -- 172.21.15.37:0/862737397 learned_addr learned my addr 172.21.15.37:0/862737397 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f720273a640 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201f39640 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 -- 172.21.15.37:0/862737397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 msgr2=0x5572b82e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 -- 172.21.15.37:0/862737397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 msgr2=0x5572b82e48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.091+0000 7f7201738640 1 -- 172.21.15.37:0/862737397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572b82d1980 con 0x5572b82d5080 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f7201738640 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82e7e20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009250 tx=0x7f71ec004f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a246276cce7c847 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:43.097 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f71e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f71e57fa640 1 -- 172.21.15.37:0/862737397 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ec03b060 con 0x5572b82d5080 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f71e57fa640 1 -- 172.21.15.37:0/862737397 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f71ec004880 con 0x5572b82d5080 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71f809c440 con 0x5572b82d5080 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f71e57fa640 1 -- 172.21.15.37:0/862737397 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f71ec028040 con 0x5572b82d5080 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f7206576740 10 client.28325 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5572b835a570 con 0x5572b82d5080 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f71e57fa640 1 -- 172.21.15.37:0/862737397 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f71f809c440 con 0x5572b82d5080 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.092+0000 7f71e57fa640 1 -- 172.21.15.37:0/862737397 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f71ec0053a0 con 0x5572b82d5080 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7206576740 10 client.28325 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7206576740 10 client.28325 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7206576740 20 client.28325 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7206576740 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b835ac40 0x5572b835d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7206576740 4 client.28325 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T06:54:43.098 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x5572b8241d20 con 0x5572b835ac40 2024-09-19T06:54:43.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7201f39640 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b835ac40 0x5572b835d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:43.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f7201f39640 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b835ac40 0x5572b835d060 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28fef186cf4a51ba server_cookie=1b92e532a3bfb7ae in_seq=0 out_seq=0 2024-09-19T06:54:43.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.093+0000 7f71e57fa640 10 client.28325 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f71e57fa640 1 -- 172.21.15.37:0/862737397 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x5572b8241d20 con 0x5572b835ac40 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f71e57fa640 10 client.28325 handle_command_reply: tid=0 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 client.28325 shutdown 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 msgr2=0x5572b82e7e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82e7e20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f71ec009250 tx=0x7f71ec004f00 comp rx=0 tx=0).stop 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 shutdown_connections 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5572b835ac40 0x5572b835d060 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5572b82df940 0x5572b82e48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5572b82d5080 0x5572b82e7e20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.184 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f71e57fa640 0 client.28325 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 --2- 172.21.15.37:0/862737397 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5572b82d4760 0x5572b82e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 >> 172.21.15.37:0/862737397 conn(0x5572b82d0cf0 msgr2=0x5572b82dc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 shutdown_connections 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 1 -- 172.21.15.37:0/862737397 wait complete. 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.182+0000 7f7206576740 20 client.28325 trim_cache size 0 max 16384 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 43, 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:54:43.185 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 42, 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "e6a2e2e8": { 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 42, 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 142.2, 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:54:43.186 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:54:43.187 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:54:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f71e80a33b0 msgr2=0x7f71e80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f71e80a33b0 0x7f71e80a5870 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f71f402f540 tx=0x7f71f402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71fc101de0 msgr2=0x7f71fc19a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71fc101de0 0x7f71fc19a630 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f71f800e9f0 tx=0x7f71f800eeb0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.188 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 shutdown_connections 2024-09-19T06:54:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f71e80a33b0 0x7f71e80a5870 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f71fc106f10 0x7f71fc19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71fc102790 0x7f71fc19ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 --2- 172.21.15.37:0/2443994190 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f71fc101de0 0x7f71fc19a630 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 >> 172.21.15.37:0/2443994190 conn(0x7f71fc0fd8c0 msgr2=0x7f71fc0feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 shutdown_connections 2024-09-19T06:54:43.189 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:43.183+0000 7f720419d640 1 -- 172.21.15.37:0/2443994190 wait complete. 2024-09-19T06:54:43.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.90 MB/sec execute 550 sec latency 75.563 ms 2024-09-19T06:54:44.119 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:43 smithi037 ceph-mon[31211]: pgmap v409: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 226 op/s 2024-09-19T06:54:44.121 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'e6a2e2e8', quiesce took 3.9 seconds. Will release after: 18.3 2024-09-19T06:54:44.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.89 MB/sec execute 551 sec latency 1042.779 ms 2024-09-19T06:54:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:43 smithi096 ceph-mon[35004]: pgmap v409: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 226 op/s 2024-09-19T06:54:44.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:43 smithi151 ceph-mon[35236]: pgmap v409: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.7 MiB/s wr, 226 op/s 2024-09-19T06:54:45.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.87 MB/sec execute 552 sec latency 2042.867 ms 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 -- 172.21.15.37:0/1949760530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c107b70 msgr2=0x7f241c107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 --2- 172.21.15.37:0/1949760530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c107b70 0x7f241c107f50 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7f240c009930 tx=0x7f240c02a420 comp rx=0 tx=0).stop 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 -- 172.21.15.37:0/1949760530 shutdown_connections 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 --2- 172.21.15.37:0/1949760530 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f241c102470 0x7f241c10e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 --2- 172.21.15.37:0/1949760530 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c101ad0 0x7f241c101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 --2- 172.21.15.37:0/1949760530 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c107b70 0x7f241c107f50 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 -- 172.21.15.37:0/1949760530 >> 172.21.15.37:0/1949760530 conn(0x7f241c0fd7d0 msgr2=0x7f241c0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 -- 172.21.15.37:0/1949760530 shutdown_connections 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 -- 172.21.15.37:0/1949760530 wait complete. 2024-09-19T06:54:45.830 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 Processor -- start 2024-09-19T06:54:45.831 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.828+0000 7f2422627640 1 -- start start 2024-09-19T06:54:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2422627640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c101ad0 0x7f241c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2422627640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c102470 0x7f241c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2422627640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f241c107b70 0x7f241c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2422627640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f241c111f20 con 0x7f241c101ad0 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2422627640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f241c111da0 con 0x7f241c102470 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2422627640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1120a0 con 0x7f241c107b70 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c101ad0 0x7f241c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c101ad0 0x7f241c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46598/0 (socket says 172.21.15.37:46598) 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 -- 172.21.15.37:0/2067163693 learned_addr learned my addr 172.21.15.37:0/2067163693 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241b7fe640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c102470 0x7f241c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2420bc4640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f241c107b70 0x7f241c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 -- 172.21.15.37:0/2067163693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f241c107b70 msgr2=0x7f241c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f241c107b70 0x7f241c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 -- 172.21.15.37:0/2067163693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c102470 msgr2=0x7f241c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c102470 0x7f241c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241bfff640 1 -- 172.21.15.37:0/2067163693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f241c1a6890 con 0x7f241c101ad0 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f241b7fe640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c102470 0x7f241c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.829+0000 7f2420bc4640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f241c107b70 0x7f241c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.830+0000 7f241bfff640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c101ad0 0x7f241c0744f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f240c009a50 tx=0x7f240c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.830+0000 7f24197fa640 1 -- 172.21.15.37:0/2067163693 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c03f030 con 0x7f241c101ad0 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.830+0000 7f24197fa640 1 -- 172.21.15.37:0/2067163693 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f240c003ce0 con 0x7f241c101ad0 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.830+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f241c1a6b20 con 0x7f241c101ad0 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.830+0000 7f24197fa640 1 -- 172.21.15.37:0/2067163693 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f240c035070 con 0x7f241c101ad0 2024-09-19T06:54:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.830+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f241c1a7010 con 0x7f241c101ad0 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.832+0000 7f24197fa640 1 -- 172.21.15.37:0/2067163693 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f240c042030 con 0x7f241c101ad0 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 Processor -- start 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 -- start start 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c988af0 0x55873c988ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c989410 0x55873c993770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c993cb0 0x55873c9960a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55873c8ed2f0 con 0x55873c988af0 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55873c895270 con 0x55873c993cb0 2024-09-19T06:54:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24253c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55873c8e92b0 con 0x55873c989410 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24197fa640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24040a3460 0x7f24040a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c988af0 0x55873c988ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c988af0 0x55873c988ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46604/0 (socket says 172.21.15.37:46604) 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 -- 172.21.15.37:0/4003073435 learned_addr learned my addr 172.21.15.37:0/4003073435 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24040a3460 0x7f24040a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f24197fa640 1 -- 172.21.15.37:0/2067163693 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f240c0f0350 con 0x7f241c101ad0 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f241bfff640 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c989410 0x55873c993770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f241b7fe640 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c993cb0 0x55873c9960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 -- 172.21.15.37:0/4003073435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c989410 msgr2=0x55873c993770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c989410 0x55873c993770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 -- 172.21.15.37:0/4003073435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c993cb0 msgr2=0x55873c9960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c993cb0 0x55873c9960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.833+0000 7f2420bc4640 1 -- 172.21.15.37:0/4003073435 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873c894340 con 0x55873c988af0 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.834+0000 7f2420bc4640 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c988af0 0x55873c988ed0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2414005fd0 tx=0x7f241400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e005184f5ced2bc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.834+0000 7f2420bc4640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24040a3460 0x7f24040a5920 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f241400fc30 tx=0x7f241401a000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.834+0000 7f24027fc640 1 -- 172.21.15.37:0/4003073435 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414020070 con 0x55873c988af0 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.834+0000 7f24027fc640 1 -- 172.21.15.37:0/4003073435 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241400abc0 con 0x55873c988af0 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.834+0000 7f24027fc640 1 -- 172.21.15.37:0/4003073435 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140115e0 con 0x55873c988af0 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 -- 172.21.15.37:0/4003073435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c988af0 msgr2=0x55873c988ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c988af0 0x55873c988ed0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2414005fd0 tx=0x7f241400f730 comp rx=0 tx=0).stop 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 -- 172.21.15.37:0/4003073435 shutdown_connections 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c993cb0 0x55873c9960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c989410 0x55873c993770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 --2- 172.21.15.37:0/4003073435 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c988af0 0x55873c988ed0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 -- 172.21.15.37:0/4003073435 >> 172.21.15.37:0/4003073435 conn(0x55873c985060 msgr2=0x55873c985440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 -- 172.21.15.37:0/4003073435 shutdown_connections 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 -- 172.21.15.37:0/4003073435 wait complete. 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 -- start start 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.835+0000 7f24253c2740 1 Processor -- start 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c990c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c991140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c991680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55873c996880 con 0x55873c993cb0 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55873c876290 con 0x55873c989410 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55873c996a00 con 0x55873c988af0 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 10 client.0 _ll_register_callbacks cb 0x55873c8a7140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c991680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c991680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46610/0 (socket says 172.21.15.37:46610) 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 -- 172.21.15.37:0/3714686953 learned_addr learned my addr 172.21.15.37:0/3714686953 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f241b7fe640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c991140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f241bfff640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c990c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 -- 172.21.15.37:0/3714686953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 msgr2=0x55873c990c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c990c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 -- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 msgr2=0x55873c991140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c991140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 -- 172.21.15.37:0/3714686953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873c600210 con 0x55873c993cb0 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f241b7fe640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c991140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f2420bc4640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c991680 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2414019cc0 tx=0x7f2414019cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7193e348020f478f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f23fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3714686953 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414020050 con 0x55873c993cb0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3714686953 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f241401c640 con 0x55873c993cb0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.836+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f241400abc0 con 0x55873c993cb0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3714686953 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f241401c9c0 con 0x55873c993cb0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 10 client.19035 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55873c6cc620 con 0x55873c993cb0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3714686953 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241400abc0 con 0x55873c993cb0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3714686953 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f241401b070 con 0x55873c993cb0 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 10 client.19035 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 10 client.19035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 20 client.19035 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873c9a3c70 0x55873c9a6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 4 client.19035 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.837+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55873c8faa20 con 0x55873c9a3c70 2024-09-19T06:54:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.838+0000 7f241bfff640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873c9a3c70 0x55873c9a6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.838+0000 7f241bfff640 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873c9a3c70 0x55873c9a6060 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c811f3c704fadf0 server_cookie=4e291fcba329aa18 in_seq=0 out_seq=0 2024-09-19T06:54:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.838+0000 7f23fb7fe640 10 client.19035 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3714686953 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55873c8faa20 con 0x55873c9a3c70 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f23fb7fe640 10 client.19035 handle_command_reply: tid=0 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 client.19035 shutdown 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 msgr2=0x55873c991680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c991680 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f2414019cc0 tx=0x7f2414019cf0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 shutdown_connections 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873c9a3c70 0x55873c9a6060 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c991680 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c991140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 --2- 172.21.15.37:0/3714686953 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c990c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f23fb7fe640 0 client.19035 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 >> 172.21.15.37:0/3714686953 conn(0x55873c985060 msgr2=0x55873c98ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 shutdown_connections 2024-09-19T06:54:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 1 -- 172.21.15.37:0/3714686953 wait complete. 2024-09-19T06:54:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.839+0000 7f24253c2740 20 client.19035 trim_cache size 0 max 16384 2024-09-19T06:54:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 Processor -- start 2024-09-19T06:54:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 -- start start 2024-09-19T06:54:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c98fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c9914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c9ab860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1b22e0 con 0x55873c993cb0 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1afbb0 con 0x55873c989410 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f24253c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1ada00 con 0x55873c988af0 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f2420bc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c9ab860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f2420bc4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c9ab860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46616/0 (socket says 172.21.15.37:46616) 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.854+0000 7f2420bc4640 1 -- 172.21.15.37:0/875382287 learned_addr learned my addr 172.21.15.37:0/875382287 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:45.861 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f241bfff640 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c98fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f241b7fe640 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c9914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2420bc4640 1 -- 172.21.15.37:0/875382287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 msgr2=0x55873c98fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2420bc4640 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c98fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2420bc4640 1 -- 172.21.15.37:0/875382287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 msgr2=0x55873c9914b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2420bc4640 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c9914b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2420bc4640 1 -- 172.21.15.37:0/875382287 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55873c986110 con 0x55873c993cb0 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2420bc4640 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c9ab860 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2414012480 tx=0x7f241400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb471f4ac7d2632 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2401ffb640 1 -- 172.21.15.37:0/875382287 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2414020070 con 0x55873c993cb0 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2401ffb640 1 -- 172.21.15.37:0/875382287 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241400abc0 con 0x55873c993cb0 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.855+0000 7f2401ffb640 1 -- 172.21.15.37:0/875382287 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24140256b0 con 0x55873c993cb0 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 -- 172.21.15.37:0/875382287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 msgr2=0x55873c9ab860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c9ab860 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f2414012480 tx=0x7f241400f730 comp rx=0 tx=0).stop 2024-09-19T06:54:45.862 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 -- 172.21.15.37:0/875382287 shutdown_connections 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c993cb0 0x55873c9ab860 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c989410 0x55873c9914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 --2- 172.21.15.37:0/875382287 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c988af0 0x55873c98fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 -- 172.21.15.37:0/875382287 >> 172.21.15.37:0/875382287 conn(0x55873c985060 msgr2=0x55873c992c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 -- 172.21.15.37:0/875382287 shutdown_connections 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 -- 172.21.15.37:0/875382287 wait complete. 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 -- start start 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.856+0000 7f24253c2740 1 Processor -- start 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c988af0 0x55873c997230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c989410 0x55873c997770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c993cb0 0x55873ca0c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1ae070 con 0x55873c989410 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1af4e0 con 0x55873c988af0 2024-09-19T06:54:45.863 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f241c1ae740 con 0x55873c993cb0 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 10 client.0 _ll_register_callbacks cb 0x7f2414012b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c989410 0x55873c997770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c989410 0x55873c997770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:46620/0 (socket says 172.21.15.37:46620) 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 -- 172.21.15.37:0/3127609845 learned_addr learned my addr 172.21.15.37:0/3127609845 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241bfff640 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c988af0 0x55873c997230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 -- 172.21.15.37:0/3127609845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c993cb0 msgr2=0x55873ca0c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c993cb0 0x55873ca0c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 -- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c988af0 msgr2=0x55873c997230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c988af0 0x55873c997230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 -- 172.21.15.37:0/3127609845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55873ca0cdb0 con 0x55873c989410 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f241b7fe640 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c989410 0x55873c997770 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f241c107070 tx=0x7f2410012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=544f306464a8c7a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:45.864 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f23fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3127609845 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2410020060 con 0x55873c989410 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f241400abc0 con 0x55873c989410 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3127609845 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f241000fb80 con 0x55873c989410 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.857+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3127609845 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f241000fd80 con 0x55873c989410 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 10 client.19041 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55873ca0e120 con 0x55873c989410 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3127609845 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f241400abc0 con 0x55873c989410 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3127609845 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f2410023610 con 0x55873c989410 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 10 client.19041 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 10 client.19041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 20 client.19041 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873ca0e900 0x55873ca10d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 4 client.19041 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:45.865 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55873c8f4d70 con 0x55873ca0e900 2024-09-19T06:54:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.858+0000 7f241bfff640 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873ca0e900 0x55873ca10d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.859+0000 7f23fb7fe640 10 client.19041 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:45.866 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.859+0000 7f241bfff640 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873ca0e900 0x55873ca10d20 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8108013f64f5e91b server_cookie=a92f875e3a04f5e7 in_seq=0 out_seq=0 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.992+0000 7f23fb7fe640 1 -- 172.21.15.37:0/3127609845 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55873c8f4d70 con 0x55873ca0e900 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.992+0000 7f23fb7fe640 10 client.19041 handle_command_reply: tid=0 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.992+0000 7f24253c2740 1 client.19041 shutdown 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c989410 msgr2=0x55873c997770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c989410 0x55873c997770 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f241c107070 tx=0x7f2410012ca0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 shutdown_connections 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55873ca0e900 0x55873ca10d20 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55873c993cb0 0x55873ca0c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f23fb7fe640 0 client.19041 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:45.995 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55873c989410 0x55873c997770 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 --2- 172.21.15.37:0/3127609845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55873c988af0 0x55873c997230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 >> 172.21.15.37:0/3127609845 conn(0x55873c985060 msgr2=0x55873c98a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 shutdown_connections 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 1 -- 172.21.15.37:0/3127609845 wait complete. 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.993+0000 7f24253c2740 20 client.19041 trim_cache size 0 max 16384 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:54:45.996 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24040a3460 msgr2=0x7f24040a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24040a3460 0x7f24040a5920 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f241400fc30 tx=0x7f241401a000 comp rx=0 tx=0).stop 2024-09-19T06:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c101ad0 msgr2=0x7f241c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c101ad0 0x7f241c0744f0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f240c009a50 tx=0x7f240c004270 comp rx=0 tx=0).stop 2024-09-19T06:54:45.997 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 shutdown_connections 2024-09-19T06:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f241c107b70 0x7f241c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f241c102470 0x7f241c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f24040a3460 0x7f24040a5920 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 --2- 172.21.15.37:0/2067163693 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f241c101ad0 0x7f241c0744f0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 >> 172.21.15.37:0/2067163693 conn(0x7f241c0fd7d0 msgr2=0x7f241c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 shutdown_connections 2024-09-19T06:54:45.998 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:45.994+0000 7f2422627640 1 -- 172.21.15.37:0/2067163693 wait complete. 2024-09-19T06:54:46.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.86 MB/sec execute 553 sec latency 3042.958 ms 2024-09-19T06:54:46.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:45 smithi096 ceph-mon[35004]: pgmap v410: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-09-19T06:54:46.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:45 smithi037 ceph-mon[31211]: pgmap v410: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-09-19T06:54:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:45 smithi151 ceph-mon[35236]: pgmap v410: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.4 MiB/s wr, 206 op/s 2024-09-19T06:54:47.235 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.84 MB/sec execute 554 sec latency 4043.001 ms 2024-09-19T06:54:48.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.83 MB/sec execute 555 sec latency 5043.086 ms 2024-09-19T06:54:48.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:47 smithi096 ceph-mon[35004]: pgmap v411: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 188 op/s 2024-09-19T06:54:48.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:47 smithi037 ceph-mon[31211]: pgmap v411: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 188 op/s 2024-09-19T06:54:48.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:47 smithi151 ceph-mon[35236]: pgmap v411: 97 pgs: 97 active+clean; 87 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.1 MiB/s wr, 188 op/s 2024-09-19T06:54:49.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.82 MB/sec execute 556 sec latency 6043.187 ms 2024-09-19T06:54:50.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.80 MB/sec execute 557 sec latency 7043.296 ms 2024-09-19T06:54:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:49 smithi096 ceph-mon[35004]: pgmap v412: 97 pgs: 97 active+clean; 74 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-09-19T06:54:50.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:49 smithi037 ceph-mon[31211]: pgmap v412: 97 pgs: 97 active+clean; 74 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-09-19T06:54:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:49 smithi151 ceph-mon[35236]: pgmap v412: 97 pgs: 97 active+clean; 74 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 300 op/s 2024-09-19T06:54:51.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.79 MB/sec execute 558 sec latency 8043.349 ms 2024-09-19T06:54:52.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.77 MB/sec execute 559 sec latency 9043.460 ms 2024-09-19T06:54:52.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:51 smithi096 ceph-mon[35004]: pgmap v413: 97 pgs: 97 active+clean; 74 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-09-19T06:54:52.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:51 smithi037 ceph-mon[31211]: pgmap v413: 97 pgs: 97 active+clean; 74 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-09-19T06:54:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:51 smithi151 ceph-mon[35236]: pgmap v413: 97 pgs: 97 active+clean; 74 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.6 MiB/s wr, 207 op/s 2024-09-19T06:54:53.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.76 MB/sec execute 560 sec latency 10043.572 ms 2024-09-19T06:54:54.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.75 MB/sec execute 561 sec latency 11043.706 ms 2024-09-19T06:54:54.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:53 smithi096 ceph-mon[35004]: pgmap v414: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-09-19T06:54:54.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:53 smithi037 ceph-mon[31211]: pgmap v414: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-09-19T06:54:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:53 smithi151 ceph-mon[35236]: pgmap v414: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.3 MiB/s wr, 237 op/s 2024-09-19T06:54:55.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.73 MB/sec execute 562 sec latency 12043.822 ms 2024-09-19T06:54:56.236 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.72 MB/sec execute 563 sec latency 13043.886 ms 2024-09-19T06:54:56.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:55 smithi096 ceph-mon[35004]: pgmap v415: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T06:54:56.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:55 smithi037 ceph-mon[31211]: pgmap v415: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T06:54:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:55 smithi151 ceph-mon[35236]: pgmap v415: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T06:54:56.496 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:54:56.757 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:57.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.70 MB/sec execute 564 sec latency 14043.939 ms 2024-09-19T06:54:57.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 -- 172.21.15.37:0/1353835661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc101410 msgr2=0x7f27bc10d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 --2- 172.21.15.37:0/1353835661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc101410 0x7f27bc10d940 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f27b4009cd0 tx=0x7f27b402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 -- 172.21.15.37:0/1353835661 shutdown_connections 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 --2- 172.21.15.37:0/1353835661 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc101410 0x7f27bc10d940 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 --2- 172.21.15.37:0/1353835661 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27bc100a70 0x7f27bc100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 --2- 172.21.15.37:0/1353835661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27bc106a70 0x7f27bc106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 -- 172.21.15.37:0/1353835661 >> 172.21.15.37:0/1353835661 conn(0x7f27bc0fc790 msgr2=0x7f27bc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 -- 172.21.15.37:0/1353835661 shutdown_connections 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.237+0000 7f27c0987640 1 -- 172.21.15.37:0/1353835661 wait complete. 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 Processor -- start 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 -- start start 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc100a70 0x7f27bc19a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27bc101410 0x7f27bc19aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27bc106a70 0x7f27bc19ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc110e90 con 0x7f27bc100a70 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc110d10 con 0x7f27bc101410 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27c0987640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f27bc111010 con 0x7f27bc106a70 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc100a70 0x7f27bc19a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc100a70 0x7f27bc19a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50710/0 (socket says 172.21.15.37:50710) 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27ba59c640 1 -- 172.21.15.37:0/1111048430 learned_addr learned my addr 172.21.15.37:0/1111048430 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27bc101410 0x7f27bc19aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.238+0000 7f27bad9d640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27bc106a70 0x7f27bc19ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ba59c640 1 -- 172.21.15.37:0/1111048430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27bc106a70 msgr2=0x7f27bc19ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ba59c640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27bc106a70 0x7f27bc19ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ba59c640 1 -- 172.21.15.37:0/1111048430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27bc101410 msgr2=0x7f27bc19aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ba59c640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27bc101410 0x7f27bc19aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ba59c640 1 -- 172.21.15.37:0/1111048430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27bc19f550 con 0x7f27bc100a70 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27bad9d640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27bc106a70 0x7f27bc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ba59c640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc100a70 0x7f27bc19a5a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c920 tx=0x7f27ac00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ab7fe640 1 -- 172.21.15.37:0/1111048430 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac007bb0 con 0x7f27bc100a70 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27bc19f840 con 0x7f27bc100a70 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27bc1a7120 con 0x7f27bc100a70 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.239+0000 7f27ab7fe640 1 -- 172.21.15.37:0/1111048430 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac007d50 con 0x7f27bc100a70 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.240+0000 7f27ab7fe640 1 -- 172.21.15.37:0/1111048430 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac00f450 con 0x7f27bc100a70 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.241+0000 7f27ab7fe640 1 -- 172.21.15.37:0/1111048430 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f27ac004ec0 con 0x7f27bc100a70 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 Processor -- start 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 -- start start 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec21a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.253 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec225140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec225680 0x5563ec227a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27ab7fe640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f27a40a33b0 0x7f27a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5563ec1802e0 con 0x5563ec225680 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5563ec127ce0 con 0x5563ec21adc0 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27c2d60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5563ec17c2a0 con 0x5563ec21a4a0 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27b9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec225680 0x5563ec227a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.242+0000 7f27b9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec225680 0x5563ec227a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50718/0 (socket says 172.21.15.37:50718) 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27b9d9b640 1 -- 172.21.15.37:0/647981045 learned_addr learned my addr 172.21.15.37:0/647981045 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27bad9d640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f27a40a33b0 0x7f27a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ba59c640 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec225140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27bad9d640 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec21a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ab7fe640 1 -- 172.21.15.37:0/1111048430 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f27ac0cfee0 con 0x7f27bc100a70 2024-09-19T06:54:57.254 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ba59c640 1 -- 172.21.15.37:0/647981045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 msgr2=0x5563ec21a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ba59c640 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec21a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ba59c640 1 -- 172.21.15.37:0/647981045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec225680 msgr2=0x5563ec227a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ba59c640 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec225680 0x5563ec227a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ba59c640 1 -- 172.21.15.37:0/647981045 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ec10c440 con 0x5563ec21adc0 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec225680 0x5563ec227a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27bad9d640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f27a40a33b0 0x7f27a40a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f27b4033040 tx=0x7f27b4009360 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27ba59c640 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec225140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0cd5d0 tx=0x7f27ac09c030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8464dc8de205f98 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.243+0000 7f27a8ff9640 1 -- 172.21.15.37:0/647981045 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0b5070 con 0x5563ec21adc0 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27a8ff9640 1 -- 172.21.15.37:0/647981045 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac09c860 con 0x5563ec21adc0 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27a8ff9640 1 -- 172.21.15.37:0/647981045 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac0046c0 con 0x5563ec21adc0 2024-09-19T06:54:57.255 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 -- 172.21.15.37:0/647981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 msgr2=0x5563ec225140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec225140 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f27ac0cd5d0 tx=0x7f27ac09c030 comp rx=0 tx=0).stop 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 -- 172.21.15.37:0/647981045 shutdown_connections 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec225680 0x5563ec227a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec225140 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 --2- 172.21.15.37:0/647981045 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec21a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 -- 172.21.15.37:0/647981045 >> 172.21.15.37:0/647981045 conn(0x5563ec216a30 msgr2=0x5563ec216e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 -- 172.21.15.37:0/647981045 shutdown_connections 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.244+0000 7f27c2d60740 1 -- 172.21.15.37:0/647981045 wait complete. 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 -- start start 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 Processor -- start 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec224fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec222270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec225680 0x5563ec2227b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5563ec228250 con 0x5563ec21adc0 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5563ec13b050 con 0x5563ec225680 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5563ec2283d0 con 0x5563ec21a4a0 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27c2d60740 10 client.0 _ll_register_callbacks cb 0x5563ec17b160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27b9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec222270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27b9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec222270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50720/0 (socket says 172.21.15.37:50720) 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2962799223 learned_addr learned my addr 172.21.15.37:0/2962799223 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27bad9d640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec225680 0x5563ec2227b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27ba59c640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec224fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2962799223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 msgr2=0x5563ec224fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec224fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.245+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec225680 msgr2=0x5563ec2227b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec225680 0x5563ec2227b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2962799223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ebef13c0 con 0x5563ec21adc0 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27ba59c640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec224fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27bad9d640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec225680 0x5563ec2227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec222270 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f27b0011540 tx=0x7f27b0011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab93e7be485d07f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27a1ffb640 1 -- 172.21.15.37:0/2962799223 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b000f6d0 con 0x5563ec21adc0 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27a1ffb640 1 -- 172.21.15.37:0/2962799223 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f27b000f870 con 0x5563ec21adc0 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27ac09c860 con 0x5563ec21adc0 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27a1ffb640 1 -- 172.21.15.37:0/2962799223 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f27b000fa70 con 0x5563ec21adc0 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27c2d60740 10 client.19053 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:57.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563ec233cc0 con 0x5563ec21adc0 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27a1ffb640 1 -- 172.21.15.37:0/2962799223 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27ac09c860 con 0x5563ec21adc0 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.246+0000 7f27a1ffb640 1 -- 172.21.15.37:0/2962799223 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f27b001d400 con 0x5563ec21adc0 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27c2d60740 10 client.19053 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27c2d60740 10 client.19053 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27c2d60740 20 client.19053 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec235250 0x5563ec237670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27c2d60740 4 client.19053 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563ec18d850 con 0x5563ec235250 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27ba59c640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec235250 0x5563ec237670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27ba59c640 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec235250 0x5563ec237670 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40ca956565032740 server_cookie=df2405ffef7f3160 in_seq=0 out_seq=0 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.247+0000 7f27a1ffb640 10 client.19053 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27a1ffb640 1 -- 172.21.15.37:0/2962799223 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5563ec18d850 con 0x5563ec235250 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27a1ffb640 10 client.19053 handle_command_reply: tid=0 2024-09-19T06:54:57.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 client.19053 shutdown 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 msgr2=0x5563ec222270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec222270 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f27b0011540 tx=0x7f27b0011a00 comp rx=0 tx=0).stop 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 shutdown_connections 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec235250 0x5563ec237670 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec225680 0x5563ec2227b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec222270 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2962799223 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec21a4a0 0x5563ec224fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27a1ffb640 0 client.19053 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 >> 172.21.15.37:0/2962799223 conn(0x5563ec216a30 msgr2=0x5563ec21c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 shutdown_connections 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 1 -- 172.21.15.37:0/2962799223 wait complete. 2024-09-19T06:54:57.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.248+0000 7f27c2d60740 20 client.19053 trim_cache size 0 max 16384 2024-09-19T06:54:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.265+0000 7f27c2d60740 1 Processor -- start 2024-09-19T06:54:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27c2d60740 1 -- start start 2024-09-19T06:54:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21a4a0 0x5563ec2314e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec23cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec23d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27c2d60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f279c0110c0 con 0x5563ec21adc0 2024-09-19T06:54:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27c2d60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f279c00e990 con 0x5563ec21a4a0 2024-09-19T06:54:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27c2d60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f279c00c7e0 con 0x5563ec225680 2024-09-19T06:54:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec23cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec23cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50734/0 (socket says 172.21.15.37:50734) 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2539293239 learned_addr learned my addr 172.21.15.37:0/2539293239 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27bad9d640 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec23d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27ba59c640 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21a4a0 0x5563ec2314e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2539293239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 msgr2=0x5563ec23d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec23d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2539293239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21a4a0 msgr2=0x5563ec2314e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21a4a0 0x5563ec2314e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27b9d9b640 1 -- 172.21.15.37:0/2539293239 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563ec227dd0 con 0x5563ec21adc0 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27ba59c640 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21a4a0 0x5563ec2314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.266+0000 7f27bad9d640 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec23d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.267+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec23cf80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f27b001da10 tx=0x7f27b0011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f732eeeb04ff20a1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.267+0000 7f27a3fff640 1 -- 172.21.15.37:0/2539293239 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b002a030 con 0x5563ec21adc0 2024-09-19T06:54:57.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.267+0000 7f27a3fff640 1 -- 172.21.15.37:0/2539293239 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b000f100 con 0x5563ec21adc0 2024-09-19T06:54:57.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.267+0000 7f27a3fff640 1 -- 172.21.15.37:0/2539293239 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27b0014440 con 0x5563ec21adc0 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 -- 172.21.15.37:0/2539293239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 msgr2=0x5563ec23cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec23cf80 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f27b001da10 tx=0x7f27b0011f80 comp rx=0 tx=0).stop 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 -- 172.21.15.37:0/2539293239 shutdown_connections 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec23d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21adc0 0x5563ec23cf80 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 --2- 172.21.15.37:0/2539293239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21a4a0 0x5563ec2314e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 -- 172.21.15.37:0/2539293239 >> 172.21.15.37:0/2539293239 conn(0x5563ec216a30 msgr2=0x5563ec21d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 -- 172.21.15.37:0/2539293239 shutdown_connections 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 -- 172.21.15.37:0/2539293239 wait complete. 2024-09-19T06:54:57.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 -- start start 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 Processor -- start 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21a4a0 0x5563ec22dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec2290a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.268+0000 7f27c2d60740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec2295e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27c2d60740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f279c00ce50 con 0x5563ec21a4a0 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27c2d60740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f279c00e2c0 con 0x5563ec21adc0 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27c2d60740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f279c00d520 con 0x5563ec225680 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21a4a0 0x5563ec22dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27c2d60740 10 client.0 _ll_register_callbacks cb 0x7f27b0009ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21a4a0 0x5563ec22dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50744/0 (socket says 172.21.15.37:50744) 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 -- 172.21.15.37:0/3678698026 learned_addr learned my addr 172.21.15.37:0/3678698026 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec2290a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 -- 172.21.15.37:0/3678698026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 msgr2=0x5563ec2295e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec2295e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 -- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 msgr2=0x5563ec2290a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec2290a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 -- 172.21.15.37:0/3678698026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563ec229e10 con 0x5563ec21a4a0 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec2290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27ba59c640 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21a4a0 0x5563ec22dcd0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f27ac09c0f0 tx=0x7f27ac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74183760735822b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27a1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27a1ffb640 1 -- 172.21.15.37:0/3678698026 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ac07f060 con 0x5563ec21a4a0 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27a1ffb640 1 -- 172.21.15.37:0/3678698026 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f27ac0046c0 con 0x5563ec21a4a0 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27b000f100 con 0x5563ec21a4a0 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27a1ffb640 1 -- 172.21.15.37:0/3678698026 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f27ac0b5040 con 0x5563ec21a4a0 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27c2d60740 10 client.19059 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:57.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.269+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5563ec29fea0 con 0x5563ec21a4a0 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27a1ffb640 1 -- 172.21.15.37:0/3678698026 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27b000f100 con 0x5563ec21a4a0 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27a1ffb640 1 -- 172.21.15.37:0/3678698026 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f27ac0850c0 con 0x5563ec21a4a0 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27c2d60740 10 client.19059 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27c2d60740 10 client.19059 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27c2d60740 20 client.19059 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27c2d60740 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec2a0130 0x5563ec2a2520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27c2d60740 4 client.19059 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5563ec187d60 con 0x5563ec2a0130 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.270+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec2a0130 0x5563ec2a2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27a1ffb640 10 client.19059 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27b9d9b640 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec2a0130 0x5563ec2a2520 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb6a8babb24937bf server_cookie=efc2ad8ded199d1e in_seq=0 out_seq=0 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27a1ffb640 1 -- 172.21.15.37:0/3678698026 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5563ec187d60 con 0x5563ec2a0130 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27a1ffb640 10 client.19059 handle_command_reply: tid=0 2024-09-19T06:54:57.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 client.19059 shutdown 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21a4a0 msgr2=0x5563ec22dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21a4a0 0x5563ec22dcd0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f27ac09c0f0 tx=0x7f27ac000bc0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 shutdown_connections 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5563ec2a0130 0x5563ec2a2520 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5563ec225680 0x5563ec2295e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5563ec21adc0 0x5563ec2290a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 --2- 172.21.15.37:0/3678698026 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5563ec21a4a0 0x5563ec22dcd0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 >> 172.21.15.37:0/3678698026 conn(0x5563ec216a30 msgr2=0x5563ec21cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27a1ffb640 0 client.19059 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 shutdown_connections 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 1 -- 172.21.15.37:0/3678698026 wait complete. 2024-09-19T06:54:57.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.271+0000 7f27c2d60740 20 client.19059 trim_cache size 0 max 16384 2024-09-19T06:54:57.279 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:54:57.279 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:54:57.279 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:54:57.279 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f27a40a33b0 msgr2=0x7f27a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f27a40a33b0 0x7f27a40a5870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f27b4033040 tx=0x7f27b4009360 comp rx=0 tx=0).stop 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc100a70 msgr2=0x7f27bc19a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc100a70 0x7f27bc19a5a0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f27ac00c920 tx=0x7f27ac00cd20 comp rx=0 tx=0).stop 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 shutdown_connections 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f27bc106a70 0x7f27bc19ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27bc101410 0x7f27bc19aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f27a40a33b0 0x7f27a40a5870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 --2- 172.21.15.37:0/1111048430 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f27bc100a70 0x7f27bc19a5a0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 >> 172.21.15.37:0/1111048430 conn(0x7f27bc0fc790 msgr2=0x7f27bc10c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 shutdown_connections 2024-09-19T06:54:57.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:57.272+0000 7f27c0987640 1 -- 172.21.15.37:0/1111048430 wait complete. 2024-09-19T06:54:58.011 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:54:58.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:54:58.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.69 MB/sec execute 565 sec latency 15044.286 ms 2024-09-19T06:54:58.269 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:54:58.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:57 smithi096 ceph-mon[35004]: pgmap v416: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T06:54:58.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:57 smithi037 ceph-mon[31211]: pgmap v416: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T06:54:58.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:57 smithi151 ceph-mon[35236]: pgmap v416: 97 pgs: 97 active+clean; 73 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 229 KiB/s rd, 1.6 MiB/s wr, 141 op/s 2024-09-19T06:54:58.752 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 -- 172.21.15.37:0/4258108821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 msgr2=0x7fe548100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 --2- 172.21.15.37:0/4258108821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe548100cb0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7fe544009900 tx=0x7fe54402a410 comp rx=0 tx=0).stop 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 -- 172.21.15.37:0/4258108821 shutdown_connections 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 --2- 172.21.15.37:0/4258108821 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54810e480 0x7fe548110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 --2- 172.21.15.37:0/4258108821 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe548101280 0x7fe54810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 --2- 172.21.15.37:0/4258108821 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe548100cb0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 -- 172.21.15.37:0/4258108821 >> 172.21.15.37:0/4258108821 conn(0x7fe5480fc7a0 msgr2=0x7fe5480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 -- 172.21.15.37:0/4258108821 shutdown_connections 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.733+0000 7fe54f360640 1 -- 172.21.15.37:0/4258108821 wait complete. 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 Processor -- start 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 -- start start 2024-09-19T06:54:58.753 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe54819a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe548101280 0x7fe54819ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54810e480 0x7fe54819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe548114200 con 0x7fe5481008d0 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe548114080 con 0x7fe54810e480 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54f360640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe548114380 con 0x7fe548101280 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54d0fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe54819a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54d0fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe54819a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50758/0 (socket says 172.21.15.37:50758) 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54d0fc640 1 -- 172.21.15.37:0/3803949722 learned_addr learned my addr 172.21.15.37:0/3803949722 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.734+0000 7fe54d8fd640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54810e480 0x7fe54819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54c8fb640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe548101280 0x7fe54819ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54d0fc640 1 -- 172.21.15.37:0/3803949722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe548101280 msgr2=0x7fe54819ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54d0fc640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe548101280 0x7fe54819ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.754 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54d0fc640 1 -- 172.21.15.37:0/3803949722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54810e480 msgr2=0x7fe54819eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54d0fc640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54810e480 0x7fe54819eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54d0fc640 1 -- 172.21.15.37:0/3803949722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe54819f6d0 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54d0fc640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe54819a720 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe544006010 tx=0x7fe5440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe53e7fc640 1 -- 172.21.15.37:0/3803949722 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54403f030 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe53e7fc640 1 -- 172.21.15.37:0/3803949722 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe544003ce0 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54819f960 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe53e7fc640 1 -- 172.21.15.37:0/3803949722 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe544035070 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.735+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5481a72a0 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.736+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe548100cb0 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.737+0000 7fe53e7fc640 1 -- 172.21.15.37:0/3803949722 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe544042030 con 0x7fe5481008d0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.738+0000 7fe53e7fc640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5340a3430 0x7fe5340a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.738+0000 7fe54c8fb640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5340a3430 0x7fe5340a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:58.755 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.738+0000 7fe53e7fc640 1 -- 172.21.15.37:0/3803949722 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7fe5440f02b0 con 0x7fe5481008d0 2024-09-19T06:54:58.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.739+0000 7fe54c8fb640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5340a3430 0x7fe5340a58f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe548102850 tx=0x7fe538008040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:58.756 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.743+0000 7fe53e7fc640 1 -- 172.21.15.37:0/3803949722 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fe5440bcd50 con 0x7fe5481008d0 2024-09-19T06:54:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.962+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe548109610 con 0x7fe5481008d0 2024-09-19T06:54:58.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.963+0000 7fe53e7fc640 1 -- 172.21.15.37:0/3803949722 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11903 (secure 0 0 0) 0x7fe5440bc650 con 0x7fe5481008d0 2024-09-19T06:54:58.965 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:54:58.965 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-19T06:54:33:862495+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:54:33.862475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27958":{"gid":27958,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/1806889964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1806889964},{"type":"v1","addr":"172.21.15.151:6839","nonce":1806889964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:54:58.965 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-19T06:54:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5340a3430 msgr2=0x7fe5340a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5340a3430 0x7fe5340a58f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe548102850 tx=0x7fe538008040 comp rx=0 tx=0).stop 2024-09-19T06:54:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 msgr2=0x7fe54819a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe54819a720 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fe544006010 tx=0x7fe5440042f0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 shutdown_connections 2024-09-19T06:54:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe54810e480 0x7fe54819eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe548101280 0x7fe54819ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe5340a3430 0x7fe5340a58f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 --2- 172.21.15.37:0/3803949722 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe5481008d0 0x7fe54819a720 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 >> 172.21.15.37:0/3803949722 conn(0x7fe5480fc7a0 msgr2=0x7fe5480feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 shutdown_connections 2024-09-19T06:54:58.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:54:58.965+0000 7fe54f360640 1 -- 172.21.15.37:0/3803949722 wait complete. 2024-09-19T06:54:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 -- 172.21.15.37:0/890383238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c101350 msgr2=0x7f435c10d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 --2- 172.21.15.37:0/890383238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c101350 0x7f435c10d880 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f4354009cd0 tx=0x7f435402a8e0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 -- 172.21.15.37:0/890383238 shutdown_connections 2024-09-19T06:54:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 --2- 172.21.15.37:0/890383238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c101350 0x7f435c10d880 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 --2- 172.21.15.37:0/890383238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f435c1009b0 0x7f435c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 --2- 172.21.15.37:0/890383238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1069c0 0x7f435c106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 -- 172.21.15.37:0/890383238 >> 172.21.15.37:0/890383238 conn(0x7f435c077990 msgr2=0x7f435c0fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 -- 172.21.15.37:0/890383238 shutdown_connections 2024-09-19T06:54:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 -- 172.21.15.37:0/890383238 wait complete. 2024-09-19T06:54:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.086+0000 7f436254b640 1 Processor -- start 2024-09-19T06:54:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f436254b640 1 -- start start 2024-09-19T06:54:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f436254b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1009b0 0x7f435c1991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f436254b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f435c101350 0x7f435c1996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f436254b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c1069c0 0x7f435c1a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f436254b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f435c110e00 con 0x7f435c1069c0 2024-09-19T06:54:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f436254b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f435c110c80 con 0x7f435c1009b0 2024-09-19T06:54:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f436254b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f435c110f80 con 0x7f435c101350 2024-09-19T06:54:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c1069c0 0x7f435c1a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c1069c0 0x7f435c1a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50774/0 (socket says 172.21.15.37:50774) 2024-09-19T06:54:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 -- 172.21.15.37:0/1916446671 learned_addr learned my addr 172.21.15.37:0/1916446671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f435bfff640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1009b0 0x7f435c1991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f435b7fe640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f435c101350 0x7f435c1996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 -- 172.21.15.37:0/1916446671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f435c101350 msgr2=0x7f435c1996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f435c101350 0x7f435c1996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 -- 172.21.15.37:0/1916446671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1009b0 msgr2=0x7f435c1991a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1009b0 0x7f435c1991a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f4360ae8640 1 -- 172.21.15.37:0/1916446671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f435c1a4760 con 0x7f435c1069c0 2024-09-19T06:54:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.087+0000 7f435b7fe640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f435c101350 0x7f435c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.088+0000 7f4360ae8640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c1069c0 0x7f435c1a4220 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4354006010 tx=0x7f4354003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.088+0000 7f43597fa640 1 -- 172.21.15.37:0/1916446671 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354004450 con 0x7f435c1069c0 2024-09-19T06:54:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.088+0000 7f43597fa640 1 -- 172.21.15.37:0/1916446671 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4354003e10 con 0x7f435c1069c0 2024-09-19T06:54:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.088+0000 7f43597fa640 1 -- 172.21.15.37:0/1916446671 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4354046480 con 0x7f435c1069c0 2024-09-19T06:54:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.088+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f435c1a4930 con 0x7f435c1069c0 2024-09-19T06:54:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.088+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f435c1a4df0 con 0x7f435c1069c0 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.090+0000 7f43653c6740 1 Processor -- start 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43597fa640 1 -- 172.21.15.37:0/1916446671 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f43540045f0 con 0x7f435c1069c0 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43653c6740 1 -- start start 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43653c6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43653c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43653c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 0x561a018fa4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43653c6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a0184d2f0 con 0x561a018e89d0 2024-09-19T06:54:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43653c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a017f5270 con 0x561a018e92f0 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43653c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a018492b0 con 0x561a018f80c0 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50782/0 (socket says 172.21.15.37:50782) 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 -- 172.21.15.37:0/3728359106 learned_addr learned my addr 172.21.15.37:0/3728359106 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435bfff640 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 -- 172.21.15.37:0/3728359106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 msgr2=0x561a018fa4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 0x561a018fa4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 -- 172.21.15.37:0/3728359106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 msgr2=0x561a018f7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f43597fa640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43440a3460 0x7f43440a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435b7fe640 1 -- 172.21.15.37:0/3728359106 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a017f4340 con 0x561a018e89d0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.091+0000 7f435bfff640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43440a3460 0x7f43440a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f435b7fe640 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e8db0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4350007a10 tx=0x7f4350007ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b17b66df77229f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f43597fa640 1 -- 172.21.15.37:0/1916446671 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f435402e030 con 0x7f435c1069c0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f43427fc640 1 -- 172.21.15.37:0/3728359106 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f435000fc90 con 0x561a018e89d0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f43427fc640 1 -- 172.21.15.37:0/3728359106 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350010040 con 0x561a018e89d0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f43427fc640 1 -- 172.21.15.37:0/3728359106 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43500136a0 con 0x561a018e89d0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f43653c6740 1 -- 172.21.15.37:0/3728359106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 msgr2=0x561a018e8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f435bfff640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43440a3460 0x7f43440a5920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561a018eb6c0 tx=0x7f434c00a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.092+0000 7f43653c6740 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e8db0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f4350007a10 tx=0x7f4350007ed0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- 172.21.15.37:0/3728359106 shutdown_connections 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 0x561a018fa4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 --2- 172.21.15.37:0/3728359106 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e8db0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- 172.21.15.37:0/3728359106 >> 172.21.15.37:0/3728359106 conn(0x561a018e5060 msgr2=0x561a018e5440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- 172.21.15.37:0/3728359106 shutdown_connections 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- 172.21.15.37:0/3728359106 wait complete. 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- start start 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 Processor -- start 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e89d0 0x561a018f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e92f0 0x561a018f5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 0x561a018f5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561a018facf0 con 0x561a018e92f0 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561a017d6290 con 0x561a018f80c0 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561a018fae70 con 0x561a018e89d0 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.093+0000 7f43653c6740 10 client.0 _ll_register_callbacks cb 0x561a01807140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e92f0 0x561a018f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e92f0 0x561a018f5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50794/0 (socket says 172.21.15.37:50794) 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 -- 172.21.15.37:0/2751141473 learned_addr learned my addr 172.21.15.37:0/2751141473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435bfff640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e89d0 0x561a018f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f4360ae8640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 0x561a018f5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 -- 172.21.15.37:0/2751141473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e89d0 msgr2=0x561a018f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e89d0 0x561a018f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 -- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 msgr2=0x561a018f5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 0x561a018f5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 -- 172.21.15.37:0/2751141473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a01560210 con 0x561a018e92f0 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435bfff640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e89d0 0x561a018f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f435b7fe640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e92f0 0x561a018f5520 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4350009040 tx=0x7f4350014b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95232ed59d252ebc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f433b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f433b7fe640 1 -- 172.21.15.37:0/2751141473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4350023070 con 0x561a018e92f0 2024-09-19T06:54:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f433b7fe640 1 -- 172.21.15.37:0/2751141473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f4350014430 con 0x561a018e92f0 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4350010040 con 0x561a018e92f0 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f433b7fe640 1 -- 172.21.15.37:0/2751141473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f4350014630 con 0x561a018e92f0 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f43653c6740 10 client.19077 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.094+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561a0162c620 con 0x561a018e92f0 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f433b7fe640 1 -- 172.21.15.37:0/2751141473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4350010040 con 0x561a018e92f0 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f433b7fe640 1 -- 172.21.15.37:0/2751141473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f435001f420 con 0x561a018e92f0 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f43653c6740 10 client.19077 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f43653c6740 10 client.19077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f43653c6740 20 client.19077 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f43653c6740 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01908050 0x561a0190a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f43653c6740 4 client.19077 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a0185aa20 con 0x561a01908050 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f435bfff640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01908050 0x561a0190a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f433b7fe640 10 client.19077 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.095+0000 7f435bfff640 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01908050 0x561a0190a440 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894a8306d0988bbf server_cookie=223eca4e87ad4c9e in_seq=0 out_seq=0 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f433b7fe640 1 -- 172.21.15.37:0/2751141473 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561a0185aa20 con 0x561a01908050 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f433b7fe640 10 client.19077 handle_command_reply: tid=0 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 client.19077 shutdown 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e92f0 msgr2=0x561a018f5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e92f0 0x561a018f5520 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f4350009040 tx=0x7f4350014b70 comp rx=0 tx=0).stop 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 shutdown_connections 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01908050 0x561a0190a440 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 0x561a018f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e92f0 0x561a018f5520 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f433b7fe640 0 client.19077 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 --2- 172.21.15.37:0/2751141473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e89d0 0x561a018f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 >> 172.21.15.37:0/2751141473 conn(0x561a018e5060 msgr2=0x561a018eadc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 shutdown_connections 2024-09-19T06:54:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.096+0000 7f43653c6740 1 -- 172.21.15.37:0/2751141473 wait complete. 2024-09-19T06:54:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.097+0000 7f43653c6740 20 client.19077 trim_cache size 0 max 16384 2024-09-19T06:54:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 Processor -- start 2024-09-19T06:54:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 -- start start 2024-09-19T06:54:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e92f0 0x561a0190fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 0x561a019100f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.114 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1addc0 con 0x561a018e89d0 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1ab690 con 0x561a018f80c0 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f43653c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1a94e0 con 0x561a018e92f0 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50802/0 (socket says 172.21.15.37:50802) 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 -- 172.21.15.37:0/1142050701 learned_addr learned my addr 172.21.15.37:0/1142050701 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 -- 172.21.15.37:0/1142050701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e92f0 msgr2=0x561a0190fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 --2- 172.21.15.37:0/1142050701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e92f0 0x561a0190fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 -- 172.21.15.37:0/1142050701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 msgr2=0x561a019100f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 --2- 172.21.15.37:0/1142050701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 0x561a019100f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 -- 172.21.15.37:0/1142050701 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a018e4e90 con 0x561a018e89d0 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.111+0000 7f435bfff640 1 --2- 172.21.15.37:0/1142050701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e40b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x561a0190aaf0 tx=0x7f434c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e188aa08208f6e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:59.115 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f4341ffb640 1 -- 172.21.15.37:0/1142050701 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c020830 con 0x561a018e89d0 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f4341ffb640 1 -- 172.21.15.37:0/1142050701 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c02f070 con 0x561a018e89d0 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f4341ffb640 1 -- 172.21.15.37:0/1142050701 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c01f450 con 0x561a018e89d0 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 -- 172.21.15.37:0/1142050701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 msgr2=0x561a018e40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 --2- 172.21.15.37:0/1142050701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e40b0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x561a0190aaf0 tx=0x7f434c014000 comp rx=0 tx=0).stop 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 -- 172.21.15.37:0/1142050701 shutdown_connections 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 --2- 172.21.15.37:0/1142050701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018f80c0 0x561a019100f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 --2- 172.21.15.37:0/1142050701 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018e92f0 0x561a0190fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 --2- 172.21.15.37:0/1142050701 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a018e40b0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 -- 172.21.15.37:0/1142050701 >> 172.21.15.37:0/1142050701 conn(0x561a018e5060 msgr2=0x561a018e39a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 -- 172.21.15.37:0/1142050701 shutdown_connections 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.112+0000 7f43653c6740 1 -- 172.21.15.37:0/1142050701 wait complete. 2024-09-19T06:54:59.116 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 -- start start 2024-09-19T06:54:59.117 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 Processor -- start 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a01900980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018fbca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 0x561a018fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1a9b50 con 0x561a018e89d0 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1aafc0 con 0x561a018e92f0 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f435c1aa220 con 0x561a018f80c0 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f43653c6740 10 client.0 _ll_register_callbacks cb 0x7f4350007910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a01900980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a01900980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50804/0 (socket says 172.21.15.37:50804) 2024-09-19T06:54:59.119 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 -- 172.21.15.37:0/2869285834 learned_addr learned my addr 172.21.15.37:0/2869285834 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f4360ae8640 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 0x561a018fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435b7fe640 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018fbca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 -- 172.21.15.37:0/2869285834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 msgr2=0x561a018fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 0x561a018fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 -- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 msgr2=0x561a018fbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018fbca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.113+0000 7f435bfff640 1 -- 172.21.15.37:0/2869285834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a01971b80 con 0x561a018e89d0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f435bfff640 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a01900980 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f434c011f90 tx=0x7f434c029420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a09b86c382edb0f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f433b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f433b7fe640 1 -- 172.21.15.37:0/2869285834 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c020830 con 0x561a018e89d0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f433b7fe640 1 -- 172.21.15.37:0/2869285834 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f434c0209d0 con 0x561a018e89d0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f434c02f070 con 0x561a018e89d0 2024-09-19T06:54:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f433b7fe640 1 -- 172.21.15.37:0/2869285834 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f434c020bd0 con 0x561a018e89d0 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 10 client.19083 fetch_fsmap learned FSMap version 43 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x561a01972d10 con 0x561a018e89d0 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f433b7fe640 1 -- 172.21.15.37:0/2869285834 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f434c02f070 con 0x561a018e89d0 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f433b7fe640 1 -- 172.21.15.37:0/2869285834 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f434c043400 con 0x561a018e89d0 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 10 client.19083 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 10 client.19083 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 20 client.19083 populate_metadata read hostname 'smithi037' 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01973460 0x561a01975880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 4 client.19083 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:54:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.114+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561a01854d70 con 0x561a01973460 2024-09-19T06:54:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.115+0000 7f435b7fe640 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01973460 0x561a01975880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:54:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.115+0000 7f433b7fe640 10 client.19083 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.115+0000 7f435b7fe640 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01973460 0x561a01975880 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1be6bee727e27a server_cookie=7fdba31f59f81100 in_seq=0 out_seq=0 2024-09-19T06:54:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f433b7fe640 1 -- 172.21.15.37:0/2869285834 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561a01854d70 con 0x561a01973460 2024-09-19T06:54:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f433b7fe640 10 client.19083 handle_command_reply: tid=0 2024-09-19T06:54:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 client.19083 shutdown 2024-09-19T06:54:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 msgr2=0x561a01900980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.160 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a01900980 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f434c011f90 tx=0x7f434c029420 comp rx=0 tx=0).stop 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 shutdown_connections 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561a01973460 0x561a01975880 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561a018f80c0 0x561a018fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561a018e92f0 0x561a018fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f433b7fe640 0 client.19083 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 --2- 172.21.15.37:0/2869285834 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561a018e89d0 0x561a01900980 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 >> 172.21.15.37:0/2869285834 conn(0x561a018e5060 msgr2=0x561a018f7060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 shutdown_connections 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 1 -- 172.21.15.37:0/2869285834 wait complete. 2024-09-19T06:54:59.161 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.158+0000 7f43653c6740 20 client.19083 trim_cache size 0 max 16384 2024-09-19T06:54:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:54:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:54:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:54:59.162 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43440a3460 msgr2=0x7f43440a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43440a3460 0x7f43440a5920 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x561a018eb6c0 tx=0x7f434c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c1069c0 msgr2=0x7f435c1a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c1069c0 0x7f435c1a4220 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f4354006010 tx=0x7f4354003c20 comp rx=0 tx=0).stop 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 shutdown_connections 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f43440a3460 0x7f43440a5920 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f435c1069c0 0x7f435c1a4220 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f435c101350 0x7f435c1996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.163 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 --2- 172.21.15.37:0/1916446671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f435c1009b0 0x7f435c1991a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:54:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 >> 172.21.15.37:0/1916446671 conn(0x7f435c077990 msgr2=0x7f435c103660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:54:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 shutdown_connections 2024-09-19T06:54:59.164 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:54:59.159+0000 7f436254b640 1 -- 172.21.15.37:0/1916446671 wait complete. 2024-09-19T06:54:59.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.68 MB/sec execute 566 sec latency 16044.386 ms 2024-09-19T06:54:59.761 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:55:00.019 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:00.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:59 smithi037 ceph-mon[31211]: pgmap v417: 97 pgs: 97 active+clean; 72 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-19T06:55:00.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:54:59 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3803949722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:00.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.66 MB/sec execute 567 sec latency 17044.425 ms 2024-09-19T06:55:00.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:59 smithi096 ceph-mon[35004]: pgmap v417: 97 pgs: 97 active+clean; 72 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-19T06:55:00.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:54:59 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3803949722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:59 smithi151 ceph-mon[35236]: pgmap v417: 97 pgs: 97 active+clean; 72 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 2.4 MiB/s wr, 171 op/s 2024-09-19T06:55:00.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:54:59 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3803949722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:00.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 -- 172.21.15.37:0/1325676095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318100a80 msgr2=0x7f5318100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 --2- 172.21.15.37:0/1325676095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318100a80 0x7f5318100ee0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f530c009860 tx=0x7f530c02a340 comp rx=0 tx=0).stop 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 -- 172.21.15.37:0/1325676095 shutdown_connections 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 --2- 172.21.15.37:0/1325676095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5318101420 0x7f531810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 --2- 172.21.15.37:0/1325676095 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318100a80 0x7f5318100ee0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 --2- 172.21.15.37:0/1325676095 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318106a80 0x7f5318106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 -- 172.21.15.37:0/1325676095 >> 172.21.15.37:0/1325676095 conn(0x7f53180fc7a0 msgr2=0x7f53180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 -- 172.21.15.37:0/1325676095 shutdown_connections 2024-09-19T06:55:00.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.471+0000 7f531d1c6640 1 -- 172.21.15.37:0/1325676095 wait complete. 2024-09-19T06:55:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.472+0000 7f531d1c6640 1 Processor -- start 2024-09-19T06:55:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531d1c6640 1 -- start start 2024-09-19T06:55:00.476 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531d1c6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318100a80 0x7f531819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531d1c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318101420 0x7f531819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531d1c6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5318106a80 0x7f531819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531d1c6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5318110e40 con 0x7f5318100a80 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531d1c6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5318110cc0 con 0x7f5318101420 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531d1c6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5318110fc0 con 0x7f5318106a80 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f5316d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318100a80 0x7f531819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f5316d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318100a80 0x7f531819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34746/0 (socket says 172.21.15.37:34746) 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f5316d9d640 1 -- 172.21.15.37:0/3483030600 learned_addr learned my addr 172.21.15.37:0/3483030600 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:00.477 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f531659c640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318101420 0x7f531819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.473+0000 7f5316d9d640 1 -- 172.21.15.37:0/3483030600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5318106a80 msgr2=0x7f531819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f5316d9d640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5318106a80 0x7f531819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f5316d9d640 1 -- 172.21.15.37:0/3483030600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318101420 msgr2=0x7f531819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f5316d9d640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318101420 0x7f531819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f5316d9d640 1 -- 172.21.15.37:0/3483030600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f531819f5f0 con 0x7f5318100a80 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f5316d9d640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318100a80 0x7f531819a640 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f530800e9c0 tx=0x7f530800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f52fffff640 1 -- 172.21.15.37:0/3483030600 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530800cde0 con 0x7f5318100a80 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f531819f8e0 con 0x7f5318100a80 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f52fffff640 1 -- 172.21.15.37:0/3483030600 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53080035f0 con 0x7f5318100a80 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f52fffff640 1 -- 172.21.15.37:0/3483030600 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5308010650 con 0x7f5318100a80 2024-09-19T06:55:00.478 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.474+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53181a71c0 con 0x7f5318100a80 2024-09-19T06:55:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.476+0000 7f52fffff640 1 -- 172.21.15.37:0/3483030600 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f53080037a0 con 0x7f5318100a80 2024-09-19T06:55:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 Processor -- start 2024-09-19T06:55:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 -- start start 2024-09-19T06:55:00.484 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1d6090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f52fffff640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f53000a33b0 0x7f53000a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1e3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d3fe13c2f0 con 0x55d3fe1e1370 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d3fe0e3cf0 con 0x55d3fe1d65d0 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.477+0000 7f531f59f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3fe1382b0 con 0x55d3fe1d7b40 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1e3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34748/0 (socket says 172.21.15.37:34748) 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531759e640 1 -- 172.21.15.37:0/246468825 learned_addr learned my addr 172.21.15.37:0/246468825 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f52fffff640 1 -- 172.21.15.37:0/3483030600 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f53080d0070 con 0x7f5318100a80 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531659c640 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f5316d9d640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f53000a33b0 0x7f53000a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.485 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531759e640 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1d6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531659c640 1 -- 172.21.15.37:0/246468825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 msgr2=0x55d3fe1d6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531659c640 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1d6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531659c640 1 -- 172.21.15.37:0/246468825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 msgr2=0x55d3fe1e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531659c640 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531659c640 1 -- 172.21.15.37:0/246468825 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3fe0c83a0 con 0x55d3fe1d65d0 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531759e640 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1d6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f5316d9d640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f53000a33b0 0x7f53000a5870 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f53080cd870 tx=0x7f53080baf90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.478+0000 7f531659c640 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d6a10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f530c005940 tx=0x7f530c009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66fc9f790aa504c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f52fd7fa640 1 -- 172.21.15.37:0/246468825 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c037040 con 0x55d3fe1d65d0 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f52fd7fa640 1 -- 172.21.15.37:0/246468825 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c030e40 con 0x55d3fe1d65d0 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f52fd7fa640 1 -- 172.21.15.37:0/246468825 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c003970 con 0x55d3fe1d65d0 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 -- 172.21.15.37:0/246468825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 msgr2=0x55d3fe1d6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.486 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d6a10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f530c005940 tx=0x7f530c009510 comp rx=0 tx=0).stop 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 -- 172.21.15.37:0/246468825 shutdown_connections 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d6a10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 --2- 172.21.15.37:0/246468825 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1d6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 -- 172.21.15.37:0/246468825 >> 172.21.15.37:0/246468825 conn(0x55d3fe19f310 msgr2=0x55d3fe1d2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 -- 172.21.15.37:0/246468825 shutdown_connections 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.479+0000 7f531f59f740 1 -- 172.21.15.37:0/246468825 wait complete. 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 -- start start 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 Processor -- start 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1de030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1de640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.487 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d3fe1e4010 con 0x55d3fe1e1370 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d3fe0f7060 con 0x55d3fe1d7b40 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3fe1e4190 con 0x55d3fe1d65d0 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.480+0000 7f531f59f740 10 client.0 _ll_register_callbacks cb 0x55d3fde4a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1de030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1de640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1de640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34758/0 (socket says 172.21.15.37:34758) 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 -- 172.21.15.37:0/364107876 learned_addr learned my addr 172.21.15.37:0/364107876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 -- 172.21.15.37:0/364107876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d65d0 msgr2=0x55d3fe1e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 -- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d7b40 msgr2=0x55d3fe1de030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1de030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.488 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 -- 172.21.15.37:0/364107876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3fdead3c0 con 0x55d3fe1e1370 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531659c640 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1de030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531759e640 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1de640 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5310005fd0 tx=0x7f5310009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91111747addcb183 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f52faff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f52faff5640 1 -- 172.21.15.37:0/364107876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531001b020 con 0x55d3fe1e1370 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f52faff5640 1 -- 172.21.15.37:0/364107876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f5310003970 con 0x55d3fe1e1370 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.481+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f530c030e40 con 0x55d3fe1e1370 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.482+0000 7f52faff5640 1 -- 172.21.15.37:0/364107876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5310026050 con 0x55d3fe1e1370 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.482+0000 7f531f59f740 10 client.19095 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.482+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d3fe1efab0 con 0x55d3fe1e1370 2024-09-19T06:55:00.489 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.482+0000 7f52faff5640 1 -- 172.21.15.37:0/364107876 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c030e40 con 0x55d3fe1e1370 2024-09-19T06:55:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.482+0000 7f52faff5640 1 -- 172.21.15.37:0/364107876 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f531000f040 con 0x55d3fe1e1370 2024-09-19T06:55:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.483+0000 7f531f59f740 10 client.19095 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.483+0000 7f531f59f740 10 client.19095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:55:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.483+0000 7f531f59f740 20 client.19095 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.483+0000 7f531f59f740 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe1f3360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.491 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.483+0000 7f531f59f740 4 client.19095 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.483+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3fe149860 con 0x55d3fe1f0f70 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.483+0000 7f531659c640 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe1f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.484+0000 7f531659c640 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe1f3360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d03abc3bbf50836d server_cookie=cdfda73757e72054 in_seq=0 out_seq=0 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.484+0000 7f52faff5640 10 client.19095 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.484+0000 7f52faff5640 1 -- 172.21.15.37:0/364107876 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3fe149860 con 0x55d3fe1f0f70 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.484+0000 7f52faff5640 10 client.19095 handle_command_reply: tid=0 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.484+0000 7f531f59f740 1 client.19095 shutdown 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 msgr2=0x55d3fe1de640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1de640 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7f5310005fd0 tx=0x7f5310009500 comp rx=0 tx=0).stop 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 shutdown_connections 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe1f3360 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1de640 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.492 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f52faff5640 0 client.19095 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:00.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1de030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 --2- 172.21.15.37:0/364107876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 >> 172.21.15.37:0/364107876 conn(0x55d3fe19f310 msgr2=0x55d3fe1d8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:00.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 shutdown_connections 2024-09-19T06:55:00.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 1 -- 172.21.15.37:0/364107876 wait complete. 2024-09-19T06:55:00.493 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.485+0000 7f531f59f740 20 client.19095 trim_cache size 0 max 16384 2024-09-19T06:55:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.499+0000 7f531f59f740 1 Processor -- start 2024-09-19T06:55:00.501 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.499+0000 7f531f59f740 1 -- start start 2024-09-19T06:55:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531f59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531f59f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1ed2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531f59f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1d9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.502 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531f59f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53181b2400 con 0x55d3fe1e1370 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531f59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53181afcd0 con 0x55d3fe1d65d0 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531f59f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53181adb20 con 0x55d3fe1d7b40 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531759e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1ed2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1d9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1ed2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:34692/0 (socket says 172.21.15.37:34692) 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 -- 172.21.15.37:0/2487390172 learned_addr learned my addr 172.21.15.37:0/2487390172 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1d9120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34760/0 (socket says 172.21.15.37:34760) 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 -- 172.21.15.37:0/2487390172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 msgr2=0x55d3fe1d2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.503 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 -- 172.21.15.37:0/2487390172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 msgr2=0x55d3fe1d9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1d9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531659c640 1 -- 172.21.15.37:0/2487390172 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3fe1f36d0 con 0x55d3fe1d7b40 2024-09-19T06:55:00.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.500+0000 7f531759e640 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:00.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.501+0000 7f531659c640 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1ed2b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f530c005e00 tx=0x7f530c02c040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3466214f68920887 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:00.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.501+0000 7f52fcff9640 1 -- 172.21.15.37:0/2487390172 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c037040 con 0x55d3fe1d7b40 2024-09-19T06:55:00.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.501+0000 7f52fcff9640 1 -- 172.21.15.37:0/2487390172 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c005050 con 0x55d3fe1d7b40 2024-09-19T06:55:00.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.501+0000 7f52fcff9640 1 -- 172.21.15.37:0/2487390172 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530c004900 con 0x55d3fe1d7b40 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 -- 172.21.15.37:0/2487390172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 msgr2=0x55d3fe1ed2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1ed2b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f530c005e00 tx=0x7f530c02c040 comp rx=0 tx=0).stop 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 -- 172.21.15.37:0/2487390172 shutdown_connections 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1e1370 0x55d3fe1d9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1ed2b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 --2- 172.21.15.37:0/2487390172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1d2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 -- 172.21.15.37:0/2487390172 >> 172.21.15.37:0/2487390172 conn(0x55d3fe19f310 msgr2=0x55d3fe1d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 -- 172.21.15.37:0/2487390172 shutdown_connections 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 -- 172.21.15.37:0/2487390172 wait complete. 2024-09-19T06:55:00.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.502+0000 7f531f59f740 1 -- start start 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 1 Processor -- start 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1daf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1e1370 0x55d3fe1db670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f53181ae190 con 0x55d3fe1d65d0 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53181af600 con 0x55d3fe1e1370 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f53181ae860 con 0x55d3fe1d7b40 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531f59f740 10 client.0 _ll_register_callbacks cb 0x7f53100098b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531759e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34770/0 (socket says 172.21.15.37:34770) 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531759e640 1 -- 172.21.15.37:0/2778793468 learned_addr learned my addr 172.21.15.37:0/2778793468 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531659c640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1daf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.503+0000 7f531759e640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1e1370 0x55d3fe1db670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531759e640 1 -- 172.21.15.37:0/2778793468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 msgr2=0x55d3fe1daf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531759e640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1daf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531759e640 1 -- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1e1370 msgr2=0x55d3fe1db670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531759e640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1e1370 0x55d3fe1db670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531759e640 1 -- 172.21.15.37:0/2778793468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3fe25abb0 con 0x55d3fe1d65d0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531659c640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1daf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531759e640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e9d30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5310009f20 tx=0x7f53100126e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca518fc838643c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f52faff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f52faff5640 1 -- 172.21.15.37:0/2778793468 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f531001b070 con 0x55d3fe1d65d0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f52faff5640 1 -- 172.21.15.37:0/2778793468 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f531002a840 con 0x55d3fe1d65d0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f530c005050 con 0x55d3fe1d65d0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f52faff5640 1 -- 172.21.15.37:0/2778793468 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5310004b20 con 0x55d3fe1d65d0 2024-09-19T06:55:00.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531f59f740 10 client.19101 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.504+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55d3fe25bd70 con 0x55d3fe1d65d0 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f52faff5640 1 -- 172.21.15.37:0/2778793468 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530c005050 con 0x55d3fe1d65d0 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f52faff5640 1 -- 172.21.15.37:0/2778793468 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f5310005390 con 0x55d3fe1d65d0 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531f59f740 10 client.19101 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531f59f740 10 client.19101 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531f59f740 20 client.19101 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531f59f740 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe25e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531f59f740 4 client.19101 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3fe143d70 con 0x55d3fe1f0f70 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531659c640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe25e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f52faff5640 10 client.19101 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.505+0000 7f531659c640 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe25e500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d015c389495d482e server_cookie=6044bc83005731dc in_seq=0 out_seq=0 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f52faff5640 1 -- 172.21.15.37:0/2778793468 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3fe143d70 con 0x55d3fe1f0f70 2024-09-19T06:55:00.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f52faff5640 10 client.19101 handle_command_reply: tid=0 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 client.19101 shutdown 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1d65d0 msgr2=0x55d3fe1e9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e9d30 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f5310009f20 tx=0x7f53100126e0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 shutdown_connections 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3fe1f0f70 0x55d3fe25e500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3fe1e1370 0x55d3fe1db670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3fe1d7b40 0x55d3fe1daf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 --2- 172.21.15.37:0/2778793468 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3fe1d65d0 0x55d3fe1e9d30 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f52faff5640 0 client.19101 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 >> 172.21.15.37:0/2778793468 conn(0x55d3fe19f310 msgr2=0x55d3fe1d1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 shutdown_connections 2024-09-19T06:55:00.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 1 -- 172.21.15.37:0/2778793468 wait complete. 2024-09-19T06:55:00.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.506+0000 7f531f59f740 20 client.19101 trim_cache size 0 max 16384 2024-09-19T06:55:00.513 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f53000a33b0 msgr2=0x7f53000a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f53000a33b0 0x7f53000a5870 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f53080cd870 tx=0x7f53080baf90 comp rx=0 tx=0).stop 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318100a80 msgr2=0x7f531819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318100a80 0x7f531819a640 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f530800e9c0 tx=0x7f530800ee80 comp rx=0 tx=0).stop 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 shutdown_connections 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5318106a80 0x7f531819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5318101420 0x7f531819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f53000a33b0 0x7f53000a5870 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 --2- 172.21.15.37:0/3483030600 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5318100a80 0x7f531819a640 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 >> 172.21.15.37:0/3483030600 conn(0x7f53180fc7a0 msgr2=0x7f531806a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 shutdown_connections 2024-09-19T06:55:00.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:00.507+0000 7f531d1c6640 1 -- 172.21.15.37:0/3483030600 wait complete. 2024-09-19T06:55:01.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.65 MB/sec execute 568 sec latency 18044.488 ms 2024-09-19T06:55:01.361 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:55:01.652 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:02.038 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:01 smithi037 ceph-mon[31211]: pgmap v418: 97 pgs: 97 active+clean; 72 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-09-19T06:55:02.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 -- 172.21.15.37:0/247762271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102570 msgr2=0x7f195c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 --2- 172.21.15.37:0/247762271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102570 0x7f195c1029b0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f1948009900 tx=0x7f194802a480 comp rx=0 tx=0).stop 2024-09-19T06:55:02.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 -- 172.21.15.37:0/247762271 shutdown_connections 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 --2- 172.21.15.37:0/247762271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f195c102ef0 0x7f195c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 --2- 172.21.15.37:0/247762271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102570 0x7f195c1029b0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 --2- 172.21.15.37:0/247762271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f195c100e20 0x7f195c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 -- 172.21.15.37:0/247762271 >> 172.21.15.37:0/247762271 conn(0x7f195c0fc7d0 msgr2=0x7f195c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 -- 172.21.15.37:0/247762271 shutdown_connections 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 -- 172.21.15.37:0/247762271 wait complete. 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 Processor -- start 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.121+0000 7f1962eee640 1 -- start start 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1962eee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f195c100e20 0x7f195c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1962eee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f195c102570 0x7f195c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1962eee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102ef0 0x7f195c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.140 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1962eee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f195c110f50 con 0x7f195c102ef0 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1962eee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f195c110dd0 con 0x7f195c100e20 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1962eee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1110d0 con 0x7f195c102570 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102ef0 0x7f195c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102ef0 0x7f195c19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34786/0 (socket says 172.21.15.37:34786) 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 -- 172.21.15.37:0/1731473631 learned_addr learned my addr 172.21.15.37:0/1731473631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f195bfff640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f195c102570 0x7f195c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1960c8a640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f195c100e20 0x7f195c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 -- 172.21.15.37:0/1731473631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f195c102570 msgr2=0x7f195c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f195c102570 0x7f195c198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 -- 172.21.15.37:0/1731473631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f195c100e20 msgr2=0x7f195c198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f195c100e20 0x7f195c198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 -- 172.21.15.37:0/1731473631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f195c19d540 con 0x7f195c102ef0 2024-09-19T06:55:02.141 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f196148b640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102ef0 0x7f195c19ce60 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f195400eaa0 tx=0x7f195400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1959ffb640 1 -- 172.21.15.37:0/1731473631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195400cdb0 con 0x7f195c102ef0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1959ffb640 1 -- 172.21.15.37:0/1731473631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1954005020 con 0x7f195c102ef0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f195c19d830 con 0x7f195c102ef0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.122+0000 7f1959ffb640 1 -- 172.21.15.37:0/1731473631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19540116b0 con 0x7f195c102ef0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.123+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f195c1a5110 con 0x7f195c102ef0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f1959ffb640 1 -- 172.21.15.37:0/1731473631 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1954005560 con 0x7f195c102ef0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 Processor -- start 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 -- start start 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd3ab40 0x55e39bd39090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd395d0 0x55e39bd39a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd4ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e39bc9f2f0 con 0x55e39bd395d0 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e39bc46cf0 con 0x55e39bd48780 2024-09-19T06:55:02.142 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f19652c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e39bc9b2b0 con 0x55e39bd3ab40 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.125+0000 7f1959ffb640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f194c0a33b0 0x7f194c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd395d0 0x55e39bd39a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd395d0 0x55e39bd39a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34788/0 (socket says 172.21.15.37:34788) 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 -- 172.21.15.37:0/3272400748 learned_addr learned my addr 172.21.15.37:0/3272400748 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f195bfff640 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd3ab40 0x55e39bd39090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1959ffb640 1 -- 172.21.15.37:0/1731473631 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f19540cff10 con 0x7f195c102ef0 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f196148b640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f194c0a33b0 0x7f194c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f195bfff640 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd4ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 -- 172.21.15.37:0/3272400748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd3ab40 msgr2=0x55e39bd39090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd3ab40 0x55e39bd39090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 -- 172.21.15.37:0/3272400748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 msgr2=0x55e39bd4ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd4ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.143 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 -- 172.21.15.37:0/3272400748 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39bc2b3a0 con 0x55e39bd395d0 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f195bfff640 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd4ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1960c8a640 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd395d0 0x55e39bd39a10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f195000bcb0 tx=0x7f195000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5a56b1487a02f1f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1946ffd640 1 -- 172.21.15.37:0/3272400748 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950015070 con 0x55e39bd395d0 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f196148b640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f194c0a33b0 0x7f194c0a5870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f195400efd0 tx=0x7f1954009270 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1946ffd640 1 -- 172.21.15.37:0/3272400748 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1950011920 con 0x55e39bd395d0 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.126+0000 7f1946ffd640 1 -- 172.21.15.37:0/3272400748 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950010930 con 0x55e39bd395d0 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 -- 172.21.15.37:0/3272400748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd395d0 msgr2=0x55e39bd39a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd395d0 0x55e39bd39a10 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f195000bcb0 tx=0x7f195000f730 comp rx=0 tx=0).stop 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 -- 172.21.15.37:0/3272400748 shutdown_connections 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd4ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd395d0 0x55e39bd39a10 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 --2- 172.21.15.37:0/3272400748 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd3ab40 0x55e39bd39090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.144 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 -- 172.21.15.37:0/3272400748 >> 172.21.15.37:0/3272400748 conn(0x55e39bd02310 msgr2=0x55e39bd35c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 -- 172.21.15.37:0/3272400748 shutdown_connections 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 -- 172.21.15.37:0/3272400748 wait complete. 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 -- start start 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.127+0000 7f19652c7740 1 Processor -- start 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f19652c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd48160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f19652c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd45410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f19652c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd459e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f19652c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e39bd4b480 con 0x55e39bd3ab40 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f19652c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e39bc5a060 con 0x55e39bd48780 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f19652c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e39bd4b600 con 0x55e39bd395d0 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f19652c7740 10 client.0 _ll_register_callbacks cb 0x55e39b9ad8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd45410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd45410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34798/0 (socket says 172.21.15.37:34798) 2024-09-19T06:55:02.145 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 -- 172.21.15.37:0/326292251 learned_addr learned my addr 172.21.15.37:0/326292251 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f195bfff640 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd48160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f195bfff640 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 -- 172.21.15.37:0/326292251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 msgr2=0x55e39bd48160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd48160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 -- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 msgr2=0x55e39bd459e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd459e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.128+0000 7f1960c8a640 1 -- 172.21.15.37:0/326292251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39ba103c0 con 0x55e39bd3ab40 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f1960c8a640 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd45410 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f195000bfd0 tx=0x7f195000b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110c3da1311386ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f193bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f193bfff640 1 -- 172.21.15.37:0/326292251 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950028060 con 0x55e39bd3ab40 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f193bfff640 1 -- 172.21.15.37:0/326292251 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f1950011650 con 0x55e39bd3ab40 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1950011920 con 0x55e39bd3ab40 2024-09-19T06:55:02.146 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f193bfff640 1 -- 172.21.15.37:0/326292251 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1950015070 con 0x55e39bd3ab40 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 10 client.19113 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e39bd56e70 con 0x55e39bd3ab40 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f193bfff640 1 -- 172.21.15.37:0/326292251 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1950011920 con 0x55e39bd3ab40 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f193bfff640 1 -- 172.21.15.37:0/326292251 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f19500047e0 con 0x55e39bd3ab40 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 10 client.19113 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 10 client.19113 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 20 client.19113 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bd58270 0x55e39bd5a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 4 client.19113 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.129+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e39bcac860 con 0x55e39bd58270 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.130+0000 7f195bfff640 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bd58270 0x55e39bd5a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.130+0000 7f195bfff640 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bd58270 0x55e39bd5a660 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d639905be1863ad3 server_cookie=ddd4480f40403703 in_seq=0 out_seq=0 2024-09-19T06:55:02.147 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.130+0000 7f193bfff640 10 client.19113 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f193bfff640 1 -- 172.21.15.37:0/326292251 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e39bcac860 con 0x55e39bd58270 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f193bfff640 10 client.19113 handle_command_reply: tid=0 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 client.19113 shutdown 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 msgr2=0x55e39bd45410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd45410 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7f195000bfd0 tx=0x7f195000b7c0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 shutdown_connections 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bd58270 0x55e39bd5a660 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd48780 0x55e39bd459e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd45410 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 --2- 172.21.15.37:0/326292251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd48160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f193bfff640 0 client.19113 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 >> 172.21.15.37:0/326292251 conn(0x55e39bd02310 msgr2=0x55e39bd3b960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:02.148 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 shutdown_connections 2024-09-19T06:55:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 1 -- 172.21.15.37:0/326292251 wait complete. 2024-09-19T06:55:02.149 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.131+0000 7f19652c7740 20 client.19113 trim_cache size 0 max 16384 2024-09-19T06:55:02.153 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 Processor -- start 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 -- start start 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd54720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd3ab40 0x55e39bd44410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd48780 0x55e39bd5fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1ae020 con 0x55e39bd48780 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1ab8f0 con 0x55e39bd3ab40 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f19652c7740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1a9740 con 0x55e39bd395d0 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd48780 0x55e39bd5fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd48780 0x55e39bd5fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34810/0 (socket says 172.21.15.37:34810) 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 -- 172.21.15.37:0/4108062603 learned_addr learned my addr 172.21.15.37:0/4108062603 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd54720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.154 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f1960c8a640 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd3ab40 0x55e39bd44410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 -- 172.21.15.37:0/4108062603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 msgr2=0x55e39bd54720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd54720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 -- 172.21.15.37:0/4108062603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd3ab40 msgr2=0x55e39bd44410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd3ab40 0x55e39bd44410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f195bfff640 1 -- 172.21.15.37:0/4108062603 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e39bd36960 con 0x55e39bd48780 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.145+0000 7f1960c8a640 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd3ab40 0x55e39bd44410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.146+0000 7f195bfff640 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd48780 0x55e39bd5fc30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1948000c00 tx=0x7f19480058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962b45c20d36d5a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.146+0000 7f19467fc640 1 -- 172.21.15.37:0/4108062603 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948050040 con 0x55e39bd48780 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.146+0000 7f19467fc640 1 -- 172.21.15.37:0/4108062603 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948005020 con 0x55e39bd48780 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.146+0000 7f19467fc640 1 -- 172.21.15.37:0/4108062603 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1948035040 con 0x55e39bd48780 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.146+0000 7f19652c7740 1 -- 172.21.15.37:0/4108062603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd48780 msgr2=0x55e39bd5fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.146+0000 7f19652c7740 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd48780 0x55e39bd5fc30 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f1948000c00 tx=0x7f19480058d0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 -- 172.21.15.37:0/4108062603 shutdown_connections 2024-09-19T06:55:02.155 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd48780 0x55e39bd5fc30 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd3ab40 0x55e39bd44410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 --2- 172.21.15.37:0/4108062603 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd395d0 0x55e39bd54720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 -- 172.21.15.37:0/4108062603 >> 172.21.15.37:0/4108062603 conn(0x55e39bd02310 msgr2=0x55e39bd43d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 -- 172.21.15.37:0/4108062603 shutdown_connections 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 -- 172.21.15.37:0/4108062603 wait complete. 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 -- start start 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.147+0000 7f19652c7740 1 Processor -- start 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f19652c7740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd395d0 0x55e39bd510e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f19652c7740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd42250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd42250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd42250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34812/0 (socket says 172.21.15.37:34812) 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f19652c7740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd48780 0x55e39bd42820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.156 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f19652c7740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1a9db0 con 0x55e39bd3ab40 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 -- 172.21.15.37:0/283055937 learned_addr learned my addr 172.21.15.37:0/283055937 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1ab220 con 0x55e39bd395d0 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f195c1aa480 con 0x55e39bd48780 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f19652c7740 10 client.0 _ll_register_callbacks cb 0x7f1948005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f195bfff640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd395d0 0x55e39bd510e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f195bfff640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd48780 0x55e39bd42820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 -- 172.21.15.37:0/283055937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd48780 msgr2=0x55e39bd42820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd48780 0x55e39bd42820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 -- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd395d0 msgr2=0x55e39bd510e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd395d0 0x55e39bd510e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 -- 172.21.15.37:0/283055937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e39bd3c120 con 0x55e39bd3ab40 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f195bfff640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd48780 0x55e39bd42820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:02.157 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f1960c8a640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd42250 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x55e39bd46a50 tx=0x7f195000ff60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265e4b5557cff6e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.148+0000 7f193bfff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f193bfff640 1 -- 172.21.15.37:0/283055937 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1950028070 con 0x55e39bd3ab40 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f193bfff640 1 -- 172.21.15.37:0/283055937 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f1950004030 con 0x55e39bd3ab40 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1948005020 con 0x55e39bd3ab40 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f193bfff640 1 -- 172.21.15.37:0/283055937 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f1950004230 con 0x55e39bd3ab40 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 10 client.19119 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55e39bdc2b20 con 0x55e39bd3ab40 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f193bfff640 1 -- 172.21.15.37:0/283055937 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1948005020 con 0x55e39bd3ab40 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f193bfff640 1 -- 172.21.15.37:0/283055937 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f1950011650 con 0x55e39bd3ab40 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 10 client.19119 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 10 client.19119 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 20 client.19119 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bdc3180 0x55e39bdc5570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:02.158 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 4 client.19119 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.149+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e39bca6d70 con 0x55e39bdc3180 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f195bfff640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bdc3180 0x55e39bdc5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f193bfff640 10 client.19119 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f195bfff640 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bdc3180 0x55e39bdc5570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48bb3340d1582358 server_cookie=47a3e67b1452e0a3 in_seq=0 out_seq=0 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f193bfff640 1 -- 172.21.15.37:0/283055937 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e39bca6d70 con 0x55e39bdc3180 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f193bfff640 10 client.19119 handle_command_reply: tid=0 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f19652c7740 1 client.19119 shutdown 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 msgr2=0x55e39bd42250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.150+0000 7f19652c7740 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd42250 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x55e39bd46a50 tx=0x7f195000ff60 comp rx=0 tx=0).stop 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 shutdown_connections 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55e39bdc3180 0x55e39bdc5570 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e39bd48780 0x55e39bd42820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.159 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e39bd3ab40 0x55e39bd42250 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 --2- 172.21.15.37:0/283055937 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e39bd395d0 0x55e39bd510e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 >> 172.21.15.37:0/283055937 conn(0x55e39bd02310 msgr2=0x55e39bd44520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:02.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f193bfff640 0 client.19119 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:02.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 shutdown_connections 2024-09-19T06:55:02.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 1 -- 172.21.15.37:0/283055937 wait complete. 2024-09-19T06:55:02.160 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f19652c7740 20 client.19119 trim_cache size 0 max 16384 2024-09-19T06:55:02.160 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:55:02.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f194c0a33b0 msgr2=0x7f194c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f1962eee640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f194c0a33b0 0x7f194c0a5870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f195400efd0 tx=0x7f1954009270 comp rx=0 tx=0).stop 2024-09-19T06:55:02.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102ef0 msgr2=0x7f195c19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:02.161 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f1962eee640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102ef0 0x7f195c19ce60 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f195400eaa0 tx=0x7f195400ef60 comp rx=0 tx=0).stop 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.151+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 shutdown_connections 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.152+0000 7f1962eee640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f194c0a33b0 0x7f194c0a5870 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.152+0000 7f1962eee640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f195c102ef0 0x7f195c19ce60 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.152+0000 7f1962eee640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f195c102570 0x7f195c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.152+0000 7f1962eee640 1 --2- 172.21.15.37:0/1731473631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f195c100e20 0x7f195c198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.152+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 >> 172.21.15.37:0/1731473631 conn(0x7f195c0fc7d0 msgr2=0x7f195c06a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.152+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 shutdown_connections 2024-09-19T06:55:02.162 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:02.152+0000 7f1962eee640 1 -- 172.21.15.37:0/1731473631 wait complete. 2024-09-19T06:55:02.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.64 MB/sec execute 569 sec latency 19044.590 ms 2024-09-19T06:55:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:01 smithi096 ceph-mon[35004]: pgmap v418: 97 pgs: 97 active+clean; 72 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-09-19T06:55:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:01 smithi151 ceph-mon[35236]: pgmap v418: 97 pgs: 97 active+clean; 72 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 1.5 MiB/s wr, 59 op/s 2024-09-19T06:55:02.422 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:55:02.930 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:55:03.020 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:03.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.62 MB/sec execute 570 sec latency 20044.712 ms 2024-09-19T06:55:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 -- 172.21.15.37:0/555967767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41036e0 msgr2=0x7fd9d4107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 --2- 172.21.15.37:0/555967767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41036e0 0x7fd9d4107b90 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8009930 tx=0x7fd9c802a420 comp rx=0 tx=0).stop 2024-09-19T06:55:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 -- 172.21.15.37:0/555967767 shutdown_connections 2024-09-19T06:55:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 --2- 172.21.15.37:0/555967767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9d41082a0 0x7fd9d410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 --2- 172.21.15.37:0/555967767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41036e0 0x7fd9d4107b90 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 --2- 172.21.15.37:0/555967767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9d4102d30 0x7fd9d4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 -- 172.21.15.37:0/555967767 >> 172.21.15.37:0/555967767 conn(0x7fd9d40fc7d0 msgr2=0x7fd9d40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 -- 172.21.15.37:0/555967767 shutdown_connections 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 -- 172.21.15.37:0/555967767 wait complete. 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.487+0000 7fd9db69e640 1 Processor -- start 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9db69e640 1 -- start start 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9db69e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9d4102d30 0x7fd9d4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9db69e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9d41036e0 0x7fd9d4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9db69e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41082a0 0x7fd9d419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9db69e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d41120c0 con 0x7fd9d41082a0 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9db69e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4111f40 con 0x7fd9d4102d30 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9db69e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fd9d4112240 con 0x7fd9d41036e0 2024-09-19T06:55:03.516 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41082a0 0x7fd9d419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41082a0 0x7fd9d419cef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34836/0 (socket says 172.21.15.37:34836) 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 -- 172.21.15.37:0/4205809886 learned_addr learned my addr 172.21.15.37:0/4205809886 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d943a640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9d4102d30 0x7fd9d4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d8c39640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9d41036e0 0x7fd9d4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 -- 172.21.15.37:0/4205809886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9d41036e0 msgr2=0x7fd9d4198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9d41036e0 0x7fd9d4198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 -- 172.21.15.37:0/4205809886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9d4102d30 msgr2=0x7fd9d4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9d4102d30 0x7fd9d4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d9c3b640 1 -- 172.21.15.37:0/4205809886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9d419d5d0 con 0x7fd9d41082a0 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.488+0000 7fd9d8c39640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9d41036e0 0x7fd9d4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.489+0000 7fd9d9c3b640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41082a0 0x7fd9d419cef0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b970 tx=0x7fd9cc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.489+0000 7fd9d27fc640 1 -- 172.21.15.37:0/4205809886 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc019070 con 0x7fd9d41082a0 2024-09-19T06:55:03.517 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.489+0000 7fd9d27fc640 1 -- 172.21.15.37:0/4205809886 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd9cc00cb40 con 0x7fd9d41082a0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.489+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9d419d8c0 con 0x7fd9d41082a0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.489+0000 7fd9d27fc640 1 -- 172.21.15.37:0/4205809886 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9cc007650 con 0x7fd9d41082a0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.489+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9d41a51a0 con 0x7fd9d41082a0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.490+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9d4103110 con 0x7fd9d41082a0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.491+0000 7fd9d27fc640 1 -- 172.21.15.37:0/4205809886 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fd9cc01c070 con 0x7fd9d41082a0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.491+0000 7fd9d27fc640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9bc0a33b0 0x7fd9bc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.492+0000 7fd9d27fc640 1 -- 172.21.15.37:0/4205809886 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7fd9cc0cf260 con 0x7fd9d41082a0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.492+0000 7fd9d943a640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9bc0a33b0 0x7fd9bc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.492+0000 7fd9d943a640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9bc0a33b0 0x7fd9bc0a5870 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4005d00 tx=0x7fd9c400b410 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:03.518 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.496+0000 7fd9d27fc640 1 -- 172.21.15.37:0/4205809886 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fd9cc09b6a0 con 0x7fd9d41082a0 2024-09-19T06:55:03.551 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:03.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.718+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd9d410a2b0 con 0x7fd9d41082a0 2024-09-19T06:55:03.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.719+0000 7fd9d27fc640 1 -- 172.21.15.37:0/4205809886 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11903 (secure 0 0 0) 0x7fd9cc09f9a0 con 0x7fd9d41082a0 2024-09-19T06:55:03.721 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:55:03.721 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-19T06:54:33:862495+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:54:33.862475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27958":{"gid":27958,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/1806889964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1806889964},{"type":"v1","addr":"172.21.15.151:6839","nonce":1806889964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:55:03.721 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9bc0a33b0 msgr2=0x7fd9bc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9bc0a33b0 0x7fd9bc0a5870 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd9c4005d00 tx=0x7fd9c400b410 comp rx=0 tx=0).stop 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41082a0 msgr2=0x7fd9d419cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41082a0 0x7fd9d419cef0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fd9cc00b970 tx=0x7fd9cc00be30 comp rx=0 tx=0).stop 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 shutdown_connections 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fd9bc0a33b0 0x7fd9bc0a5870 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fd9d41082a0 0x7fd9d419cef0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd9d41036e0 0x7fd9d4198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 --2- 172.21.15.37:0/4205809886 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd9d4102d30 0x7fd9d4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:03.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 >> 172.21.15.37:0/4205809886 conn(0x7fd9d40fc7d0 msgr2=0x7fd9d4105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:03.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 shutdown_connections 2024-09-19T06:55:03.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:03.721+0000 7fd9db69e640 1 -- 172.21.15.37:0/4205809886 wait complete. 2024-09-19T06:55:04.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:03 smithi037 ceph-mon[31211]: pgmap v419: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.3 MiB/s wr, 117 op/s 2024-09-19T06:55:04.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:03 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4205809886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:04.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.61 MB/sec execute 571 sec latency 21044.794 ms 2024-09-19T06:55:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:03 smithi096 ceph-mon[35004]: pgmap v419: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.3 MiB/s wr, 117 op/s 2024-09-19T06:55:04.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:03 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4205809886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:03 smithi151 ceph-mon[35236]: pgmap v419: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 162 KiB/s rd, 2.3 MiB/s wr, 117 op/s 2024-09-19T06:55:04.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:03 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4205809886' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:04.494 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id e6a2e2e8 --release --await' 2024-09-19T06:55:04.494 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id e6a2e2e8 --release --await 2024-09-19T06:55:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 -- 172.21.15.37:0/3472843654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 msgr2=0x7f0f8810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3472843654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f8810da40 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f0f80009ca0 tx=0x7f0f8002a950 comp rx=0 tx=0).stop 2024-09-19T06:55:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 -- 172.21.15.37:0/3472843654 shutdown_connections 2024-09-19T06:55:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3472843654 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f8810da40 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3472843654 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88102570 0x7f0f881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3472843654 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f88100e20 0x7f0f88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 -- 172.21.15.37:0/3472843654 >> 172.21.15.37:0/3472843654 conn(0x7f0f880fc7d0 msgr2=0x7f0f880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 -- 172.21.15.37:0/3472843654 shutdown_connections 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.700+0000 7f0f901f8640 1 -- 172.21.15.37:0/3472843654 wait complete. 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 Processor -- start 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 -- start start 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88100e20 0x7f0f881993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f88102570 0x7f0f881998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f881a4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88110fa0 con 0x7f0f88102ef0 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88110e20 con 0x7f0f88100e20 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.701+0000 7f0f901f8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f88111120 con 0x7f0f88102570 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f881a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f881a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34848/0 (socket says 172.21.15.37:34848) 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 -- 172.21.15.37:0/3393055432 learned_addr learned my addr 172.21.15.37:0/3393055432 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8df94640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88100e20 0x7f0f881993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8d793640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f88102570 0x7f0f881998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 -- 172.21.15.37:0/3393055432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f88102570 msgr2=0x7f0f881998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f88102570 0x7f0f881998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 -- 172.21.15.37:0/3393055432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88100e20 msgr2=0x7f0f881993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88100e20 0x7f0f881993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 -- 172.21.15.37:0/3393055432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f881a4970 con 0x7f0f88102ef0 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8d793640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f88102570 0x7f0f881998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f8e795640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f881a4430 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002ae70 tx=0x7f0f8002f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f7effd640 1 -- 172.21.15.37:0/3393055432 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f80034040 con 0x7f0f88102ef0 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f881a4b40 con 0x7f0f88102ef0 2024-09-19T06:55:04.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f7effd640 1 -- 172.21.15.37:0/3393055432 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f80030920 con 0x7f0f88102ef0 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f7effd640 1 -- 172.21.15.37:0/3393055432 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f800047f0 con 0x7f0f88102ef0 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.702+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f881a4f90 con 0x7f0f88102ef0 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f7effd640 1 -- 172.21.15.37:0/3393055432 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0f80037020 con 0x7f0f88102ef0 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 Processor -- start 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 -- start start 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece55090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece55a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece62760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecdbb2f0 con 0x55c0ece56b40 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecd62cf0 con 0x55c0ece555d0 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f925d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecdb72b0 con 0x55c0ece60370 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f8d793640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece55090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f8d793640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece55090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34852/0 (socket says 172.21.15.37:34852) 2024-09-19T06:55:04.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.705+0000 7f0f8d793640 1 -- 172.21.15.37:0/266494492 learned_addr learned my addr 172.21.15.37:0/266494492 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8df94640 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece55a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f7effd640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f740a33b0 0x7f0f740a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8d793640 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece62760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8e795640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f740a33b0 0x7f0f740a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8d793640 1 -- 172.21.15.37:0/266494492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 msgr2=0x55c0ece62760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8d793640 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece62760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8d793640 1 -- 172.21.15.37:0/266494492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 msgr2=0x55c0ece55a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8d793640 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece55a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8d793640 1 -- 172.21.15.37:0/266494492 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ecd473a0 con 0x55c0ece56b40 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8df94640 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece55a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f7effd640 1 -- 172.21.15.37:0/3393055432 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f0f800f0b40 con 0x7f0f88102ef0 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8d793640 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece55090 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800bfd0 tx=0x7f0f7800b730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88a7e8566c6fc439 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:04.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f73fff640 1 -- 172.21.15.37:0/266494492 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78004030 con 0x55c0ece56b40 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f73fff640 1 -- 172.21.15.37:0/266494492 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f78011c30 con 0x55c0ece56b40 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f8e795640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f740a33b0 0x7f0f740a5870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0f80082bd0 tx=0x7f0f8008bc20 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.706+0000 7f0f73fff640 1 -- 172.21.15.37:0/266494492 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f78003850 con 0x55c0ece56b40 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 -- 172.21.15.37:0/266494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 msgr2=0x55c0ece55090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece55090 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800bfd0 tx=0x7f0f7800b730 comp rx=0 tx=0).stop 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 -- 172.21.15.37:0/266494492 shutdown_connections 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece62760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece55a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 --2- 172.21.15.37:0/266494492 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece55090 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 -- 172.21.15.37:0/266494492 >> 172.21.15.37:0/266494492 conn(0x55c0ece1e310 msgr2=0x55c0ece51c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 -- 172.21.15.37:0/266494492 shutdown_connections 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.707+0000 7f0f925d1740 1 -- 172.21.15.37:0/266494492 wait complete. 2024-09-19T06:55:04.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 -- start start 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 Processor -- start 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece5fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece56b40 0x55c0ece5cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece60370 0x55c0ece5d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c0ece63010 con 0x55c0ece60370 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c0ecd76060 con 0x55c0ece56b40 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0ece63190 con 0x55c0ece555d0 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f925d1740 10 client.0 _ll_register_callbacks cb 0x55c0ecac98c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f8d793640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece60370 0x55c0ece5d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f8df94640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece56b40 0x55c0ece5cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f8d793640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece60370 0x55c0ece5d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34868/0 (socket says 172.21.15.37:34868) 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f8d793640 1 -- 172.21.15.37:0/2103784635 learned_addr learned my addr 172.21.15.37:0/2103784635 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.708+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece5fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.722 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f8df94640 1 -- 172.21.15.37:0/2103784635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 msgr2=0x55c0ece5fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f8df94640 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece5fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f8df94640 1 -- 172.21.15.37:0/2103784635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece60370 msgr2=0x55c0ece5d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f8df94640 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece60370 0x55c0ece5d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f8df94640 1 -- 172.21.15.37:0/2103784635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ecb2c3c0 con 0x55c0ece56b40 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece5fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f8df94640 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece56b40 0x55c0ece5cfc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f84011540 tx=0x7f0f84011a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd4ebd9c8408329 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f717fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f717fa640 1 -- 172.21.15.37:0/2103784635 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84013670 con 0x55c0ece56b40 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f78011c30 con 0x55c0ece56b40 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f717fa640 1 -- 172.21.15.37:0/2103784635 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f0f84013810 con 0x55c0ece56b40 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f717fa640 1 -- 172.21.15.37:0/2103784635 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0f84013a10 con 0x55c0ece56b40 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f925d1740 10 client.28418 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.709+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c0ece72f50 con 0x55c0ece56b40 2024-09-19T06:55:04.723 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f717fa640 1 -- 172.21.15.37:0/2103784635 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f78011c30 con 0x55c0ece56b40 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f717fa640 1 -- 172.21.15.37:0/2103784635 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f0f8401d400 con 0x55c0ece56b40 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f925d1740 10 client.28418 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f925d1740 10 client.28418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f925d1740 20 client.28418 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ece743b0 0x55c0ece767d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f925d1740 4 client.28418 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0ecdc8860 con 0x55c0ece743b0 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ece743b0 0x55c0ece767d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ece743b0 0x55c0ece767d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671236931d32b176 server_cookie=f0b252cfd3f02928 in_seq=0 out_seq=0 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.710+0000 7f0f717fa640 10 client.28418 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.711+0000 7f0f717fa640 1 -- 172.21.15.37:0/2103784635 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0ecdc8860 con 0x55c0ece743b0 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.711+0000 7f0f717fa640 10 client.28418 handle_command_reply: tid=0 2024-09-19T06:55:04.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.711+0000 7f0f925d1740 1 client.28418 shutdown 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.711+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece56b40 msgr2=0x55c0ece5cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.711+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece56b40 0x55c0ece5cfc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0f84011540 tx=0x7f0f84011a00 comp rx=0 tx=0).stop 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 shutdown_connections 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ece743b0 0x55c0ece767d0 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f717fa640 0 client.28418 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece60370 0x55c0ece5d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece56b40 0x55c0ece5cfc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2103784635 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece5fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 >> 172.21.15.37:0/2103784635 conn(0x55c0ece1e310 msgr2=0x55c0ece57930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 shutdown_connections 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 1 -- 172.21.15.37:0/2103784635 wait complete. 2024-09-19T06:55:04.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.712+0000 7f0f925d1740 20 client.28418 trim_cache size 0 max 16384 2024-09-19T06:55:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 Processor -- start 2024-09-19T06:55:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 -- start start 2024-09-19T06:55:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece7c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece7c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881adf60 con 0x55c0ece56b40 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ab830 con 0x55c0ece555d0 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f925d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a9680 con 0x55c0ece60370 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f8df94640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece7c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f8df94640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece7c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34884/0 (socket says 172.21.15.37:34884) 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.725+0000 7f0f8df94640 1 -- 172.21.15.37:0/3632080778 learned_addr learned my addr 172.21.15.37:0/3632080778 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8d793640 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8d793640 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece7c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8df94640 1 -- 172.21.15.37:0/3632080778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 msgr2=0x55c0ece7c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8df94640 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece7c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8df94640 1 -- 172.21.15.37:0/3632080778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 msgr2=0x55c0ece70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8df94640 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece70770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8df94640 1 -- 172.21.15.37:0/3632080778 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0ece5d8d0 con 0x55c0ece56b40 2024-09-19T06:55:04.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f8df94640 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece7c030 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0f84004810 tx=0x7f0f840094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf5051cc071b0bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f737fe640 1 -- 172.21.15.37:0/3632080778 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402a070 con 0x55c0ece56b40 2024-09-19T06:55:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f737fe640 1 -- 172.21.15.37:0/3632080778 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f84013670 con 0x55c0ece56b40 2024-09-19T06:55:04.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.726+0000 7f0f737fe640 1 -- 172.21.15.37:0/3632080778 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84000e60 con 0x55c0ece56b40 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 -- 172.21.15.37:0/3632080778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 msgr2=0x55c0ece7c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece7c030 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f0f84004810 tx=0x7f0f840094f0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 -- 172.21.15.37:0/3632080778 shutdown_connections 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece60370 0x55c0ece7c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece7c030 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 --2- 172.21.15.37:0/3632080778 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece555d0 0x55c0ece70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 -- 172.21.15.37:0/3632080778 >> 172.21.15.37:0/3632080778 conn(0x55c0ece1e310 msgr2=0x55c0ece5c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 -- 172.21.15.37:0/3632080778 shutdown_connections 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 -- 172.21.15.37:0/3632080778 wait complete. 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.727+0000 7f0f925d1740 1 -- start start 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 1 Processor -- start 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece6cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece6d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece60370 0x55c0ece698b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881a9cf0 con 0x55c0ece56b40 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881ab160 con 0x55c0ece60370 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0f881aa3c0 con 0x55c0ece555d0 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f925d1740 10 client.0 _ll_register_callbacks cb 0x7f0f840095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f8df94640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece6d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f8df94640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece6d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34894/0 (socket says 172.21.15.37:34894) 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f8df94640 1 -- 172.21.15.37:0/2646260325 learned_addr learned my addr 172.21.15.37:0/2646260325 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece60370 0x55c0ece698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.728+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece6cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f8df94640 1 -- 172.21.15.37:0/2646260325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 msgr2=0x55c0ece6cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f8df94640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece6cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f8df94640 1 -- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece60370 msgr2=0x55c0ece698b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:04.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f8df94640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece60370 0x55c0ece698b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f8df94640 1 -- 172.21.15.37:0/2646260325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0ece69fa0 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece60370 0x55c0ece698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f8df94640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece6d000 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0f84011a40 tx=0x7f0f84009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8550ccdae2595c2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f717fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f717fa640 1 -- 172.21.15.37:0/2646260325 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402a040 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f717fa640 1 -- 172.21.15.37:0/2646260325 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7f0f84014650 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f84013670 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f717fa640 1 -- 172.21.15.37:0/2646260325 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f0f840059c0 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f925d1740 10 client.19143 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.729+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55c0ecedf9f0 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f717fa640 1 -- 172.21.15.37:0/2646260325 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0f84013670 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f717fa640 1 -- 172.21.15.37:0/2646260325 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f0f84006020 con 0x55c0ece56b40 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f925d1740 10 client.19143 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:04.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f925d1740 10 client.19143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f925d1740 20 client.19143 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ecee0050 0x55c0ecee2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f925d1740 4 client.19143 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:04.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c0ecdc2d70 con 0x55c0ecee0050 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.730+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ecee0050 0x55c0ecee2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f8d793640 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ecee0050 0x55c0ecee2440 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fab1951bd6a64161 server_cookie=aae6a093ead8e346 in_seq=0 out_seq=0 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f717fa640 10 client.19143 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f717fa640 1 -- 172.21.15.37:0/2646260325 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c0ecdc2d70 con 0x55c0ecee0050 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f717fa640 10 client.19143 handle_command_reply: tid=0 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 client.19143 shutdown 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 msgr2=0x55c0ece6d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece6d000 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f0f84011a40 tx=0x7f0f84009110 comp rx=0 tx=0).stop 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 shutdown_connections 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0ecee0050 0x55c0ecee2440 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0ece60370 0x55c0ece698b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f717fa640 0 client.19143 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0ece56b40 0x55c0ece6d000 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 --2- 172.21.15.37:0/2646260325 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0ece555d0 0x55c0ece6cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 >> 172.21.15.37:0/2646260325 conn(0x55c0ece1e310 msgr2=0x55c0ece5e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 shutdown_connections 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.731+0000 7f0f925d1740 1 -- 172.21.15.37:0/2646260325 wait complete. 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f925d1740 20 client.19143 trim_cache size 0 max 16384 2024-09-19T06:55:04.737 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:55:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f740a33b0 msgr2=0x7f0f740a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f740a33b0 0x7f0f740a5870 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0f80082bd0 tx=0x7f0f8008bc20 comp rx=0 tx=0).stop 2024-09-19T06:55:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 msgr2=0x7f0f881a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f881a4430 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f0f8002ae70 tx=0x7f0f8002f5d0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 shutdown_connections 2024-09-19T06:55:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0f740a33b0 0x7f0f740a5870 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.738 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0f88102ef0 0x7f0f881a4430 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0f88102570 0x7f0f881998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 --2- 172.21.15.37:0/3393055432 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0f88100e20 0x7f0f881993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 >> 172.21.15.37:0/3393055432 conn(0x7f0f880fc7d0 msgr2=0x7f0f88106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 shutdown_connections 2024-09-19T06:55:04.739 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:04.732+0000 7f0f901f8640 1 -- 172.21.15.37:0/3393055432 wait complete. 2024-09-19T06:55:05.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1025322 7.60 MB/sec execute 572 sec latency 22044.906 ms 2024-09-19T06:55:05.453 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 -- 172.21.15.37:0/3312277268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec101036c0 msgr2=0x7fec1010bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 --2- 172.21.15.37:0/3312277268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec101036c0 0x7fec1010bbb0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fec00009900 tx=0x7fec0002a480 comp rx=0 tx=0).stop 2024-09-19T06:55:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 -- 172.21.15.37:0/3312277268 shutdown_connections 2024-09-19T06:55:05.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 --2- 172.21.15.37:0/3312277268 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec1010c180 0x7fec1010e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 --2- 172.21.15.37:0/3312277268 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec101036c0 0x7fec1010bbb0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 --2- 172.21.15.37:0/3312277268 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec10102da0 0x7fec10103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 -- 172.21.15.37:0/3312277268 >> 172.21.15.37:0/3312277268 conn(0x7fec100fc7a0 msgr2=0x7fec100febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 -- 172.21.15.37:0/3312277268 shutdown_connections 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.895+0000 7fec16b02640 1 -- 172.21.15.37:0/3312277268 wait complete. 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 Processor -- start 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 -- start start 2024-09-19T06:55:05.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec10102da0 0x7fec101983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec101036c0 0x7fec10198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec1010c180 0x7fec1019cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec10111f60 con 0x7fec10102da0 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec10111de0 con 0x7fec1010c180 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec16b02640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec101120e0 con 0x7fec101036c0 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec10102da0 0x7fec101983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec10102da0 0x7fec101983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34910/0 (socket says 172.21.15.37:34910) 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 -- 172.21.15.37:0/3764093215 learned_addr learned my addr 172.21.15.37:0/3764093215 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec07fff640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec101036c0 0x7fec10198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1509f640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec1010c180 0x7fec1019cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 -- 172.21.15.37:0/3764093215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec101036c0 msgr2=0x7fec10198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec101036c0 0x7fec10198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 -- 172.21.15.37:0/3764093215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec1010c180 msgr2=0x7fec1019cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec1010c180 0x7fec1019cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1489e640 1 -- 172.21.15.37:0/3764093215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec1019d370 con 0x7fec10102da0 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec07fff640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec101036c0 0x7fec10198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.896+0000 7fec1509f640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec1010c180 0x7fec1019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.897+0000 7fec1489e640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec10102da0 0x7fec101983c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00d950 tx=0x7fec0c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.897+0000 7fec05ffb640 1 -- 172.21.15.37:0/3764093215 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c017030 con 0x7fec10102da0 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.897+0000 7fec05ffb640 1 -- 172.21.15.37:0/3764093215 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0c004020 con 0x7fec10102da0 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.897+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec1019d660 con 0x7fec10102da0 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.897+0000 7fec05ffb640 1 -- 172.21.15.37:0/3764093215 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0047a0 con 0x7fec10102da0 2024-09-19T06:55:05.903 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.897+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec101a4f40 con 0x7fec10102da0 2024-09-19T06:55:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.899+0000 7fec05ffb640 1 -- 172.21.15.37:0/3764093215 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fec0c0041d0 con 0x7fec10102da0 2024-09-19T06:55:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 Processor -- start 2024-09-19T06:55:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 -- start start 2024-09-19T06:55:05.909 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319b2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319bf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x555831917620 con 0x5558319b2060 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5558318bee50 con 0x5558319bd240 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec18edb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558319135e0 con 0x5558319b2980 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec05ffb640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7febf80a33b0 0x7febf80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec1509f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec1509f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319b2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34920/0 (socket says 172.21.15.37:34920) 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec1509f640 1 -- 172.21.15.37:0/1925654969 learned_addr learned my addr 172.21.15.37:0/1925654969 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:05.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec1509f640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7febf80a33b0 0x7febf80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec07fff640 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319bf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec05ffb640 1 -- 172.21.15.37:0/3764093215 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7fec0c0cf820 con 0x7fec10102da0 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.900+0000 7fec1489e640 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7fec1509f640 1 -- 172.21.15.37:0/1925654969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 msgr2=0x5558319bcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7fec1509f640 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319bcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7fec1509f640 1 -- 172.21.15.37:0/1925654969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 msgr2=0x5558319bf630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7fec1509f640 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319bf630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7fec1509f640 1 -- 172.21.15.37:0/1925654969 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558318afd40 con 0x5558319b2060 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7fec1509f640 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319b2440 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fec08012d80 tx=0x7fec0800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff57e554c9c6a1a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7fec1509f640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7febf80a33b0 0x7febf80a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec0800fc30 tx=0x7fec0801a000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7febf6ffd640 1 -- 172.21.15.37:0/1925654969 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08020070 con 0x5558319b2060 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7febf6ffd640 1 -- 172.21.15.37:0/1925654969 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0800ab70 con 0x5558319b2060 2024-09-19T06:55:05.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.901+0000 7febf6ffd640 1 -- 172.21.15.37:0/1925654969 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0800d3d0 con 0x5558319b2060 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 -- 172.21.15.37:0/1925654969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 msgr2=0x5558319b2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319b2440 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fec08012d80 tx=0x7fec0800f730 comp rx=0 tx=0).stop 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 -- 172.21.15.37:0/1925654969 shutdown_connections 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319bf630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319bcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 --2- 172.21.15.37:0/1925654969 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319b2440 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 -- 172.21.15.37:0/1925654969 >> 172.21.15.37:0/1925654969 conn(0x5558319ae5f0 msgr2=0x5558319ae9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 -- 172.21.15.37:0/1925654969 shutdown_connections 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 -- 172.21.15.37:0/1925654969 wait complete. 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 -- start start 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.902+0000 7fec18edb740 1 Processor -- start 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec18edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319ba1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.912 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec18edb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319ba6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec18edb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319bd240 0x5558319bac20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec18edb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5558319bfe10 con 0x5558319bd240 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec18edb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55583189ff10 con 0x5558319b2060 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec18edb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5558319bff90 con 0x5558319b2980 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec18edb740 10 client.0 _ll_register_callbacks cb 0x5558318c3db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319bd240 0x5558319bac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319bd240 0x5558319bac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34922/0 (socket says 172.21.15.37:34922) 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 -- 172.21.15.37:0/3718708312 learned_addr learned my addr 172.21.15.37:0/3718708312 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1489e640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319ba1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 -- 172.21.15.37:0/3718708312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 msgr2=0x5558319ba6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec07fff640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319ba6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319ba6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.913 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 -- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 msgr2=0x5558319ba1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319ba1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1509f640 1 -- 172.21.15.37:0/3718708312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555831626580 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.903+0000 7fec1489e640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec1509f640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319bd240 0x5558319bac20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fec08005fd0 tx=0x7fec08012500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e923390c6a7d494 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7febeffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7febeffff640 1 -- 172.21.15.37:0/3718708312 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec08020070 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7febeffff640 1 -- 172.21.15.37:0/3718708312 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7fec0801c410 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec0800ab70 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7febeffff640 1 -- 172.21.15.37:0/3718708312 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fec0801c610 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 10 client.19155 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5558316f0960 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7febeffff640 1 -- 172.21.15.37:0/3718708312 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec0800ab70 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7febeffff640 1 -- 172.21.15.37:0/3718708312 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fec0801b050 con 0x5558319bd240 2024-09-19T06:55:05.914 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 10 client.19155 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 10 client.19155 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 20 client.19155 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558319ccd20 0x5558319cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 4 client.19155 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.904+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555831924b90 con 0x5558319ccd20 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.905+0000 7fec1489e640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558319ccd20 0x5558319cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.905+0000 7febeffff640 10 client.19155 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.905+0000 7fec1489e640 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558319ccd20 0x5558319cf110 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea1a3ea699a911e5 server_cookie=9e0f1bc96b98926b in_seq=0 out_seq=0 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7febeffff640 1 -- 172.21.15.37:0/3718708312 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x555831924b90 con 0x5558319ccd20 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7febeffff640 10 client.19155 handle_command_reply: tid=0 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 client.19155 shutdown 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319bd240 msgr2=0x5558319bac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.915 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319bd240 0x5558319bac20 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7fec08005fd0 tx=0x7fec08012500 comp rx=0 tx=0).stop 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 shutdown_connections 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5558319ccd20 0x5558319cf110 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319bd240 0x5558319bac20 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319ba6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 --2- 172.21.15.37:0/3718708312 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319ba1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7febeffff640 0 client.19155 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 >> 172.21.15.37:0/3718708312 conn(0x5558319ae5f0 msgr2=0x5558319b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 shutdown_connections 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.906+0000 7fec18edb740 1 -- 172.21.15.37:0/3718708312 wait complete. 2024-09-19T06:55:05.916 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.907+0000 7fec18edb740 20 client.19155 trim_cache size 0 max 16384 2024-09-19T06:55:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 Processor -- start 2024-09-19T06:55:05.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 -- start start 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2980 0x5558319d4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319bd240 0x5558319d4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec101b01e0 con 0x5558319b2980 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec101adab0 con 0x5558319b2060 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec18edb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec101ab900 con 0x5558319bd240 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2980 0x5558319d4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2980 0x5558319d4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34928/0 (socket says 172.21.15.37:34928) 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 -- 172.21.15.37:0/2085834075 learned_addr learned my addr 172.21.15.37:0/2085834075 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:05.927 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec1489e640 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec1509f640 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319bd240 0x5558319d4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 -- 172.21.15.37:0/2085834075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319bd240 msgr2=0x5558319d4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319bd240 0x5558319d4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 -- 172.21.15.37:0/2085834075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 msgr2=0x5558319b41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319b41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.924+0000 7fec07fff640 1 -- 172.21.15.37:0/2085834075 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558319bf9a0 con 0x5558319b2980 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.925+0000 7fec07fff640 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2980 0x5558319d4710 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fec00009900 tx=0x7fec00005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=871692681973e02c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.925+0000 7febf67fc640 1 -- 172.21.15.37:0/2085834075 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec00035040 con 0x5558319b2980 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.925+0000 7febf67fc640 1 -- 172.21.15.37:0/2085834075 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec000317b0 con 0x5558319b2980 2024-09-19T06:55:05.928 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.925+0000 7febf67fc640 1 -- 172.21.15.37:0/2085834075 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec000046c0 con 0x5558319b2980 2024-09-19T06:55:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 -- 172.21.15.37:0/2085834075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2980 msgr2=0x5558319d4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.931 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2980 0x5558319d4710 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7fec00009900 tx=0x7fec00005c00 comp rx=0 tx=0).stop 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 -- 172.21.15.37:0/2085834075 shutdown_connections 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319bd240 0x5558319d4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2980 0x5558319d4710 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 --2- 172.21.15.37:0/2085834075 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319b2060 0x5558319b41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 -- 172.21.15.37:0/2085834075 >> 172.21.15.37:0/2085834075 conn(0x5558319ae5f0 msgr2=0x5558319b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 -- 172.21.15.37:0/2085834075 shutdown_connections 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 -- 172.21.15.37:0/2085834075 wait complete. 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 -- start start 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.926+0000 7fec18edb740 1 Processor -- start 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319c54c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319c5a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.932 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319c22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fec101abf70 con 0x5558319b2060 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec101ad3e0 con 0x5558319bd240 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fec101ac640 con 0x5558319b2980 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 10 client.0 _ll_register_callbacks cb 0x7fec08012b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319c54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319c54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34930/0 (socket says 172.21.15.37:34930) 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 -- 172.21.15.37:0/3285969491 learned_addr learned my addr 172.21.15.37:0/3285969491 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec07fff640 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319c5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1509f640 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319c22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 -- 172.21.15.37:0/3285969491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 msgr2=0x5558319c5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319c5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 -- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 msgr2=0x5558319c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:05.933 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319c22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 -- 172.21.15.37:0/3285969491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558319c2a70 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec1489e640 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319c54c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009610 tx=0x7fec0c097df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7929b56fdde886e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7febeffff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7febeffff640 1 -- 172.21.15.37:0/3285969491 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec0c0a4710 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7febeffff640 1 -- 172.21.15.37:0/3285969491 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8527+0+0 (secure 0 0 0) 0x7fec0c0860b0 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.927+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec000317b0 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7febeffff640 1 -- 172.21.15.37:0/3285969491 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7fec0c0ba300 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 10 client.19161 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x555831a3c760 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7febeffff640 1 -- 172.21.15.37:0/3285969491 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec000317b0 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7febeffff640 1 -- 172.21.15.37:0/3285969491 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7fec0c0a4280 con 0x5558319b2060 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 10 client.19161 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:05.934 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 10 client.19161 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:55:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 20 client.19161 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555831a3ce80 0x555831a3f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 4 client.19161 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "e6a2e2e8", "release": true, "await": true}] 2024-09-19T06:55:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "e6a2e2e8", "release": true, "await": true}) -- 0x55583191f0a0 con 0x555831a3ce80 2024-09-19T06:55:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.928+0000 7fec07fff640 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555831a3ce80 0x555831a3f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.929+0000 7fec07fff640 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555831a3ce80 0x555831a3f2a0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c66971714f8af9ac server_cookie=987570a4aa9966a0 in_seq=0 out_seq=0 2024-09-19T06:55:05.935 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.929+0000 7febeffff640 10 client.19161 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7febeffff640 1 -- 172.21.15.37:0/3285969491 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x55583191f0a0 con 0x555831a3ce80 2024-09-19T06:55:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7febeffff640 10 client.19161 handle_command_reply: tid=0 2024-09-19T06:55:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 client.19161 shutdown 2024-09-19T06:55:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 msgr2=0x5558319c54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319c54c0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7fec0c009610 tx=0x7fec0c097df0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 shutdown_connections 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x555831a3ce80 0x555831a3f2a0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5558319bd240 0x5558319c22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5558319b2980 0x5558319c5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 --2- 172.21.15.37:0/3285969491 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5558319b2060 0x5558319c54c0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 >> 172.21.15.37:0/3285969491 conn(0x5558319ae5f0 msgr2=0x5558319b42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7febeffff640 0 client.19161 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 shutdown_connections 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 1 -- 172.21.15.37:0/3285969491 wait complete. 2024-09-19T06:55:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.938+0000 7fec18edb740 20 client.19161 trim_cache size 0 max 16384 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 43, 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 44, 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "e6a2e2e8": { 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 44, 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:55:05.942 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 142.2, 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:05.943 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:05.944 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:05.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7febf80a33b0 msgr2=0x7febf80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7febf80a33b0 0x7febf80a5870 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fec0800fc30 tx=0x7fec0801a000 comp rx=0 tx=0).stop 2024-09-19T06:55:05.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec10102da0 msgr2=0x7fec101983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:05.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec10102da0 0x7fec101983c0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fec0c00d950 tx=0x7fec0c00de10 comp rx=0 tx=0).stop 2024-09-19T06:55:05.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 shutdown_connections 2024-09-19T06:55:05.944 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec1010c180 0x7fec1019cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fec101036c0 0x7fec10198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7febf80a33b0 0x7febf80a5870 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 --2- 172.21.15.37:0/3764093215 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fec10102da0 0x7fec101983c0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 >> 172.21.15.37:0/3764093215 conn(0x7fec100fc7a0 msgr2=0x7fec100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 shutdown_connections 2024-09-19T06:55:05.945 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:05.939+0000 7fec16b02640 1 -- 172.21.15.37:0/3764093215 wait complete. 2024-09-19T06:55:06.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:05 smithi037 ceph-mon[31211]: pgmap v420: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-19T06:55:06.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1026153 7.60 MB/sec execute 573 sec latency 22750.734 ms 2024-09-19T06:55:06.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:05 smithi096 ceph-mon[35004]: pgmap v420: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-19T06:55:06.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:05 smithi151 ceph-mon[35236]: pgmap v420: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-19T06:55:06.408 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:55:06.408 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:55:06.717 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'e6a2e2e8', total seconds elapsed: 26.5 2024-09-19T06:55:06.717 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 22.2 2024-09-19T06:55:06.717 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:55:06.760 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:07.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1027081 7.59 MB/sec execute 574 sec latency 90.165 ms 2024-09-19T06:55:07.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.289+0000 7ffa98d8b640 1 -- 172.21.15.37:0/1332459238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109830 msgr2=0x7ffa94109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.289+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/1332459238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109830 0x7ffa94109c90 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009900 tx=0x7ffa8802a420 comp rx=0 tx=0).stop 2024-09-19T06:55:07.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.290+0000 7ffa98d8b640 1 -- 172.21.15.37:0/1332459238 shutdown_connections 2024-09-19T06:55:07.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.290+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/1332459238 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa9411b140 0x7ffa9411d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.290+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/1332459238 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109830 0x7ffa94109c90 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7ffa88009900 tx=0x7ffa8802a420 comp rx=0 tx=0).stop 2024-09-19T06:55:07.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.290+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/1332459238 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa94109450 0x7ffa94110250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.290+0000 7ffa98d8b640 1 -- 172.21.15.37:0/1332459238 >> 172.21.15.37:0/1332459238 conn(0x7ffa9406e140 msgr2=0x7ffa9406e550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:07.293 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.291+0000 7ffa98d8b640 1 -- 172.21.15.37:0/1332459238 shutdown_connections 2024-09-19T06:55:07.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.293+0000 7ffa98d8b640 1 -- 172.21.15.37:0/1332459238 wait complete. 2024-09-19T06:55:07.295 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.294+0000 7ffa98d8b640 1 Processor -- start 2024-09-19T06:55:07.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa98d8b640 1 -- start start 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa98d8b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109450 0x7ffa9411a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa98d8b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9411b140 0x7ffa94113930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa98d8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa94113e70 0x7ffa94114320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa98d8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94120da0 con 0x7ffa94113e70 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa98d8b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94120c20 con 0x7ffa94109450 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa98d8b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa94120f20 con 0x7ffa9411b140 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.297+0000 7ffa9259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109450 0x7ffa9411a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.298+0000 7ffa9259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109450 0x7ffa9411a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54682/0 (socket says 172.21.15.37:54682) 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.298+0000 7ffa9259c640 1 -- 172.21.15.37:0/2722171767 learned_addr learned my addr 172.21.15.37:0/2722171767 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:07.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.298+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9411b140 0x7ffa94113930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.298+0000 7ffa9259c640 1 -- 172.21.15.37:0/2722171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9411b140 msgr2=0x7ffa94113930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.299+0000 7ffa9259c640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9411b140 0x7ffa94113930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.299+0000 7ffa9259c640 1 -- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa94113e70 msgr2=0x7ffa94114320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.299+0000 7ffa9259c640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa94113e70 0x7ffa94114320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.299+0000 7ffa9259c640 1 -- 172.21.15.37:0/2722171767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa94118450 con 0x7ffa94109450 2024-09-19T06:55:07.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.299+0000 7ffa9259c640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109450 0x7ffa9411a8a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400c890 tx=0x7ffa8400cd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:07.302 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.300+0000 7ffa837fe640 1 -- 172.21.15.37:0/2722171767 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84004220 con 0x7ffa94109450 2024-09-19T06:55:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.300+0000 7ffa837fe640 1 -- 172.21.15.37:0/2722171767 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa840043c0 con 0x7ffa94109450 2024-09-19T06:55:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.300+0000 7ffa837fe640 1 -- 172.21.15.37:0/2722171767 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840049e0 con 0x7ffa94109450 2024-09-19T06:55:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.302+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa94118740 con 0x7ffa94109450 2024-09-19T06:55:07.303 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.302+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa94118c00 con 0x7ffa94109450 2024-09-19T06:55:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.306+0000 7ffa9b164740 1 Processor -- start 2024-09-19T06:55:07.310 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.307+0000 7ffa9b164740 1 -- start start 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.307+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06b9ffb60 0x55f06b9fff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.307+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba00480 0x55f06ba0a800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.307+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba0ad40 0x55f06ba0d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.307+0000 7ffa9b164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f06b965280 con 0x55f06ba00480 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.307+0000 7ffa9b164740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f06b90cb10 con 0x55f06b9ffb60 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.307+0000 7ffa9b164740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f06b961240 con 0x55f06ba0ad40 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.308+0000 7ffa837fe640 1 -- 172.21.15.37:0/2722171767 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7ffa840079b0 con 0x7ffa94109450 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.309+0000 7ffa91d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba0ad40 0x55f06ba0d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.309+0000 7ffa91d9b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba0ad40 0x55f06ba0d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:34900/0 (socket says 172.21.15.37:34900) 2024-09-19T06:55:07.311 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.309+0000 7ffa91d9b640 1 -- 172.21.15.37:0/3636966989 learned_addr learned my addr 172.21.15.37:0/3636966989 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.309+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06b9ffb60 0x55f06b9fff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.310+0000 7ffa91d9b640 1 -- 172.21.15.37:0/3636966989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06b9ffb60 msgr2=0x55f06b9fff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.310+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06b9ffb60 0x55f06b9fff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.310+0000 7ffa91d9b640 1 -- 172.21.15.37:0/3636966989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba00480 msgr2=0x55f06ba0a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.310+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba00480 0x55f06ba0a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.312 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.310+0000 7ffa91d9b640 1 -- 172.21.15.37:0/3636966989 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06b9191f0 con 0x55f06ba0ad40 2024-09-19T06:55:07.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.311+0000 7ffa837fe640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa7c0a3580 0x7ffa7c0a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.311+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba0ad40 0x55f06ba0d130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa88000c00 tx=0x7ffa88031630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ca53ff4b780f5fc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:07.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.312+0000 7ffa837fe640 1 -- 172.21.15.37:0/2722171767 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7ffa840cfd10 con 0x7ffa94109450 2024-09-19T06:55:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.312+0000 7ffa80ff9640 1 -- 172.21.15.37:0/3636966989 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88035020 con 0x55f06ba0ad40 2024-09-19T06:55:07.315 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.313+0000 7ffa80ff9640 1 -- 172.21.15.37:0/3636966989 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa88004040 con 0x55f06ba0ad40 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.314+0000 7ffa9b164740 1 -- 172.21.15.37:0/3636966989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba0ad40 msgr2=0x55f06ba0d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.314+0000 7ffa9b164740 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba0ad40 0x55f06ba0d130 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa88000c00 tx=0x7ffa88031630 comp rx=0 tx=0).stop 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.314+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa7c0a3580 0x7ffa7c0a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.315+0000 7ffa9b164740 1 -- 172.21.15.37:0/3636966989 shutdown_connections 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.315+0000 7ffa9b164740 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba0ad40 0x55f06ba0d130 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ffa88000c00 tx=0x7ffa88031630 comp rx=0 tx=0).stop 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.315+0000 7ffa9b164740 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba00480 0x55f06ba0a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.315+0000 7ffa9b164740 1 --2- 172.21.15.37:0/3636966989 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06b9ffb60 0x55f06b9fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.315+0000 7ffa9b164740 1 -- 172.21.15.37:0/3636966989 >> 172.21.15.37:0/3636966989 conn(0x55f06b9fc0f0 msgr2=0x55f06b9fc4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:07.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.315+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa7c0a3580 0x7ffa7c0a5a40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c005c40 tx=0x7ffa8c005bd0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:07.319 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.317+0000 7ffa9b164740 1 -- 172.21.15.37:0/3636966989 shutdown_connections 2024-09-19T06:55:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.318+0000 7ffa9b164740 1 -- 172.21.15.37:0/3636966989 wait complete. 2024-09-19T06:55:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.319+0000 7ffa9b164740 1 -- start start 2024-09-19T06:55:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.319+0000 7ffa9b164740 1 Processor -- start 2024-09-19T06:55:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.319+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba07e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba08360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba16e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.322 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa9b164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f06ba0d910 con 0x55f06b9ffb60 2024-09-19T06:55:07.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa9b164740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f06b91ff10 con 0x55f06ba088a0 2024-09-19T06:55:07.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa9b164740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f06ba0da90 con 0x55f06ba00480 2024-09-19T06:55:07.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa9b164740 10 client.0 _ll_register_callbacks cb 0x55f06b8e8a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:07.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa92d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba16e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa92d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba16e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54700/0 (socket says 172.21.15.37:54700) 2024-09-19T06:55:07.323 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.320+0000 7ffa92d9d640 1 -- 172.21.15.37:0/958550191 learned_addr learned my addr 172.21.15.37:0/958550191 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:07.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.322+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba08360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.324+0000 7ffa92d9d640 1 -- 172.21.15.37:0/958550191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 msgr2=0x55f06ba08360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.324+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba08360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.324+0000 7ffa92d9d640 1 -- 172.21.15.37:0/958550191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 msgr2=0x55f06ba07e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.324+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba07e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.324+0000 7ffa92d9d640 1 -- 172.21.15.37:0/958550191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f06b6d66a0 con 0x55f06ba088a0 2024-09-19T06:55:07.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.329+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba16e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c011c00 tx=0x7ffa8c004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bec3503f169c40b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:07.330 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.329+0000 7ffa79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:55:07.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.330+0000 7ffa79ffb640 1 -- 172.21.15.37:0/958550191 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c00ed90 con 0x55f06ba088a0 2024-09-19T06:55:07.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.330+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f06b9727f0 con 0x55f06ba088a0 2024-09-19T06:55:07.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.330+0000 7ffa79ffb640 1 -- 172.21.15.37:0/958550191 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7ffa8c003970 con 0x55f06ba088a0 2024-09-19T06:55:07.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.330+0000 7ffa79ffb640 1 -- 172.21.15.37:0/958550191 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ffa8c003bb0 con 0x55f06ba088a0 2024-09-19T06:55:07.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.330+0000 7ffa9b164740 10 client.28448 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:07.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.330+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f06ba19d50 con 0x55f06ba088a0 2024-09-19T06:55:07.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.331+0000 7ffa79ffb640 1 -- 172.21.15.37:0/958550191 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f06b9727f0 con 0x55f06ba088a0 2024-09-19T06:55:07.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.331+0000 7ffa79ffb640 1 -- 172.21.15.37:0/958550191 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7ffa8c01e4d0 con 0x55f06ba088a0 2024-09-19T06:55:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.332+0000 7ffa9b164740 10 client.28448 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.332+0000 7ffa9b164740 10 client.28448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.332+0000 7ffa9b164740 20 client.28448 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.332+0000 7ffa9b164740 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba1b630 0x55f06ba1dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.332+0000 7ffa9b164740 4 client.28448 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:07.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.332+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f06b96cd00 con 0x55f06ba1b630 2024-09-19T06:55:07.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.333+0000 7ffa9259c640 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba1b630 0x55f06ba1dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.340 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.339+0000 7ffa79ffb640 10 client.28448 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:07.341 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.339+0000 7ffa9259c640 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba1b630 0x55f06ba1dac0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a363d84b528d9b77 server_cookie=cee40f1481a73e22 in_seq=0 out_seq=0 2024-09-19T06:55:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.343+0000 7ffa79ffb640 1 -- 172.21.15.37:0/958550191 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f06b96cd00 con 0x55f06ba1b630 2024-09-19T06:55:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.343+0000 7ffa79ffb640 10 client.28448 handle_command_reply: tid=0 2024-09-19T06:55:07.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.343+0000 7ffa9b164740 1 client.28448 shutdown 2024-09-19T06:55:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 msgr2=0x55f06ba16e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba16e00 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c011c00 tx=0x7ffa8c004a40 comp rx=0 tx=0).stop 2024-09-19T06:55:07.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 shutdown_connections 2024-09-19T06:55:07.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba1b630 0x55f06ba1dac0 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba16e00 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba08360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 --2- 172.21.15.37:0/958550191 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba07e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.346+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 >> 172.21.15.37:0/958550191 conn(0x55f06b9fc0f0 msgr2=0x55f06ba01e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:07.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.347+0000 7ffa79ffb640 0 client.28448 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:07.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.347+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 shutdown_connections 2024-09-19T06:55:07.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.347+0000 7ffa9b164740 1 -- 172.21.15.37:0/958550191 wait complete. 2024-09-19T06:55:07.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.347+0000 7ffa9b164740 20 client.28448 trim_cache size 0 max 16384 2024-09-19T06:55:07.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.383+0000 7ffa9b164740 1 Processor -- start 2024-09-19T06:55:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.383+0000 7ffa9b164740 1 -- start start 2024-09-19T06:55:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba16bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba066f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba06c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9b164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941c7f30 con 0x55f06b9ffb60 2024-09-19T06:55:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9b164740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941c5800 con 0x55f06ba088a0 2024-09-19T06:55:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba16bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9259c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba16bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:34964/0 (socket says 172.21.15.37:34964) 2024-09-19T06:55:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa9259c640 1 -- 172.21.15.37:0/2384174451 learned_addr learned my addr 172.21.15.37:0/2384174451 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.384+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba06c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa9b164740 1 -- 172.21.15.37:0/2384174451 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941c3670 con 0x55f06ba00480 2024-09-19T06:55:07.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba066f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa92d9d640 1 -- 172.21.15.37:0/2384174451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 msgr2=0x55f06ba066f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba066f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa92d9d640 1 -- 172.21.15.37:0/2384174451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 msgr2=0x55f06ba16bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba16bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa92d9d640 1 -- 172.21.15.37:0/2384174451 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f06ba002b0 con 0x55f06ba088a0 2024-09-19T06:55:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:07.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.385+0000 7ffa92d9d640 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba06c30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c01e8c0 tx=0x7ffa8c00e790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32ff0156bb669fc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.386+0000 7ffa7bfff640 1 -- 172.21.15.37:0/2384174451 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c03d070 con 0x55f06ba088a0 2024-09-19T06:55:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.386+0000 7ffa7bfff640 1 -- 172.21.15.37:0/2384174451 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffa8c009d80 con 0x55f06ba088a0 2024-09-19T06:55:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.386+0000 7ffa7bfff640 1 -- 172.21.15.37:0/2384174451 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c032040 con 0x55f06ba088a0 2024-09-19T06:55:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.387+0000 7ffa9b164740 1 -- 172.21.15.37:0/2384174451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 msgr2=0x55f06ba06c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.387+0000 7ffa9b164740 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba06c30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c01e8c0 tx=0x7ffa8c00e790 comp rx=0 tx=0).stop 2024-09-19T06:55:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.387+0000 7ffa9b164740 1 -- 172.21.15.37:0/2384174451 shutdown_connections 2024-09-19T06:55:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.387+0000 7ffa9b164740 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba088a0 0x55f06ba06c30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.387+0000 7ffa9b164740 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06ba00480 0x55f06ba066f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.387+0000 7ffa9b164740 1 --2- 172.21.15.37:0/2384174451 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06b9ffb60 0x55f06ba16bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.387+0000 7ffa9b164740 1 -- 172.21.15.37:0/2384174451 >> 172.21.15.37:0/2384174451 conn(0x55f06b9fc0f0 msgr2=0x55f06ba03100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.388+0000 7ffa9b164740 1 -- 172.21.15.37:0/2384174451 shutdown_connections 2024-09-19T06:55:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.388+0000 7ffa9b164740 1 -- 172.21.15.37:0/2384174451 wait complete. 2024-09-19T06:55:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.389+0000 7ffa9b164740 1 -- start start 2024-09-19T06:55:07.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.389+0000 7ffa9b164740 1 Processor -- start 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06b9ffb60 0x55f06ba0dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba00480 0x55f06ba0e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa9b164740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba088a0 0x55f06ba0e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa9b164740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941c3cc0 con 0x55f06ba088a0 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa91d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba00480 0x55f06ba0e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa91d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba00480 0x55f06ba0e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54720/0 (socket says 172.21.15.37:54720) 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa91d9b640 1 -- 172.21.15.37:0/4091735713 learned_addr learned my addr 172.21.15.37:0/4091735713 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941c5130 con 0x55f06ba00480 2024-09-19T06:55:07.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.390+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7ffa941c4390 con 0x55f06b9ffb60 2024-09-19T06:55:07.393 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.391+0000 7ffa9b164740 10 client.0 _ll_register_callbacks cb 0x7ffa94110870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:07.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.391+0000 7ffa91d9b640 1 -- 172.21.15.37:0/4091735713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06b9ffb60 msgr2=0x55f06ba0dd60 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:55:07.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.391+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06b9ffb60 0x55f06ba0dd60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.391+0000 7ffa91d9b640 1 -- 172.21.15.37:0/4091735713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba088a0 msgr2=0x55f06ba0e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:07.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.391+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba088a0 0x55f06ba0e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.394 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.391+0000 7ffa91d9b640 1 -- 172.21.15.37:0/4091735713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f06ba84650 con 0x55f06ba00480 2024-09-19T06:55:07.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.391+0000 7ffa91d9b640 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba00480 0x55f06ba0e2a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802a460 tx=0x7ffa88031ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd5951c910aee23a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:07.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.392+0000 7ffa79ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:55:07.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.392+0000 7ffa79ffb640 1 -- 172.21.15.37:0/4091735713 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa88054060 con 0x55f06ba00480 2024-09-19T06:55:07.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.392+0000 7ffa79ffb640 1 -- 172.21.15.37:0/4091735713 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7ffa8803d860 con 0x55f06ba00480 2024-09-19T06:55:07.395 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.393+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f06b8e9110 con 0x55f06ba00480 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.393+0000 7ffa79ffb640 1 -- 172.21.15.37:0/4091735713 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7ffa88035040 con 0x55f06ba00480 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.393+0000 7ffa9b164740 10 client.28454 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.393+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55f06ba85900 con 0x55f06ba00480 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.395+0000 7ffa79ffb640 1 -- 172.21.15.37:0/4091735713 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55f06b8e9110 con 0x55f06ba00480 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.395+0000 7ffa79ffb640 1 -- 172.21.15.37:0/4091735713 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7ffa8803a040 con 0x55f06ba00480 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.396+0000 7ffa9b164740 10 client.28454 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.396+0000 7ffa9b164740 10 client.28454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.396+0000 7ffa9b164740 20 client.28454 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.396+0000 7ffa9b164740 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba85b90 0x55f06ba87fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:07.398 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.396+0000 7ffa9b164740 4 client.28454 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:55:07.399 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.396+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f06b936940 con 0x55f06ba85b90 2024-09-19T06:55:07.400 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.398+0000 7ffa9259c640 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba85b90 0x55f06ba87fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.402+0000 7ffa9259c640 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba85b90 0x55f06ba87fa0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45d4dafa9b96ca23 server_cookie=11045b602604dcdb in_seq=0 out_seq=0 2024-09-19T06:55:07.404 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.402+0000 7ffa79ffb640 10 client.28454 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:07.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.412+0000 7ffa79ffb640 1 -- 172.21.15.37:0/4091735713 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f06b936940 con 0x55f06ba85b90 2024-09-19T06:55:07.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.412+0000 7ffa79ffb640 10 client.28454 handle_command_reply: tid=0 2024-09-19T06:55:07.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.412+0000 7ffa9b164740 1 client.28454 shutdown 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.413+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba00480 msgr2=0x55f06ba0e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.413+0000 7ffa9b164740 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba00480 0x55f06ba0e2a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ffa8802a460 tx=0x7ffa88031ab0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.413+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 shutdown_connections 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.413+0000 7ffa9b164740 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f06ba85b90 0x55f06ba87fa0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.413+0000 7ffa9b164740 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f06ba088a0 0x55f06ba0e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.413+0000 7ffa79ffb640 0 client.28454 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.413+0000 7ffa9b164740 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f06ba00480 0x55f06ba0e2a0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.414+0000 7ffa9b164740 1 --2- 172.21.15.37:0/4091735713 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f06b9ffb60 0x55f06ba0dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.414+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 >> 172.21.15.37:0/4091735713 conn(0x55f06b9fc0f0 msgr2=0x55f06ba066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.414+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 shutdown_connections 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.414+0000 7ffa9b164740 1 -- 172.21.15.37:0/4091735713 wait complete. 2024-09-19T06:55:07.416 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.414+0000 7ffa9b164740 20 client.28454 trim_cache size 0 max 16384 2024-09-19T06:55:07.417 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:07.417 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:55:07.417 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "eb5d42ad-afda-4e57-a55f-b02f1f8ecb54", 2024-09-19T06:55:07.417 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:55:07.417 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:07.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa7c0a3580 msgr2=0x7ffa7c0a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa7c0a3580 0x7ffa7c0a5a40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c005c40 tx=0x7ffa8c005bd0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109450 msgr2=0x7ffa9411a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:07.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109450 0x7ffa9411a8a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400c890 tx=0x7ffa8400cd50 comp rx=0 tx=0).stop 2024-09-19T06:55:07.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 shutdown_connections 2024-09-19T06:55:07.418 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7ffa7c0a3580 0x7ffa7c0a5a40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7ffa94113e70 0x7ffa94114320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.416+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffa9411b140 0x7ffa94113930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.417+0000 7ffa98d8b640 1 --2- 172.21.15.37:0/2722171767 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa94109450 0x7ffa9411a8a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:07.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.417+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 >> 172.21.15.37:0/2722171767 conn(0x7ffa9406e140 msgr2=0x7ffa94070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:07.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.417+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 shutdown_connections 2024-09-19T06:55:07.419 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:07.417+0000 7ffa98d8b640 1 -- 172.21.15.37:0/2722171767 wait complete. 2024-09-19T06:55:08.220 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:07 smithi037 ceph-mon[31211]: pgmap v421: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-19T06:55:08.221 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:07 smithi037 ceph-mon[31211]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T06:55:08.237 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1027572 7.58 MB/sec execute 575 sec latency 56.320 ms 2024-09-19T06:55:08.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:07 smithi096 ceph-mon[35004]: pgmap v421: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-19T06:55:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:07 smithi096 ceph-mon[35004]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T06:55:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:07 smithi151 ceph-mon[35236]: pgmap v421: 97 pgs: 97 active+clean; 67 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 1.6 MiB/s wr, 88 op/s 2024-09-19T06:55:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:07 smithi151 ceph-mon[35236]: osdmap e103: 12 total, 12 up, 12 in 2024-09-19T06:55:08.557 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 -- 172.21.15.37:0/4132670447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578106f20 msgr2=0x7f757810eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 --2- 172.21.15.37:0/4132670447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578106f20 0x7f757810eaa0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f7570009ca0 tx=0x7f757002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 -- 172.21.15.37:0/4132670447 shutdown_connections 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 --2- 172.21.15.37:0/4132670447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578106f20 0x7f757810eaa0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 --2- 172.21.15.37:0/4132670447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f75781027a0 0x7f75781067f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 --2- 172.21.15.37:0/4132670447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7578101df0 0x7f75781021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 -- 172.21.15.37:0/4132670447 >> 172.21.15.37:0/4132670447 conn(0x7f75780fd8f0 msgr2=0x7f75780ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 -- 172.21.15.37:0/4132670447 shutdown_connections 2024-09-19T06:55:08.765 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.757+0000 7f757d255640 1 -- 172.21.15.37:0/4132670447 wait complete. 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.758+0000 7f757d255640 1 Processor -- start 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.758+0000 7f757d255640 1 -- start start 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.758+0000 7f757d255640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578101df0 0x7f75781983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.758+0000 7f757d255640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75781027a0 0x7f7578198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757d255640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7578106f20 0x7f757819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757d255640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7578111fd0 con 0x7f7578101df0 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757d255640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7578111e50 con 0x7f75781027a0 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757d255640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7578112150 con 0x7f7578106f20 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75781027a0 0x7f7578198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f7576d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578101df0 0x7f75781983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75781027a0 0x7f7578198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:54726/0 (socket says 172.21.15.37:54726) 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 -- 172.21.15.37:0/1163166994 learned_addr learned my addr 172.21.15.37:0/1163166994 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:08.766 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757759e640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7578106f20 0x7f757819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 -- 172.21.15.37:0/1163166994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7578106f20 msgr2=0x7f757819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7578106f20 0x7f757819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 -- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578101df0 msgr2=0x7f75781983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578101df0 0x7f75781983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 -- 172.21.15.37:0/1163166994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f757819d380 con 0x7f75781027a0 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f7576d9d640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578101df0 0x7f75781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.759+0000 7f757659c640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75781027a0 0x7f7578198910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f756c00d770 tx=0x7f756c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.760+0000 7f755ffff640 1 -- 172.21.15.37:0/1163166994 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c017070 con 0x7f75781027a0 2024-09-19T06:55:08.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.760+0000 7f755ffff640 1 -- 172.21.15.37:0/1163166994 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f756c005500 con 0x7f75781027a0 2024-09-19T06:55:08.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.760+0000 7f755ffff640 1 -- 172.21.15.37:0/1163166994 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c004e00 con 0x7f75781027a0 2024-09-19T06:55:08.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.760+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f757819d670 con 0x7f75781027a0 2024-09-19T06:55:08.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.760+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75781a4f50 con 0x7f75781027a0 2024-09-19T06:55:08.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.761+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75781021d0 con 0x7f75781027a0 2024-09-19T06:55:08.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.763+0000 7f755ffff640 1 -- 172.21.15.37:0/1163166994 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f756c004fa0 con 0x7f75781027a0 2024-09-19T06:55:08.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.764+0000 7f755ffff640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75600a3430 0x7f75600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:08.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.764+0000 7f755ffff640 1 -- 172.21.15.37:0/1163166994 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f756c0cf670 con 0x7f75781027a0 2024-09-19T06:55:08.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.764+0000 7f7576d9d640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75600a3430 0x7f75600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:08.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.764+0000 7f7576d9d640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75600a3430 0x7f75600a58f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7568009fc0 tx=0x7f7568005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:08.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:08.768+0000 7f755ffff640 1 -- 172.21.15.37:0/1163166994 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f756c09b9a0 con 0x7f75781027a0 2024-09-19T06:55:08.905 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:09.021 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.019+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f757810a100 con 0x7f75781027a0 2024-09-19T06:55:09.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:08 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:55:09.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:08 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.022 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:08 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:09.022 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:08 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:09.022 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:08 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.022 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:08 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.022 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.020+0000 7f755ffff640 1 -- 172.21.15.37:0/1163166994 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) ==== 94+0+11903 (secure 0 0 0) 0x7f756c09fca0 con 0x7f75781027a0 2024-09-19T06:55:09.022 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:55:09.023 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":43,"btime":"2024-09-19T06:54:33:862495+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:54:33.862475+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_27958":{"gid":27958,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/1806889964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":1806889964},{"type":"v1","addr":"172.21.15.151:6839","nonce":1806889964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:55:09.023 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 43 2024-09-19T06:55:09.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.022+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75600a3430 msgr2=0x7f75600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:09.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.022+0000 7f757d255640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75600a3430 0x7f75600a58f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f7568009fc0 tx=0x7f7568005f90 comp rx=0 tx=0).stop 2024-09-19T06:55:09.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.022+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75781027a0 msgr2=0x7f7578198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:09.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.022+0000 7f757d255640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75781027a0 0x7f7578198910 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f756c00d770 tx=0x7f756c00dc30 comp rx=0 tx=0).stop 2024-09-19T06:55:09.024 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 shutdown_connections 2024-09-19T06:55:09.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7578106f20 0x7f757819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:09.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f75781027a0 0x7f7578198910 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:09.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f75600a3430 0x7f75600a58f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:09.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 --2- 172.21.15.37:0/1163166994 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7578101df0 0x7f75781983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:09.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 >> 172.21.15.37:0/1163166994 conn(0x7f75780fd8f0 msgr2=0x7f75780fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:09.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 shutdown_connections 2024-09-19T06:55:09.025 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:09.023+0000 7f757d255640 1 -- 172.21.15.37:0/1163166994 wait complete. 2024-09-19T06:55:09.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1029657 7.59 MB/sec execute 576 sec latency 38.948 ms 2024-09-19T06:55:09.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:08 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:55:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:08 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:08 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:08 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:08 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:08 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:08 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:55:09.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:08 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:08 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:08 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:08 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:09.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:08 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:10.050 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0' 2024-09-19T06:55:10.050 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db / --timeout 90 --expiration 142.2 --await-for 45.0 2024-09-19T06:55:10.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1031168 7.59 MB/sec execute 577 sec latency 17.272 ms 2024-09-19T06:55:10.265 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:09 smithi037 ceph-mon[31211]: pgmap v423: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-09-19T06:55:10.266 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:09 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1163166994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:10.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:09 smithi096 ceph-mon[35004]: pgmap v423: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-09-19T06:55:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:09 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1163166994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:09 smithi151 ceph-mon[35236]: pgmap v423: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-09-19T06:55:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:09 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1163166994' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:10.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 -- 172.21.15.37:0/692206625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1082a0 msgr2=0x7f34ec10ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 --2- 172.21.15.37:0/692206625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1082a0 0x7f34ec10ec40 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f34e4009cd0 tx=0x7f34e402a910 comp rx=0 tx=0).stop 2024-09-19T06:55:10.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 -- 172.21.15.37:0/692206625 shutdown_connections 2024-09-19T06:55:10.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 --2- 172.21.15.37:0/692206625 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1082a0 0x7f34ec10ec40 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 --2- 172.21.15.37:0/692206625 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34ec1036e0 0x7f34ec107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 --2- 172.21.15.37:0/692206625 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 0x7f34ec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 -- 172.21.15.37:0/692206625 >> 172.21.15.37:0/692206625 conn(0x7f34ec0fc7d0 msgr2=0x7f34ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:10.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.363+0000 7f34f2652640 1 -- 172.21.15.37:0/692206625 shutdown_connections 2024-09-19T06:55:10.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.364+0000 7f34f2652640 1 -- 172.21.15.37:0/692206625 wait complete. 2024-09-19T06:55:10.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34f2652640 1 Processor -- start 2024-09-19T06:55:10.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34f2652640 1 -- start start 2024-09-19T06:55:10.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34f2652640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 0x7f34ec1a05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34f2652640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1036e0 0x7f34ec1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1036e0 0x7f34ec1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 0x7f34ec1a05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 0x7f34ec1a05c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:56000/0 (socket says 172.21.15.37:56000) 2024-09-19T06:55:10.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1036e0 0x7f34ec1a0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59828/0 (socket says 172.21.15.37:59828) 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.365+0000 7f34ebfff640 1 -- 172.21.15.37:0/709071253 learned_addr learned my addr 172.21.15.37:0/709071253 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34f2652640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34ec1082a0 0x7f34ec19a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec112100 con 0x7f34ec1036e0 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec111f80 con 0x7f34ec1082a0 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34ec112280 con 0x7f34ec102d30 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34f0bef640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34ec1082a0 0x7f34ec19a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34eb7fe640 1 -- 172.21.15.37:0/709071253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 msgr2=0x7f34ec1a05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 0x7f34ec1a05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34eb7fe640 1 -- 172.21.15.37:0/709071253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34ec1082a0 msgr2=0x7f34ec19a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34ec1082a0 0x7f34ec19a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34ebfff640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 0x7f34ec1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34eb7fe640 1 -- 172.21.15.37:0/709071253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34ec19af00 con 0x7f34ec1036e0 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34f0bef640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34ec1082a0 0x7f34ec19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.366+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1036e0 0x7f34ec1a0b00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0697d0 tx=0x7f34e000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.367+0000 7f34e97fa640 1 -- 172.21.15.37:0/709071253 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e0015020 con 0x7f34ec1036e0 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.367+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34ec19b1f0 con 0x7f34ec1036e0 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.367+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34ec1a7540 con 0x7f34ec1036e0 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.367+0000 7f34e97fa640 1 -- 172.21.15.37:0/709071253 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e0011d10 con 0x7f34ec1036e0 2024-09-19T06:55:10.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.367+0000 7f34e97fa640 1 -- 172.21.15.37:0/709071253 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e000f2e0 con 0x7f34ec1036e0 2024-09-19T06:55:10.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.370+0000 7f34f4a2b740 1 Processor -- start 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.370+0000 7f34f4a2b740 1 -- start start 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef7113c0 0x5599ef7117a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 0x5599ef724950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef724e90 0x5599ef727280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5599ef6772e0 con 0x5599ef724e90 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5599ef61ece0 con 0x5599ef711ce0 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5599ef6732a0 con 0x5599ef7113c0 2024-09-19T06:55:10.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef724e90 0x5599ef727280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef724e90 0x5599ef727280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59836/0 (socket says 172.21.15.37:59836) 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.371+0000 7f34eb7fe640 1 -- 172.21.15.37:0/401848593 learned_addr learned my addr 172.21.15.37:0/401848593 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.372+0000 7f34e97fa640 1 -- 172.21.15.37:0/709071253 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f34e0018020 con 0x7f34ec1036e0 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.372+0000 7f34f0bef640 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef7113c0 0x5599ef7117a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.372+0000 7f34eb7fe640 1 -- 172.21.15.37:0/401848593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef7113c0 msgr2=0x5599ef7117a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.372+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef7113c0 0x5599ef7117a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.372+0000 7f34eb7fe640 1 -- 172.21.15.37:0/401848593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 msgr2=0x5599ef724950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.373+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 0x5599ef724950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.373+0000 7f34eb7fe640 1 -- 172.21.15.37:0/401848593 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ef603440 con 0x5599ef724e90 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.374+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef724e90 0x5599ef727280 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ccb60 tx=0x7f34e00d5a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27103adb18cfbfac server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.374+0000 7f34d27fc640 1 -- 172.21.15.37:0/401848593 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e00e2070 con 0x5599ef724e90 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.374+0000 7f34d27fc640 1 -- 172.21.15.37:0/401848593 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e00d15d0 con 0x5599ef724e90 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.374+0000 7f34e97fa640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34d40a3500 0x7f34d40a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.374+0000 7f34f0bef640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34d40a3500 0x7f34d40a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.375+0000 7f34e97fa640 1 -- 172.21.15.37:0/709071253 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f34e00cf450 con 0x7f34ec1036e0 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.375+0000 7f34f4a2b740 1 -- 172.21.15.37:0/401848593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef724e90 msgr2=0x5599ef727280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.375+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef724e90 0x5599ef727280 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f34e00ccb60 tx=0x7f34e00d5a70 comp rx=0 tx=0).stop 2024-09-19T06:55:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.375+0000 7f34f0bef640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34d40a3500 0x7f34d40a59c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006010 tx=0x7f34e4005d80 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.376+0000 7f34f4a2b740 1 -- 172.21.15.37:0/401848593 shutdown_connections 2024-09-19T06:55:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.376+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef724e90 0x5599ef727280 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.376+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 0x5599ef724950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.376+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/401848593 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef7113c0 0x5599ef7117a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.376+0000 7f34f4a2b740 1 -- 172.21.15.37:0/401848593 >> 172.21.15.37:0/401848593 conn(0x5599ef70da30 msgr2=0x5599ef70de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.376+0000 7f34f4a2b740 1 -- 172.21.15.37:0/401848593 shutdown_connections 2024-09-19T06:55:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.376+0000 7f34f4a2b740 1 -- 172.21.15.37:0/401848593 wait complete. 2024-09-19T06:55:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.377+0000 7f34f4a2b740 1 -- start start 2024-09-19T06:55:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.377+0000 7f34f4a2b740 1 Processor -- start 2024-09-19T06:55:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef71e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 0x5599ef71eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 0x5599ef724290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5599ef7141f0 con 0x5599ef7113c0 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5599ef632050 con 0x5599ef724e90 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5599ef714370 con 0x5599ef711ce0 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34f4a2b740 10 client.0 _ll_register_callbacks cb 0x5599ef672160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef71e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef71e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59846/0 (socket says 172.21.15.37:59846) 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.378+0000 7f34ebfff640 1 -- 172.21.15.37:0/1786234932 learned_addr learned my addr 172.21.15.37:0/1786234932 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.379+0000 7f34ebfff640 1 -- 172.21.15.37:0/1786234932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 msgr2=0x5599ef71eac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.379+0000 7f34ebfff640 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 0x5599ef71eac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.379+0000 7f34ebfff640 1 -- 172.21.15.37:0/1786234932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 msgr2=0x5599ef724290 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:55:10.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.379+0000 7f34ebfff640 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 0x5599ef724290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.379+0000 7f34ebfff640 1 -- 172.21.15.37:0/1786234932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ef3e83c0 con 0x5599ef7113c0 2024-09-19T06:55:10.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.379+0000 7f34ebfff640 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef71e580 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f34dc010660 tx=0x7f34dc010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaae14c66af65125 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:10.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.379+0000 7f34cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.381+0000 7f34cb7fe640 1 -- 172.21.15.37:0/1786234932 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00ea50 con 0x5599ef7113c0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.381+0000 7f34cb7fe640 1 -- 172.21.15.37:0/1786234932 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f34dc00ebf0 con 0x5599ef7113c0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.381+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34e00d15d0 con 0x5599ef7113c0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.381+0000 7f34cb7fe640 1 -- 172.21.15.37:0/1786234932 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f34dc00edf0 con 0x5599ef7113c0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.381+0000 7f34f4a2b740 10 client.19185 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.381+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5599ef71a690 con 0x5599ef7113c0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34cb7fe640 1 -- 172.21.15.37:0/1786234932 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34e00d15d0 con 0x5599ef7113c0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34cb7fe640 1 -- 172.21.15.37:0/1786234932 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f34dc00f040 con 0x5599ef7113c0 2024-09-19T06:55:10.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34f4a2b740 10 client.19185 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:10.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34f4a2b740 10 client.19185 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:10.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34f4a2b740 20 client.19185 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:10.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef73d140 0x5599ef73f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34f4a2b740 4 client.19185 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:10.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.382+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5599ef684850 con 0x5599ef73d140 2024-09-19T06:55:10.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.383+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef73d140 0x5599ef73f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.386+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef73d140 0x5599ef73f600 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f515ee8e116d6c94 server_cookie=e8ed7a835a5a33ed in_seq=0 out_seq=0 2024-09-19T06:55:10.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.386+0000 7f34cb7fe640 10 client.19185 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:10.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.388+0000 7f34cb7fe640 1 -- 172.21.15.37:0/1786234932 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5599ef684850 con 0x5599ef73d140 2024-09-19T06:55:10.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.388+0000 7f34cb7fe640 10 client.19185 handle_command_reply: tid=0 2024-09-19T06:55:10.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.388+0000 7f34f4a2b740 1 client.19185 shutdown 2024-09-19T06:55:10.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 msgr2=0x5599ef71e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.390 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef71e580 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f34dc010660 tx=0x7f34dc010b20 comp rx=0 tx=0).stop 2024-09-19T06:55:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 shutdown_connections 2024-09-19T06:55:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef73d140 0x5599ef73f600 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 0x5599ef724290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 0x5599ef71eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1786234932 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef71e580 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 >> 172.21.15.37:0/1786234932 conn(0x5599ef70da30 msgr2=0x5599ef7136c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:10.391 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.389+0000 7f34cb7fe640 0 client.19185 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:10.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.390+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 shutdown_connections 2024-09-19T06:55:10.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.390+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1786234932 wait complete. 2024-09-19T06:55:10.392 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.390+0000 7f34f4a2b740 20 client.19185 trim_cache size 0 max 16384 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 Processor -- start 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 -- start start 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef724200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 0x5599ef744ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 0x5599ef745410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34d80112d0 con 0x5599ef7113c0 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34d800eba0 con 0x5599ef724e90 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34d800ca10 con 0x5599ef711ce0 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.416+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef724200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.422 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34f0bef640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 0x5599ef745410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef724200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59848/0 (socket says 172.21.15.37:59848) 2024-09-19T06:55:10.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 -- 172.21.15.37:0/1570688346 learned_addr learned my addr 172.21.15.37:0/1570688346 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:10.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 0x5599ef744ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 -- 172.21.15.37:0/1570688346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 msgr2=0x5599ef744ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:10.423 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 0x5599ef744ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 -- 172.21.15.37:0/1570688346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 msgr2=0x5599ef745410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 0x5599ef745410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 -- 172.21.15.37:0/1570688346 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5599ef70d860 con 0x5599ef7113c0 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34ebfff640 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef724200 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f34dc01a6f0 tx=0x7f34dc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b769e2783383b54 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34d1ffb640 1 -- 172.21.15.37:0/1570688346 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc010070 con 0x5599ef7113c0 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34d1ffb640 1 -- 172.21.15.37:0/1570688346 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34dc010210 con 0x5599ef7113c0 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.417+0000 7f34d1ffb640 1 -- 172.21.15.37:0/1570688346 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34dc0052c0 con 0x5599ef7113c0 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1570688346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 msgr2=0x5599ef724200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef724200 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f34dc01a6f0 tx=0x7f34dc005c50 comp rx=0 tx=0).stop 2024-09-19T06:55:10.424 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1570688346 shutdown_connections 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef724e90 0x5599ef745410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef711ce0 0x5599ef744ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/1570688346 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef7113c0 0x5599ef724200 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1570688346 >> 172.21.15.37:0/1570688346 conn(0x5599ef70da30 msgr2=0x5599ef718170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1570688346 shutdown_connections 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 -- 172.21.15.37:0/1570688346 wait complete. 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.418+0000 7f34f4a2b740 1 -- start start 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 1 Processor -- start 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 0x5599ef74cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef724e90 0x5599ef74d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef74d670 0x5599ef7a6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.425 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f34d800d060 con 0x5599ef74d670 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f34d800e4d0 con 0x5599ef711ce0 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f34d800d730 con 0x5599ef724e90 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f4a2b740 10 client.0 _ll_register_callbacks cb 0x7f34dc0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f0bef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef74d670 0x5599ef7a6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34ebfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 0x5599ef74cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f0bef640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef74d670 0x5599ef7a6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59854/0 (socket says 172.21.15.37:59854) 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34f0bef640 1 -- 172.21.15.37:0/2654511386 learned_addr learned my addr 172.21.15.37:0/2654511386 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.419+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef724e90 0x5599ef74d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f0bef640 1 -- 172.21.15.37:0/2654511386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef724e90 msgr2=0x5599ef74d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f0bef640 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef724e90 0x5599ef74d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.426 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f0bef640 1 -- 172.21.15.37:0/2654511386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 msgr2=0x5599ef74cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f0bef640 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 0x5599ef74cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f0bef640 1 -- 172.21.15.37:0/2654511386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5599ef7a73e0 con 0x5599ef74d670 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34eb7fe640 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef724e90 0x5599ef74d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f0bef640 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef74d670 0x5599ef7a6d20 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005e30 tx=0x7f34e402a910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8037283ac989df78 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34cb7fe640 1 -- 172.21.15.37:0/2654511386 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34e4031730 con 0x5599ef74d670 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34cb7fe640 1 -- 172.21.15.37:0/2654511386 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f34e4041030 con 0x5599ef74d670 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34dc010210 con 0x5599ef74d670 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34cb7fe640 1 -- 172.21.15.37:0/2654511386 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f34e40318d0 con 0x5599ef74d670 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f4a2b740 10 client.19191 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:10.427 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.420+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x5599ef7a86d0 con 0x5599ef74d670 2024-09-19T06:55:10.432 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34cb7fe640 1 -- 172.21.15.37:0/2654511386 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34dc010210 con 0x5599ef74d670 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34cb7fe640 1 -- 172.21.15.37:0/2654511386 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f34e402f9b0 con 0x5599ef74d670 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34f4a2b740 10 client.19191 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34f4a2b740 10 client.19191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34f4a2b740 20 client.19191 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef7a8e70 0x5599ef7ab290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34f4a2b740 4 client.19191 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5599ef67ed60 con 0x5599ef7a8e70 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.421+0000 7f34ebfff640 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef7a8e70 0x5599ef7ab290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34ebfff640 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef7a8e70 0x5599ef7ab290 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ecc2effc312a52f server_cookie=f7346da3c9466d1d in_seq=0 out_seq=0 2024-09-19T06:55:10.433 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34cb7fe640 10 client.19191 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34cb7fe640 1 -- 172.21.15.37:0/2654511386 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5599ef67ed60 con 0x5599ef7a8e70 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34cb7fe640 10 client.19191 handle_command_reply: tid=0 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 client.19191 shutdown 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef74d670 msgr2=0x5599ef7a6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef74d670 0x5599ef7a6d20 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f34e4005e30 tx=0x7f34e402a910 comp rx=0 tx=0).stop 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 shutdown_connections 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5599ef7a8e70 0x5599ef7ab290 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5599ef74d670 0x5599ef7a6d20 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5599ef724e90 0x5599ef74d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 --2- 172.21.15.37:0/2654511386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5599ef711ce0 0x5599ef74cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.434 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 >> 172.21.15.37:0/2654511386 conn(0x5599ef70da30 msgr2=0x5599ef70c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:10.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.422+0000 7f34cb7fe640 0 client.19191 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:10.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.423+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 shutdown_connections 2024-09-19T06:55:10.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.423+0000 7f34f4a2b740 1 -- 172.21.15.37:0/2654511386 wait complete. 2024-09-19T06:55:10.435 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.423+0000 7f34f4a2b740 20 client.19191 trim_cache size 0 max 16384 2024-09-19T06:55:10.435 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: "eb5d42ad-afda-4e57-a55f-b02f1f8ecb54": { 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "eb5d42ad-afda-4e57-a55f-b02f1f8ecb54", 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:10.436 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.423+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34d40a3500 msgr2=0x7f34d40a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.423+0000 7f34f2652640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34d40a3500 0x7f34d40a59c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f34e4006010 tx=0x7f34e4005d80 comp rx=0 tx=0).stop 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.423+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1036e0 msgr2=0x7f34ec1a0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.423+0000 7f34f2652640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1036e0 0x7f34ec1a0b00 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f34ec0697d0 tx=0x7f34e000b850 comp rx=0 tx=0).stop 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 shutdown_connections 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f34ec1082a0 0x7f34ec19a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f34d40a3500 0x7f34d40a59c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.438 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f34ec1036e0 0x7f34ec1a0b00 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 --2- 172.21.15.37:0/709071253 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f34ec102d30 0x7f34ec1a05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:10.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 >> 172.21.15.37:0/709071253 conn(0x7f34ec0fc7d0 msgr2=0x7f34ec101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:10.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 shutdown_connections 2024-09-19T06:55:10.439 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:10.424+0000 7f34f2652640 1 -- 172.21.15.37:0/709071253 wait complete. 2024-09-19T06:55:10.439 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:11.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1033849 7.60 MB/sec execute 578 sec latency 13.052 ms 2024-09-19T06:55:11.578 INFO:tasks.cephfs.filesystem:scrub status for tag:eb5d42ad-afda-4e57-a55f-b02f1f8ecb54 - {'path': '/', 'tag': 'eb5d42ad-afda-4e57-a55f-b02f1f8ecb54', 'options': 'recursive,force'} 2024-09-19T06:55:12.087 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2024916836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102570 msgr2=0x7f5e001029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2024916836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102570 0x7f5e001029b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5df4009930 tx=0x7f5df402a4d0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2024916836 shutdown_connections 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2024916836 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00102ef0 0x7f5e0010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2024916836 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102570 0x7f5e001029b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2024916836 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e00100e20 0x7f5e00101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2024916836 >> 172.21.15.37:0/2024916836 conn(0x7f5e000fc7d0 msgr2=0x7f5e000febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2024916836 shutdown_connections 2024-09-19T06:55:12.088 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2024916836 wait complete. 2024-09-19T06:55:12.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.086+0000 7f5e05cf4640 1 Processor -- start 2024-09-19T06:55:12.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5e05cf4640 1 -- start start 2024-09-19T06:55:12.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5e05cf4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00100e20 0x7f5e00199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5e05cf4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e00102570 0x7f5e00199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5e05cf4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102ef0 0x7f5e001a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5e05cf4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00110f80 con 0x7f5e00100e20 2024-09-19T06:55:12.090 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5e05cf4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00110e00 con 0x7f5e00102ef0 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5e05cf4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e00111100 con 0x7f5e00102570 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5dff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00100e20 0x7f5e00199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5dff7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00100e20 0x7f5e00199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59880/0 (socket says 172.21.15.37:59880) 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5dff7fe640 1 -- 172.21.15.37:0/2136080519 learned_addr learned my addr 172.21.15.37:0/2136080519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5dfffff640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102ef0 0x7f5e001a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.087+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e00102570 0x7f5e00199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2136080519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102ef0 msgr2=0x7f5e001a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102ef0 0x7f5e001a44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00100e20 msgr2=0x7f5e00199450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00100e20 0x7f5e00199450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2136080519 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e001a4a10 con 0x7f5e00102570 2024-09-19T06:55:12.091 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfffff640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102ef0 0x7f5e001a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:12.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00100e20 0x7f5e00199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:12.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e00102570 0x7f5e00199990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5df4005940 tx=0x7f5df40040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfcff9640 1 -- 172.21.15.37:0/2136080519 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df403f070 con 0x7f5e00102570 2024-09-19T06:55:12.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfcff9640 1 -- 172.21.15.37:0/2136080519 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df40044c0 con 0x7f5e00102570 2024-09-19T06:55:12.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5dfcff9640 1 -- 172.21.15.37:0/2136080519 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4035070 con 0x7f5e00102570 2024-09-19T06:55:12.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e001a4be0 con 0x7f5e00102570 2024-09-19T06:55:12.092 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.088+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e001a4fe0 con 0x7f5e00102570 2024-09-19T06:55:12.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5dfcff9640 1 -- 172.21.15.37:0/2136080519 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5df403b460 con 0x7f5e00102570 2024-09-19T06:55:12.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 Processor -- start 2024-09-19T06:55:12.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 -- start start 2024-09-19T06:55:12.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592c2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cfd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.094 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563c592282a0 con 0x563c592cd940 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563c591cfb10 con 0x563c592c2760 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.091+0000 7f5e080cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563c59224260 con 0x563c592c3080 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592c2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592c2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44514/0 (socket says 172.21.15.37:44514) 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 -- 172.21.15.37:0/1145752354 learned_addr learned my addr 172.21.15.37:0/1145752354 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cfd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfcff9640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5de80a33b0 0x7f5de80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 -- 172.21.15.37:0/1145752354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 msgr2=0x563c592cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 -- 172.21.15.37:0/1145752354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 msgr2=0x563c592cfd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cfd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 -- 172.21.15.37:0/1145752354 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c58f99b90 con 0x563c592c2760 2024-09-19T06:55:12.095 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfcff9640 1 -- 172.21.15.37:0/2136080519 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f5df40f0240 con 0x7f5e00102570 2024-09-19T06:55:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5de80a33b0 0x7f5de80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5dfffff640 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592c2b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5df8011910 tx=0x7f5df8011dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=482710155886b5ad server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5de5ffb640 1 -- 172.21.15.37:0/1145752354 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df8017050 con 0x563c592c2760 2024-09-19T06:55:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.092+0000 7f5de5ffb640 1 -- 172.21.15.37:0/1145752354 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df801bb90 con 0x563c592c2760 2024-09-19T06:55:12.096 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5de5ffb640 1 -- 172.21.15.37:0/1145752354 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df800e3b0 con 0x563c592c2760 2024-09-19T06:55:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 -- 172.21.15.37:0/1145752354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 msgr2=0x563c592c2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5dfffff640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5de80a33b0 0x7f5de80a5870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5df8023000 tx=0x7f5df8024000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.099 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592c2b40 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5df8011910 tx=0x7f5df8011dd0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 -- 172.21.15.37:0/1145752354 shutdown_connections 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 --2- 172.21.15.37:0/1145752354 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592c2b40 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 -- 172.21.15.37:0/1145752354 >> 172.21.15.37:0/1145752354 conn(0x563c592becf0 msgr2=0x563c592bf0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 -- 172.21.15.37:0/1145752354 shutdown_connections 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.093+0000 7f5e080cd740 1 -- 172.21.15.37:0/1145752354 wait complete. 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 -- start start 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 Processor -- start 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.100 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x563c592d0510 con 0x563c592cd940 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563c591b06e0 con 0x563c592c2760 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x563c592d0690 con 0x563c592c3080 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5dff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5dff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44530/0 (socket says 172.21.15.37:44530) 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5dff7fe640 1 -- 172.21.15.37:0/2241617623 learned_addr learned my addr 172.21.15.37:0/2241617623 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5e080cd740 10 client.0 _ll_register_callbacks cb 0x563c591d4a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5dfffff640 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.094+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5dff7fe640 1 -- 172.21.15.37:0/2241617623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 msgr2=0x563c592cad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5dff7fe640 1 -- 172.21.15.37:0/2241617623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 msgr2=0x563c592cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.101 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5dff7fe640 1 -- 172.21.15.37:0/2241617623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c592d99d0 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592ca7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5df00118b0 tx=0x7f5df0011cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f85a8d2aa06da051 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2241617623 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df001f060 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2241617623 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f5df0010ba0 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df801bb90 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2241617623 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5df0010da0 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5e080cd740 10 client.28478 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563c592dc3a0 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.095+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2241617623 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df801bb90 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2241617623 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f5df000f500 con 0x563c592c2760 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5e080cd740 10 client.28478 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5e080cd740 10 client.28478 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5e080cd740 20 client.28478 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:12.102 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c592dd860 0x563c592dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5e080cd740 4 client.28478 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563c592359d0 con 0x563c592dd860 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c592dd860 0x563c592dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c592dd860 0x563c592dfc90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdb8a12cd1db6dce server_cookie=8a29394b883e8b96 in_seq=0 out_seq=0 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.096+0000 7f5ddeffd640 10 client.28478 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2241617623 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x563c592359d0 con 0x563c592dd860 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5ddeffd640 10 client.28478 handle_command_reply: tid=0 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 client.28478 shutdown 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 msgr2=0x563c592ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592ca7f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5df00118b0 tx=0x7f5df0011cb0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 shutdown_connections 2024-09-19T06:55:12.103 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c592dd860 0x563c592dfc90 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592cad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2241617623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592ca7f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 >> 172.21.15.37:0/2241617623 conn(0x563c592becf0 msgr2=0x563c592c49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.097+0000 7f5ddeffd640 0 client.28478 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.098+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 shutdown_connections 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.098+0000 7f5e080cd740 1 -- 172.21.15.37:0/2241617623 wait complete. 2024-09-19T06:55:12.104 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.098+0000 7f5e080cd740 20 client.28478 trim_cache size 0 max 16384 2024-09-19T06:55:12.117 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 Processor -- start 2024-09-19T06:55:12.117 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:11 smithi037 ceph-mon[31211]: pgmap v424: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 -- start start 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592e5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592e5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592c7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001b02e0 con 0x563c592cd940 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001adbb0 con 0x563c592c2760 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5e080cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001aba20 con 0x563c592c3080 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592c7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592e5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592e5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:44538/0 (socket says 172.21.15.37:44538) 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592c7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59902/0 (socket says 172.21.15.37:59902) 2024-09-19T06:55:12.119 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dff7fe640 1 -- 172.21.15.37:0/2095390578 learned_addr learned my addr 172.21.15.37:0/2095390578 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592e5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2095390578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 msgr2=0x563c592e5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592e5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2095390578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 msgr2=0x563c592c7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592c7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2095390578 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563c592beb20 con 0x563c592c3080 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.117+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.116+0000 7f5dfffff640 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.117+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592e5aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5df40ed930 tx=0x7f5df402a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e323475ab5d55cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.117+0000 7f5de57fa640 1 -- 172.21.15.37:0/2095390578 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4048040 con 0x563c592c3080 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.117+0000 7f5de57fa640 1 -- 172.21.15.37:0/2095390578 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df40c0870 con 0x563c592c3080 2024-09-19T06:55:12.120 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.117+0000 7f5de57fa640 1 -- 172.21.15.37:0/2095390578 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4031bd0 con 0x563c592c3080 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 -- 172.21.15.37:0/2095390578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 msgr2=0x563c592e5aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592e5aa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5df40ed930 tx=0x7f5df402a4d0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 -- 172.21.15.37:0/2095390578 shutdown_connections 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592c7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592e5aa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2095390578 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592e5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 -- 172.21.15.37:0/2095390578 >> 172.21.15.37:0/2095390578 conn(0x563c592becf0 msgr2=0x563c592cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 -- 172.21.15.37:0/2095390578 shutdown_connections 2024-09-19T06:55:12.123 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 -- 172.21.15.37:0/2095390578 wait complete. 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.118+0000 7f5e080cd740 1 -- start start 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 1 Processor -- start 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592d5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592d28b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592d2df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001ac070 con 0x563c592cd940 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001ad4e0 con 0x563c592c2760 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5e001ac740 con 0x563c592c3080 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5e080cd740 10 client.0 _ll_register_callbacks cb 0x7f5df000bee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5dfffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592d2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5dfeffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592d28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5dfffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592d2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59916/0 (socket says 172.21.15.37:59916) 2024-09-19T06:55:12.124 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5dfffff640 1 -- 172.21.15.37:0/2701692671 learned_addr learned my addr 172.21.15.37:0/2701692671 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.119+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592d5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 msgr2=0x563c592d5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592d5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2701692671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 msgr2=0x563c592d2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592d2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5dfeffd640 1 -- 172.21.15.37:0/2701692671 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563c592ba260 con 0x563c592c3080 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5dfeffd640 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592d28b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5df40ed930 tx=0x7f5df4009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1c5c58aaf411812c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5ddeffd640 10 client.0 ms_handle_connect on v2:172.21.15.151:3300/0 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2701692671 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5df4048040 con 0x563c592c3080 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2701692671 <== mon.1 v2:172.21.15.151:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f5df40d5310 con 0x563c592c3080 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5df40c0870 con 0x563c592c3080 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.120+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2701692671 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f5df40edbb0 con 0x563c592c3080 2024-09-19T06:55:12.125 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 10 client.28114 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x563c5935a190 con 0x563c592c3080 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2701692671 <== mon.1 v2:172.21.15.151:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5df40c0870 con 0x563c592c3080 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2701692671 <== mon.1 v2:172.21.15.151:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f5df404b060 con 0x563c592c3080 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 10 client.28114 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 10 client.28114 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 20 client.28114 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c5935a810 0x563c5935cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 4 client.28114 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}] 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 142.2, "await_for": 45.0}) -- 0x563c5922fd20 con 0x563c5935a810 2024-09-19T06:55:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.121+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c5935a810 0x563c5935cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.122+0000 7f5ddeffd640 10 client.28114 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.127 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.122+0000 7f5dff7fe640 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c5935a810 0x563c5935cc00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=585d09a7a6d723cd server_cookie=4f50d471ca1516e9 in_seq=0 out_seq=0 2024-09-19T06:55:12.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.210+0000 7f5ddeffd640 1 -- 172.21.15.37:0/2701692671 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x563c5922fd20 con 0x563c5935a810 2024-09-19T06:55:12.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.210+0000 7f5ddeffd640 10 client.28114 handle_command_reply: tid=0 2024-09-19T06:55:12.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.210+0000 7f5e080cd740 1 client.28114 shutdown 2024-09-19T06:55:12.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 msgr2=0x563c592d28b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592d28b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5df40ed930 tx=0x7f5df4009680 comp rx=0 tx=0).stop 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 shutdown_connections 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x563c5935a810 0x563c5935cc00 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x563c592cd940 0x563c592d2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x563c592c3080 0x563c592d28b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 --2- 172.21.15.37:0/2701692671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563c592c2760 0x563c592d5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5ddeffd640 0 client.28114 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 >> 172.21.15.37:0/2701692671 conn(0x563c592becf0 msgr2=0x563c592c8c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 shutdown_connections 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 1 -- 172.21.15.37:0/2701692671 wait complete. 2024-09-19T06:55:12.214 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.211+0000 7f5e080cd740 20 client.28114 trim_cache size 0 max 16384 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 43, 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 46, 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "fdb3482b": { 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 46, 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:55:12.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 142.2, 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "QUIESCED", 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:55:12.216 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:12.217 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:12.217 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:12.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5de80a33b0 msgr2=0x7f5de80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5de80a33b0 0x7f5de80a5870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5df8023000 tx=0x7f5df8024000 comp rx=0 tx=0).stop 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e00102570 msgr2=0x7f5e00199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e00102570 0x7f5e00199990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f5df4005940 tx=0x7f5df40040c0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 shutdown_connections 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5e00102ef0 0x7f5e001a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5e00102570 0x7f5e00199990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5de80a33b0 0x7f5de80a5870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 --2- 172.21.15.37:0/2136080519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5e00100e20 0x7f5e00199450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 >> 172.21.15.37:0/2136080519 conn(0x7f5e000fc7d0 msgr2=0x7f5e001048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 shutdown_connections 2024-09-19T06:55:12.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:12.212+0000 7f5e05cf4640 1 -- 172.21.15.37:0/2136080519 wait complete. 2024-09-19T06:55:12.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.60 MB/sec execute 579 sec latency 98.383 ms 2024-09-19T06:55:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 -- 172.21.15.37:0/2200623378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 msgr2=0x7f11bc1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 --2- 172.21.15.37:0/2200623378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc1030a0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f11b8009900 tx=0x7f11b802a480 comp rx=0 tx=0).stop 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 -- 172.21.15.37:0/2200623378 shutdown_connections 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 --2- 172.21.15.37:0/2200623378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11bc10c110 0x7f11bc10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 --2- 172.21.15.37:0/2200623378 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc1035e0 0x7f11bc10bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 --2- 172.21.15.37:0/2200623378 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc1030a0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 -- 172.21.15.37:0/2200623378 >> 172.21.15.37:0/2200623378 conn(0x7f11bc0fc6b0 msgr2=0x7f11bc0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 -- 172.21.15.37:0/2200623378 shutdown_connections 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 -- 172.21.15.37:0/2200623378 wait complete. 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 Processor -- start 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.255+0000 7f11c40af640 1 -- start start 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c40af640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c40af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11bc1035e0 0x7f11bc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.259 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c40af640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc10c110 0x7f11bc19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c40af640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc111ef0 con 0x7f11bc102cc0 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c40af640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc111d70 con 0x7f11bc1035e0 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c40af640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc112070 con 0x7f11bc10c110 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59936/0 (socket says 172.21.15.37:59936) 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 -- 172.21.15.37:0/16233210 learned_addr learned my addr 172.21.15.37:0/16233210 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c164a640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11bc1035e0 0x7f11bc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 -- 172.21.15.37:0/16233210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc10c110 msgr2=0x7f11bc19cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c264c640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc10c110 0x7f11bc19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc10c110 0x7f11bc19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 -- 172.21.15.37:0/16233210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11bc1035e0 msgr2=0x7f11bc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11bc1035e0 0x7f11bc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 -- 172.21.15.37:0/16233210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11bc19d410 con 0x7f11bc102cc0 2024-09-19T06:55:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc198420 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f11b8006010 tx=0x7f11b80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.256+0000 7f11b2ffd640 1 -- 172.21.15.37:0/16233210 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b803f030 con 0x7f11bc102cc0 2024-09-19T06:55:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.257+0000 7f11b2ffd640 1 -- 172.21.15.37:0/16233210 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b8003ce0 con 0x7f11bc102cc0 2024-09-19T06:55:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.257+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11bc19d6a0 con 0x7f11bc102cc0 2024-09-19T06:55:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.257+0000 7f11b2ffd640 1 -- 172.21.15.37:0/16233210 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8035070 con 0x7f11bc102cc0 2024-09-19T06:55:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.257+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11bc1a4f90 con 0x7f11bc102cc0 2024-09-19T06:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.259+0000 7f11b2ffd640 1 -- 172.21.15.37:0/16233210 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f11b8042030 con 0x7f11bc102cc0 2024-09-19T06:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.259+0000 7f11c6e40740 1 Processor -- start 2024-09-19T06:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.259+0000 7f11c6e40740 1 -- start start 2024-09-19T06:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce4ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5cb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.266 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c6e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eb0cdb22f0 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c6e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eb0cd5a270 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c6e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb0cdae2b0 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59948/0 (socket says 172.21.15.37:59948) 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 -- 172.21.15.37:0/3008799122 learned_addr learned my addr 172.21.15.37:0/3008799122 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5cb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c264c640 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce4ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11b2ffd640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11a80a34e0 0x7f11a80a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 -- 172.21.15.37:0/3008799122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 msgr2=0x55eb0ce5cb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5cb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 -- 172.21.15.37:0/3008799122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4d9d0 msgr2=0x55eb0ce4ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce4ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 -- 172.21.15.37:0/3008799122 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0cd59340 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c264c640 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce4ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11b2ffd640 1 -- 172.21.15.37:0/16233210 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f11b80f0240 con 0x7f11bc102cc0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c164a640 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5f4b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00bcd0 tx=0x7f11ac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ec5a44c886d2de server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.260+0000 7f11c264c640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11a80a34e0 0x7f11a80a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.261+0000 7f11a7fff640 1 -- 172.21.15.37:0/3008799122 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac004030 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.261+0000 7f11a7fff640 1 -- 172.21.15.37:0/3008799122 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac0041d0 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.261+0000 7f11a7fff640 1 -- 172.21.15.37:0/3008799122 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac011770 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.261+0000 7f11c6e40740 1 -- 172.21.15.37:0/3008799122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce5d0c0 msgr2=0x55eb0ce5f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.261+0000 7f11c264c640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11a80a34e0 0x7f11a80a59a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.261+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5f4b0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00bcd0 tx=0x7f11ac0107a0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 -- 172.21.15.37:0/3008799122 shutdown_connections 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5f4b0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5cb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3008799122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce4ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 -- 172.21.15.37:0/3008799122 >> 172.21.15.37:0/3008799122 conn(0x55eb0ce4a060 msgr2=0x55eb0ce4a440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 -- 172.21.15.37:0/3008799122 shutdown_connections 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 -- 172.21.15.37:0/3008799122 wait complete. 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 -- start start 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 Processor -- start 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.262+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5aae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c6e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ce5fcf0 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c6e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55eb0cd3b290 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c6e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55eb0ce5fe70 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c6e40740 10 client.0 _ll_register_callbacks cb 0x55eb0cd6c140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59952/0 (socket says 172.21.15.37:59952) 2024-09-19T06:55:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3719714677 learned_addr learned my addr 172.21.15.37:0/3719714677 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c264c640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c164a640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3719714677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 msgr2=0x55eb0ce5a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 msgr2=0x55eb0ce5aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5aae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3719714677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0cac5210 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c264c640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a060 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f11b80b89e0 tx=0x7f11b8003e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e6a58c5d62cb4bd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11a57fa640 1 -- 172.21.15.37:0/3719714677 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8049070 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11a57fa640 1 -- 172.21.15.37:0/3719714677 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f11b80bc7e0 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.274 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11ac0041d0 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11a57fa640 1 -- 172.21.15.37:0/3719714677 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f11b80bc350 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c6e40740 10 client.19221 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.263+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eb0cb91620 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11a57fa640 1 -- 172.21.15.37:0/3719714677 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11ac0041d0 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11a57fa640 1 -- 172.21.15.37:0/3719714677 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f11b80c4d60 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c6e40740 10 client.19221 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c6e40740 10 client.19221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c6e40740 20 client.19221 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ce6d0c0 0x55eb0ce6f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c6e40740 4 client.19221 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb0cdbfa20 con 0x55eb0ce6d0c0 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c164a640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ce6d0c0 0x55eb0ce6f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11a57fa640 10 client.19221 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.264+0000 7f11c164a640 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ce6d0c0 0x55eb0ce6f550 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8c7d3798b1f64e server_cookie=a51c76514ac85cf7 in_seq=0 out_seq=0 2024-09-19T06:55:12.276 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:11 smithi096 ceph-mon[35004]: pgmap v424: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-09-19T06:55:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11a57fa640 1 -- 172.21.15.37:0/3719714677 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55eb0cdbfa20 con 0x55eb0ce6d0c0 2024-09-19T06:55:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11a57fa640 10 client.19221 handle_command_reply: tid=0 2024-09-19T06:55:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 client.19221 shutdown 2024-09-19T06:55:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 msgr2=0x55eb0ce5a060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a060 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f11b80b89e0 tx=0x7f11b8003e90 comp rx=0 tx=0).stop 2024-09-19T06:55:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 shutdown_connections 2024-09-19T06:55:12.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ce6d0c0 0x55eb0ce6f550 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce5aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce5a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11a57fa640 0 client.19221 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3719714677 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a060 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 >> 172.21.15.37:0/3719714677 conn(0x55eb0ce4a060 msgr2=0x55eb0ce4fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 shutdown_connections 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.265+0000 7f11c6e40740 1 -- 172.21.15.37:0/3719714677 wait complete. 2024-09-19T06:55:12.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.266+0000 7f11c6e40740 20 client.19221 trim_cache size 0 max 16384 2024-09-19T06:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 Processor -- start 2024-09-19T06:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 -- start start 2024-09-19T06:55:12.286 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce74e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce75370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ae030 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ab900 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c6e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1a9770 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.280+0000 7f11c1e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59966/0 (socket says 172.21.15.37:59966) 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3956353859 learned_addr learned my addr 172.21.15.37:0/3956353859 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c164a640 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce74e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c264c640 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce75370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.287 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3956353859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce5d0c0 msgr2=0x55eb0ce75370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce75370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3956353859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4e2f0 msgr2=0x55eb0ce74e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce74e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 -- 172.21.15.37:0/3956353859 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb0ce59e30 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c164a640 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce74e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c264c640 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce75370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a730 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f11b80c4cc0 tx=0x7f11b8009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9046c27febb45a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.288 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11a77fe640 1 -- 172.21.15.37:0/3956353859 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8049070 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11a77fe640 1 -- 172.21.15.37:0/3956353859 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b80bcc80 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.281+0000 7f11a77fe640 1 -- 172.21.15.37:0/3956353859 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b80044e0 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 -- 172.21.15.37:0/3956353859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 msgr2=0x55eb0ce5a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a730 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f11b80c4cc0 tx=0x7f11b8009560 comp rx=0 tx=0).stop 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 -- 172.21.15.37:0/3956353859 shutdown_connections 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce75370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce74e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 --2- 172.21.15.37:0/3956353859 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce5a730 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 -- 172.21.15.37:0/3956353859 >> 172.21.15.37:0/3956353859 conn(0x55eb0ce4a060 msgr2=0x55eb0ce5a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.289 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 -- 172.21.15.37:0/3956353859 shutdown_connections 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 -- 172.21.15.37:0/3956353859 wait complete. 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 -- start start 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 Processor -- start 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.282+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce60ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce61530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce61a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1a9dc0 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1ab230 con 0x55eb0ce5d0c0 2024-09-19T06:55:12.290 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f11bc1aa490 con 0x55eb0ce4d9d0 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 10 client.0 _ll_register_callbacks cb 0x7f11b80b8910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce61530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce61530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:59978/0 (socket says 172.21.15.37:59978) 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 -- 172.21.15.37:0/649432311 learned_addr learned my addr 172.21.15.37:0/649432311 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce60ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 -- 172.21.15.37:0/649432311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4d9d0 msgr2=0x55eb0ce60ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce60ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 -- 172.21.15.37:0/649432311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 msgr2=0x55eb0ce61a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:12.291 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce61a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 -- 172.21.15.37:0/649432311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb0ce5cbf0 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c164a640 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce61530 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00bbc0 tx=0x7f11ac005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f885a0acb7dcca11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11a57fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11a57fa640 1 -- 172.21.15.37:0/649432311 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11ac005040 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11a57fa640 1 -- 172.21.15.37:0/649432311 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 8615+0+0 (secure 0 0 0) 0x7f11ac0051e0 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11b80bcc80 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11a57fa640 1 -- 172.21.15.37:0/649432311 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) ==== 24+0+0 (secure 0 0 0) 0x7f11ac0053e0 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 10 client.19227 fetch_fsmap learned FSMap version 43 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.283+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) -- 0x55eb0ced8060 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.292 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11a57fa640 1 -- 172.21.15.37:0/649432311 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f11b80bcc80 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11a57fa640 1 -- 172.21.15.37:0/649432311 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 43) ==== 8346+0+0 (secure 0 0 0) 0x7f11ac028620 con 0x55eb0ce4e2f0 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11c6e40740 10 client.19227 fetch_fsmap finished waiting for FSMap version 43 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11c6e40740 10 client.19227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11c6e40740 20 client.19227 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11c6e40740 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ced8470 0x55eb0ceda890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11c6e40740 4 client.19227 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55eb0cdb9d70 con 0x55eb0ced8470 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.284+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ced8470 0x55eb0ceda890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.285+0000 7f11a57fa640 10 client.19227 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.293 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.285+0000 7f11c1e4b640 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ced8470 0x55eb0ceda890 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9874c64553394884 server_cookie=83471a9f552f6129 in_seq=0 out_seq=0 2024-09-19T06:55:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:11 smithi151 ceph-mon[35236]: pgmap v424: 97 pgs: 40 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 35 active+clean; 66 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 731 KiB/s rd, 2.0 MiB/s wr, 128 op/s 2024-09-19T06:55:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.379+0000 7f11a57fa640 1 -- 172.21.15.37:0/649432311 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55eb0cdb9d70 con 0x55eb0ced8470 2024-09-19T06:55:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.379+0000 7f11a57fa640 10 client.19227 handle_command_reply: tid=0 2024-09-19T06:55:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.379+0000 7f11c6e40740 1 client.19227 shutdown 2024-09-19T06:55:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.380+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4e2f0 msgr2=0x55eb0ce61530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.380+0000 7f11c6e40740 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce61530 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00bbc0 tx=0x7f11ac005a70 comp rx=0 tx=0).stop 2024-09-19T06:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 shutdown_connections 2024-09-19T06:55:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55eb0ced8470 0x55eb0ceda890 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11a57fa640 0 client.19227 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55eb0ce5d0c0 0x55eb0ce61a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55eb0ce4e2f0 0x55eb0ce61530 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 --2- 172.21.15.37:0/649432311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55eb0ce4d9d0 0x55eb0ce60ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 >> 172.21.15.37:0/649432311 conn(0x55eb0ce4a060 msgr2=0x55eb0ce489a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 shutdown_connections 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.384+0000 7f11c6e40740 1 -- 172.21.15.37:0/649432311 wait complete. 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.385+0000 7f11c6e40740 20 client.19227 trim_cache size 0 max 16384 2024-09-19T06:55:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:55:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:55:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:55:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.385+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11a80a34e0 msgr2=0x7f11a80a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.385+0000 7f11c40af640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11a80a34e0 0x7f11a80a59a0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f11b4005fd0 tx=0x7f11b4004a40 comp rx=0 tx=0).stop 2024-09-19T06:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 msgr2=0x7f11bc198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc198420 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f11b8006010 tx=0x7f11b80042f0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 shutdown_connections 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f11bc10c110 0x7f11bc19cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f11bc1035e0 0x7f11bc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f11a80a34e0 0x7f11a80a59a0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 --2- 172.21.15.37:0/16233210 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f11bc102cc0 0x7f11bc198420 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 >> 172.21.15.37:0/16233210 conn(0x7f11bc0fc6b0 msgr2=0x7f11bc0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 shutdown_connections 2024-09-19T06:55:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:12.386+0000 7f11c40af640 1 -- 172.21.15.37:0/16233210 wait complete. 2024-09-19T06:55:12.895 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fdb3482b', quiesce took 6.2 seconds. Will release after: 16.0 2024-09-19T06:55:12.929 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:55:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setuser ceph since I am not root 2024-09-19T06:55:12.930 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:55:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setgroup ceph since I am not root 2024-09-19T06:55:12.930 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:55:12 smithi151 ceph-mds[66270]: -- 172.21.15.151:0/1911829330 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558a46f95860 con 0x558a46f73000 2024-09-19T06:55:12.930 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:55:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: starting mds.i at 2024-09-19T06:55:13.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.58 MB/sec execute 580 sec latency 1098.426 ms 2024-09-19T06:55:13.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:12 smithi096 ceph-mon[35004]: daemon mds.i restarted 2024-09-19T06:55:13.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:12 smithi037 ceph-mon[31211]: daemon mds.i restarted 2024-09-19T06:55:13.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:12 smithi151 ceph-mon[35236]: daemon mds.i restarted 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: pgmap v425: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6838/2835776386,v1:172.21.15.151:6839/2835776386] up:boot 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:55:13.933 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:55:13.934 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:55:13.934 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:13 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:55:14.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.57 MB/sec execute 581 sec latency 2098.528 ms 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: pgmap v425: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6838/2835776386,v1:172.21.15.151:6839/2835776386] up:boot 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:55:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:55:14.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:13 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:55:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: pgmap v425: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 2.8 MiB/s wr, 215 op/s 2024-09-19T06:55:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: osdmap e104: 12 total, 12 up, 12 in 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6838/2835776386,v1:172.21.15.151:6839/2835776386] up:boot 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:55:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:13 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:55:15.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.56 MB/sec execute 582 sec latency 3098.650 ms 2024-09-19T06:55:16.238 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.55 MB/sec execute 583 sec latency 4098.718 ms 2024-09-19T06:55:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:15 smithi096 ceph-mon[35004]: pgmap v427: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 268 op/s 2024-09-19T06:55:16.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:15 smithi096 ceph-mon[35004]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T06:55:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:15 smithi037 ceph-mon[31211]: pgmap v427: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 268 op/s 2024-09-19T06:55:16.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:15 smithi037 ceph-mon[31211]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T06:55:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:15 smithi151 ceph-mon[35236]: pgmap v427: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 3.5 MiB/s wr, 268 op/s 2024-09-19T06:55:16.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:15 smithi151 ceph-mon[35236]: osdmap e105: 12 total, 12 up, 12 in 2024-09-19T06:55:17.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.53 MB/sec execute 584 sec latency 5098.768 ms 2024-09-19T06:55:18.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.52 MB/sec execute 585 sec latency 6098.816 ms 2024-09-19T06:55:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:17 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:55:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:17 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:55:18.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:17 smithi096 ceph-mon[35004]: pgmap v429: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-19T06:55:18.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:17 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:55:18.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:17 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:55:18.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:17 smithi037 ceph-mon[31211]: pgmap v429: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-19T06:55:18.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:17 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:55:18.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:17 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:55:18.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:17 smithi151 ceph-mon[35236]: pgmap v429: 97 pgs: 97 active+clean; 62 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 2.2 MiB/s wr, 194 op/s 2024-09-19T06:55:19.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.51 MB/sec execute 586 sec latency 7098.904 ms 2024-09-19T06:55:20.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.49 MB/sec execute 587 sec latency 8099.048 ms 2024-09-19T06:55:20.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:19 smithi096 ceph-mon[35004]: pgmap v430: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 212 op/s 2024-09-19T06:55:20.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:19 smithi037 ceph-mon[31211]: pgmap v430: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 212 op/s 2024-09-19T06:55:20.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:19 smithi151 ceph-mon[35236]: pgmap v430: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 2.3 MiB/s wr, 212 op/s 2024-09-19T06:55:21.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.48 MB/sec execute 588 sec latency 9099.173 ms 2024-09-19T06:55:22.241 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.47 MB/sec execute 589 sec latency 10099.222 ms 2024-09-19T06:55:22.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:21 smithi096 ceph-mon[35004]: pgmap v431: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 121 KiB/s wr, 17 op/s 2024-09-19T06:55:22.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:21 smithi037 ceph-mon[31211]: pgmap v431: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 121 KiB/s wr, 17 op/s 2024-09-19T06:55:22.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:21 smithi151 ceph-mon[35236]: pgmap v431: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 121 KiB/s wr, 17 op/s 2024-09-19T06:55:23.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.46 MB/sec execute 590 sec latency 11099.340 ms 2024-09-19T06:55:24.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.44 MB/sec execute 591 sec latency 12099.456 ms 2024-09-19T06:55:24.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:23 smithi096 ceph-mon[35004]: pgmap v432: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 229 KiB/s wr, 23 op/s 2024-09-19T06:55:24.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:23 smithi037 ceph-mon[31211]: pgmap v432: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 229 KiB/s wr, 23 op/s 2024-09-19T06:55:24.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:23 smithi151 ceph-mon[35236]: pgmap v432: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 229 KiB/s wr, 23 op/s 2024-09-19T06:55:25.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.43 MB/sec execute 592 sec latency 13099.603 ms 2024-09-19T06:55:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 -- 172.21.15.37:0/4051107276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d481035e0 msgr2=0x7f4d4810bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 --2- 172.21.15.37:0/4051107276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d481035e0 0x7f4d4810bac0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c009900 tx=0x7f4d3c02a480 comp rx=0 tx=0).stop 2024-09-19T06:55:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 -- 172.21.15.37:0/4051107276 shutdown_connections 2024-09-19T06:55:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 --2- 172.21.15.37:0/4051107276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d4810c110 0x7f4d4810e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.495 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 --2- 172.21.15.37:0/4051107276 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d481035e0 0x7f4d4810bac0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 --2- 172.21.15.37:0/4051107276 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48102cc0 0x7f4d481030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 -- 172.21.15.37:0/4051107276 >> 172.21.15.37:0/4051107276 conn(0x7f4d480fc6b0 msgr2=0x7f4d480fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 -- 172.21.15.37:0/4051107276 shutdown_connections 2024-09-19T06:55:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.494+0000 7f4d4d482640 1 -- 172.21.15.37:0/4051107276 wait complete. 2024-09-19T06:55:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 Processor -- start 2024-09-19T06:55:25.496 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 -- start start 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48102cc0 0x7f4d481983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d481035e0 0x7f4d48198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d4810c110 0x7f4d4819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48111ef0 con 0x7f4d4810c110 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48111d70 con 0x7f4d481035e0 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d4d482640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d48112070 con 0x7f4d48102cc0 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d4810c110 0x7f4d4819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d477fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d4810c110 0x7f4d4819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41560/0 (socket says 172.21.15.37:41560) 2024-09-19T06:55:25.498 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48102cc0 0x7f4d481983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d477fe640 1 -- 172.21.15.37:0/3506240021 learned_addr learned my addr 172.21.15.37:0/3506240021 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.495+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d481035e0 0x7f4d48198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d477fe640 1 -- 172.21.15.37:0/3506240021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48102cc0 msgr2=0x7f4d481983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d477fe640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48102cc0 0x7f4d481983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d477fe640 1 -- 172.21.15.37:0/3506240021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d481035e0 msgr2=0x7f4d48198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d477fe640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d481035e0 0x7f4d48198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d477fe640 1 -- 172.21.15.37:0/3506240021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4819d3d0 con 0x7f4d4810c110 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d477fe640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d4810c110 0x7f4d4819cca0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009fd0 tx=0x7f4d4000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d2ffff640 1 -- 172.21.15.37:0/3506240021 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d40017070 con 0x7f4d4810c110 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d4819d6c0 con 0x7f4d4810c110 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d2ffff640 1 -- 172.21.15.37:0/3506240021 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d40003e50 con 0x7f4d4810c110 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d2ffff640 1 -- 172.21.15.37:0/3506240021 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d400106c0 con 0x7f4d4810c110 2024-09-19T06:55:25.499 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.496+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d481a4f50 con 0x7f4d4810c110 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d2ffff640 1 -- 172.21.15.37:0/3506240021 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4d40003970 con 0x7f4d4810c110 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 Processor -- start 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 -- start start 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492139d0 0x559249213db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x559249222b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492230c0 0x5592492254b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5592491782f0 con 0x5592492230c0 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559249120270 con 0x5592492142f0 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d50222740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5592491742b0 con 0x5592492139d0 2024-09-19T06:55:25.505 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492230c0 0x5592492254b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492230c0 0x5592492254b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41576/0 (socket says 172.21.15.37:41576) 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d2ffff640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4d300a3460 0x7f4d300a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d467fc640 1 -- 172.21.15.37:0/3813703627 learned_addr learned my addr 172.21.15.37:0/3813703627 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.499+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x559249222b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492139d0 0x559249213db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d477fe640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4d300a3460 0x7f4d300a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d2ffff640 1 -- 172.21.15.37:0/3506240021 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f4d400cfa80 con 0x7f4d4810c110 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d467fc640 1 -- 172.21.15.37:0/3813703627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492139d0 msgr2=0x559249213db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492139d0 0x559249213db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d467fc640 1 -- 172.21.15.37:0/3813703627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 msgr2=0x559249222b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x559249222b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.506 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d467fc640 1 -- 172.21.15.37:0/3813703627 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924911f340 con 0x5592492230c0 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492230c0 0x5592492254b0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c033000 tx=0x7f4d3c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db9994db11ab1b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d477fe640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4d300a3460 0x7f4d300a5920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4d4009ba30 tx=0x7f4d40004020 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d2d7fa640 1 -- 172.21.15.37:0/3813703627 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c035040 con 0x5592492230c0 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d2d7fa640 1 -- 172.21.15.37:0/3813703627 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c0035f0 con 0x5592492230c0 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.500+0000 7f4d2d7fa640 1 -- 172.21.15.37:0/3813703627 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c03d680 con 0x5592492230c0 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 -- 172.21.15.37:0/3813703627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492230c0 msgr2=0x5592492254b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492230c0 0x5592492254b0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c033000 tx=0x7f4d3c0056d0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 -- 172.21.15.37:0/3813703627 shutdown_connections 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492230c0 0x5592492254b0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x559249222b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 --2- 172.21.15.37:0/3813703627 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492139d0 0x559249213db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 -- 172.21.15.37:0/3813703627 >> 172.21.15.37:0/3813703627 conn(0x559249210060 msgr2=0x559249210440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 -- 172.21.15.37:0/3813703627 shutdown_connections 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.501+0000 7f4d50222740 1 -- 172.21.15.37:0/3813703627 wait complete. 2024-09-19T06:55:25.507 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 -- start start 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 Processor -- start 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492139d0 0x559249220040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492142f0 0x559249220580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x559249220ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x559249225c90 con 0x5592492142f0 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559249101290 con 0x5592492139d0 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x559249225e10 con 0x5592492230c0 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d50222740 10 client.0 _ll_register_callbacks cb 0x559249132140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492142f0 0x559249220580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492142f0 0x559249220580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41582/0 (socket says 172.21.15.37:41582) 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3369156730 learned_addr learned my addr 172.21.15.37:0/3369156730 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:25.508 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x559249220ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492139d0 0x559249220040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3369156730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 msgr2=0x559249220ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x559249220ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492139d0 msgr2=0x559249220040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492139d0 0x559249220040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.502+0000 7f4d46ffd640 1 -- 172.21.15.37:0/3369156730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559248e8b210 con 0x5592492142f0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492142f0 0x559249220580 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f4d38010820 tx=0x7f4d38010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422dd7ff96c978d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d2aff5640 1 -- 172.21.15.37:0/3369156730 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3801e060 con 0x5592492142f0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d2aff5640 1 -- 172.21.15.37:0/3369156730 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f4d3800fb90 con 0x5592492142f0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c0035f0 con 0x5592492142f0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d2aff5640 1 -- 172.21.15.37:0/3369156730 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4d3800fd90 con 0x5592492142f0 2024-09-19T06:55:25.509 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 10 client.19242 fetch_fsmap learned FSMap version 45 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x559248f57620 con 0x5592492142f0 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d2aff5640 1 -- 172.21.15.37:0/3369156730 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c0035f0 con 0x5592492142f0 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d2aff5640 1 -- 172.21.15.37:0/3369156730 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f4d3800e4f0 con 0x5592492142f0 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 10 client.19242 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 10 client.19242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 20 client.19242 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559249233100 0x559249235510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 4 client.19242 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.503+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559249185a20 con 0x559249233100 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.504+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559249233100 0x559249235510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.504+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559249233100 0x559249235510 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606d5b4e5556f9b3 server_cookie=ea85f69fd7f0ecf3 in_seq=0 out_seq=0 2024-09-19T06:55:25.510 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.504+0000 7f4d2aff5640 10 client.19242 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d2aff5640 1 -- 172.21.15.37:0/3369156730 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x559249185a20 con 0x559249233100 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d2aff5640 10 client.19242 handle_command_reply: tid=0 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 client.19242 shutdown 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492142f0 msgr2=0x559249220580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492142f0 0x559249220580 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f4d38010820 tx=0x7f4d38010ce0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 shutdown_connections 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x559249233100 0x559249235510 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x559249220ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492142f0 0x559249220580 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d2aff5640 0 client.19242 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 --2- 172.21.15.37:0/3369156730 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492139d0 0x559249220040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.512 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 >> 172.21.15.37:0/3369156730 conn(0x559249210060 msgr2=0x559249215dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 shutdown_connections 2024-09-19T06:55:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 1 -- 172.21.15.37:0/3369156730 wait complete. 2024-09-19T06:55:25.513 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.505+0000 7f4d50222740 20 client.19242 trim_cache size 0 max 16384 2024-09-19T06:55:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 Processor -- start 2024-09-19T06:55:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 -- start start 2024-09-19T06:55:25.524 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x5592492354f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x55924923ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x55924923b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481adfb0 con 0x5592492139d0 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481ab880 con 0x5592492142f0 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d50222740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481a96f0 con 0x5592492230c0 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x5592492354f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x5592492354f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41586/0 (socket says 172.21.15.37:41586) 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 -- 172.21.15.37:0/3022646403 learned_addr learned my addr 172.21.15.37:0/3022646403 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x55924923b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x55924923ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 -- 172.21.15.37:0/3022646403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 msgr2=0x55924923b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.525 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x55924923b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 -- 172.21.15.37:0/3022646403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 msgr2=0x55924923ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x55924923ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 -- 172.21.15.37:0/3022646403 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55924921efd0 con 0x5592492139d0 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.520+0000 7f4d467fc640 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x5592492354f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c031710 tx=0x7f4d3c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b84f374bd03ff2c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d2cff9640 1 -- 172.21.15.37:0/3022646403 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c035040 con 0x5592492139d0 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d2cff9640 1 -- 172.21.15.37:0/3022646403 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c003ad0 con 0x5592492139d0 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d2cff9640 1 -- 172.21.15.37:0/3022646403 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c004310 con 0x5592492139d0 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 -- 172.21.15.37:0/3022646403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 msgr2=0x5592492354f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x5592492354f0 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c031710 tx=0x7f4d3c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 -- 172.21.15.37:0/3022646403 shutdown_connections 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492230c0 0x55924923b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.526 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492142f0 0x55924923ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 --2- 172.21.15.37:0/3022646403 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x5592492354f0 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 -- 172.21.15.37:0/3022646403 >> 172.21.15.37:0/3022646403 conn(0x559249210060 msgr2=0x5592492220c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 -- 172.21.15.37:0/3022646403 shutdown_connections 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.521+0000 7f4d50222740 1 -- 172.21.15.37:0/3022646403 wait complete. 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 -- start start 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 Processor -- start 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x559249226f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492142f0 0x559249227470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492230c0 0x5592492279b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481a9d40 con 0x5592492139d0 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481ab1b0 con 0x5592492230c0 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4d481aa410 con 0x5592492142f0 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d50222740 10 client.0 _ll_register_callbacks cb 0x7f4d3800a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:25.527 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x559249226f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x559249226f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:41592/0 (socket says 172.21.15.37:41592) 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 -- 172.21.15.37:0/1523129855 learned_addr learned my addr 172.21.15.37:0/1523129855 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492142f0 0x559249227470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492230c0 0x5592492279b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 -- 172.21.15.37:0/1523129855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492142f0 msgr2=0x559249227470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492142f0 0x559249227470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 -- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492230c0 msgr2=0x5592492279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492230c0 0x5592492279b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 -- 172.21.15.37:0/1523129855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55924929cef0 con 0x5592492139d0 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d467fc640 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x559249226f30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c02a4c0 tx=0x7f4d3c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60fcf565fab38b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.522+0000 7f4d2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:25.528 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d2aff5640 1 -- 172.21.15.37:0/1523129855 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c035070 con 0x5592492139d0 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d2aff5640 1 -- 172.21.15.37:0/1523129855 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f4d3c003830 con 0x5592492139d0 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4d3c003ad0 con 0x5592492139d0 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d2aff5640 1 -- 172.21.15.37:0/1523129855 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7f4d3c004090 con 0x5592492139d0 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 10 client.19248 fetch_fsmap learned FSMap version 45 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55924929e1e0 con 0x5592492139d0 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d2aff5640 1 -- 172.21.15.37:0/1523129855 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4d3c003ad0 con 0x5592492139d0 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d2aff5640 1 -- 172.21.15.37:0/1523129855 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7f4d3c064020 con 0x5592492139d0 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 10 client.19248 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 10 client.19248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 20 client.19248 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55924929e3b0 0x5592492a07a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:25.529 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 4 client.19248 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.523+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55924917fd70 con 0x55924929e3b0 2024-09-19T06:55:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.524+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55924929e3b0 0x5592492a07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.524+0000 7f4d46ffd640 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55924929e3b0 0x5592492a07a0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6489aaf7824cd54 server_cookie=1ec8c57e2319334c in_seq=0 out_seq=0 2024-09-19T06:55:25.530 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.524+0000 7f4d2aff5640 10 client.19248 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:25.563 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d2aff5640 1 -- 172.21.15.37:0/1523129855 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55924917fd70 con 0x55924929e3b0 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d2aff5640 10 client.19248 handle_command_reply: tid=0 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 client.19248 shutdown 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 msgr2=0x559249226f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x559249226f30 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f4d3c02a4c0 tx=0x7f4d3c003f90 comp rx=0 tx=0).stop 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 shutdown_connections 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55924929e3b0 0x5592492a07a0 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5592492230c0 0x5592492279b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d2aff5640 0 client.19248 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5592492142f0 0x559249227470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 --2- 172.21.15.37:0/1523129855 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5592492139d0 0x559249226f30 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.564 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.562+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 >> 172.21.15.37:0/1523129855 conn(0x559249210060 msgr2=0x5592492155a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 shutdown_connections 2024-09-19T06:55:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d50222740 1 -- 172.21.15.37:0/1523129855 wait complete. 2024-09-19T06:55:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d50222740 20 client.19248 trim_cache size 0 max 16384 2024-09-19T06:55:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:55:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:55:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:55:25.565 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:55:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4d300a3460 msgr2=0x7f4d300a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4d300a3460 0x7f4d300a5920 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4d4009ba30 tx=0x7f4d40004020 comp rx=0 tx=0).stop 2024-09-19T06:55:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d4810c110 msgr2=0x7f4d4819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d4810c110 0x7f4d4819cca0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f4d40009fd0 tx=0x7f4d4000b260 comp rx=0 tx=0).stop 2024-09-19T06:55:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 shutdown_connections 2024-09-19T06:55:25.566 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f4d300a3460 0x7f4d300a5920 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4d4810c110 0x7f4d4819cca0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4d481035e0 0x7f4d48198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 --2- 172.21.15.37:0/3506240021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d48102cc0 0x7f4d481983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 >> 172.21.15.37:0/3506240021 conn(0x7f4d480fc6b0 msgr2=0x7f4d480fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 shutdown_connections 2024-09-19T06:55:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:25.563+0000 7f4d4d482640 1 -- 172.21.15.37:0/3506240021 wait complete. 2024-09-19T06:55:26.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.42 MB/sec execute 593 sec latency 14099.734 ms 2024-09-19T06:55:26.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:25 smithi096 ceph-mon[35004]: pgmap v433: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 KiB/s wr, 23 op/s 2024-09-19T06:55:26.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:25 smithi037 ceph-mon[31211]: pgmap v433: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 KiB/s wr, 23 op/s 2024-09-19T06:55:26.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:25 smithi151 ceph-mon[35236]: pgmap v433: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 226 KiB/s wr, 23 op/s 2024-09-19T06:55:27.239 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.41 MB/sec execute 594 sec latency 15099.853 ms 2024-09-19T06:55:28.240 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.39 MB/sec execute 595 sec latency 16099.972 ms 2024-09-19T06:55:28.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:27 smithi096 ceph-mon[35004]: pgmap v434: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 190 KiB/s wr, 19 op/s 2024-09-19T06:55:28.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:27 smithi037 ceph-mon[31211]: pgmap v434: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 190 KiB/s wr, 19 op/s 2024-09-19T06:55:28.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:27 smithi151 ceph-mon[35236]: pgmap v434: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 190 KiB/s wr, 19 op/s 2024-09-19T06:55:28.897 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:55:29.172 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:29.240 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.38 MB/sec execute 596 sec latency 17100.076 ms 2024-09-19T06:55:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.637+0000 7fcd645bc640 1 -- 172.21.15.37:0/2438377811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 msgr2=0x7fcd5c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.638 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.637+0000 7fcd645bc640 1 --2- 172.21.15.37:0/2438377811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c106e60 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009930 tx=0x7fcd5802a420 comp rx=0 tx=0).stop 2024-09-19T06:55:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 -- 172.21.15.37:0/2438377811 shutdown_connections 2024-09-19T06:55:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 --2- 172.21.15.37:0/2438377811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd5c101420 0x7fcd5c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 --2- 172.21.15.37:0/2438377811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd5c100a80 0x7fcd5c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 --2- 172.21.15.37:0/2438377811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c106e60 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 -- 172.21.15.37:0/2438377811 >> 172.21.15.37:0/2438377811 conn(0x7fcd5c0fc7a0 msgr2=0x7fcd5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 -- 172.21.15.37:0/2438377811 shutdown_connections 2024-09-19T06:55:29.639 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 -- 172.21.15.37:0/2438377811 wait complete. 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 Processor -- start 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 -- start start 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.638+0000 7fcd645bc640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd5c100a80 0x7fcd5c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd645bc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd5c101420 0x7fcd5c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd645bc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c19cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd645bc640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c110e80 con 0x7fcd5c106a80 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd645bc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c110d00 con 0x7fcd5c101420 2024-09-19T06:55:29.641 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd645bc640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd5c111000 con 0x7fcd5c100a80 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c19cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c19cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36622/0 (socket says 172.21.15.37:36622) 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 -- 172.21.15.37:0/482405531 learned_addr learned my addr 172.21.15.37:0/482405531 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62358640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd5c100a80 0x7fcd5c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd61b57640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd5c101420 0x7fcd5c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 -- 172.21.15.37:0/482405531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd5c100a80 msgr2=0x7fcd5c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd5c100a80 0x7fcd5c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 -- 172.21.15.37:0/482405531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd5c101420 msgr2=0x7fcd5c1989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd5c101420 0x7fcd5c1989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 -- 172.21.15.37:0/482405531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd5c19d420 con 0x7fcd5c106a80 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd61b57640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd5c101420 0x7fcd5c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:29.642 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd62b59640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c19cd40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400cce0 tx=0x7fcd5400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd537fe640 1 -- 172.21.15.37:0/482405531 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd5400f450 con 0x7fcd5c106a80 2024-09-19T06:55:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd537fe640 1 -- 172.21.15.37:0/482405531 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd5400fad0 con 0x7fcd5c106a80 2024-09-19T06:55:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.639+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd5c19d710 con 0x7fcd5c106a80 2024-09-19T06:55:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.640+0000 7fcd537fe640 1 -- 172.21.15.37:0/482405531 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd54018c10 con 0x7fcd5c106a80 2024-09-19T06:55:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.640+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd5c1a4ff0 con 0x7fcd5c106a80 2024-09-19T06:55:29.643 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.640+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd5c100ee0 con 0x7fcd5c106a80 2024-09-19T06:55:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.642+0000 7fcd537fe640 1 -- 172.21.15.37:0/482405531 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcd5400f5f0 con 0x7fcd5c106a80 2024-09-19T06:55:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.642+0000 7fcd537fe640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd480a33b0 0x7fcd480a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.643+0000 7fcd537fe640 1 -- 172.21.15.37:0/482405531 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fcd540d0460 con 0x7fcd5c106a80 2024-09-19T06:55:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.643+0000 7fcd62358640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd480a33b0 0x7fcd480a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:29.644 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.643+0000 7fcd62358640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd480a33b0 0x7fcd480a5870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009900 tx=0x7fcd580058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:29.646 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.646+0000 7fcd537fe640 1 -- 172.21.15.37:0/482405531 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcd54014030 con 0x7fcd5c106a80 2024-09-19T06:55:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.862+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcd5c1997d0 con 0x7fcd5c106a80 2024-09-19T06:55:29.863 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.863+0000 7fcd537fe640 1 -- 172.21.15.37:0/482405531 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 45 v45) ==== 94+0+11903 (secure 0 0 0) 0x7fcd5c1997d0 con 0x7fcd5c106a80 2024-09-19T06:55:29.864 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:55:29.864 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":45,"btime":"2024-09-19T06:55:12:938168+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":45,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:55:12.938155+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28132":{"gid":28132,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2835776386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2835776386},{"type":"v1","addr":"172.21.15.151:6839","nonce":2835776386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:55:29.864 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 45 2024-09-19T06:55:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.865+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd480a33b0 msgr2=0x7fcd480a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.865+0000 7fcd645bc640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd480a33b0 0x7fcd480a5870 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcd58009900 tx=0x7fcd580058d0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.866 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.865+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 msgr2=0x7fcd5c19cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.865+0000 7fcd645bc640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c19cd40 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7fcd5400cce0 tx=0x7fcd5400a620 comp rx=0 tx=0).stop 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 shutdown_connections 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd480a33b0 0x7fcd480a5870 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd5c106a80 0x7fcd5c19cd40 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd5c101420 0x7fcd5c1989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 --2- 172.21.15.37:0/482405531 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd5c100a80 0x7fcd5c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 >> 172.21.15.37:0/482405531 conn(0x7fcd5c0fc7a0 msgr2=0x7fcd5c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 shutdown_connections 2024-09-19T06:55:29.867 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:29.866+0000 7fcd645bc640 1 -- 172.21.15.37:0/482405531 wait complete. 2024-09-19T06:55:30.240 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.37 MB/sec execute 597 sec latency 18100.168 ms 2024-09-19T06:55:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:29 smithi096 ceph-mon[35004]: pgmap v435: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 220 KiB/s wr, 33 op/s 2024-09-19T06:55:30.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:29 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/482405531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:29 smithi037 ceph-mon[31211]: pgmap v435: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 220 KiB/s wr, 33 op/s 2024-09-19T06:55:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:29 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/482405531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:30.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:29 smithi151 ceph-mon[35236]: pgmap v435: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 220 KiB/s wr, 33 op/s 2024-09-19T06:55:30.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:29 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/482405531' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:30.534 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --set-id fdb3482b --release --await' 2024-09-19T06:55:30.535 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --set-id fdb3482b --release --await 2024-09-19T06:55:30.794 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:31.240 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1034957 7.36 MB/sec execute 598 sec latency 19100.233 ms 2024-09-19T06:55:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.268+0000 7feebfc97640 1 -- 172.21.15.37:0/875520498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 msgr2=0x7feeb8100cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.268+0000 7feebfc97640 1 --2- 172.21.15.37:0/875520498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb8100cb0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7feeb4009900 tx=0x7feeb402a410 comp rx=0 tx=0).stop 2024-09-19T06:55:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.268+0000 7feebfc97640 1 -- 172.21.15.37:0/875520498 shutdown_connections 2024-09-19T06:55:31.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.268+0000 7feebfc97640 1 --2- 172.21.15.37:0/875520498 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb810e480 0x7feeb8110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.268+0000 7feebfc97640 1 --2- 172.21.15.37:0/875520498 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8101280 0x7feeb810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.268+0000 7feebfc97640 1 --2- 172.21.15.37:0/875520498 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb8100cb0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.268+0000 7feebfc97640 1 -- 172.21.15.37:0/875520498 >> 172.21.15.37:0/875520498 conn(0x7feeb80fc7a0 msgr2=0x7feeb80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 -- 172.21.15.37:0/875520498 shutdown_connections 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 -- 172.21.15.37:0/875520498 wait complete. 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 Processor -- start 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 -- start start 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb81a03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8101280 0x7feeb81a0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb810e480 0x7feeb819a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8114200 con 0x7feeb81008d0 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8114080 con 0x7feeb810e480 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.269+0000 7feebfc97640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feeb8114380 con 0x7feeb8101280 2024-09-19T06:55:31.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb81a03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb81a03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36646/0 (socket says 172.21.15.37:36646) 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 -- 172.21.15.37:0/3678274382 learned_addr learned my addr 172.21.15.37:0/3678274382 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebe234640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb810e480 0x7feeb819a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebd232640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8101280 0x7feeb81a0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 -- 172.21.15.37:0/3678274382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8101280 msgr2=0x7feeb81a0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8101280 0x7feeb81a0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 -- 172.21.15.37:0/3678274382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb810e480 msgr2=0x7feeb819a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb810e480 0x7feeb819a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 -- 172.21.15.37:0/3678274382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb819acc0 con 0x7feeb81008d0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebda33640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb81a03d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006010 tx=0x7feeb40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feeaeffd640 1 -- 172.21.15.37:0/3678274382 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb403f030 con 0x7feeb81008d0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb819af50 con 0x7feeb81008d0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feeaeffd640 1 -- 172.21.15.37:0/3678274382 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb4003ce0 con 0x7feeb81008d0 2024-09-19T06:55:31.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feeaeffd640 1 -- 172.21.15.37:0/3678274382 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4035070 con 0x7feeb81008d0 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.270+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb81a72a0 con 0x7feeb81008d0 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.273+0000 7feeaeffd640 1 -- 172.21.15.37:0/3678274382 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7feeb4042030 con 0x7feeb81008d0 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.273+0000 7feec2070740 1 Processor -- start 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.273+0000 7feec2070740 1 -- start start 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feec2070740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b7368440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feec2070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7372d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feec2070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7375630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feec2070740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e2b72cd620 con 0x55e2b7368060 2024-09-19T06:55:31.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feec2070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e2b7274e50 con 0x55e2b7368980 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feec2070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e2b72c95e0 con 0x55e2b7373240 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b7368440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b7368440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36652/0 (socket says 172.21.15.37:36652) 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 -- 172.21.15.37:0/1497927046 learned_addr learned my addr 172.21.15.37:0/1497927046 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebd232640 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7375630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feeaeffd640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feea40a33b0 0x7feea40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebda33640 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7372d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 -- 172.21.15.37:0/1497927046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 msgr2=0x55e2b7375630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7375630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 -- 172.21.15.37:0/1497927046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 msgr2=0x55e2b7372d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7372d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 -- 172.21.15.37:0/1497927046 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b7265d40 con 0x55e2b7368060 2024-09-19T06:55:31.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feeaeffd640 1 -- 172.21.15.37:0/3678274382 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7feeb40f0380 con 0x7feeb81008d0 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feea40a33b0 0x7feea40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.274+0000 7feebe234640 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b7368440 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7feeb0010510 tx=0x7feeb00109d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00550e139efd97e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feea3fff640 1 -- 172.21.15.37:0/1497927046 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb001d400 con 0x55e2b7368060 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feea3fff640 1 -- 172.21.15.37:0/1497927046 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb001da80 con 0x55e2b7368060 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feea3fff640 1 -- 172.21.15.37:0/1497927046 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb001c9a0 con 0x55e2b7368060 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feec2070740 1 -- 172.21.15.37:0/1497927046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 msgr2=0x55e2b7368440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feebe234640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feea40a33b0 0x7feea40a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feeb0021000 tx=0x7feeb001e000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feec2070740 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b7368440 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7feeb0010510 tx=0x7feeb00109d0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feec2070740 1 -- 172.21.15.37:0/1497927046 shutdown_connections 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feec2070740 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7375630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feec2070740 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7372d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feec2070740 1 --2- 172.21.15.37:0/1497927046 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b7368440 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.275+0000 7feec2070740 1 -- 172.21.15.37:0/1497927046 >> 172.21.15.37:0/1497927046 conn(0x55e2b73645f0 msgr2=0x55e2b73649d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 -- 172.21.15.37:0/1497927046 shutdown_connections 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 -- 172.21.15.37:0/1497927046 wait complete. 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 -- start start 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 Processor -- start 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b7370120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7370660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7370ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55e2b7375e10 con 0x55e2b7368980 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e2b7255f10 con 0x55e2b7368060 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55e2b7375f90 con 0x55e2b7373240 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feec2070740 10 client.0 _ll_register_callbacks cb 0x55e2b7279db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.276+0000 7feebd232640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7370660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7370660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36666/0 (socket says 172.21.15.37:36666) 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 -- 172.21.15.37:0/3139361326 learned_addr learned my addr 172.21.15.37:0/3139361326 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebe234640 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7370ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebda33640 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b7370120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 -- 172.21.15.37:0/3139361326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 msgr2=0x55e2b7370ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7370ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 -- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 msgr2=0x55e2b7370120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b7370120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 -- 172.21.15.37:0/3139361326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b6fdc580 con 0x55e2b7368980 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feebd232640 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7370660 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7feea8010500 tx=0x7feea80109c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9e5ac7383dd865 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feea17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feea17fa640 1 -- 172.21.15.37:0/3139361326 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea801a020 con 0x55e2b7368980 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feea17fa640 1 -- 172.21.15.37:0/3139361326 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7feea8016690 con 0x55e2b7368980 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.277+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feeb001da80 con 0x55e2b7368980 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feea17fa640 1 -- 172.21.15.37:0/3139361326 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7feea8025050 con 0x55e2b7368980 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 10 client.19266 fetch_fsmap learned FSMap version 45 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e2b70a6960 con 0x55e2b7368980 2024-09-19T06:55:31.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feea17fa640 1 -- 172.21.15.37:0/3139361326 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb001da80 con 0x55e2b7368980 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feea17fa640 1 -- 172.21.15.37:0/3139361326 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7feea8014b60 con 0x55e2b7368980 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 10 client.19266 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 10 client.19266 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 20 client.19266 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b7382e60 0x55e2b7385300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 4 client.19266 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2b72dab90 con 0x55e2b7382e60 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.278+0000 7feebda33640 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b7382e60 0x55e2b7385300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.279+0000 7feebda33640 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b7382e60 0x55e2b7385300 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76d4e8a8cc9b7dfd server_cookie=21ec7d61d285c8c7 in_seq=0 out_seq=0 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.279+0000 7feea17fa640 10 client.19266 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.279+0000 7feea17fa640 1 -- 172.21.15.37:0/3139361326 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e2b72dab90 con 0x55e2b7382e60 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.279+0000 7feea17fa640 10 client.19266 handle_command_reply: tid=0 2024-09-19T06:55:31.290 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.279+0000 7feec2070740 1 client.19266 shutdown 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 msgr2=0x55e2b7370660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7370660 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7feea8010500 tx=0x7feea80109c0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 shutdown_connections 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b7382e60 0x55e2b7385300 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b7370ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7370660 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feea17fa640 0 client.19266 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 --2- 172.21.15.37:0/3139361326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b7370120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 >> 172.21.15.37:0/3139361326 conn(0x55e2b73645f0 msgr2=0x55e2b736a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 shutdown_connections 2024-09-19T06:55:31.291 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 1 -- 172.21.15.37:0/3139361326 wait complete. 2024-09-19T06:55:31.292 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.280+0000 7feec2070740 20 client.19266 trim_cache size 0 max 16384 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 Processor -- start 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 -- start start 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b73649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7363640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b738ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81b2490 con 0x55e2b7368980 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81afd60 con 0x55e2b7368060 2024-09-19T06:55:31.299 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.297+0000 7feec2070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81adbd0 con 0x55e2b7373240 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7363640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7363640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36674/0 (socket says 172.21.15.37:36674) 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 -- 172.21.15.37:0/4191975657 learned_addr learned my addr 172.21.15.37:0/4191975657 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebe234640 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b738ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebda33640 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b73649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 -- 172.21.15.37:0/4191975657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 msgr2=0x55e2b738ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b738ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 -- 172.21.15.37:0/4191975657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 msgr2=0x55e2b73649d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b73649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 -- 172.21.15.37:0/4191975657 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2b7372060 con 0x55e2b7368980 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebe234640 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b738ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feebd232640 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7363640 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7feea8014f80 tx=0x7feea800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdb722f20dec991 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:31.300 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feea37fe640 1 -- 172.21.15.37:0/4191975657 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea801a040 con 0x55e2b7368980 2024-09-19T06:55:31.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feea37fe640 1 -- 172.21.15.37:0/4191975657 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feea8012e10 con 0x55e2b7368980 2024-09-19T06:55:31.301 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.298+0000 7feea37fe640 1 -- 172.21.15.37:0/4191975657 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea8016c70 con 0x55e2b7368980 2024-09-19T06:55:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 -- 172.21.15.37:0/4191975657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 msgr2=0x55e2b7363640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7363640 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7feea8014f80 tx=0x7feea800b480 comp rx=0 tx=0).stop 2024-09-19T06:55:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 -- 172.21.15.37:0/4191975657 shutdown_connections 2024-09-19T06:55:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b738ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.304 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368980 0x55e2b7363640 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 --2- 172.21.15.37:0/4191975657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368060 0x55e2b73649d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 -- 172.21.15.37:0/4191975657 >> 172.21.15.37:0/4191975657 conn(0x55e2b73645f0 msgr2=0x55e2b7362f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 -- 172.21.15.37:0/4191975657 shutdown_connections 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 -- 172.21.15.37:0/4191975657 wait complete. 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 -- start start 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.299+0000 7feec2070740 1 Processor -- start 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feec2070740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b73778c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feec2070740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7377e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feec2070740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b73e6c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feec2070740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ae220 con 0x55e2b7368060 2024-09-19T06:55:31.305 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feec2070740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81af690 con 0x55e2b7368980 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feec2070740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7feeb81ae8f0 con 0x55e2b7373240 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feec2070740 10 client.0 _ll_register_callbacks cb 0x7feea800bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b73778c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b73778c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36690/0 (socket says 172.21.15.37:36690) 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 -- 172.21.15.37:0/4024292519 learned_addr learned my addr 172.21.15.37:0/4024292519 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebd232640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7377e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebe234640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b73e6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 -- 172.21.15.37:0/4024292519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 msgr2=0x55e2b73e6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b73e6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.306 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 -- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 msgr2=0x55e2b7377e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7377e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 -- 172.21.15.37:0/4024292519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2b73e72c0 con 0x55e2b7368060 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebe234640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b73e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feebda33640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b73778c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7feeb40bc680 tx=0x7feeb40bc2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b481c0f39341273b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.300+0000 7feea17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feea17fa640 1 -- 172.21.15.37:0/4024292519 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feeb4049040 con 0x55e2b7368060 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feea17fa640 1 -- 172.21.15.37:0/4024292519 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7feeb40edd20 con 0x55e2b7368060 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feea8012e10 con 0x55e2b7368060 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feea17fa640 1 -- 172.21.15.37:0/4024292519 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) ==== 24+0+0 (secure 0 0 0) 0x7feeb40edf20 con 0x55e2b7368060 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 10 client.19272 fetch_fsmap learned FSMap version 45 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) -- 0x55e2b73ee640 con 0x55e2b7368060 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feea17fa640 1 -- 172.21.15.37:0/4024292519 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feeb403b580 con 0x55e2b7368060 2024-09-19T06:55:31.307 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feea17fa640 1 -- 172.21.15.37:0/4024292519 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 45) ==== 8346+0+0 (secure 0 0 0) 0x7feeb40c4ed0 con 0x55e2b7368060 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 10 client.19272 fetch_fsmap finished waiting for FSMap version 45 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 10 client.19272 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 20 client.19272 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b73eed10 0x55e2b73f1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 4 client.19272 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fdb3482b", "release": true, "await": true}] 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.301+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fdb3482b", "release": true, "await": true}) -- 0x7feea8012e10 con 0x55e2b73eed10 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.302+0000 7feebd232640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b73eed10 0x55e2b73f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.302+0000 7feebd232640 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b73eed10 0x55e2b73f1100 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2565c193e1b7696 server_cookie=10c0027298c19f6e in_seq=0 out_seq=0 2024-09-19T06:55:31.308 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.302+0000 7feea17fa640 10 client.19272 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.311+0000 7feea17fa640 1 -- 172.21.15.37:0/4024292519 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+591 (crc 0 0 0) 0x7feea8012e10 con 0x55e2b73eed10 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.311+0000 7feea17fa640 10 client.19272 handle_command_reply: tid=0 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 client.19272 shutdown 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 msgr2=0x55e2b73778c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b73778c0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7feeb40bc680 tx=0x7feeb40bc2a0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 shutdown_connections 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55e2b73eed10 0x55e2b73f1100 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55e2b7373240 0x55e2b73e6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e2b7368980 0x55e2b7377e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.313 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feea17fa640 0 client.19272 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 --2- 172.21.15.37:0/4024292519 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55e2b7368060 0x55e2b73778c0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 >> 172.21.15.37:0/4024292519 conn(0x55e2b73645f0 msgr2=0x55e2b73708e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 shutdown_connections 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 1 -- 172.21.15.37:0/4024292519 wait complete. 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.312+0000 7feec2070740 20 client.19272 trim_cache size 0 max 16384 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 45, 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:55:31.314 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 48, 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": { 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "fdb3482b": { 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "version": 48, 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "age_ref": 0.0, 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "timeout": 90.0, 2024-09-19T06:55:31.315 INFO:teuthology.orchestra.run.smithi037.stdout: "expiration": 142.2, 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "members": { 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "file:/": { 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "excluded": false, 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "state": { 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RELEASED", 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: "age": 0.0 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:31.316 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:31.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feea40a33b0 msgr2=0x7feea40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feea40a33b0 0x7feea40a5870 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7feeb0021000 tx=0x7feeb001e000 comp rx=0 tx=0).stop 2024-09-19T06:55:31.317 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 msgr2=0x7feeb81a03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb81a03d0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7feeb4006010 tx=0x7feeb40042f0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 shutdown_connections 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feeb810e480 0x7feeb819a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feeb8101280 0x7feeb81a0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7feea40a33b0 0x7feea40a5870 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 --2- 172.21.15.37:0/3678274382 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7feeb81008d0 0x7feeb81a03d0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 >> 172.21.15.37:0/3678274382 conn(0x7feeb80fc7a0 msgr2=0x7feeb80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 shutdown_connections 2024-09-19T06:55:31.318 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:31.313+0000 7feebfc97640 1 -- 172.21.15.37:0/3678274382 wait complete. 2024-09-19T06:55:31.969 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:31 smithi037 ceph-mon[31211]: pgmap v436: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 139 KiB/s wr, 21 op/s 2024-09-19T06:55:32.001 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fdb3482b', total seconds elapsed: 25.3 2024-09-19T06:55:32.240 INFO:tasks.workunit.client.0.smithi037.stdout: 1 1037406 7.37 MB/sec execute 599 sec latency 19174.961 ms 2024-09-19T06:55:32.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:31 smithi096 ceph-mon[35004]: pgmap v436: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 139 KiB/s wr, 21 op/s 2024-09-19T06:55:32.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:31 smithi151 ceph-mon[35236]: pgmap v436: 97 pgs: 97 active+clean; 63 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 139 KiB/s wr, 21 op/s 2024-09-19T06:55:33.240 INFO:tasks.workunit.client.0.smithi037.stdout: 1 cleanup 600 sec 2024-09-19T06:55:33.382 INFO:tasks.workunit.client.0.smithi037.stdout: 0 cleanup 600 sec 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Operation Count AvgLat MaxLat 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: ---------------------------------------- 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: NTCreateX 140629 1.900 32082.052 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Close 103309 0.003 0.087 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Rename 5956 14.755 31032.986 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Unlink 28402 2.488 28101.399 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Deltree 4 78.241 104.668 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Mkdir 2 0.010 0.015 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Qpathinfo 127464 0.886 9890.914 2024-09-19T06:55:33.383 INFO:tasks.workunit.client.0.smithi037.stdout: Qfileinfo 22353 0.001 0.032 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: Qfsinfo 23378 0.178 38.440 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: Sfileinfo 11458 0.028 27.464 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: Find 49291 0.306 9507.136 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: WriteX 70213 0.054 64.652 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: ReadX 220453 0.023 51.678 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: LockX 458 0.836 3.767 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: UnlockX 458 0.701 2.093 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: Flush 9859 3.016 83.643 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-09-19T06:55:33.384 INFO:tasks.workunit.client.0.smithi037.stdout:Throughput 7.36963 MB/sec 1 clients 1 procs max_latency=32082.056 ms 2024-09-19T06:55:33.385 INFO:tasks.workunit.client.0.smithi037.stderr:+ dbench 10 2024-09-19T06:55:33.385 INFO:tasks.workunit.client.0.smithi037.stdout:dbench version 4.00 - Copyright Andrew Tridgell 1999-2004 2024-09-19T06:55:33.385 INFO:tasks.workunit.client.0.smithi037.stdout: 2024-09-19T06:55:33.385 INFO:tasks.workunit.client.0.smithi037.stdout:Running for 600 seconds with load '/usr/share/dbench/client.txt' and minimum warmup 120 secs 2024-09-19T06:55:33.385 INFO:tasks.workunit.client.0.smithi037.stdout:8 of 10 processes prepared for launch 0 sec 2024-09-19T06:55:33.484 INFO:tasks.workunit.client.0.smithi037.stdout:10 of 10 processes prepared for launch 0 sec 2024-09-19T06:55:33.484 INFO:tasks.workunit.client.0.smithi037.stdout:releasing clients 2024-09-19T06:55:34.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:33 smithi096 ceph-mon[35004]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 335 KiB/s wr, 43 op/s 2024-09-19T06:55:34.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:33 smithi037 ceph-mon[31211]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 335 KiB/s wr, 43 op/s 2024-09-19T06:55:34.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:33 smithi151 ceph-mon[35236]: pgmap v437: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 335 KiB/s wr, 43 op/s 2024-09-19T06:55:34.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 386 137.74 MB/sec warmup 1 sec latency 143.380 ms 2024-09-19T06:55:35.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 834 103.03 MB/sec warmup 2 sec latency 203.618 ms 2024-09-19T06:55:36.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:35 smithi096 ceph-mon[35004]: pgmap v438: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 227 KiB/s wr, 35 op/s 2024-09-19T06:55:36.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:35 smithi037 ceph-mon[31211]: pgmap v438: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 227 KiB/s wr, 35 op/s 2024-09-19T06:55:36.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:35 smithi151 ceph-mon[35236]: pgmap v438: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 227 KiB/s wr, 35 op/s 2024-09-19T06:55:36.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 1379 83.10 MB/sec warmup 3 sec latency 104.664 ms 2024-09-19T06:55:37.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 2192 67.39 MB/sec warmup 4 sec latency 121.183 ms 2024-09-19T06:55:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:37 smithi096 ceph-mon[35004]: pgmap v439: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 227 KiB/s wr, 35 op/s 2024-09-19T06:55:38.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:37 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:37 smithi037 ceph-mon[31211]: pgmap v439: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 227 KiB/s wr, 35 op/s 2024-09-19T06:55:38.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:37 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:38.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:37 smithi151 ceph-mon[35236]: pgmap v439: 97 pgs: 97 active+clean; 65 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 309 KiB/s rd, 227 KiB/s wr, 35 op/s 2024-09-19T06:55:38.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:37 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:38.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 2785 64.77 MB/sec warmup 5 sec latency 44.747 ms 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 -- 172.21.15.37:0/459887502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 msgr2=0x7fde50107f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 --2- 172.21.15.37:0/459887502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde50107f50 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009930 tx=0x7fde4c02a420 comp rx=0 tx=0).stop 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 -- 172.21.15.37:0/459887502 shutdown_connections 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 --2- 172.21.15.37:0/459887502 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50102470 0x7fde5010e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 --2- 172.21.15.37:0/459887502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50101ad0 0x7fde50101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 --2- 172.21.15.37:0/459887502 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde50107f50 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 -- 172.21.15.37:0/459887502 >> 172.21.15.37:0/459887502 conn(0x7fde500fd7d0 msgr2=0x7fde500ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 -- 172.21.15.37:0/459887502 shutdown_connections 2024-09-19T06:55:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.656+0000 7fde57ca5640 1 -- 172.21.15.37:0/459887502 wait complete. 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 Processor -- start 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 -- start start 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50101ad0 0x7fde50198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102470 0x7fde501988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde5019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde50111f20 con 0x7fde50107b70 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde50111da0 con 0x7fde50102470 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde57ca5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde501120a0 con 0x7fde50101ad0 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde56242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde5019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde56242640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde5019cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36718/0 (socket says 172.21.15.37:36718) 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde55a41640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50101ad0 0x7fde50198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde56242640 1 -- 172.21.15.37:0/3363229487 learned_addr learned my addr 172.21.15.37:0/3363229487 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.657+0000 7fde55240640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102470 0x7fde501988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde56242640 1 -- 172.21.15.37:0/3363229487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50101ad0 msgr2=0x7fde50198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde56242640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50101ad0 0x7fde50198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde56242640 1 -- 172.21.15.37:0/3363229487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102470 msgr2=0x7fde501988b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde56242640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102470 0x7fde501988b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde56242640 1 -- 172.21.15.37:0/3363229487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde5019d360 con 0x7fde50107b70 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde56242640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde5019cc30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fde4800eb10 tx=0x7fde4800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde46ffd640 1 -- 172.21.15.37:0/3363229487 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48004b20 con 0x7fde50107b70 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5019d650 con 0x7fde50107b70 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde46ffd640 1 -- 172.21.15.37:0/3363229487 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde48004cc0 con 0x7fde50107b70 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde46ffd640 1 -- 172.21.15.37:0/3363229487 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48010760 con 0x7fde50107b70 2024-09-19T06:55:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.658+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde501a4ee0 con 0x7fde50107b70 2024-09-19T06:55:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.661+0000 7fde5aa44740 1 Processor -- start 2024-09-19T06:55:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.661+0000 7fde46ffd640 1 -- 172.21.15.37:0/3363229487 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fde48005020 con 0x7fde50107b70 2024-09-19T06:55:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.661+0000 7fde5aa44740 1 -- start start 2024-09-19T06:55:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283e9fbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea0ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde5aa44740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56283e9602f0 con 0x56283e9fb9f0 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde5aa44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56283e908270 con 0x56283e9fc310 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde5aa44740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56283e95c2b0 con 0x56283ea0b0c0 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283e9fbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283e9fbdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36720/0 (socket says 172.21.15.37:36720) 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 -- 172.21.15.37:0/422645536 learned_addr learned my addr 172.21.15.37:0/422645536 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55a41640 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea0ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 -- 172.21.15.37:0/422645536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 msgr2=0x56283ea0d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 -- 172.21.15.37:0/422645536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 msgr2=0x56283ea0ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea0ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 -- 172.21.15.37:0/422645536 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283e907340 con 0x56283e9fb9f0 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde46ffd640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde3c0a3460 0x7fde3c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.662+0000 7fde55240640 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283e9fbdd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fde40005bc0 tx=0x7fde40005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92000d7abaec37dc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.663+0000 7fde46ffd640 1 -- 172.21.15.37:0/3363229487 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fde480d0380 con 0x7fde50107b70 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.663+0000 7fde55a41640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde3c0a3460 0x7fde3c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.663+0000 7fde3bfff640 1 -- 172.21.15.37:0/422645536 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4000f480 con 0x56283e9fb9f0 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.663+0000 7fde3bfff640 1 -- 172.21.15.37:0/422645536 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4000fb00 con 0x56283e9fb9f0 2024-09-19T06:55:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.663+0000 7fde3bfff640 1 -- 172.21.15.37:0/422645536 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40010b30 con 0x56283e9fb9f0 2024-09-19T06:55:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 -- 172.21.15.37:0/422645536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 msgr2=0x56283e9fbdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde55a41640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde3c0a3460 0x7fde3c0a5920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009900 tx=0x7fde4c0058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283e9fbdd0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fde40005bc0 tx=0x7fde40005f90 comp rx=0 tx=0).stop 2024-09-19T06:55:38.669 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 -- 172.21.15.37:0/422645536 shutdown_connections 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea0ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 --2- 172.21.15.37:0/422645536 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283e9fbdd0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 -- 172.21.15.37:0/422645536 >> 172.21.15.37:0/422645536 conn(0x56283e9f8060 msgr2=0x56283e9f8440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 -- 172.21.15.37:0/422645536 shutdown_connections 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 -- 172.21.15.37:0/422645536 wait complete. 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 -- start start 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.664+0000 7fde5aa44740 1 Processor -- start 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea07f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283e9fc310 0x56283ea084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283ea0b0c0 0x56283ea089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde5aa44740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56283ea0dd20 con 0x56283e9fb9f0 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde5aa44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56283e8e9290 con 0x56283ea0b0c0 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde5aa44740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56283ea0dea0 con 0x56283e9fc310 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde5aa44740 10 client.0 _ll_register_callbacks cb 0x56283e91a140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde55a41640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea07f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde55a41640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea07f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36726/0 (socket says 172.21.15.37:36726) 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde55a41640 1 -- 172.21.15.37:0/1916332535 learned_addr learned my addr 172.21.15.37:0/1916332535 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde56242640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283ea0b0c0 0x56283ea089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde55240640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283e9fc310 0x56283ea084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde56242640 1 -- 172.21.15.37:0/1916332535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283e9fc310 msgr2=0x56283ea084a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde56242640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283e9fc310 0x56283ea084a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde56242640 1 -- 172.21.15.37:0/1916332535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 msgr2=0x56283ea07f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde56242640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea07f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde56242640 1 -- 172.21.15.37:0/1916332535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283e673210 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde55240640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283e9fc310 0x56283ea084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde56242640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283ea0b0c0 0x56283ea089e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fde480cda70 tx=0x7fde4809c170 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad015fa7b65b1b91 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.665+0000 7fde397fa640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde397fa640 1 -- 172.21.15.37:0/1916332535 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde48005510 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4000fb00 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde397fa640 1 -- 172.21.15.37:0/1916332535 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fde4809ba80 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde397fa640 1 -- 172.21.15.37:0/1916332535 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fde4809c630 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 10 client.28547 fetch_fsmap learned FSMap version 46 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56283e73f620 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde397fa640 1 -- 172.21.15.37:0/1916332535 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4000fb00 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde397fa640 1 -- 172.21.15.37:0/1916332535 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 46) ==== 8354+0+0 (secure 0 0 0) 0x7fde480b0be0 con 0x56283ea0b0c0 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 10 client.28547 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:55:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 10 client.28547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 20 client.28547 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea1b0b0 0x56283ea1d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 4 client.28547 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283e96da20 con 0x56283ea1b0b0 2024-09-19T06:55:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.666+0000 7fde55a41640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea1b0b0 0x56283ea1d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.667+0000 7fde397fa640 10 client.28547 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.667+0000 7fde55a41640 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea1b0b0 0x56283ea1d4a0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9845ade2ac21d45b server_cookie=214462721c257060 in_seq=0 out_seq=0 2024-09-19T06:55:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.669+0000 7fde397fa640 1 -- 172.21.15.37:0/1916332535 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56283e96da20 con 0x56283ea1b0b0 2024-09-19T06:55:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.669+0000 7fde397fa640 10 client.28547 handle_command_reply: tid=0 2024-09-19T06:55:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 client.28547 shutdown 2024-09-19T06:55:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283ea0b0c0 msgr2=0x56283ea089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283ea0b0c0 0x56283ea089e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fde480cda70 tx=0x7fde4809c170 comp rx=0 tx=0).stop 2024-09-19T06:55:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 shutdown_connections 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea1b0b0 0x56283ea1d4a0 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283ea0b0c0 0x56283ea089e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283e9fc310 0x56283ea084a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 --2- 172.21.15.37:0/1916332535 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea07f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 >> 172.21.15.37:0/1916332535 conn(0x56283e9f8060 msgr2=0x56283e9fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde397fa640 0 client.28547 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 shutdown_connections 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 1 -- 172.21.15.37:0/1916332535 wait complete. 2024-09-19T06:55:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.670+0000 7fde5aa44740 20 client.28547 trim_cache size 0 max 16384 2024-09-19T06:55:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 Processor -- start 2024-09-19T06:55:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 -- start start 2024-09-19T06:55:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea09ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea22d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde501adf10 con 0x56283e9fb9f0 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde501ab780 con 0x56283e9fc310 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde5aa44740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde501a95f0 con 0x56283ea0b0c0 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde55a41640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea09ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde55a41640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea09ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36728/0 (socket says 172.21.15.37:36728) 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde55a41640 1 -- 172.21.15.37:0/493580324 learned_addr learned my addr 172.21.15.37:0/493580324 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.686+0000 7fde55240640 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea22d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde56242640 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde55a41640 1 -- 172.21.15.37:0/493580324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 msgr2=0x56283ea232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde55a41640 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea232d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde55a41640 1 -- 172.21.15.37:0/493580324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 msgr2=0x56283ea22d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde55a41640 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea22d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde55a41640 1 -- 172.21.15.37:0/493580324 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283e9fc140 con 0x56283e9fb9f0 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde55a41640 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea09ed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fde4c005940 tx=0x7fde4c03f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f2ace62a9fe1c4e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde3b7fe640 1 -- 172.21.15.37:0/493580324 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c035040 con 0x56283e9fb9f0 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde3b7fe640 1 -- 172.21.15.37:0/493580324 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c05a070 con 0x56283e9fb9f0 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.687+0000 7fde3b7fe640 1 -- 172.21.15.37:0/493580324 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde4c03d450 con 0x56283e9fb9f0 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 -- 172.21.15.37:0/493580324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 msgr2=0x56283ea09ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea09ed0 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fde4c005940 tx=0x7fde4c03f070 comp rx=0 tx=0).stop 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 -- 172.21.15.37:0/493580324 shutdown_connections 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fc310 0x56283ea22d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 --2- 172.21.15.37:0/493580324 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fb9f0 0x56283ea09ed0 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 -- 172.21.15.37:0/493580324 >> 172.21.15.37:0/493580324 conn(0x56283e9f8060 msgr2=0x56283ea097c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 -- 172.21.15.37:0/493580324 shutdown_connections 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 -- 172.21.15.37:0/493580324 wait complete. 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 -- start start 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.688+0000 7fde5aa44740 1 Processor -- start 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fb9f0 0x56283ea13450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fc310 0x56283ea13990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde5aa44740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde5aa44740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fde501a9c40 con 0x56283e9fc310 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde5aa44740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fde501ab0b0 con 0x56283e9fb9f0 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde5aa44740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fde501aa310 con 0x56283ea0b0c0 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde5aa44740 10 client.0 _ll_register_callbacks cb 0x7fde480988d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fc310 0x56283ea13990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fc310 0x56283ea13990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36740/0 (socket says 172.21.15.37:36740) 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 -- 172.21.15.37:0/2553563509 learned_addr learned my addr 172.21.15.37:0/2553563509 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55a41640 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fb9f0 0x56283ea13450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde56242640 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 -- 172.21.15.37:0/2553563509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 msgr2=0x56283ea0e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 -- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fb9f0 msgr2=0x56283ea13450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fb9f0 0x56283ea13450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.689+0000 7fde55240640 1 -- 172.21.15.37:0/2553563509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283ea0f050 con 0x56283e9fc310 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde55240640 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fc310 0x56283ea13990 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fde40009250 tx=0x7fde40009e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5cacc85b34b2cef server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde397fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde397fa640 1 -- 172.21.15.37:0/2553563509 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde40009ea0 con 0x56283e9fc310 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde397fa640 1 -- 172.21.15.37:0/2553563509 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fde4000f480 con 0x56283e9fc310 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde4c05a070 con 0x56283e9fc310 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde397fa640 1 -- 172.21.15.37:0/2553563509 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7fde40030050 con 0x56283e9fc310 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde5aa44740 10 client.19290 fetch_fsmap learned FSMap version 46 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=4+}) -- 0x56283ea85950 con 0x56283e9fc310 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde397fa640 1 -- 172.21.15.37:0/2553563509 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fde4c05a070 con 0x56283e9fc310 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.690+0000 7fde397fa640 1 -- 172.21.15.37:0/2553563509 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 46) ==== 8354+0+0 (secure 0 0 0) 0x7fde400035f0 con 0x56283e9fc310 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde5aa44740 10 client.19290 fetch_fsmap finished waiting for FSMap version 46 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde5aa44740 10 client.19290 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde5aa44740 20 client.19290 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde5aa44740 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea85f20 0x56283ea88340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde5aa44740 4 client.19290 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56283e967d70 con 0x56283ea85f20 2024-09-19T06:55:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde55a41640 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea85f20 0x56283ea88340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde55a41640 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea85f20 0x56283ea88340 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d96cdac90d6f889d server_cookie=7e74bdfecafa32ab in_seq=0 out_seq=0 2024-09-19T06:55:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:38.691+0000 7fde397fa640 10 client.19290 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.166+0000 7fde397fa640 1 -- 172.21.15.37:0/2553563509 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56283e967d70 con 0x56283ea85f20 2024-09-19T06:55:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde397fa640 10 client.19290 handle_command_reply: tid=0 2024-09-19T06:55:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 client.19290 shutdown 2024-09-19T06:55:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fc310 msgr2=0x56283ea13990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fc310 0x56283ea13990 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fde40009250 tx=0x7fde40009e30 comp rx=0 tx=0).stop 2024-09-19T06:55:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 shutdown_connections 2024-09-19T06:55:39.168 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x56283ea85f20 0x56283ea88340 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56283ea0b0c0 0x56283ea0e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde397fa640 0 client.19290 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56283e9fc310 0x56283ea13990 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 --2- 172.21.15.37:0/2553563509 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56283e9fb9f0 0x56283ea13450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 >> 172.21.15.37:0/2553563509 conn(0x56283e9f8060 msgr2=0x56283ea08740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 shutdown_connections 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 1 -- 172.21.15.37:0/2553563509 wait complete. 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.167+0000 7fde5aa44740 20 client.19290 trim_cache size 0 max 16384 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:55:39.169 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:55:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:55:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:55:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde3c0a3460 msgr2=0x7fde3c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:39.170 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde3c0a3460 0x7fde3c0a5920 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fde4c009900 tx=0x7fde4c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 msgr2=0x7fde5019cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde5019cc30 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fde4800eb10 tx=0x7fde4800c2d0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 shutdown_connections 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fde3c0a3460 0x7fde3c0a5920 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fde50107b70 0x7fde5019cc30 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fde50102470 0x7fde501988b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 --2- 172.21.15.37:0/3363229487 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fde50101ad0 0x7fde50198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 >> 172.21.15.37:0/3363229487 conn(0x7fde500fd7d0 msgr2=0x7fde500fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 shutdown_connections 2024-09-19T06:55:39.171 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:39.168+0000 7fde57ca5640 1 -- 172.21.15.37:0/3363229487 wait complete. 2024-09-19T06:55:39.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3240 59.92 MB/sec warmup 6 sec latency 347.365 ms 2024-09-19T06:55:40.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:39 smithi096 ceph-mon[35004]: pgmap v440: 97 pgs: 97 active+clean; 78 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-09-19T06:55:40.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:39 smithi096 ceph-mon[35004]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:40.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:39 smithi037 ceph-mon[31211]: pgmap v440: 97 pgs: 97 active+clean; 78 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-09-19T06:55:40.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:39 smithi037 ceph-mon[31211]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:39 smithi151 ceph-mon[35236]: pgmap v440: 97 pgs: 97 active+clean; 78 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 891 KiB/s rd, 1.7 MiB/s wr, 131 op/s 2024-09-19T06:55:40.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:39 smithi151 ceph-mon[35236]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-19T06:55:40.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3605 54.87 MB/sec warmup 7 sec latency 89.533 ms 2024-09-19T06:55:41.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 3829 48.41 MB/sec warmup 8 sec latency 98.648 ms 2024-09-19T06:55:41.580 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:55:41.849 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:42.231 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:41 smithi037 ceph-mon[31211]: pgmap v441: 97 pgs: 97 active+clean; 78 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-09-19T06:55:42.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:41 smithi096 ceph-mon[35004]: pgmap v441: 97 pgs: 97 active+clean; 78 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 -- 172.21.15.37:0/1113439671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 msgr2=0x7f414010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 --2- 172.21.15.37:0/1113439671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414010da40 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f4138009ca0 tx=0x7f413802a950 comp rx=0 tx=0).stop 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 -- 172.21.15.37:0/1113439671 shutdown_connections 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 --2- 172.21.15.37:0/1113439671 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414010da40 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 --2- 172.21.15.37:0/1113439671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4140102570 0x7f41401029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 --2- 172.21.15.37:0/1113439671 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4140100e20 0x7f4140101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 -- 172.21.15.37:0/1113439671 >> 172.21.15.37:0/1113439671 conn(0x7f41400fc7d0 msgr2=0x7f41400febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:42.324 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 -- 172.21.15.37:0/1113439671 shutdown_connections 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.317+0000 7f4144cc2640 1 -- 172.21.15.37:0/1113439671 wait complete. 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 Processor -- start 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 -- start start 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4140100e20 0x7f414019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4140102570 0x7f414019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4140110f80 con 0x7f4140102ef0 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4140110e00 con 0x7f4140100e20 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f4144cc2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4140111100 con 0x7f4140102570 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f413e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4140100e20 0x7f414019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.325 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f413ed9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414019efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60258/0 (socket says 172.21.15.37:60258) 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f413ed9d640 1 -- 172.21.15.37:0/35663192 learned_addr learned my addr 172.21.15.37:0/35663192 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.318+0000 7f413dd9b640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4140102570 0x7f414019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f413ed9d640 1 -- 172.21.15.37:0/35663192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4140102570 msgr2=0x7f414019ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f413ed9d640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4140102570 0x7f414019ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f413ed9d640 1 -- 172.21.15.37:0/35663192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4140100e20 msgr2=0x7f414019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f413ed9d640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4140100e20 0x7f414019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f413ed9d640 1 -- 172.21.15.37:0/35663192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414019f6c0 con 0x7f4140102ef0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f413ed9d640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414019efe0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4138004af0 tx=0x7f413802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f41377fe640 1 -- 172.21.15.37:0/35663192 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138034040 con 0x7f4140102ef0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414019f950 con 0x7f4140102ef0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f41377fe640 1 -- 172.21.15.37:0/35663192 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4138030920 con 0x7f4140102ef0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f41377fe640 1 -- 172.21.15.37:0/35663192 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4138004820 con 0x7f4140102ef0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.319+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41401a7290 con 0x7f4140102ef0 2024-09-19T06:55:42.326 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.321+0000 7f41377fe640 1 -- 172.21.15.37:0/35663192 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4138037020 con 0x7f4140102ef0 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 Processor -- start 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 -- start start 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24c57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24db280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5616f242b2e0 con 0x5616f24c5ce0 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616f23d2ce0 con 0x5616f24d8e90 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f414709b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5616f24272a0 con 0x5616f24c53c0 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f413dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24db280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f413e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f413e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60268/0 (socket says 172.21.15.37:60268) 2024-09-19T06:55:42.327 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.322+0000 7f413e59c640 1 -- 172.21.15.37:0/4119950774 learned_addr learned my addr 172.21.15.37:0/4119950774 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413dd9b640 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24c57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f41377fe640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f412c0a33b0 0x7f412c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413e59c640 1 -- 172.21.15.37:0/4119950774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 msgr2=0x5616f24c57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413e59c640 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24c57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413e59c640 1 -- 172.21.15.37:0/4119950774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 msgr2=0x5616f24db280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413e59c640 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24db280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413e59c640 1 -- 172.21.15.37:0/4119950774 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f23b7440 con 0x5616f24c5ce0 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413ed9d640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f412c0a33b0 0x7f412c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413dd9b640 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24db280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f41377fe640 1 -- 172.21.15.37:0/35663192 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f41380f0aa0 con 0x7f4140102ef0 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f413e59c640 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d8950 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f413000ac10 tx=0x7f413000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd688f09e195516 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f4134ff9640 1 -- 172.21.15.37:0/4119950774 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413001d070 con 0x5616f24c5ce0 2024-09-19T06:55:42.328 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f4134ff9640 1 -- 172.21.15.37:0/4119950774 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4130018b20 con 0x5616f24c5ce0 2024-09-19T06:55:42.329 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.323+0000 7f4134ff9640 1 -- 172.21.15.37:0/4119950774 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4130010ad0 con 0x5616f24c5ce0 2024-09-19T06:55:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 -- 172.21.15.37:0/4119950774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 msgr2=0x5616f24d8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f413ed9d640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f412c0a33b0 0x7f412c0a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f41380ee7c0 tx=0x7f413808ebe0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d8950 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f413000ac10 tx=0x7f413000f790 comp rx=0 tx=0).stop 2024-09-19T06:55:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 -- 172.21.15.37:0/4119950774 shutdown_connections 2024-09-19T06:55:42.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24db280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d8950 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 --2- 172.21.15.37:0/4119950774 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24c57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 -- 172.21.15.37:0/4119950774 >> 172.21.15.37:0/4119950774 conn(0x5616f24c1a30 msgr2=0x5616f24c1e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 -- 172.21.15.37:0/4119950774 shutdown_connections 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.324+0000 7f414709b740 1 -- 172.21.15.37:0/4119950774 wait complete. 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 -- start start 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 Processor -- start 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24d8850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d5b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24d60f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5616f24dba00 con 0x5616f24c5ce0 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5616f23e6050 con 0x5616f24d8e90 2024-09-19T06:55:42.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5616f24dbb80 con 0x5616f24c53c0 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f414709b740 10 client.0 _ll_register_callbacks cb 0x5616f2426160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f413e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f413e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60282/0 (socket says 172.21.15.37:60282) 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f413e59c640 1 -- 172.21.15.37:0/2085429854 learned_addr learned my addr 172.21.15.37:0/2085429854 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f413dd9b640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24d60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.325+0000 7f413dd9b640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24d8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f413e59c640 1 -- 172.21.15.37:0/2085429854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 msgr2=0x5616f24d8850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f413e59c640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24d8850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f413e59c640 1 -- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 msgr2=0x5616f24d60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f413e59c640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24d60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f413e59c640 1 -- 172.21.15.37:0/2085429854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f219c3c0 con 0x5616f24c5ce0 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f413dd9b640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:42.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f413e59c640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d5b00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f41300105f0 tx=0x7f4130005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37dbc32a1a65a4bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f4125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f4125ffb640 1 -- 172.21.15.37:0/2085429854 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413001d050 con 0x5616f24c5ce0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f4125ffb640 1 -- 172.21.15.37:0/2085429854 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f4130005240 con 0x5616f24c5ce0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4130018b20 con 0x5616f24c5ce0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f4125ffb640 1 -- 172.21.15.37:0/2085429854 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4130032050 con 0x5616f24c5ce0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f414709b740 10 client.19302 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5616f24e7460 con 0x5616f24c5ce0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.326+0000 7f4125ffb640 1 -- 172.21.15.37:0/2085429854 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4130018b20 con 0x5616f24c5ce0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f4125ffb640 1 -- 172.21.15.37:0/2085429854 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f4130004660 con 0x5616f24c5ce0 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f414709b740 10 client.19302 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f414709b740 10 client.19302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f414709b740 20 client.19302 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f414709b740 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f24e8860 0x5616f24eac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f414709b740 4 client.19302 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5616f2438850 con 0x5616f24e8860 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f413dd9b640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f24e8860 0x5616f24eac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f413dd9b640 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f24e8860 0x5616f24eac50 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed20cae853536614 server_cookie=b2c4322b126545fd in_seq=0 out_seq=0 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.327+0000 7f4125ffb640 10 client.19302 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f4125ffb640 1 -- 172.21.15.37:0/2085429854 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5616f2438850 con 0x5616f24e8860 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f4125ffb640 10 client.19302 handle_command_reply: tid=0 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 client.19302 shutdown 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 msgr2=0x5616f24d5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d5b00 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f41300105f0 tx=0x7f4130005e30 comp rx=0 tx=0).stop 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 shutdown_connections 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f24e8860 0x5616f24eac50 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24d8e90 0x5616f24d60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d5b00 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f4125ffb640 0 client.19302 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 --2- 172.21.15.37:0/2085429854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24d8850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 >> 172.21.15.37:0/2085429854 conn(0x5616f24c1a30 msgr2=0x5616f24c76c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 shutdown_connections 2024-09-19T06:55:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 1 -- 172.21.15.37:0/2085429854 wait complete. 2024-09-19T06:55:42.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.331+0000 7f414709b740 20 client.19302 trim_cache size 0 max 16384 2024-09-19T06:55:42.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 Processor -- start 2024-09-19T06:55:42.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 -- start start 2024-09-19T06:55:42.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c53c0 0x5616f24e4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24d8e90 0x5616f24c0a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41401ae0c0 con 0x5616f24c5ce0 2024-09-19T06:55:42.352 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41401ab930 con 0x5616f24c53c0 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.350+0000 7f414709b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41401a97e0 con 0x5616f24d8e90 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413e59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60292/0 (socket says 172.21.15.37:60292) 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413e59c640 1 -- 172.21.15.37:0/3081966652 learned_addr learned my addr 172.21.15.37:0/3081966652 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c53c0 0x5616f24e4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24d8e90 0x5616f24c0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 -- 172.21.15.37:0/3081966652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24d8e90 msgr2=0x5616f24c0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24d8e90 0x5616f24c0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 -- 172.21.15.37:0/3081966652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 msgr2=0x5616f24d43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 -- 172.21.15.37:0/3081966652 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616f24d6460 con 0x5616f24c53c0 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413e59c640 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:42.353 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f413dd9b640 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c53c0 0x5616f24e4ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5616f24eb220 tx=0x7f4128009a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebed1525045dcfd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f4127fff640 1 -- 172.21.15.37:0/3081966652 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f412801a040 con 0x5616f24c53c0 2024-09-19T06:55:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.351+0000 7f4127fff640 1 -- 172.21.15.37:0/3081966652 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412800ee90 con 0x5616f24c53c0 2024-09-19T06:55:42.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f4127fff640 1 -- 172.21.15.37:0/3081966652 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4128004e30 con 0x5616f24c53c0 2024-09-19T06:55:42.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 -- 172.21.15.37:0/3081966652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c53c0 msgr2=0x5616f24e4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c53c0 0x5616f24e4ca0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x5616f24eb220 tx=0x7f4128009a20 comp rx=0 tx=0).stop 2024-09-19T06:55:42.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 -- 172.21.15.37:0/3081966652 shutdown_connections 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24d8e90 0x5616f24c0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24c5ce0 0x5616f24d43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 --2- 172.21.15.37:0/3081966652 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c53c0 0x5616f24e4ca0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 -- 172.21.15.37:0/3081966652 >> 172.21.15.37:0/3081966652 conn(0x5616f24c1a30 msgr2=0x5616f24c0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 -- 172.21.15.37:0/3081966652 shutdown_connections 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.352+0000 7f414709b740 1 -- 172.21.15.37:0/3081966652 wait complete. 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 -- start start 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 Processor -- start 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24e1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c5ce0 0x5616f24dc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24d8e90 0x5616f24dccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41401a9e30 con 0x5616f24d8e90 2024-09-19T06:55:42.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41401ab260 con 0x5616f24c5ce0 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41401aa4c0 con 0x5616f24c53c0 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f414709b740 10 client.0 _ll_register_callbacks cb 0x7f413001b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f413e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c5ce0 0x5616f24dc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f413dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24d8e90 0x5616f24dccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f413e59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c5ce0 0x5616f24dc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36232/0 (socket says 172.21.15.37:36232) 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f413dd9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24d8e90 0x5616f24dccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60306/0 (socket says 172.21.15.37:60306) 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f413e59c640 1 -- 172.21.15.37:0/1218905715 learned_addr learned my addr 172.21.15.37:0/1218905715 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.353+0000 7f413dd9b640 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24e1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f413e59c640 1 -- 172.21.15.37:0/1218905715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 msgr2=0x5616f24e1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f413e59c640 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24e1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f413e59c640 1 -- 172.21.15.37:0/1218905715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24d8e90 msgr2=0x5616f24dccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f413e59c640 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24d8e90 0x5616f24dccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f413e59c640 1 -- 172.21.15.37:0/1218905715 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5616f24dd520 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f413dd9b640 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24d8e90 0x5616f24dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f413e59c640 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c5ce0 0x5616f24dc7b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4130004ba0 tx=0x7f413000abc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22f15ce69c4f7545 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f4125ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f4125ffb640 1 -- 172.21.15.37:0/1218905715 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f413003a070 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f4125ffb640 1 -- 172.21.15.37:0/1218905715 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f4130004660 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f412800ee90 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f4125ffb640 1 -- 172.21.15.37:0/1218905715 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f4130045050 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f414709b740 10 client.28571 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.354+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5616f25531f0 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f4125ffb640 1 -- 172.21.15.37:0/1218905715 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f412800ee90 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f4125ffb640 1 -- 172.21.15.37:0/1218905715 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f41300108d0 con 0x5616f24c5ce0 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f414709b740 10 client.28571 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f414709b740 10 client.28571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:42.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f414709b740 20 client.28571 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:42.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f414709b740 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f2553940 0x5616f2555d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:42.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f414709b740 4 client.28571 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:55:42.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5616f2432d60 con 0x5616f2553940 2024-09-19T06:55:42.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f413dd9b640 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f2553940 0x5616f2555d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:42.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f4125ffb640 10 client.28571 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:42.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.355+0000 7f413dd9b640 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f2553940 0x5616f2555d60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a5d0b9d9c0e22 server_cookie=f49c3777258dba71 in_seq=0 out_seq=0 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f4125ffb640 1 -- 172.21.15.37:0/1218905715 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5616f2432d60 con 0x5616f2553940 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f4125ffb640 10 client.28571 handle_command_reply: tid=0 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 client.28571 shutdown 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c5ce0 msgr2=0x5616f24dc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c5ce0 0x5616f24dc7b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f4130004ba0 tx=0x7f413000abc0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 shutdown_connections 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5616f2553940 0x5616f2555d60 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5616f24d8e90 0x5616f24dccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5616f24c5ce0 0x5616f24dc7b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f4125ffb640 0 client.28571 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 --2- 172.21.15.37:0/1218905715 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5616f24c53c0 0x5616f24e1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 >> 172.21.15.37:0/1218905715 conn(0x5616f24c1a30 msgr2=0x5616f24c0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 shutdown_connections 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 1 -- 172.21.15.37:0/1218905715 wait complete. 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f414709b740 20 client.28571 trim_cache size 0 max 16384 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:55:42.362 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:55:42.363 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:42.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f412c0a33b0 msgr2=0x7f412c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f4144cc2640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f412c0a33b0 0x7f412c0a5870 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f41380ee7c0 tx=0x7f413808ebe0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 msgr2=0x7f414019efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:42.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.358+0000 7f4144cc2640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414019efe0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f4138004af0 tx=0x7f413802f5b0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 shutdown_connections 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f412c0a33b0 0x7f412c0a5870 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4140102ef0 0x7f414019efe0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4140102570 0x7f414019ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 --2- 172.21.15.37:0/35663192 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4140100e20 0x7f414019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 >> 172.21.15.37:0/35663192 conn(0x7f41400fc7d0 msgr2=0x7f414006a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 shutdown_connections 2024-09-19T06:55:42.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:42.359+0000 7f4144cc2640 1 -- 172.21.15.37:0/35663192 wait complete. 2024-09-19T06:55:42.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:41 smithi151 ceph-mon[35236]: pgmap v441: 97 pgs: 97 active+clean; 78 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 870 KiB/s rd, 1.7 MiB/s wr, 117 op/s 2024-09-19T06:55:42.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 4023 43.28 MB/sec warmup 9 sec latency 152.559 ms 2024-09-19T06:55:43.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 4413 40.78 MB/sec warmup 10 sec latency 140.906 ms 2024-09-19T06:55:43.596 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:55:43.596 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:55:43.869 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:44.267 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:43 smithi037 ceph-mon[31211]: pgmap v442: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 437 op/s 2024-09-19T06:55:44.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:43 smithi096 ceph-mon[35004]: pgmap v442: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 437 op/s 2024-09-19T06:55:44.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:43 smithi151 ceph-mon[35236]: pgmap v442: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.3 MiB/s wr, 437 op/s 2024-09-19T06:55:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 -- 172.21.15.37:0/194455175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 msgr2=0x7f9d9410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/194455175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9410ea90 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f9d8c009ca0 tx=0x7f9d8c02a8c0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 -- 172.21.15.37:0/194455175 shutdown_connections 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/194455175 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9410ea90 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/194455175 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d94102790 0x7f9d941067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/194455175 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 0x7f9d941021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 -- 172.21.15.37:0/194455175 >> 172.21.15.37:0/194455175 conn(0x7f9d940fd8c0 msgr2=0x7f9d940ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 -- 172.21.15.37:0/194455175 shutdown_connections 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 -- 172.21.15.37:0/194455175 wait complete. 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.355+0000 7f9d98c3d640 1 Processor -- start 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d98c3d640 1 -- start start 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d98c3d640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 0x7f9d941983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d98c3d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d94102790 0x7f9d94198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d98c3d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9419ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:44.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d98c3d640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111f70 con 0x7f9d94106f10 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d98c3d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d94111df0 con 0x7f9d94102790 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d98c3d640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9d941120f0 con 0x7f9d94101de0 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d92d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9419ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 0x7f9d941983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d8bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d94102790 0x7f9d94198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 0x7f9d941983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:36428/0 (socket says 172.21.15.37:36428) 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d92d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9419ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60328/0 (socket says 172.21.15.37:60328) 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 -- 172.21.15.37:0/2867958790 learned_addr learned my addr 172.21.15.37:0/2867958790 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 -- 172.21.15.37:0/2867958790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d94102790 msgr2=0x7f9d94198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d94102790 0x7f9d94198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 -- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 msgr2=0x7f9d9419ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9419ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d9259c640 1 -- 172.21.15.37:0/2867958790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d9419d3a0 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.356+0000 7f9d92d9d640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.357+0000 7f9d9259c640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 0x7f9d941983f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400d950 tx=0x7f9d8400de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.357+0000 7f9d8b7fe640 1 -- 172.21.15.37:0/2867958790 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d84017070 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.357+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d9419d690 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.357+0000 7f9d8b7fe640 1 -- 172.21.15.37:0/2867958790 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d84009c30 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.357+0000 7f9d8b7fe640 1 -- 172.21.15.37:0/2867958790 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8400f6b0 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.357+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d941a4f70 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.358+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d941021c0 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.359+0000 7f9d8b7fe640 1 -- 172.21.15.37:0/2867958790 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f9d840109d0 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.360+0000 7f9d8b7fe640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9d800a33b0 0x7f9d800a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.360+0000 7f9d8b7fe640 1 -- 172.21.15.37:0/2867958790 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f9d840cf8e0 con 0x7f9d94101de0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.360+0000 7f9d8bfff640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9d800a33b0 0x7f9d800a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:44.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.360+0000 7f9d8bfff640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9d800a33b0 0x7f9d800a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d940fdd70 tx=0x7f9d7c005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:44.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.363+0000 7f9d8b7fe640 1 -- 172.21.15.37:0/2867958790 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f9d84009de0 con 0x7f9d94101de0 2024-09-19T06:55:44.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 4994 40.52 MB/sec warmup 11 sec latency 63.999 ms 2024-09-19T06:55:44.593 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.592+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9d9410a120 con 0x7f9d94101de0 2024-09-19T06:55:44.594 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.593+0000 7f9d8b7fe640 1 -- 172.21.15.37:0/2867958790 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11907 (secure 0 0 0) 0x7f9d8409bc40 con 0x7f9d94101de0 2024-09-19T06:55:44.595 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:55:44.595 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"btime":"2024-09-19T06:55:38:930626+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:55:38.046955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28132":{"gid":28132,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2835776386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2835776386},{"type":"v1","addr":"172.21.15.151:6839","nonce":2835776386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:55:44.595 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 47 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9d800a33b0 msgr2=0x7f9d800a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9d800a33b0 0x7f9d800a5870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9d940fdd70 tx=0x7f9d7c005f90 comp rx=0 tx=0).stop 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 msgr2=0x7f9d941983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 0x7f9d941983f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f9d8400d950 tx=0x7f9d8400de10 comp rx=0 tx=0).stop 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 shutdown_connections 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f9d800a33b0 0x7f9d800a5870 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f9d94106f10 0x7f9d9419ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d94102790 0x7f9d94198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 --2- 172.21.15.37:0/2867958790 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9d94101de0 0x7f9d941983f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 >> 172.21.15.37:0/2867958790 conn(0x7f9d940fd8c0 msgr2=0x7f9d940fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:44.597 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 shutdown_connections 2024-09-19T06:55:44.598 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:44.595+0000 7f9d98c3d640 1 -- 172.21.15.37:0/2867958790 wait complete. 2024-09-19T06:55:45.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:44 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2867958790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:45.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:44 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2867958790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:45.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:44 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2867958790' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:55:45.431 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:55:45.484 INFO:tasks.workunit.client.0.smithi037.stdout: 10 5763 38.76 MB/sec warmup 12 sec latency 155.487 ms 2024-09-19T06:55:45.709 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:46.128 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:45 smithi037 ceph-mon[31211]: pgmap v443: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 416 op/s 2024-09-19T06:55:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 -- 172.21.15.37:0/3708011980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41082a0 msgr2=0x7f92a410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 --2- 172.21.15.37:0/3708011980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41082a0 0x7f92a410ec40 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f929c009c10 tx=0x7f929c02a6f0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 -- 172.21.15.37:0/3708011980 shutdown_connections 2024-09-19T06:55:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 --2- 172.21.15.37:0/3708011980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41082a0 0x7f92a410ec40 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.265 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 --2- 172.21.15.37:0/3708011980 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a41036e0 0x7f92a4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 --2- 172.21.15.37:0/3708011980 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92a4102d30 0x7f92a4103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 -- 172.21.15.37:0/3708011980 >> 172.21.15.37:0/3708011980 conn(0x7f92a40fc7d0 msgr2=0x7f92a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 -- 172.21.15.37:0/3708011980 shutdown_connections 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.228+0000 7f92aa976640 1 -- 172.21.15.37:0/3708011980 wait complete. 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 Processor -- start 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 -- start start 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a4102d30 0x7f92a4198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41036e0 0x7f92a4198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92a41082a0 0x7f92a419cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41120a0 con 0x7f92a4102d30 2024-09-19T06:55:46.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4111f20 con 0x7f92a41036e0 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92aa976640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92a4112220 con 0x7f92a41082a0 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a4102d30 0x7f92a4198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92a37fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41036e0 0x7f92a4198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92a3fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a4102d30 0x7f92a4198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60352/0 (socket says 172.21.15.37:60352) 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92a3fff640 1 -- 172.21.15.37:0/1899060827 learned_addr learned my addr 172.21.15.37:0/1899060827 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.229+0000 7f92a8f13640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92a41082a0 0x7f92a419cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a3fff640 1 -- 172.21.15.37:0/1899060827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92a41082a0 msgr2=0x7f92a419cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a3fff640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92a41082a0 0x7f92a419cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a3fff640 1 -- 172.21.15.37:0/1899060827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41036e0 msgr2=0x7f92a4198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a3fff640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41036e0 0x7f92a4198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a3fff640 1 -- 172.21.15.37:0/1899060827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a419d4d0 con 0x7f92a4102d30 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a37fe640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41036e0 0x7f92a4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a3fff640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a4102d30 0x7f92a4198520 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f929400ea60 tx=0x7f929400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a17fa640 1 -- 172.21.15.37:0/1899060827 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929400cd20 con 0x7f92a4102d30 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a17fa640 1 -- 172.21.15.37:0/1899060827 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92940035f0 con 0x7f92a4102d30 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92a17fa640 1 -- 172.21.15.37:0/1899060827 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9294010690 con 0x7f92a4102d30 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a419d7c0 con 0x7f92a4102d30 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.230+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a41008c0 con 0x7f92a4102d30 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92a17fa640 1 -- 172.21.15.37:0/1899060827 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f92940037a0 con 0x7f92a4102d30 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 Processor -- start 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 -- start start 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dce5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dce5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcf2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55612dc4b2f0 con 0x55612dce6b40 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55612dbf2cf0 con 0x55612dce55d0 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92acd4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55612dc472b0 con 0x55612dcf0370 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92a8f13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dce5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92a8f13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dce5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60360/0 (socket says 172.21.15.37:60360) 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92a8f13640 1 -- 172.21.15.37:0/742873897 learned_addr learned my addr 172.21.15.37:0/742873897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.233+0000 7f92a37fe640 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcf2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a3fff640 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dce5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a8f13640 1 -- 172.21.15.37:0/742873897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 msgr2=0x55612dcf2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.269 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a8f13640 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcf2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a8f13640 1 -- 172.21.15.37:0/742873897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 msgr2=0x55612dce5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a8f13640 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dce5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a8f13640 1 -- 172.21.15.37:0/742873897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612dbd73a0 con 0x55612dce6b40 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a17fa640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f928c0a7840 0x7f928c0a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a3fff640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f928c0a7840 0x7f928c0a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a8f13640 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dce5090 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f929c009be0 tx=0x7f929c030a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916326b9437c7ecc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f92a17fa640 1 -- 172.21.15.37:0/1899060827 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f92940cffd0 con 0x7f92a4102d30 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.234+0000 7f928a7fc640 1 -- 172.21.15.37:0/742873897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c041030 con 0x55612dce6b40 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f928a7fc640 1 -- 172.21.15.37:0/742873897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929c030c60 con 0x55612dce6b40 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f928a7fc640 1 -- 172.21.15.37:0/742873897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c036040 con 0x55612dce6b40 2024-09-19T06:55:46.270 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 -- 172.21.15.37:0/742873897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 msgr2=0x55612dce5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dce5090 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f929c009be0 tx=0x7f929c030a30 comp rx=0 tx=0).stop 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92a3fff640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f928c0a7840 0x7f928c0a9d00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f92940cd7a0 tx=0x7f92940bc040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 -- 172.21.15.37:0/742873897 shutdown_connections 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcf2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dce5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 --2- 172.21.15.37:0/742873897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dce5090 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 -- 172.21.15.37:0/742873897 >> 172.21.15.37:0/742873897 conn(0x55612dcae310 msgr2=0x55612dce1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.271 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 -- 172.21.15.37:0/742873897 shutdown_connections 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.235+0000 7f92acd4f740 1 -- 172.21.15.37:0/742873897 wait complete. 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 -- start start 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 Processor -- start 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce55d0 0x55612dcefd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcecfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 0x55612dced530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55612dcf3060 con 0x55612dce6b40 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55612dc06060 con 0x55612dcf0370 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55612dcf31e0 con 0x55612dce55d0 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.236+0000 7f92acd4f740 10 client.0 _ll_register_callbacks cb 0x55612d9598c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:46.272 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcecfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcecfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60362/0 (socket says 172.21.15.37:60362) 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 -- 172.21.15.37:0/3826288201 learned_addr learned my addr 172.21.15.37:0/3826288201 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a8f13640 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce55d0 0x55612dcefd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 -- 172.21.15.37:0/3826288201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce55d0 msgr2=0x55612dcefd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce55d0 0x55612dcefd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 -- 172.21.15.37:0/3826288201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 msgr2=0x55612dced530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 0x55612dced530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.273 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 -- 172.21.15.37:0/3826288201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612d9bc3c0 con 0x55612dce6b40 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92a37fe640 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcecfb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9298012cf0 tx=0x7f929800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4464233b02faacce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92837fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92837fe640 1 -- 172.21.15.37:0/3826288201 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298012110 con 0x55612dce6b40 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92837fe640 1 -- 172.21.15.37:0/3826288201 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f9298004050 con 0x55612dce6b40 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f929c030c60 con 0x55612dce6b40 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92837fe640 1 -- 172.21.15.37:0/3826288201 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f92980122b0 con 0x55612dce6b40 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92acd4f740 10 client.19326 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.237+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55612dd07300 con 0x55612dce6b40 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92837fe640 1 -- 172.21.15.37:0/3826288201 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f929c030c60 con 0x55612dce6b40 2024-09-19T06:55:46.274 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92837fe640 1 -- 172.21.15.37:0/3826288201 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f9298023070 con 0x55612dce6b40 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92acd4f740 10 client.19326 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92acd4f740 10 client.19326 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92acd4f740 20 client.19326 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92acd4f740 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd08890 0x55612dd0acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92acd4f740 4 client.19326 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55612dc58860 con 0x55612dd08890 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92a8f13640 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd08890 0x55612dd0acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92837fe640 10 client.19326 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.238+0000 7f92a8f13640 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd08890 0x55612dd0acb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b8b6af41e08039bd server_cookie=f6a8b9d6a3e24aa6 in_seq=0 out_seq=0 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.239+0000 7f92837fe640 1 -- 172.21.15.37:0/3826288201 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55612dc58860 con 0x55612dd08890 2024-09-19T06:55:46.275 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.239+0000 7f92837fe640 10 client.19326 handle_command_reply: tid=0 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.239+0000 7f92acd4f740 1 client.19326 shutdown 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.239+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 msgr2=0x55612dcecfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.239+0000 7f92acd4f740 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcecfb0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f9298012cf0 tx=0x7f929800f790 comp rx=0 tx=0).stop 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.240+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 shutdown_connections 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.240+0000 7f92acd4f740 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd08890 0x55612dd0acb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.240+0000 7f92acd4f740 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 0x55612dced530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.240+0000 7f92acd4f740 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcecfb0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.240+0000 7f92acd4f740 1 --2- 172.21.15.37:0/3826288201 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce55d0 0x55612dcefd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.276 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.240+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 >> 172.21.15.37:0/3826288201 conn(0x55612dcae310 msgr2=0x55612dce7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.241+0000 7f92837fe640 0 client.19326 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:46.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.241+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 shutdown_connections 2024-09-19T06:55:46.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.241+0000 7f92acd4f740 1 -- 172.21.15.37:0/3826288201 wait complete. 2024-09-19T06:55:46.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.241+0000 7f92acd4f740 20 client.19326 trim_cache size 0 max 16384 2024-09-19T06:55:46.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 Processor -- start 2024-09-19T06:55:46.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 -- start start 2024-09-19T06:55:46.277 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dd04b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcdcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcec870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41ae090 con 0x55612dce6b40 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41ab900 con 0x55612dce55d0 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.257+0000 7f92acd4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41a9770 con 0x55612dcf0370 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcdcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a8f13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dd04b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcdcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60370/0 (socket says 172.21.15.37:60370) 2024-09-19T06:55:46.278 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a8f13640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dd04b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36288/0 (socket says 172.21.15.37:36288) 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 -- 172.21.15.37:0/1274701680 learned_addr learned my addr 172.21.15.37:0/1274701680 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a8f13640 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcec870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 -- 172.21.15.37:0/1274701680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 msgr2=0x55612dcec870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcec870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 -- 172.21.15.37:0/1274701680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 msgr2=0x55612dd04b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dd04b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 -- 172.21.15.37:0/1274701680 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55612dced8a0 con 0x55612dce6b40 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a8f13640 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dd04b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:46.279 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f92a37fe640 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcdcad0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f9298000c00 tx=0x7f92980057d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a50aee38504af1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f9289ffb640 1 -- 172.21.15.37:0/1274701680 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298029070 con 0x55612dce6b40 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f9289ffb640 1 -- 172.21.15.37:0/1274701680 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92980055d0 con 0x55612dce6b40 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.258+0000 7f9289ffb640 1 -- 172.21.15.37:0/1274701680 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9298003850 con 0x55612dce6b40 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 -- 172.21.15.37:0/1274701680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 msgr2=0x55612dcdcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcdcad0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7f9298000c00 tx=0x7f92980057d0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 -- 172.21.15.37:0/1274701680 shutdown_connections 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dcf0370 0x55612dcec870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce6b40 0x55612dcdcad0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1274701680 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dce55d0 0x55612dd04b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 -- 172.21.15.37:0/1274701680 >> 172.21.15.37:0/1274701680 conn(0x55612dcae310 msgr2=0x55612dceb880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 -- 172.21.15.37:0/1274701680 shutdown_connections 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 -- 172.21.15.37:0/1274701680 wait complete. 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.259+0000 7f92acd4f740 1 -- start start 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 1 Processor -- start 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce55d0 0x55612dcfc220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce6b40 0x55612dcfc760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.281 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 0x55612dd71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41a9dc0 con 0x55612dce55d0 2024-09-19T06:55:46.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41ab230 con 0x55612dcf0370 2024-09-19T06:55:46.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f92a41aa490 con 0x55612dce6b40 2024-09-19T06:55:46.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92acd4f740 10 client.0 _ll_register_callbacks cb 0x7f9298009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:46.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce55d0 0x55612dcfc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce55d0 0x55612dcfc220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60384/0 (socket says 172.21.15.37:60384) 2024-09-19T06:55:46.282 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 -- 172.21.15.37:0/1854018885 learned_addr learned my addr 172.21.15.37:0/1854018885 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:46.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 0x55612dd71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 -- 172.21.15.37:0/1854018885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce6b40 msgr2=0x55612dcfc760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce6b40 0x55612dcfc760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 -- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 msgr2=0x55612dd71580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:46.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 0x55612dd71580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.260+0000 7f92a8f13640 1 -- 172.21.15.37:0/1854018885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55612dd71b70 con 0x55612dce55d0 2024-09-19T06:55:46.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92a8f13640 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce55d0 0x55612dcfc220 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f929c006010 tx=0x7f929c048040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9226ab3808f6d11 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:46.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92837fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:46.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92837fe640 1 -- 172.21.15.37:0/1854018885 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f929c036040 con 0x55612dce55d0 2024-09-19T06:55:46.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92837fe640 1 -- 172.21.15.37:0/1854018885 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f929c0303a0 con 0x55612dce55d0 2024-09-19T06:55:46.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92980055d0 con 0x55612dce55d0 2024-09-19T06:55:46.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92837fe640 1 -- 172.21.15.37:0/1854018885 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f929c041040 con 0x55612dce55d0 2024-09-19T06:55:46.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92acd4f740 10 client.19332 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:46.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55612dd72e50 con 0x55612dce55d0 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92837fe640 1 -- 172.21.15.37:0/1854018885 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f92980055d0 con 0x55612dce55d0 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.261+0000 7f92837fe640 1 -- 172.21.15.37:0/1854018885 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f929c060020 con 0x55612dce55d0 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92acd4f740 10 client.19332 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92acd4f740 10 client.19332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92acd4f740 20 client.19332 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd73180 0x55612dd755c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92acd4f740 4 client.19332 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55612dc52d70 con 0x55612dd73180 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92a37fe640 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd73180 0x55612dd755c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:46.285 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92a37fe640 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd73180 0x55612dd755c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6809a2fc81320573 server_cookie=c889d92a81fce3fa in_seq=0 out_seq=0 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92837fe640 10 client.19332 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92837fe640 1 -- 172.21.15.37:0/1854018885 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55612dc52d70 con 0x55612dd73180 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92837fe640 10 client.19332 handle_command_reply: tid=0 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.262+0000 7f92acd4f740 1 client.19332 shutdown 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce55d0 msgr2=0x55612dcfc220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce55d0 0x55612dcfc220 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f929c006010 tx=0x7f929c048040 comp rx=0 tx=0).stop 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 shutdown_connections 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55612dd73180 0x55612dd755c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55612dcf0370 0x55612dd71580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55612dce6b40 0x55612dcfc760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 --2- 172.21.15.37:0/1854018885 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55612dce55d0 0x55612dcfc220 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.286 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 >> 172.21.15.37:0/1854018885 conn(0x55612dcae310 msgr2=0x55612dcec800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92837fe640 0 client.19332 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:55:46.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 shutdown_connections 2024-09-19T06:55:46.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 1 -- 172.21.15.37:0/1854018885 wait complete. 2024-09-19T06:55:46.287 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92acd4f740 20 client.19332 trim_cache size 0 max 16384 2024-09-19T06:55:46.287 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:55:46.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f928c0a7840 msgr2=0x7f928c0a9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92aa976640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f928c0a7840 0x7f928c0a9d00 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f92940cd7a0 tx=0x7f92940bc040 comp rx=0 tx=0).stop 2024-09-19T06:55:46.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a4102d30 msgr2=0x7f92a4198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:46.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.263+0000 7f92aa976640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a4102d30 0x7f92a4198520 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f929400ea60 tx=0x7f929400ee60 comp rx=0 tx=0).stop 2024-09-19T06:55:46.288 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 shutdown_connections 2024-09-19T06:55:46.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f92a41082a0 0x7f92a419cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f92a41036e0 0x7f92a4198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f928c0a7840 0x7f928c0a9d00 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 --2- 172.21.15.37:0/1899060827 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f92a4102d30 0x7f92a4198520 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:46.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 >> 172.21.15.37:0/1899060827 conn(0x7f92a40fc7d0 msgr2=0x7f92a4105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:46.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 shutdown_connections 2024-09-19T06:55:46.289 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:46.264+0000 7f92aa976640 1 -- 172.21.15.37:0/1899060827 wait complete. 2024-09-19T06:55:46.290 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:45 smithi096 ceph-mon[35004]: pgmap v443: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 416 op/s 2024-09-19T06:55:46.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:45 smithi151 ceph-mon[35236]: pgmap v443: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 416 op/s 2024-09-19T06:55:46.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 6362 39.72 MB/sec warmup 13 sec latency 157.122 ms 2024-09-19T06:55:47.004 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:55:47.296 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:47.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 6920 40.20 MB/sec warmup 14 sec latency 151.802 ms 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 -- 172.21.15.37:0/1545970447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 msgr2=0x7fe94c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 --2- 172.21.15.37:0/1545970447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c101200 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fe93c009900 tx=0x7fe93c02a480 comp rx=0 tx=0).stop 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 -- 172.21.15.37:0/1545970447 shutdown_connections 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 --2- 172.21.15.37:0/1545970447 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe94c102ef0 0x7fe94c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 --2- 172.21.15.37:0/1545970447 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c102570 0x7fe94c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 --2- 172.21.15.37:0/1545970447 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c101200 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 -- 172.21.15.37:0/1545970447 >> 172.21.15.37:0/1545970447 conn(0x7fe94c0fc7d0 msgr2=0x7fe94c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:47.799 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 -- 172.21.15.37:0/1545970447 shutdown_connections 2024-09-19T06:55:47.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.797+0000 7fe9511c8640 1 -- 172.21.15.37:0/1545970447 wait complete. 2024-09-19T06:55:47.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.798+0000 7fe9511c8640 1 Processor -- start 2024-09-19T06:55:47.800 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.798+0000 7fe9511c8640 1 -- start start 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe9511c8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe9511c8640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c102570 0x7fe94c197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe9511c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe94c102ef0 0x7fe94c197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe9511c8640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c110fd0 con 0x7fe94c100e20 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe9511c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c110e50 con 0x7fe94c102ef0 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe9511c8640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c111150 con 0x7fe94c102570 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe94ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe94b59e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe94c102ef0 0x7fe94c197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe94a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c102570 0x7fe94c197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.801 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe94ad9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60394/0 (socket says 172.21.15.37:60394) 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe94a59c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c102570 0x7fe94c197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:36492/0 (socket says 172.21.15.37:36492) 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.799+0000 7fe94ad9d640 1 -- 172.21.15.37:0/2525016876 learned_addr learned my addr 172.21.15.37:0/2525016876 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe94ad9d640 1 -- 172.21.15.37:0/2525016876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c102570 msgr2=0x7fe94c197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c102570 0x7fe94c197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe94ad9d640 1 -- 172.21.15.37:0/2525016876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe94c102ef0 msgr2=0x7fe94c197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe94c102ef0 0x7fe94c197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe94ad9d640 1 -- 172.21.15.37:0/2525016876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe94c1a6980 con 0x7fe94c100e20 2024-09-19T06:55:47.802 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe94b59e640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe94c102ef0 0x7fe94c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:47.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c197110 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe93c02aa70 tx=0x7fe93c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:47.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe933fff640 1 -- 172.21.15.37:0/2525016876 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c03f070 con 0x7fe94c100e20 2024-09-19T06:55:47.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe94c1a6c10 con 0x7fe94c100e20 2024-09-19T06:55:47.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe933fff640 1 -- 172.21.15.37:0/2525016876 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe93c031c30 con 0x7fe94c100e20 2024-09-19T06:55:47.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe933fff640 1 -- 172.21.15.37:0/2525016876 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe93c035030 con 0x7fe94c100e20 2024-09-19T06:55:47.803 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.800+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe94c1a70f0 con 0x7fe94c100e20 2024-09-19T06:55:47.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe933fff640 1 -- 172.21.15.37:0/2525016876 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fe93c042070 con 0x7fe94c100e20 2024-09-19T06:55:47.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 Processor -- start 2024-09-19T06:55:47.804 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 -- start start 2024-09-19T06:55:47.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 0x56426a0f0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a0f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.805 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0fd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56426a0562f0 con 0x56426a0fb370 2024-09-19T06:55:47.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564269ffdcf0 con 0x56426a0f05d0 2024-09-19T06:55:47.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.803+0000 7fe9535a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56426a0522b0 con 0x56426a0f1b40 2024-09-19T06:55:47.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0fd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0fd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60398/0 (socket says 172.21.15.37:60398) 2024-09-19T06:55:47.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 -- 172.21.15.37:0/3352516631 learned_addr learned my addr 172.21.15.37:0/3352516631 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:47.806 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a0f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 -- 172.21.15.37:0/3352516631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 msgr2=0x56426a0f0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 0x56426a0f0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 -- 172.21.15.37:0/3352516631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 msgr2=0x56426a0f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a0f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.804+0000 7fe94a59c640 1 -- 172.21.15.37:0/3352516631 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564269fe23a0 con 0x56426a0fb370 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.805+0000 7fe933fff640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9340a33b0 0x7fe9340a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.805+0000 7fe94a59c640 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0fd760 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b740 tx=0x7fe94000bc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbcf2b7a8d57311 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.805+0000 7fe94b59e640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9340a33b0 0x7fe9340a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.805+0000 7fe933fff640 1 -- 172.21.15.37:0/2525016876 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fe93c0f03a0 con 0x7fe94c100e20 2024-09-19T06:55:47.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.805+0000 7fe9317fa640 1 -- 172.21.15.37:0/3352516631 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940003dd0 con 0x56426a0fb370 2024-09-19T06:55:47.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.805+0000 7fe9317fa640 1 -- 172.21.15.37:0/3352516631 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe940015070 con 0x56426a0fb370 2024-09-19T06:55:47.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.805+0000 7fe9317fa640 1 -- 172.21.15.37:0/3352516631 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940004e00 con 0x56426a0fb370 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 -- 172.21.15.37:0/3352516631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 msgr2=0x56426a0fd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe94b59e640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9340a33b0 0x7fe9340a5870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe944005fd0 tx=0x7fe944009f00 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0fd760 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7fe94000b740 tx=0x7fe94000bc00 comp rx=0 tx=0).stop 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 -- 172.21.15.37:0/3352516631 shutdown_connections 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0fd760 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a0f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3352516631 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 0x56426a0f0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 -- 172.21.15.37:0/3352516631 >> 172.21.15.37:0/3352516631 conn(0x56426a0b9310 msgr2=0x56426a0ecc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 -- 172.21.15.37:0/3352516631 shutdown_connections 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 -- 172.21.15.37:0/3352516631 wait complete. 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.806+0000 7fe9535a1740 1 -- start start 2024-09-19T06:55:47.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 1 Processor -- start 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 0x56426a0fad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f1b40 0x56426a0f7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0f8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x56426a0fe070 con 0x56426a0fb370 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56426a011060 con 0x56426a0f1b40 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x56426a0fe1f0 con 0x56426a0f05d0 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe9535a1740 10 client.0 _ll_register_callbacks cb 0x564269d648c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0f8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0f8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60404/0 (socket says 172.21.15.37:60404) 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 -- 172.21.15.37:0/3955074461 learned_addr learned my addr 172.21.15.37:0/3955074461 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94a59c640 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f1b40 0x56426a0f7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 -- 172.21.15.37:0/3955074461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 msgr2=0x56426a0fad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 0x56426a0fad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 -- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f1b40 msgr2=0x56426a0f7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f1b40 0x56426a0f7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94b59e640 1 -- 172.21.15.37:0/3955074461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564269dc73c0 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.807+0000 7fe94a59c640 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f1b40 0x56426a0f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe94b59e640 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0f8560 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe944004a80 tx=0x7fe944009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1596d1706af10c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe92eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe92eff5640 1 -- 172.21.15.37:0/3955074461 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94401e420 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe92eff5640 1 -- 172.21.15.37:0/3955074461 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fe94401e5c0 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe940015070 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe92eff5640 1 -- 172.21.15.37:0/3955074461 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe94401e7c0 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe9535a1740 10 client.19344 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56426a109b20 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe92eff5640 1 -- 172.21.15.37:0/3955074461 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe940015070 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.808+0000 7fe92eff5640 1 -- 172.21.15.37:0/3955074461 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fe944026a50 con 0x56426a0fb370 2024-09-19T06:55:47.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe9535a1740 10 client.19344 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe9535a1740 10 client.19344 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe9535a1740 20 client.19344 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a10afe0 0x56426a10d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe9535a1740 4 client.19344 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56426a063860 con 0x56426a10afe0 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a10afe0 0x56426a10d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a10afe0 0x56426a10d450 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6283274b9733cdc server_cookie=724715e0e8c03381 in_seq=0 out_seq=0 2024-09-19T06:55:47.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.809+0000 7fe92eff5640 10 client.19344 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe92eff5640 1 -- 172.21.15.37:0/3955074461 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56426a063860 con 0x56426a10afe0 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe92eff5640 10 client.19344 handle_command_reply: tid=0 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 client.19344 shutdown 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 msgr2=0x56426a0f8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0f8560 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7fe944004a80 tx=0x7fe944009500 comp rx=0 tx=0).stop 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 shutdown_connections 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a10afe0 0x56426a10d450 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a0f8560 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f1b40 0x56426a0f7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 --2- 172.21.15.37:0/3955074461 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 0x56426a0fad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe92eff5640 0 client.19344 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:47.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 >> 172.21.15.37:0/3955074461 conn(0x56426a0b9310 msgr2=0x56426a0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:47.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 shutdown_connections 2024-09-19T06:55:47.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 1 -- 172.21.15.37:0/3955074461 wait complete. 2024-09-19T06:55:47.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.810+0000 7fe9535a1740 20 client.19344 trim_cache size 0 max 16384 2024-09-19T06:55:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 Processor -- start 2024-09-19T06:55:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 -- start start 2024-09-19T06:55:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a107310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 0x56426a0f9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a112d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1b2460 con 0x56426a0fb370 2024-09-19T06:55:47.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1afcd0 con 0x56426a0f05d0 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe9535a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1adb40 con 0x56426a0f1b40 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a112d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60406/0 (socket says 172.21.15.37:60406) 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94ad9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 -- 172.21.15.37:0/2453363526 learned_addr learned my addr 172.21.15.37:0/2453363526 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94a59c640 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 0x56426a0f9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 -- 172.21.15.37:0/2453363526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 msgr2=0x56426a0f9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 0x56426a0f9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 -- 172.21.15.37:0/2453363526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 msgr2=0x56426a107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.825+0000 7fe94b59e640 1 -- 172.21.15.37:0/2453363526 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56426a0f6510 con 0x56426a0fb370 2024-09-19T06:55:47.829 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.826+0000 7fe94b59e640 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a112d20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe944004010 tx=0x7fe94400f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1ca0ece4a08f8b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.826+0000 7fe930ff9640 1 -- 172.21.15.37:0/2453363526 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944005bd0 con 0x56426a0fb370 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.826+0000 7fe930ff9640 1 -- 172.21.15.37:0/2453363526 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe944034070 con 0x56426a0fb370 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.826+0000 7fe930ff9640 1 -- 172.21.15.37:0/2453363526 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe944026400 con 0x56426a0fb370 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- 172.21.15.37:0/2453363526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 msgr2=0x56426a112d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a112d20 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fe944004010 tx=0x7fe94400f5b0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- 172.21.15.37:0/2453363526 shutdown_connections 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0fb370 0x56426a112d20 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f1b40 0x56426a0f9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2453363526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0f05d0 0x56426a107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- 172.21.15.37:0/2453363526 >> 172.21.15.37:0/2453363526 conn(0x56426a0b9310 msgr2=0x56426a0f9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- 172.21.15.37:0/2453363526 shutdown_connections 2024-09-19T06:55:47.832 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- 172.21.15.37:0/2453363526 wait complete. 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- start start 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 Processor -- start 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 0x56426a103780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0f1b40 0x56426a103cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0fb370 0x56426a1004b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ae190 con 0x56426a0f1b40 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1af600 con 0x56426a0fb370 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fe94c1ae860 con 0x56426a0f05d0 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe9535a1740 10 client.0 _ll_register_callbacks cb 0x7fe94400f500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.827+0000 7fe94a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0f1b40 0x56426a103cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0f1b40 0x56426a103cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60410/0 (socket says 172.21.15.37:60410) 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 -- 172.21.15.37:0/2412246979 learned_addr learned my addr 172.21.15.37:0/2412246979 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94b59e640 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0fb370 0x56426a1004b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.833 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 0x56426a103780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 -- 172.21.15.37:0/2412246979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 msgr2=0x56426a103780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 0x56426a103780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 -- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0fb370 msgr2=0x56426a1004b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0fb370 0x56426a1004b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 -- 172.21.15.37:0/2412246979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56426a100ce0 con 0x56426a0f1b40 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe94a59c640 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0f1b40 0x56426a103cc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x56426a0f1070 tx=0x7fe940003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b37698a990a45d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe92eff5640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe92eff5640 1 -- 172.21.15.37:0/2412246979 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940011850 con 0x56426a0f1b40 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe92eff5640 1 -- 172.21.15.37:0/2412246979 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fe9400119f0 con 0x56426a0f1b40 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe944034070 con 0x56426a0f1b40 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe92eff5640 1 -- 172.21.15.37:0/2412246979 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fe940033050 con 0x56426a0f1b40 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe9535a1740 10 client.19350 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.828+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x56426a176880 con 0x56426a0f1b40 2024-09-19T06:55:47.834 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe92eff5640 1 -- 172.21.15.37:0/2412246979 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe944034070 con 0x56426a0f1b40 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe92eff5640 1 -- 172.21.15.37:0/2412246979 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fe940025bd0 con 0x56426a0f1b40 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe9535a1740 10 client.19350 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe9535a1740 10 client.19350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe9535a1740 20 client.19350 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a176da0 0x56426a1791c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe9535a1740 4 client.19350 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56426a05dd70 con 0x56426a176da0 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a176da0 0x56426a1791c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe94ad9d640 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a176da0 0x56426a1791c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d861822071c9ad56 server_cookie=2cf03433e826646d in_seq=0 out_seq=0 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.829+0000 7fe92eff5640 10 client.19350 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe92eff5640 1 -- 172.21.15.37:0/2412246979 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56426a05dd70 con 0x56426a176da0 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe92eff5640 10 client.19350 handle_command_reply: tid=0 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 client.19350 shutdown 2024-09-19T06:55:47.835 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0f1b40 msgr2=0x56426a103cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0f1b40 0x56426a103cc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x56426a0f1070 tx=0x7fe940003e00 comp rx=0 tx=0).stop 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 shutdown_connections 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x56426a176da0 0x56426a1791c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56426a0fb370 0x56426a1004b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x56426a0f1b40 0x56426a103cc0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe92eff5640 0 client.19350 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 --2- 172.21.15.37:0/2412246979 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x56426a0f05d0 0x56426a103780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 >> 172.21.15.37:0/2412246979 conn(0x56426a0b9310 msgr2=0x56426a0f2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 shutdown_connections 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 1 -- 172.21.15.37:0/2412246979 wait complete. 2024-09-19T06:55:47.836 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9535a1740 20 client.19350 trim_cache size 0 max 16384 2024-09-19T06:55:47.837 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:55:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9340a33b0 msgr2=0x7fe9340a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9511c8640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9340a33b0 0x7fe9340a5870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe944005fd0 tx=0x7fe944009f00 comp rx=0 tx=0).stop 2024-09-19T06:55:47.837 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 msgr2=0x7fe94c197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.830+0000 7fe9511c8640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c197110 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fe93c02aa70 tx=0x7fe93c003c10 comp rx=0 tx=0).stop 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 shutdown_connections 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe94c102ef0 0x7fe94c197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe94c102570 0x7fe94c197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fe9340a33b0 0x7fe9340a5870 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 --2- 172.21.15.37:0/2525016876 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fe94c100e20 0x7fe94c197110 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 >> 172.21.15.37:0/2525016876 conn(0x7fe94c0fc7d0 msgr2=0x7fe94c0fe050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 shutdown_connections 2024-09-19T06:55:47.838 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:47.831+0000 7fe9511c8640 1 -- 172.21.15.37:0/2525016876 wait complete. 2024-09-19T06:55:48.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:47 smithi096 ceph-mon[35004]: pgmap v444: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 416 op/s 2024-09-19T06:55:48.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:47 smithi037 ceph-mon[31211]: pgmap v444: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 416 op/s 2024-09-19T06:55:48.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:47 smithi151 ceph-mon[35236]: pgmap v444: 97 pgs: 97 active+clean; 85 MiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.1 MiB/s wr, 416 op/s 2024-09-19T06:55:48.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 7141 38.18 MB/sec warmup 15 sec latency 68.152 ms 2024-09-19T06:55:48.597 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:55:48.869 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:49.354 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 -- 172.21.15.37:0/2329306904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341082a0 msgr2=0x7fae3410ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 --2- 172.21.15.37:0/2329306904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341082a0 0x7fae3410ec40 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009cd0 tx=0x7fae2c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 -- 172.21.15.37:0/2329306904 shutdown_connections 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 --2- 172.21.15.37:0/2329306904 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341082a0 0x7fae3410ec40 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 --2- 172.21.15.37:0/2329306904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae341036e0 0x7fae34107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 --2- 172.21.15.37:0/2329306904 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae34102d30 0x7fae34103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 -- 172.21.15.37:0/2329306904 >> 172.21.15.37:0/2329306904 conn(0x7fae340fc7d0 msgr2=0x7fae340febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 -- 172.21.15.37:0/2329306904 shutdown_connections 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.335+0000 7fae3b22f640 1 -- 172.21.15.37:0/2329306904 wait complete. 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 Processor -- start 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 -- start start 2024-09-19T06:55:49.355 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae34102d30 0x7fae3419a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341036e0 0x7fae3419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae341082a0 0x7fae3419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae341120a0 con 0x7fae341036e0 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae34111f20 con 0x7fae341082a0 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae3b22f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae34112220 con 0x7fae34102d30 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341036e0 0x7fae3419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae397cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae341082a0 0x7fae3419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341036e0 0x7fae3419ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60428/0 (socket says 172.21.15.37:60428) 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 -- 172.21.15.37:0/3052045357 learned_addr learned my addr 172.21.15.37:0/3052045357 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae38fcb640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae34102d30 0x7fae3419a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 -- 172.21.15.37:0/3052045357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae34102d30 msgr2=0x7fae3419a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae34102d30 0x7fae3419a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.356 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 -- 172.21.15.37:0/3052045357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae341082a0 msgr2=0x7fae3419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae341082a0 0x7fae3419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.336+0000 7fae33fff640 1 -- 172.21.15.37:0/3052045357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae3419f6c0 con 0x7fae341036e0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.337+0000 7fae33fff640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341036e0 0x7fae3419ac50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fae2000e970 tx=0x7fae2000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.337+0000 7fae31ffb640 1 -- 172.21.15.37:0/3052045357 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2000cd80 con 0x7fae341036e0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.337+0000 7fae31ffb640 1 -- 172.21.15.37:0/3052045357 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae200035f0 con 0x7fae341036e0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.337+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3419f9b0 con 0x7fae341036e0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.337+0000 7fae31ffb640 1 -- 172.21.15.37:0/3052045357 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20010690 con 0x7fae341036e0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.337+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae341a7290 con 0x7fae341036e0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.339+0000 7fae31ffb640 1 -- 172.21.15.37:0/3052045357 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fae200037a0 con 0x7fae341036e0 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 Processor -- start 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 -- start start 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c4b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.357 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c4ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c58760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f866bb12f0 con 0x55f866c4cb40 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f866b58cf0 con 0x55f866c56370 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae3d608740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f866bad2b0 con 0x55f866c4b5d0 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c58760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae31ffb640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fae240a33b0 0x7fae240a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae397cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c4b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae397cc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c4b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60440/0 (socket says 172.21.15.37:60440) 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae33fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c58760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36364/0 (socket says 172.21.15.37:36364) 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.340+0000 7fae397cc640 1 -- 172.21.15.37:0/3757209686 learned_addr learned my addr 172.21.15.37:0/3757209686 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae38fcb640 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c4ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae397cc640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fae240a33b0 0x7fae240a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae31ffb640 1 -- 172.21.15.37:0/3052045357 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fae200cffe0 con 0x7fae341036e0 2024-09-19T06:55:49.358 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae33fff640 1 -- 172.21.15.37:0/3757209686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 msgr2=0x55f866c4ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae33fff640 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c4ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae33fff640 1 -- 172.21.15.37:0/3757209686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 msgr2=0x55f866c4b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae33fff640 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c4b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae33fff640 1 -- 172.21.15.37:0/3757209686 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f866b3d3a0 con 0x55f866c56370 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae397cc640 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae33fff640 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c58760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fae200cd6a0 tx=0x7fae2009bb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1c9ecb398817ce server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae397cc640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fae240a33b0 0x7fae240a5870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009a60 tx=0x7fae2c02a8e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae1effd640 1 -- 172.21.15.37:0/3757209686 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2009c930 con 0x55f866c56370 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae1effd640 1 -- 172.21.15.37:0/3757209686 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2009c230 con 0x55f866c56370 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.341+0000 7fae1effd640 1 -- 172.21.15.37:0/3757209686 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae20098a30 con 0x55f866c56370 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 -- 172.21.15.37:0/3757209686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 msgr2=0x55f866c58760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c58760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fae200cd6a0 tx=0x7fae2009bb90 comp rx=0 tx=0).stop 2024-09-19T06:55:49.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 -- 172.21.15.37:0/3757209686 shutdown_connections 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c58760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c4ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 --2- 172.21.15.37:0/3757209686 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c4b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 -- 172.21.15.37:0/3757209686 >> 172.21.15.37:0/3757209686 conn(0x55f866c14310 msgr2=0x55f866c47c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 -- 172.21.15.37:0/3757209686 shutdown_connections 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 -- 172.21.15.37:0/3757209686 wait complete. 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 -- start start 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.342+0000 7fae3d608740 1 Processor -- start 2024-09-19T06:55:49.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae3d608740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c55da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae3d608740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c53050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae3d608740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae3d608740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55f866c58fb0 con 0x55f866c4cb40 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae3d608740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f866b6c060 con 0x55f866c56370 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae3d608740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55f866c59130 con 0x55f866c4b5d0 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae3d608740 10 client.0 _ll_register_callbacks cb 0x55f8668bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c53050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c53050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60442/0 (socket says 172.21.15.37:60442) 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 -- 172.21.15.37:0/2694328482 learned_addr learned my addr 172.21.15.37:0/2694328482 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae397cc640 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 -- 172.21.15.37:0/2694328482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 msgr2=0x55f866c55da0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:55:49.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c55da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 -- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 msgr2=0x55f866c537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.343+0000 7fae33fff640 1 -- 172.21.15.37:0/2694328482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8669223c0 con 0x55f866c4cb40 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae33fff640 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c53050 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fae2009bf30 tx=0x7fae200056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f498b06056682a6d server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae13fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae13fff640 1 -- 172.21.15.37:0/2694328482 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae200a4ea0 con 0x55f866c4cb40 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae2009c230 con 0x55f866c4cb40 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae13fff640 1 -- 172.21.15.37:0/2694328482 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fae200a0170 con 0x55f866c4cb40 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae13fff640 1 -- 172.21.15.37:0/2694328482 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fae20098a10 con 0x55f866c4cb40 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 10 client.19362 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f866c64a90 con 0x55f866c4cb40 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae13fff640 1 -- 172.21.15.37:0/2694328482 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae2009c230 con 0x55f866c4cb40 2024-09-19T06:55:49.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae13fff640 1 -- 172.21.15.37:0/2694328482 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fae200bc150 con 0x55f866c4cb40 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 10 client.19362 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 10 client.19362 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 20 client.19362 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866c65de0 0x55f866c681d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 4 client.19362 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.344+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f866bbe860 con 0x55f866c65de0 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.345+0000 7fae38fcb640 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866c65de0 0x55f866c681d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.345+0000 7fae13fff640 10 client.19362 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.345+0000 7fae38fcb640 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866c65de0 0x55f866c681d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a7e236874967b99 server_cookie=5624acae2af894e7 in_seq=0 out_seq=0 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.346+0000 7fae13fff640 1 -- 172.21.15.37:0/2694328482 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f866bbe860 con 0x55f866c65de0 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.346+0000 7fae13fff640 10 client.19362 handle_command_reply: tid=0 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 client.19362 shutdown 2024-09-19T06:55:49.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 msgr2=0x55f866c53050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c53050 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fae2009bf30 tx=0x7fae200056f0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 shutdown_connections 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866c65de0 0x55f866c681d0 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c53050 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 --2- 172.21.15.37:0/2694328482 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c55da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae13fff640 0 client.19362 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 >> 172.21.15.37:0/2694328482 conn(0x55f866c14310 msgr2=0x55f866c4d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 shutdown_connections 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 1 -- 172.21.15.37:0/2694328482 wait complete. 2024-09-19T06:55:49.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.347+0000 7fae3d608740 20 client.19362 trim_cache size 0 max 16384 2024-09-19T06:55:49.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 Processor -- start 2024-09-19T06:55:49.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 -- start start 2024-09-19T06:55:49.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c4b5d0 0x55f866c47da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c54fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c56370 0x55f866c4e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae341b25a0 con 0x55f866c4cb40 2024-09-19T06:55:49.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae341afe10 con 0x55f866c4b5d0 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae3d608740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae341adc80 con 0x55f866c56370 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c54fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c4b5d0 0x55f866c47da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c4b5d0 0x55f866c47da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36380/0 (socket says 172.21.15.37:36380) 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c54fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60444/0 (socket says 172.21.15.37:60444) 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 -- 172.21.15.37:0/2592512811 learned_addr learned my addr 172.21.15.37:0/2592512811 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae397cc640 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c56370 0x55f866c4e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 -- 172.21.15.37:0/2592512811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c56370 msgr2=0x55f866c4e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c56370 0x55f866c4e120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 -- 172.21.15.37:0/2592512811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 msgr2=0x55f866c54fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c54fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae38fcb640 1 -- 172.21.15.37:0/2592512811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f866c62730 con 0x55f866c4b5d0 2024-09-19T06:55:49.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.361+0000 7fae33fff640 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c54fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.362+0000 7fae38fcb640 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c4b5d0 0x55f866c47da0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fae280043d0 tx=0x7fae2800bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4008f09d15b72caa server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.362+0000 7fae1e7fc640 1 -- 172.21.15.37:0/2592512811 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae28003b50 con 0x55f866c4b5d0 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.362+0000 7fae1e7fc640 1 -- 172.21.15.37:0/2592512811 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae28005500 con 0x55f866c4b5d0 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.362+0000 7fae1e7fc640 1 -- 172.21.15.37:0/2592512811 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2801a330 con 0x55f866c4b5d0 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.362+0000 7fae3d608740 1 -- 172.21.15.37:0/2592512811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c4b5d0 msgr2=0x55f866c47da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c4b5d0 0x55f866c47da0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fae280043d0 tx=0x7fae2800bb40 comp rx=0 tx=0).stop 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 -- 172.21.15.37:0/2592512811 shutdown_connections 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c56370 0x55f866c4e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c54fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 --2- 172.21.15.37:0/2592512811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c4b5d0 0x55f866c47da0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 -- 172.21.15.37:0/2592512811 >> 172.21.15.37:0/2592512811 conn(0x55f866c14310 msgr2=0x55f866c548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 -- 172.21.15.37:0/2592512811 shutdown_connections 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 -- 172.21.15.37:0/2592512811 wait complete. 2024-09-19T06:55:49.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 -- start start 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.363+0000 7fae3d608740 1 Processor -- start 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae3d608740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c5e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae3d608740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c5e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae3d608740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c5b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae3d608740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fae341ae2d0 con 0x55f866c4cb40 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae3d608740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fae341af740 con 0x55f866c56370 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae3d608740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fae341ae9a0 con 0x55f866c4b5d0 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae3d608740 10 client.0 _ll_register_callbacks cb 0x7fae2c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c5e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae397cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c5b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae397cc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c5b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:36396/0 (socket says 172.21.15.37:36396) 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae397cc640 1 -- 172.21.15.37:0/1729934883 learned_addr learned my addr 172.21.15.37:0/1729934883 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:49.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c5e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60454/0 (socket says 172.21.15.37:60454) 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae38fcb640 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c5e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 -- 172.21.15.37:0/1729934883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 msgr2=0x55f866c5e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c5e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 -- 172.21.15.37:0/1729934883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 msgr2=0x55f866c5b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c5b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 -- 172.21.15.37:0/1729934883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f866c5eff0 con 0x55f866c4cb40 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae33fff640 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c5e900 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x55f866c54600 tx=0x7fae2009bb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f95667f74a43796 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.364+0000 7fae13fff640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae13fff640 1 -- 172.21.15.37:0/1729934883 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae2009bd70 con 0x55f866c4cb40 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae13fff640 1 -- 172.21.15.37:0/1729934883 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fae200b5120 con 0x55f866c4cb40 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fae28005500 con 0x55f866c4cb40 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae13fff640 1 -- 172.21.15.37:0/1729934883 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fae200984d0 con 0x55f866c4cb40 2024-09-19T06:55:49.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 10 client.19368 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55f866c5eff0 con 0x55f866c4cb40 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae13fff640 1 -- 172.21.15.37:0/1729934883 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fae28005500 con 0x55f866c4cb40 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae13fff640 1 -- 172.21.15.37:0/1729934883 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fae2009b0f0 con 0x55f866c4cb40 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 10 client.19368 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 10 client.19368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 20 client.19368 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866cd10b0 0x55f866cd34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 4 client.19368 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f866bb8d70 con 0x55f866cd10b0 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.365+0000 7fae38fcb640 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866cd10b0 0x55f866cd34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.366+0000 7fae38fcb640 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866cd10b0 0x55f866cd34d0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4a627873e791c2 server_cookie=4c5f95e0fd54cafa in_seq=0 out_seq=0 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.366+0000 7fae13fff640 10 client.19368 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:49.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae13fff640 1 -- 172.21.15.37:0/1729934883 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f866bb8d70 con 0x55f866cd10b0 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae13fff640 10 client.19368 handle_command_reply: tid=0 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 client.19368 shutdown 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 msgr2=0x55f866c5e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c5e900 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x55f866c54600 tx=0x7fae2009bb90 comp rx=0 tx=0).stop 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 shutdown_connections 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55f866cd10b0 0x55f866cd34d0 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f866c56370 0x55f866c5b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55f866c4cb40 0x55f866c5e900 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae13fff640 0 client.19368 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 --2- 172.21.15.37:0/1729934883 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55f866c4b5d0 0x55f866c5e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 >> 172.21.15.37:0/1729934883 conn(0x55f866c14310 msgr2=0x55f866c46d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 shutdown_connections 2024-09-19T06:55:49.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 1 -- 172.21.15.37:0/1729934883 wait complete. 2024-09-19T06:55:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.367+0000 7fae3d608740 20 client.19368 trim_cache size 0 max 16384 2024-09-19T06:55:49.376 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:55:49.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fae240a33b0 msgr2=0x7fae240a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fae240a33b0 0x7fae240a5870 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fae2c009a60 tx=0x7fae2c02a8e0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341036e0 msgr2=0x7fae3419ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341036e0 0x7fae3419ac50 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fae2000e970 tx=0x7fae2000ee30 comp rx=0 tx=0).stop 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 shutdown_connections 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fae341082a0 0x7fae3419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fae240a33b0 0x7fae240a5870 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fae341036e0 0x7fae3419ac50 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 --2- 172.21.15.37:0/3052045357 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fae34102d30 0x7fae3419a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 >> 172.21.15.37:0/3052045357 conn(0x7fae340fc7d0 msgr2=0x7fae34100c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 shutdown_connections 2024-09-19T06:55:49.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:49.368+0000 7fae3b22f640 1 -- 172.21.15.37:0/3052045357 wait complete. 2024-09-19T06:55:49.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 7351 35.87 MB/sec warmup 16 sec latency 104.689 ms 2024-09-19T06:55:50.021 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:49 smithi037 ceph-mon[31211]: pgmap v445: 97 pgs: 97 active+clean; 92 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 532 op/s 2024-09-19T06:55:50.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:49 smithi096 ceph-mon[35004]: pgmap v445: 97 pgs: 97 active+clean; 92 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 532 op/s 2024-09-19T06:55:50.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:49 smithi151 ceph-mon[35236]: pgmap v445: 97 pgs: 97 active+clean; 92 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.4 MiB/s wr, 532 op/s 2024-09-19T06:55:50.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 7629 34.11 MB/sec warmup 17 sec latency 199.998 ms 2024-09-19T06:55:51.050 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-19T06:55:51.051 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-19T06:55:51.318 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:51.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 8161 34.32 MB/sec warmup 18 sec latency 109.493 ms 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 -- 172.21.15.37:0/3414482900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c101de0 msgr2=0x7f3c9c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/3414482900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c101de0 0x7f3c9c1021c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f3c98009900 tx=0x7f3c9802a410 comp rx=0 tx=0).stop 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 -- 172.21.15.37:0/3414482900 shutdown_connections 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/3414482900 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c9c106f10 0x7f3c9c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/3414482900 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c9c102790 0x7f3c9c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/3414482900 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c101de0 0x7f3c9c1021c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 -- 172.21.15.37:0/3414482900 >> 172.21.15.37:0/3414482900 conn(0x7f3c9c0fd8c0 msgr2=0x7f3c9c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:51.814 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 -- 172.21.15.37:0/3414482900 shutdown_connections 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.809+0000 7f3ca40f6640 1 -- 172.21.15.37:0/3414482900 wait complete. 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 Processor -- start 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 -- start start 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c9c101de0 0x7f3c9c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c9c102790 0x7f3c9c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c106f10 0x7f3c9c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c111f80 con 0x7f3c9c106f10 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c111e00 con 0x7f3c9c102790 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca40f6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c112100 con 0x7f3c9c101de0 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.810+0000 7f3ca2693640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c106f10 0x7f3c9c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c106f10 0x7f3c9c074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50386/0 (socket says 172.21.15.37:50386) 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 -- 172.21.15.37:0/537514063 learned_addr learned my addr 172.21.15.37:0/537514063 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:51.815 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c9c101de0 0x7f3c9c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca1691640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c9c102790 0x7f3c9c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 -- 172.21.15.37:0/537514063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c9c101de0 msgr2=0x7f3c9c0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c9c101de0 0x7f3c9c0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 -- 172.21.15.37:0/537514063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c9c102790 msgr2=0x7f3c9c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c9c102790 0x7f3c9c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 -- 172.21.15.37:0/537514063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c9c1a6920 con 0x7f3c9c106f10 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca2693640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c106f10 0x7f3c9c074f70 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400eaa0 tx=0x7f3c9400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3c92ffd640 1 -- 172.21.15.37:0/537514063 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9400cdb0 con 0x7f3c9c106f10 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3c92ffd640 1 -- 172.21.15.37:0/537514063 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c94005020 con 0x7f3c9c106f10 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c9c1a6c10 con 0x7f3c9c106f10 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3c92ffd640 1 -- 172.21.15.37:0/537514063 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c940116b0 con 0x7f3c9c106f10 2024-09-19T06:55:51.816 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.811+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c9c1a7150 con 0x7f3c9c106f10 2024-09-19T06:55:51.818 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.814+0000 7f3c92ffd640 1 -- 172.21.15.37:0/537514063 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f3c94005560 con 0x7f3c9c106f10 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.814+0000 7f3ca64cf740 1 Processor -- start 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca64cf740 1 -- start start 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3c92ffd640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3c880a3430 0x7f3c880a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b55f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b60800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b63130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561118abb280 con 0x561118b56480 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561118a62b10 con 0x561118b60d40 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561118ab7240 con 0x561118b55b60 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca1691640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3c880a3430 0x7f3c880a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca1e92640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b55f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca1e92640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b55f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:37480/0 (socket says 172.21.15.37:37480) 2024-09-19T06:55:51.819 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca1e92640 1 -- 172.21.15.37:0/3557357260 learned_addr learned my addr 172.21.15.37:0/3557357260 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3c92ffd640 1 -- 172.21.15.37:0/537514063 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f3c940cfe00 con 0x7f3c9c106f10 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca2693640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b63130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca1691640 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b60800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca2693640 1 -- 172.21.15.37:0/3557357260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 msgr2=0x561118b55f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca2693640 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b55f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca2693640 1 -- 172.21.15.37:0/3557357260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 msgr2=0x561118b60800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca2693640 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b60800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca2693640 1 -- 172.21.15.37:0/3557357260 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561118a6f1f0 con 0x561118b60d40 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.815+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b55f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.816+0000 7f3ca1691640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3c880a3430 0x7f3c880a58f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x561118b52f50 tx=0x7f3c8c009850 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.816+0000 7f3ca2693640 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b63130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3c940cd520 tx=0x7f3c9409c060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b99f1a899294d3d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.816+0000 7f3c87fff640 1 -- 172.21.15.37:0/3557357260 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c9401f040 con 0x561118b60d40 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.816+0000 7f3c87fff640 1 -- 172.21.15.37:0/3557357260 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9409a460 con 0x561118b60d40 2024-09-19T06:55:51.820 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.816+0000 7f3c87fff640 1 -- 172.21.15.37:0/3557357260 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c94086be0 con 0x561118b60d40 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3557357260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 msgr2=0x561118b63130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b63130 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3c940cd520 tx=0x7f3c9409c060 comp rx=0 tx=0).stop 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3557357260 shutdown_connections 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b63130 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b60800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3557357260 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b55f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3557357260 >> 172.21.15.37:0/3557357260 conn(0x561118b520f0 msgr2=0x561118b524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3557357260 shutdown_connections 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3557357260 wait complete. 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 -- start start 2024-09-19T06:55:51.821 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.817+0000 7f3ca64cf740 1 Processor -- start 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b5dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b5e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b5e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561118b63910 con 0x561118b56480 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561118a75f10 con 0x561118b60d40 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561118b63a90 con 0x561118b55b60 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca64cf740 10 client.0 _ll_register_callbacks cb 0x561118a3ea40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b5e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b5e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50406/0 (socket says 172.21.15.37:50406) 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 -- 172.21.15.37:0/3254415773 learned_addr learned my addr 172.21.15.37:0/3254415773 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:51.824 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca2693640 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b5e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 -- 172.21.15.37:0/3254415773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 msgr2=0x561118b5dbf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b5dbf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 -- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 msgr2=0x561118b5e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b5e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 -- 172.21.15.37:0/3254415773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56111882c6a0 con 0x561118b56480 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca2693640 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3ca1691640 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b5e130 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00fd70 tx=0x7f3c8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdec4e44525f83e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3c857fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.818+0000 7f3c857fa640 1 -- 172.21.15.37:0/3254415773 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c026070 con 0x561118b56480 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3c857fa640 1 -- 172.21.15.37:0/3254415773 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f3c8c00fda0 con 0x561118b56480 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c9409a460 con 0x561118b56480 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3c857fa640 1 -- 172.21.15.37:0/3254415773 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3c8c020450 con 0x561118b56480 2024-09-19T06:55:51.825 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 10 client.19380 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:51.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561118b6f510 con 0x561118b56480 2024-09-19T06:55:51.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3c857fa640 1 -- 172.21.15.37:0/3254415773 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c9409a460 con 0x561118b56480 2024-09-19T06:55:51.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3c857fa640 1 -- 172.21.15.37:0/3254415773 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f3c8c01d040 con 0x561118b56480 2024-09-19T06:55:51.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 10 client.19380 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:51.826 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 10 client.19380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 20 client.19380 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118b70a90 0x561118b72ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 4 client.19380 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561118ac87f0 con 0x561118b70a90 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.819+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118b70a90 0x561118b72ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.820+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118b70a90 0x561118b72ee0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43d3a25d6743d20d server_cookie=74b4d99c47a3bcb2 in_seq=0 out_seq=0 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.820+0000 7f3c857fa640 10 client.19380 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3c857fa640 1 -- 172.21.15.37:0/3254415773 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561118ac87f0 con 0x561118b70a90 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3c857fa640 10 client.19380 handle_command_reply: tid=0 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 client.19380 shutdown 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 msgr2=0x561118b5e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b5e130 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c00fd70 tx=0x7f3c8c004730 comp rx=0 tx=0).stop 2024-09-19T06:55:51.827 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 shutdown_connections 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118b70a90 0x561118b72ee0 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b5e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b5e130 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/3254415773 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b5dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3c857fa640 0 client.19380 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 >> 172.21.15.37:0/3254415773 conn(0x561118b520f0 msgr2=0x561118b57e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 shutdown_connections 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 1 -- 172.21.15.37:0/3254415773 wait complete. 2024-09-19T06:55:51.828 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.822+0000 7f3ca64cf740 20 client.19380 trim_cache size 0 max 16384 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.841+0000 7f3ca64cf740 1 Processor -- start 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.841+0000 7f3ca64cf740 1 -- start start 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b5cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b78d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1b2520 con 0x561118b56480 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1afd90 con 0x561118b60d40 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1adc00 con 0x561118b55b60 2024-09-19T06:55:51.844 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b787f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50412/0 (socket says 172.21.15.37:50412) 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca2693640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b78d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 -- 172.21.15.37:0/1341270973 learned_addr learned my addr 172.21.15.37:0/1341270973 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b5cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 -- 172.21.15.37:0/1341270973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 msgr2=0x561118b5cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b5cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 -- 172.21.15.37:0/1341270973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 msgr2=0x561118b78d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b78d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 -- 172.21.15.37:0/1341270973 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561118b562b0 con 0x561118b56480 2024-09-19T06:55:51.845 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.842+0000 7f3ca1691640 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b787f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c02a7c0 tx=0x7f3c8c009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d780741f0894e server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:51.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.843+0000 7f3c877fe640 1 -- 172.21.15.37:0/1341270973 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c026070 con 0x561118b56480 2024-09-19T06:55:51.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.843+0000 7f3c877fe640 1 -- 172.21.15.37:0/1341270973 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c8c00fb40 con 0x561118b56480 2024-09-19T06:55:51.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.843+0000 7f3c877fe640 1 -- 172.21.15.37:0/1341270973 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c8c020850 con 0x561118b56480 2024-09-19T06:55:51.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.843+0000 7f3ca64cf740 1 -- 172.21.15.37:0/1341270973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 msgr2=0x561118b787f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.843+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b787f0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f3c8c02a7c0 tx=0x7f3c8c009f80 comp rx=0 tx=0).stop 2024-09-19T06:55:51.848 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 -- 172.21.15.37:0/1341270973 shutdown_connections 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b78d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b56480 0x561118b787f0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/1341270973 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b55b60 0x561118b5cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 -- 172.21.15.37:0/1341270973 >> 172.21.15.37:0/1341270973 conn(0x561118b520f0 msgr2=0x561118b5c3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 -- 172.21.15.37:0/1341270973 shutdown_connections 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 -- 172.21.15.37:0/1341270973 wait complete. 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 -- start start 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.844+0000 7f3ca64cf740 1 Processor -- start 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b55b60 0x561118b68d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b56480 0x561118b69250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca64cf740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b65ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ae250 con 0x561118b55b60 2024-09-19T06:55:51.849 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1af6c0 con 0x561118b60d40 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca64cf740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f3c9c1ae920 con 0x561118b56480 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca64cf740 10 client.0 _ll_register_callbacks cb 0x7f3c8c009a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b55b60 0x561118b68d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b55b60 0x561118b68d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50416/0 (socket says 172.21.15.37:50416) 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 -- 172.21.15.37:0/270660684 learned_addr learned my addr 172.21.15.37:0/270660684 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca2693640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b65ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1691640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b56480 0x561118b69250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 -- 172.21.15.37:0/270660684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b56480 msgr2=0x561118b69250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b56480 0x561118b69250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 -- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 msgr2=0x561118b65ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b65ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.850 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 -- 172.21.15.37:0/270660684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561118b66220 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1691640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b56480 0x561118b69250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3ca1e92640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b55b60 0x561118b68d10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3c980096c0 tx=0x7f3c98004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5372fdc3f71a02da server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3c857fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.845+0000 7f3c857fa640 1 -- 172.21.15.37:0/270660684 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c98035040 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3c857fa640 1 -- 172.21.15.37:0/270660684 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7f3c98005510 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c8c00fb40 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3c857fa640 1 -- 172.21.15.37:0/270660684 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f3c980046c0 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 10 client.19386 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561118bdbea0 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3c857fa640 1 -- 172.21.15.37:0/270660684 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3c8c00fb40 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3c857fa640 1 -- 172.21.15.37:0/270660684 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f3c98055070 con 0x561118b55b60 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 10 client.19386 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 10 client.19386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:51.851 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 20 client.19386 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:51.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118bdc600 0x561118bdea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:51.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 4 client.19386 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-19T06:55:51.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561118ac2d00 con 0x561118bdc600 2024-09-19T06:55:51.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.846+0000 7f3ca1691640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118bdc600 0x561118bdea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:51.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.847+0000 7f3ca1691640 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118bdc600 0x561118bdea20 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff34453d84a775c server_cookie=385b67259034be72 in_seq=0 out_seq=0 2024-09-19T06:55:51.852 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.847+0000 7f3c857fa640 10 client.19386 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:51.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3c857fa640 1 -- 172.21.15.37:0/270660684 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561118ac2d00 con 0x561118bdc600 2024-09-19T06:55:51.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3c857fa640 10 client.19386 handle_command_reply: tid=0 2024-09-19T06:55:51.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 client.19386 shutdown 2024-09-19T06:55:51.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b55b60 msgr2=0x561118b68d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b55b60 0x561118b68d10 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f3c980096c0 tx=0x7f3c98004460 comp rx=0 tx=0).stop 2024-09-19T06:55:51.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 shutdown_connections 2024-09-19T06:55:51.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561118bdc600 0x561118bdea20 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561118b60d40 0x561118b65ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561118b56480 0x561118b69250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3c857fa640 0 client.19386 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 --2- 172.21.15.37:0/270660684 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561118b55b60 0x561118b68d10 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 >> 172.21.15.37:0/270660684 conn(0x561118b520f0 msgr2=0x561118b5cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 shutdown_connections 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.850+0000 7f3ca64cf740 1 -- 172.21.15.37:0/270660684 wait complete. 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca64cf740 20 client.19386 trim_cache size 0 max 16384 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stdout: "return_code": 0, 2024-09-19T06:55:51.854 INFO:teuthology.orchestra.run.smithi037.stdout: "scrub_tag": "5e8ef1dc-4f46-4049-93ee-de66b7381fb0", 2024-09-19T06:55:51.855 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": "asynchronous" 2024-09-19T06:55:51.855 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:51.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3c880a3430 msgr2=0x7f3c880a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3c880a3430 0x7f3c880a58f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x561118b52f50 tx=0x7f3c8c009850 comp rx=0 tx=0).stop 2024-09-19T06:55:51.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c106f10 msgr2=0x7f3c9c074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c106f10 0x7f3c9c074f70 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f3c9400eaa0 tx=0x7f3c9400ef60 comp rx=0 tx=0).stop 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 shutdown_connections 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f3c880a3430 0x7f3c880a58f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f3c9c106f10 0x7f3c9c074f70 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3c9c102790 0x7f3c9c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 --2- 172.21.15.37:0/537514063 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3c9c101de0 0x7f3c9c0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 >> 172.21.15.37:0/537514063 conn(0x7f3c9c0fd8c0 msgr2=0x7f3c9c0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 shutdown_connections 2024-09-19T06:55:51.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:51.851+0000 7f3ca40f6640 1 -- 172.21.15.37:0/537514063 wait complete. 2024-09-19T06:55:52.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:51 smithi037 ceph-mon[31211]: pgmap v446: 97 pgs: 97 active+clean; 92 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 436 op/s 2024-09-19T06:55:52.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:51 smithi037 ceph-mon[31211]: scrub queued for path: / 2024-09-19T06:55:52.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:51 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:52.025 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:51 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 -- 172.21.15.37:0/1849315925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 msgr2=0x7fc928100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 --2- 172.21.15.37:0/1849315925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928100e50 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7fc918009930 tx=0x7fc91802a420 comp rx=0 tx=0).stop 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 -- 172.21.15.37:0/1849315925 shutdown_connections 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 --2- 172.21.15.37:0/1849315925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928101390 0x7fc92810d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 --2- 172.21.15.37:0/1849315925 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928100e50 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 --2- 172.21.15.37:0/1849315925 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928106a00 0x7fc928106de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 -- 172.21.15.37:0/1849315925 >> 172.21.15.37:0/1849315925 conn(0x7fc9280fc6b0 msgr2=0x7fc9280fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 -- 172.21.15.37:0/1849315925 shutdown_connections 2024-09-19T06:55:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.263+0000 7fc930166640 1 -- 172.21.15.37:0/1849315925 wait complete. 2024-09-19T06:55:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 Processor -- start 2024-09-19T06:55:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 -- start start 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc928199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928106a00 0x7fc9281a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc928110e90 con 0x7fc9281009f0 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc928110d10 con 0x7fc928101390 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc930166640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc928111010 con 0x7fc928106a00 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50432/0 (socket says 172.21.15.37:50432) 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 -- 172.21.15.37:0/3279554815 learned_addr learned my addr 172.21.15.37:0/3279554815 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92d701640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc928199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92e703640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928106a00 0x7fc9281a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 -- 172.21.15.37:0/3279554815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928106a00 msgr2=0x7fc9281a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928106a00 0x7fc9281a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 -- 172.21.15.37:0/3279554815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 msgr2=0x7fc928199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc928199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.264+0000 7fc92df02640 1 -- 172.21.15.37:0/3279554815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9281a47c0 con 0x7fc9281009f0 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.265+0000 7fc92df02640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928199200 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc92400e9f0 tx=0x7fc92400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.265+0000 7fc91effd640 1 -- 172.21.15.37:0/3279554815 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92400cd80 con 0x7fc9281009f0 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.265+0000 7fc91effd640 1 -- 172.21.15.37:0/3279554815 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc924010910 con 0x7fc9281009f0 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.265+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9281a4990 con 0x7fc9281009f0 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.265+0000 7fc91effd640 1 -- 172.21.15.37:0/3279554815 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924018990 con 0x7fc9281009f0 2024-09-19T06:55:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.265+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9281a4f00 con 0x7fc9281009f0 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.267+0000 7fc91effd640 1 -- 172.21.15.37:0/3279554815 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc924010ac0 con 0x7fc9281009f0 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 Processor -- start 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 -- start start 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5621dd72c2f0 con 0x5621dd7c7af0 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621dd6d4270 con 0x5621dd7c8410 2024-09-19T06:55:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc932fde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5621dd7282b0 con 0x5621dd7d2cb0 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc92e703640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc92df02640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc92e703640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50434/0 (socket says 172.21.15.37:50434) 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc92e703640 1 -- 172.21.15.37:0/2207984267 learned_addr learned my addr 172.21.15.37:0/2207984267 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc92d701640 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.268+0000 7fc91effd640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc9140a34e0 0x7fc9140a59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc92e703640 1 -- 172.21.15.37:0/2207984267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 msgr2=0x5621dd7d50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc92e703640 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc92e703640 1 -- 172.21.15.37:0/2207984267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 msgr2=0x5621dd7d2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc92e703640 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc92e703640 1 -- 172.21.15.37:0/2207984267 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621dd6d3340 con 0x5621dd7c7af0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc91effd640 1 -- 172.21.15.37:0/3279554815 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fc9240cff20 con 0x7fc9281009f0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc92e703640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc9140a34e0 0x7fc9140a59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc92e703640 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7c7ed0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc920011590 tx=0x7fc920011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5c26b4167190199 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc913fff640 1 -- 172.21.15.37:0/2207984267 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920017050 con 0x5621dd7c7af0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc913fff640 1 -- 172.21.15.37:0/2207984267 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc92000fd20 con 0x5621dd7c7af0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.269+0000 7fc913fff640 1 -- 172.21.15.37:0/2207984267 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92001ba70 con 0x5621dd7c7af0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 -- 172.21.15.37:0/2207984267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 msgr2=0x5621dd7c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7c7ed0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7fc920011590 tx=0x7fc920011a50 comp rx=0 tx=0).stop 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc92e703640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc9140a34e0 0x7fc9140a59a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc920022000 tx=0x7fc920023000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 -- 172.21.15.37:0/2207984267 shutdown_connections 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 --2- 172.21.15.37:0/2207984267 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7c7ed0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 -- 172.21.15.37:0/2207984267 >> 172.21.15.37:0/2207984267 conn(0x5621dd7c4060 msgr2=0x5621dd7c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 -- 172.21.15.37:0/2207984267 shutdown_connections 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 -- 172.21.15.37:0/2207984267 wait complete. 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 -- start start 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.270+0000 7fc932fde740 1 Processor -- start 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc932fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7cfb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc932fde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc932fde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc932fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5621dd7d5880 con 0x5621dd7c7af0 2024-09-19T06:55:52.278 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc932fde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5621dd6b5290 con 0x5621dd7c8410 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc932fde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5621dd7d5a00 con 0x5621dd7d2cb0 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc932fde740 10 client.0 _ll_register_callbacks cb 0x5621dd6e6140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7cfb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7cfb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50438/0 (socket says 172.21.15.37:50438) 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 -- 172.21.15.37:0/1958603978 learned_addr learned my addr 172.21.15.37:0/1958603978 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92d701640 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92e703640 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 -- 172.21.15.37:0/1958603978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 msgr2=0x5621dd7d05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 -- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 msgr2=0x5621dd7d00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d00a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.271+0000 7fc92df02640 1 -- 172.21.15.37:0/1958603978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621dd43f210 con 0x5621dd7c7af0 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc92df02640 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7cfb60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc9240986e0 tx=0x7fc924009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e85aa0c98a9ead9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:52.279 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc9117fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc9117fa640 1 -- 172.21.15.37:0/1958603978 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9240a0580 con 0x5621dd7c7af0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc9117fa640 1 -- 172.21.15.37:0/1958603978 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fc9240bc0e0 con 0x5621dd7c7af0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc92000fd20 con 0x5621dd7c7af0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc9117fa640 1 -- 172.21.15.37:0/1958603978 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc9240a4e40 con 0x5621dd7c7af0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 10 client.19398 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5621dd50b620 con 0x5621dd7c7af0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc9117fa640 1 -- 172.21.15.37:0/1958603978 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc92000fd20 con 0x5621dd7c7af0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc9117fa640 1 -- 172.21.15.37:0/1958603978 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fc924018c50 con 0x5621dd7c7af0 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 10 client.19398 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 10 client.19398 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 20 client.19398 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd7e2c60 0x5621dd7e5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.280 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 4 client.19398 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.272+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621dd739a20 con 0x5621dd7e2c60 2024-09-19T06:55:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.273+0000 7fc92d701640 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd7e2c60 0x5621dd7e5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.273+0000 7fc9117fa640 10 client.19398 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:52.281 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.273+0000 7fc92d701640 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd7e2c60 0x5621dd7e5050 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299f48688a1915b3 server_cookie=c5e339c0fa3fd043 in_seq=0 out_seq=0 2024-09-19T06:55:52.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:51 smithi096 ceph-mon[35004]: pgmap v446: 97 pgs: 97 active+clean; 92 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 436 op/s 2024-09-19T06:55:52.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:51 smithi096 ceph-mon[35004]: scrub queued for path: / 2024-09-19T06:55:52.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:51 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:52.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:51 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc9117fa640 1 -- 172.21.15.37:0/1958603978 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5621dd739a20 con 0x5621dd7e2c60 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc9117fa640 10 client.19398 handle_command_reply: tid=0 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 client.19398 shutdown 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 msgr2=0x5621dd7cfb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7cfb60 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc9240986e0 tx=0x7fc924009310 comp rx=0 tx=0).stop 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 shutdown_connections 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd7e2c60 0x5621dd7e5050 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc9117fa640 0 client.19398 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 --2- 172.21.15.37:0/1958603978 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7cfb60 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.283 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 >> 172.21.15.37:0/1958603978 conn(0x5621dd7c4060 msgr2=0x5621dd7c9dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 shutdown_connections 2024-09-19T06:55:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 1 -- 172.21.15.37:0/1958603978 wait complete. 2024-09-19T06:55:52.284 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.275+0000 7fc932fde740 20 client.19398 trim_cache size 0 max 16384 2024-09-19T06:55:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.291+0000 7fc932fde740 1 Processor -- start 2024-09-19T06:55:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.291+0000 7fc932fde740 1 -- start start 2024-09-19T06:55:52.295 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc932fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7d1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc932fde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c8410 0x5621dd7d0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc932fde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc932fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281b0200 con 0x5621dd7c7af0 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc932fde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281ada70 con 0x5621dd7d2cb0 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc932fde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281ab8e0 con 0x5621dd7c8410 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7d1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7d1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50446/0 (socket says 172.21.15.37:50446) 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 -- 172.21.15.37:0/2873232470 learned_addr learned my addr 172.21.15.37:0/2873232470 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92e703640 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92d701640 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c8410 0x5621dd7d0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 -- 172.21.15.37:0/2873232470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c8410 msgr2=0x5621dd7d0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c8410 0x5621dd7d0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.296 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 -- 172.21.15.37:0/2873232470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7d2cb0 msgr2=0x5621dd7d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 -- 172.21.15.37:0/2873232470 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621dd7c3e90 con 0x5621dd7c7af0 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92e703640 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc92df02640 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7d1e90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cdc40 tx=0x7fc924018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d35b91a0f43536 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc9137fe640 1 -- 172.21.15.37:0/2873232470 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc92409bbe0 con 0x5621dd7c7af0 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc9137fe640 1 -- 172.21.15.37:0/2873232470 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9240988f0 con 0x5621dd7c7af0 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.292+0000 7fc9137fe640 1 -- 172.21.15.37:0/2873232470 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc924018d90 con 0x5621dd7c7af0 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 -- 172.21.15.37:0/2873232470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 msgr2=0x5621dd7d1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7d1e90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cdc40 tx=0x7fc924018420 comp rx=0 tx=0).stop 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 -- 172.21.15.37:0/2873232470 shutdown_connections 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c8410 0x5621dd7d0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.297 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 --2- 172.21.15.37:0/2873232470 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7c7af0 0x5621dd7d1e90 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 -- 172.21.15.37:0/2873232470 >> 172.21.15.37:0/2873232470 conn(0x5621dd7c4060 msgr2=0x5621dd7c31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 -- 172.21.15.37:0/2873232470 shutdown_connections 2024-09-19T06:55:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.293+0000 7fc932fde740 1 -- 172.21.15.37:0/2873232470 wait complete. 2024-09-19T06:55:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 -- start start 2024-09-19T06:55:52.298 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 Processor -- start 2024-09-19T06:55:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c7af0 0x5621dd7db350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.300 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281abf30 con 0x5621dd7d2cb0 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281ad3a0 con 0x5621dd7c8410 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc9281ac600 con 0x5621dd7c7af0 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc932fde740 10 client.0 _ll_register_callbacks cb 0x7fc92400cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc92e703640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc92df02640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c7af0 0x5621dd7db350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc92e703640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50456/0 (socket says 172.21.15.37:50456) 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc92e703640 1 -- 172.21.15.37:0/1984108678 learned_addr learned my addr 172.21.15.37:0/1984108678 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.294+0000 7fc92d701640 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc92e703640 1 -- 172.21.15.37:0/1984108678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c7af0 msgr2=0x5621dd7db350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc92e703640 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c7af0 0x5621dd7db350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc92e703640 1 -- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 msgr2=0x5621dd7d6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:52.301 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc92e703640 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc92e703640 1 -- 172.21.15.37:0/1984108678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621dd7d73c0 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc92e703640 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d6bb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc920011560 tx=0x7fc92001bb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7292a8f22ada69b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc9117fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc9117fa640 1 -- 172.21.15.37:0/1984108678 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc920037060 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc9117fa640 1 -- 172.21.15.37:0/1984108678 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fc9200049e0 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9240988f0 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc9117fa640 1 -- 172.21.15.37:0/1984108678 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fc920009680 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc932fde740 10 client.19404 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5621dd84d890 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.295+0000 7fc9117fa640 1 -- 172.21.15.37:0/1984108678 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc9240988f0 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc9117fa640 1 -- 172.21.15.37:0/1984108678 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fc92001a070 con 0x5621dd7d2cb0 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc932fde740 10 client.19404 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc932fde740 10 client.19404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc932fde740 20 client.19404 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc932fde740 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd84df50 0x5621dd850370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc932fde740 4 client.19404 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:55:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5621dd733d70 con 0x5621dd84df50 2024-09-19T06:55:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc92df02640 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd84df50 0x5621dd850370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc92df02640 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd84df50 0x5621dd850370 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68ea11109a661421 server_cookie=a7d6240b185b48e in_seq=0 out_seq=0 2024-09-19T06:55:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.296+0000 7fc9117fa640 10 client.19404 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:51 smithi151 ceph-mon[35236]: pgmap v446: 97 pgs: 97 active+clean; 92 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.9 MiB/s wr, 436 op/s 2024-09-19T06:55:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:51 smithi151 ceph-mon[35236]: scrub queued for path: / 2024-09-19T06:55:52.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:51 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:52.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:51 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:52.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 8745 33.12 MB/sec warmup 19 sec latency 171.166 ms 2024-09-19T06:55:52.678 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.748+0000 7fc9117fa640 1 -- 172.21.15.37:0/1984108678 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5621dd733d70 con 0x5621dd84df50 2024-09-19T06:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.748+0000 7fc9117fa640 10 client.19404 handle_command_reply: tid=0 2024-09-19T06:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.748+0000 7fc932fde740 1 client.19404 shutdown 2024-09-19T06:55:52.750 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.748+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7d2cb0 msgr2=0x5621dd7d6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.748+0000 7fc932fde740 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d6bb0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fc920011560 tx=0x7fc92001bb50 comp rx=0 tx=0).stop 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 shutdown_connections 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5621dd84df50 0x5621dd850370 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5621dd7d2cb0 0x5621dd7d6bb0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5621dd7c8410 0x5621dd7d6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 --2- 172.21.15.37:0/1984108678 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5621dd7c7af0 0x5621dd7db350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 >> 172.21.15.37:0/1984108678 conn(0x5621dd7c4060 msgr2=0x5621dd7ddb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc9117fa640 0 client.19404 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 shutdown_connections 2024-09-19T06:55:52.751 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 1 -- 172.21.15.37:0/1984108678 wait complete. 2024-09-19T06:55:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.749+0000 7fc932fde740 20 client.19404 trim_cache size 0 max 16384 2024-09-19T06:55:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:55:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:55:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:55:52.752 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc9140a34e0 msgr2=0x7fc9140a59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc9140a34e0 0x7fc9140a59a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fc920022000 tx=0x7fc920023000 comp rx=0 tx=0).stop 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 msgr2=0x7fc928199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928199200 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fc92400e9f0 tx=0x7fc92400eeb0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 shutdown_connections 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc928106a00 0x7fc9281a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc928101390 0x7fc928199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc9140a34e0 0x7fc9140a59a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.753 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 --2- 172.21.15.37:0/3279554815 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc9281009f0 0x7fc928199200 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 >> 172.21.15.37:0/3279554815 conn(0x7fc9280fc6b0 msgr2=0x7fc9280fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 shutdown_connections 2024-09-19T06:55:52.754 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:55:52.750+0000 7fc930166640 1 -- 172.21.15.37:0/3279554815 wait complete. 2024-09-19T06:55:52.996 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: active paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:52 smithi096 ceph-mon[35004]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: active paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.276 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:52 smithi037 ceph-mon[31211]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:53.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: active paths [/] 2024-09-19T06:55:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.364 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:52 smithi151 ceph-mon[35236]: scrub summary: idle+waiting paths [/] 2024-09-19T06:55:53.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 9255 32.58 MB/sec warmup 20 sec latency 284.687 ms 2024-09-19T06:55:53.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 -- 172.21.15.37:0/1289706607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c100e20 msgr2=0x7fb77c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1289706607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c100e20 0x7fb77c101200 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009900 tx=0x7fb76c02a480 comp rx=0 tx=0).stop 2024-09-19T06:55:53.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 -- 172.21.15.37:0/1289706607 shutdown_connections 2024-09-19T06:55:53.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1289706607 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb77c102ef0 0x7fb77c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.504 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1289706607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 0x7fb77c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1289706607 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c100e20 0x7fb77c101200 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 -- 172.21.15.37:0/1289706607 >> 172.21.15.37:0/1289706607 conn(0x7fb77c0fc7d0 msgr2=0x7fb77c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 -- 172.21.15.37:0/1289706607 shutdown_connections 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.492+0000 7fb7840a2640 1 -- 172.21.15.37:0/1289706607 wait complete. 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 Processor -- start 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 -- start start 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb77c100e20 0x7fb77c1973c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 0x7fb77c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c102ef0 0x7fb77c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c110f80 con 0x7fb77c102ef0 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c110e00 con 0x7fb77c102570 2024-09-19T06:55:53.505 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb7840a2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c111100 con 0x7fb77c100e20 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb78163d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 0x7fb77c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb78263f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c102ef0 0x7fb77c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb78263f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c102ef0 0x7fb77c19bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50470/0 (socket says 172.21.15.37:50470) 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb78163d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 0x7fb77c197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48290/0 (socket says 172.21.15.37:48290) 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb78263f640 1 -- 172.21.15.37:0/1735354601 learned_addr learned my addr 172.21.15.37:0/1735354601 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.493+0000 7fb781e3e640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb77c100e20 0x7fb77c1973c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb78263f640 1 -- 172.21.15.37:0/1735354601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb77c100e20 msgr2=0x7fb77c1973c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb78263f640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb77c100e20 0x7fb77c1973c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb78263f640 1 -- 172.21.15.37:0/1735354601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 msgr2=0x7fb77c197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb78263f640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 0x7fb77c197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb78263f640 1 -- 172.21.15.37:0/1735354601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb77c19c370 con 0x7fb77c102ef0 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb78163d640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 0x7fb77c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb78263f640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c102ef0 0x7fb77c19bc90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb77400eaa0 tx=0x7fb77400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:53.506 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb772ffd640 1 -- 172.21.15.37:0/1735354601 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb77400cdb0 con 0x7fb77c102ef0 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb772ffd640 1 -- 172.21.15.37:0/1735354601 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb774005020 con 0x7fb77c102ef0 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb77c19c660 con 0x7fb77c102ef0 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb772ffd640 1 -- 172.21.15.37:0/1735354601 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740116b0 con 0x7fb77c102ef0 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.494+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb77c1a3f40 con 0x7fb77c102ef0 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.496+0000 7fb772ffd640 1 -- 172.21.15.37:0/1735354601 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb774005560 con 0x7fb77c102ef0 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 Processor -- start 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 -- start start 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3ea570 0x561ccd3ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb772ffd640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7680a33b0 0x7fb7680a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd3f7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3fa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561ccd34e2e0 con 0x561ccd3f7c80 2024-09-19T06:55:53.507 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ccd2f5ce0 con 0x561ccd3ea570 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.497+0000 7fb78647b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561ccd34a2a0 con 0x561ccd3eae90 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb781e3e640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7680a33b0 0x7fb7680a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3ea570 0x561ccd3ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3ea570 0x561ccd3ea950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48300/0 (socket says 172.21.15.37:48300) 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 -- 172.21.15.37:0/406710770 learned_addr learned my addr 172.21.15.37:0/406710770 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78263f640 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3fa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb772ffd640 1 -- 172.21.15.37:0/1735354601 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fb7740cff10 con 0x7fb77c102ef0 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd3f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 -- 172.21.15.37:0/406710770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 msgr2=0x561ccd3f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd3f7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 -- 172.21.15.37:0/406710770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 msgr2=0x561ccd3fa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3fa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.508 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 -- 172.21.15.37:0/406710770 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccd2da440 con 0x561ccd3ea570 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb781e3e640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7680a33b0 0x7fb7680a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009230 tx=0x7fb76c0058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb78163d640 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3ea570 0x561ccd3ea950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb778013990 tx=0x7fb778009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab9300ffd204072f server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.498+0000 7fb767fff640 1 -- 172.21.15.37:0/406710770 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778004030 con 0x561ccd3ea570 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb767fff640 1 -- 172.21.15.37:0/406710770 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77800fe20 con 0x561ccd3ea570 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb767fff640 1 -- 172.21.15.37:0/406710770 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb778003c20 con 0x561ccd3ea570 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 -- 172.21.15.37:0/406710770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3ea570 msgr2=0x561ccd3ea950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3ea570 0x561ccd3ea950 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb778013990 tx=0x7fb778009c50 comp rx=0 tx=0).stop 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 -- 172.21.15.37:0/406710770 shutdown_connections 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3fa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd3f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 --2- 172.21.15.37:0/406710770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3ea570 0x561ccd3ea950 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 -- 172.21.15.37:0/406710770 >> 172.21.15.37:0/406710770 conn(0x561ccd3e6c40 msgr2=0x561ccd3e7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.499+0000 7fb78647b740 1 -- 172.21.15.37:0/406710770 shutdown_connections 2024-09-19T06:55:53.509 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 -- 172.21.15.37:0/406710770 wait complete. 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 -- start start 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 Processor -- start 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3ea570 0x561ccd3f7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd3f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3f4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x561ccd3fa670 con 0x561ccd3f7c80 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561ccd309050 con 0x561ccd3eae90 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x561ccd3fa7f0 con 0x561ccd3ea570 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.500+0000 7fb78647b740 10 client.0 _ll_register_callbacks cb 0x561ccd349160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3f4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78163d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd3f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.510 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3f4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50492/0 (socket says 172.21.15.37:50492) 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78163d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd3f48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.37:48314/0 (socket says 172.21.15.37:48314) 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 -- 172.21.15.37:0/2398315754 learned_addr learned my addr 172.21.15.37:0/2398315754 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb781e3e640 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3ea570 0x561ccd3f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 -- 172.21.15.37:0/2398315754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3ea570 msgr2=0x561ccd3f7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3ea570 0x561ccd3f7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 -- 172.21.15.37:0/2398315754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 msgr2=0x561ccd3f48f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd3f48f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 -- 172.21.15.37:0/2398315754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccd0bf3c0 con 0x561ccd3f7c80 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78263f640 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3f4ee0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb774097f50 tx=0x7fb77401f3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c444e3b4fc72cb22 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb7657fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb7657fa640 1 -- 172.21.15.37:0/2398315754 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7740aead0 con 0x561ccd3f7c80 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb7657fa640 1 -- 172.21.15.37:0/2398315754 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fb7740051d0 con 0x561ccd3f7c80 2024-09-19T06:55:53.511 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb77800fe20 con 0x561ccd3f7c80 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb7657fa640 1 -- 172.21.15.37:0/2398315754 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb77409c270 con 0x561ccd3f7c80 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.501+0000 7fb78647b740 10 client.19416 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561ccd4060d0 con 0x561ccd3f7c80 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb7657fa640 1 -- 172.21.15.37:0/2398315754 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb77800fe20 con 0x561ccd3f7c80 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb7657fa640 1 -- 172.21.15.37:0/2398315754 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fb77400f040 con 0x561ccd3f7c80 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb78647b740 10 client.19416 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb78647b740 10 client.19416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb78647b740 20 client.19416 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb78647b740 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd407530 0x561ccd409970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb78647b740 4 client.19416 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561ccd35b850 con 0x561ccd407530 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.502+0000 7fb781e3e640 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd407530 0x561ccd409970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.512 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.503+0000 7fb781e3e640 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd407530 0x561ccd409970 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d227b49c90e9b2cb server_cookie=4cf00b3b93240737 in_seq=0 out_seq=0 2024-09-19T06:55:53.513 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.503+0000 7fb7657fa640 10 client.19416 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.504+0000 7fb7657fa640 1 -- 172.21.15.37:0/2398315754 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561ccd35b850 con 0x561ccd407530 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.504+0000 7fb7657fa640 10 client.19416 handle_command_reply: tid=0 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.504+0000 7fb78647b740 1 client.19416 shutdown 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 msgr2=0x561ccd3f4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3f4ee0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb774097f50 tx=0x7fb77401f3f0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 shutdown_connections 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd407530 0x561ccd409970 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3f7c80 0x561ccd3f4ee0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd3f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 --2- 172.21.15.37:0/2398315754 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3ea570 0x561ccd3f7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 >> 172.21.15.37:0/2398315754 conn(0x561ccd3e6c40 msgr2=0x561ccd3ec9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:53.514 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb7657fa640 0 client.19416 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 shutdown_connections 2024-09-19T06:55:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 1 -- 172.21.15.37:0/2398315754 wait complete. 2024-09-19T06:55:53.515 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.505+0000 7fb78647b740 20 client.19416 trim_cache size 0 max 16384 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.522+0000 7fb78647b740 1 Processor -- start 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.522+0000 7fb78647b740 1 -- start start 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd3e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78647b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd403910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78647b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3f7c80 0x561ccd409540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1ae010 con 0x561ccd3ea570 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78647b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1ab880 con 0x561ccd3f7c80 2024-09-19T06:55:53.525 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78647b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1a96f0 con 0x561ccd3eae90 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd3e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd3e3a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50498/0 (socket says 172.21.15.37:50498) 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 -- 172.21.15.37:0/1583461670 learned_addr learned my addr 172.21.15.37:0/1583461670 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78263f640 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3f7c80 0x561ccd409540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb78163d640 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd403910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 -- 172.21.15.37:0/1583461670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 msgr2=0x561ccd403910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd403910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 -- 172.21.15.37:0/1583461670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3f7c80 msgr2=0x561ccd409540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3f7c80 0x561ccd409540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 -- 172.21.15.37:0/1583461670 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ccd3f79b0 con 0x561ccd3ea570 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb781e3e640 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd3e3a60 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02a9a0 tx=0x7fb76c03e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a395bf98ea9ca0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb7677fe640 1 -- 172.21.15.37:0/1583461670 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c035050 con 0x561ccd3ea570 2024-09-19T06:55:53.526 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb7677fe640 1 -- 172.21.15.37:0/1583461670 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c058070 con 0x561ccd3ea570 2024-09-19T06:55:53.527 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.523+0000 7fb7677fe640 1 -- 172.21.15.37:0/1583461670 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c053860 con 0x561ccd3ea570 2024-09-19T06:55:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 -- 172.21.15.37:0/1583461670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 msgr2=0x561ccd3e3a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd3e3a60 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb76c02a9a0 tx=0x7fb76c03e000 comp rx=0 tx=0).stop 2024-09-19T06:55:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 -- 172.21.15.37:0/1583461670 shutdown_connections 2024-09-19T06:55:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3f7c80 0x561ccd409540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3eae90 0x561ccd403910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.529 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 --2- 172.21.15.37:0/1583461670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd3e3a60 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 -- 172.21.15.37:0/1583461670 >> 172.21.15.37:0/1583461670 conn(0x561ccd3e6c40 msgr2=0x561ccd3e3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 -- 172.21.15.37:0/1583461670 shutdown_connections 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.524+0000 7fb78647b740 1 -- 172.21.15.37:0/1583461670 wait complete. 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 -- start start 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 Processor -- start 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd4000b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd40ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3f7c80 0x561ccd40efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1a9d40 con 0x561ccd3ea570 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1ab1b0 con 0x561ccd3eae90 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb77c1aa410 con 0x561ccd3f7c80 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.525+0000 7fb78647b740 10 client.0 _ll_register_callbacks cb 0x7fb7740985a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:55:53.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd4000b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd4000b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:50502/0 (socket says 172.21.15.37:50502) 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 -- 172.21.15.37:0/3266250033 learned_addr learned my addr 172.21.15.37:0/3266250033 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb78263f640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3f7c80 0x561ccd40efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb78163d640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd40ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 -- 172.21.15.37:0/3266250033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3f7c80 msgr2=0x561ccd40efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3f7c80 0x561ccd40efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 -- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 msgr2=0x561ccd40ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd40ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 -- 172.21.15.37:0/3266250033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ccd40f700 con 0x561ccd3ea570 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb78163d640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd40ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb78263f640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3f7c80 0x561ccd40efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb781e3e640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd4000b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb76c003ef0 tx=0x7fb76c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3e74643b9d18f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:55:53.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb7657fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb7657fa640 1 -- 172.21.15.37:0/3266250033 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb76c035040 con 0x561ccd3ea570 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb7657fa640 1 -- 172.21.15.37:0/3266250033 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8571+0+0 (secure 0 0 0) 0x7fb76c053ab0 con 0x561ccd3ea570 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.526+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb76c058070 con 0x561ccd3ea570 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb7657fa640 1 -- 172.21.15.37:0/3266250033 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fb76c06c050 con 0x561ccd3ea570 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 10 client.19422 fetch_fsmap learned FSMap version 47 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x561ccd472150 con 0x561ccd3ea570 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb7657fa640 1 -- 172.21.15.37:0/3266250033 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb76c058070 con 0x561ccd3ea570 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb7657fa640 1 -- 172.21.15.37:0/3266250033 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fb76c0675f0 con 0x561ccd3ea570 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 10 client.19422 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 10 client.19422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:55:53.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 20 client.19422 populate_metadata read hostname 'smithi037' 2024-09-19T06:55:53.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd4728c0 0x561ccd474ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:55:53.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 4 client.19422 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:55:53.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561ccd355d60 con 0x561ccd4728c0 2024-09-19T06:55:53.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.527+0000 7fb78163d640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd4728c0 0x561ccd474ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.528+0000 7fb78163d640 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd4728c0 0x561ccd474ce0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555f8b7d1200a74d server_cookie=521fc8a280dc814f in_seq=0 out_seq=0 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.528+0000 7fb7657fa640 10 client.19422 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb7657fa640 1 -- 172.21.15.37:0/3266250033 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561ccd355d60 con 0x561ccd4728c0 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb7657fa640 10 client.19422 handle_command_reply: tid=0 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 client.19422 shutdown 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 msgr2=0x561ccd4000b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd4000b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fb76c003ef0 tx=0x7fb76c009260 comp rx=0 tx=0).stop 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 shutdown_connections 2024-09-19T06:55:53.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x561ccd4728c0 0x561ccd474ce0 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x561ccd3f7c80 0x561ccd40efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb7657fa640 0 client.19422 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561ccd3eae90 0x561ccd40ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 --2- 172.21.15.37:0/3266250033 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x561ccd3ea570 0x561ccd4000b0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 >> 172.21.15.37:0/3266250033 conn(0x561ccd3e6c40 msgr2=0x561ccd3f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 shutdown_connections 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.530+0000 7fb78647b740 1 -- 172.21.15.37:0/3266250033 wait complete. 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb78647b740 20 client.19422 trim_cache size 0 max 16384 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:55:53.535 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": { 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout: "5e8ef1dc-4f46-4049-93ee-de66b7381fb0": { 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout: "path": "/", 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tag": "5e8ef1dc-4f46-4049-93ee-de66b7381fb0", 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout: "options": "recursive,force" 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:55:53.536 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7680a33b0 msgr2=0x7fb7680a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7680a33b0 0x7fb7680a5870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb76c009230 tx=0x7fb76c0058d0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c102ef0 msgr2=0x7fb77c19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c102ef0 0x7fb77c19bc90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7fb77400eaa0 tx=0x7fb77400ef60 comp rx=0 tx=0).stop 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 shutdown_connections 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb7680a33b0 0x7fb7680a5870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb77c102ef0 0x7fb77c19bc90 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb77c102570 0x7fb77c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 --2- 172.21.15.37:0/1735354601 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb77c100e20 0x7fb77c1973c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:55:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 >> 172.21.15.37:0/1735354601 conn(0x7fb77c0fc7d0 msgr2=0x7fb77c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:55:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 shutdown_connections 2024-09-19T06:55:53.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:55:53.531+0000 7fb7840a2640 1 -- 172.21.15.37:0/1735354601 wait complete. 2024-09-19T06:55:54.208 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:53 smithi037 ceph-mon[31211]: pgmap v447: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 775 op/s 2024-09-19T06:55:54.241 INFO:tasks.cephfs.filesystem:scrub status for tag:5e8ef1dc-4f46-4049-93ee-de66b7381fb0 - {'path': '/', 'tag': '5e8ef1dc-4f46-4049-93ee-de66b7381fb0', 'options': 'recursive,force'} 2024-09-19T06:55:54.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:53 smithi096 ceph-mon[35004]: pgmap v447: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 775 op/s 2024-09-19T06:55:54.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:53 smithi151 ceph-mon[35236]: pgmap v447: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.1 MiB/s wr, 775 op/s 2024-09-19T06:55:54.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 10008 33.97 MB/sec warmup 21 sec latency 74.283 ms 2024-09-19T06:55:55.486 INFO:tasks.workunit.client.0.smithi037.stdout: 10 10476 34.34 MB/sec warmup 22 sec latency 50.190 ms 2024-09-19T06:55:56.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:55 smithi096 ceph-mon[35004]: pgmap v448: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 454 op/s 2024-09-19T06:55:56.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:55 smithi037 ceph-mon[31211]: pgmap v448: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 454 op/s 2024-09-19T06:55:56.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:55 smithi151 ceph-mon[35236]: pgmap v448: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 454 op/s 2024-09-19T06:55:56.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 10645 33.14 MB/sec warmup 23 sec latency 210.399 ms 2024-09-19T06:55:57.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 10832 31.79 MB/sec warmup 24 sec latency 222.006 ms 2024-09-19T06:55:58.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:57 smithi096 ceph-mon[35004]: pgmap v449: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 454 op/s 2024-09-19T06:55:58.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:57 smithi037 ceph-mon[31211]: pgmap v449: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 454 op/s 2024-09-19T06:55:58.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:57 smithi151 ceph-mon[35236]: pgmap v449: 97 pgs: 97 active+clean; 94 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.4 MiB/s wr, 454 op/s 2024-09-19T06:55:58.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 11101 30.77 MB/sec warmup 25 sec latency 295.469 ms 2024-09-19T06:55:59.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 11553 30.69 MB/sec warmup 26 sec latency 42.578 ms 2024-09-19T06:56:00.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:55:59 smithi096 ceph-mon[35004]: pgmap v450: 97 pgs: 97 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 537 op/s 2024-09-19T06:56:00.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:55:59 smithi037 ceph-mon[31211]: pgmap v450: 97 pgs: 97 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 537 op/s 2024-09-19T06:56:00.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:55:59 smithi151 ceph-mon[35236]: pgmap v450: 97 pgs: 97 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 3.3 MiB/s wr, 537 op/s 2024-09-19T06:56:00.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 12326 30.33 MB/sec warmup 27 sec latency 118.088 ms 2024-09-19T06:56:01.485 INFO:tasks.workunit.client.0.smithi037.stdout: 10 12726 30.18 MB/sec warmup 28 sec latency 99.375 ms 2024-09-19T06:56:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:01 smithi096 ceph-mon[35004]: pgmap v451: 97 pgs: 97 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 421 op/s 2024-09-19T06:56:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:01 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:56:02.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:01 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:56:02.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:01 smithi037 ceph-mon[31211]: pgmap v451: 97 pgs: 97 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 421 op/s 2024-09-19T06:56:02.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:01 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:56:02.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:01 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:56:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:01 smithi151 ceph-mon[35236]: pgmap v451: 97 pgs: 97 active+clean; 90 MiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.1 MiB/s wr, 421 op/s 2024-09-19T06:56:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:01 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:56:02.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:01 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:56:02.486 INFO:tasks.workunit.client.0.smithi037.stdout: 10 13082 30.05 MB/sec warmup 29 sec latency 375.913 ms 2024-09-19T06:56:03.154 INFO:tasks.workunit.client.0.smithi037.stdout:[13885] write failed on handle 12771 (Resource temporarily unavailable) 2024-09-19T06:56:03.154 INFO:tasks.workunit.client.0.smithi037.stdout:Child failed with status 1 2024-09-19T06:56:03.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T06:56:03.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:02 smithi096 ceph-mon[35004]: scrub summary: idle 2024-09-19T06:56:03.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:02 smithi096 ceph-mon[35004]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T06:56:03.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:02 smithi037 ceph-mon[31211]: scrub summary: idle 2024-09-19T06:56:03.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:02 smithi037 ceph-mon[31211]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T06:56:03.312 INFO:tasks.workunit:Stopping ['suites/dbench.sh'] on client.0... 2024-09-19T06:56:03.312 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-19T06:56:03.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:02 smithi151 ceph-mon[35236]: scrub summary: idle 2024-09-19T06:56:03.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:02 smithi151 ceph-mon[35236]: osdmap e106: 12 total, 12 up, 12 in 2024-09-19T06:56:03.847 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-19T06:56:05.982 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=87213e1fdee9413aa0945c20cfb19ca6 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 293, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/workunit.py", line 434, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh' 2024-09-19T06:56:05.985 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-19T06:56:06.592 INFO:tasks.quiescer:joining Quiescers 2024-09-19T06:56:06.593 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-19T06:56:06.593 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:56:06.600 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:03 smithi037 ceph-mon[31211]: pgmap v453: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 110 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 437 op/s 2024-09-19T06:56:06.600 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:03 smithi151 ceph-mon[35236]: pgmap v453: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 110 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 437 op/s 2024-09-19T06:56:06.600 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:06 smithi151 ceph-mon[35236]: pgmap v454: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 111 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 454 op/s 2024-09-19T06:56:06.600 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:03 smithi096 ceph-mon[35004]: pgmap v453: 97 pgs: 21 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 38 active+clean; 110 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.4 MiB/s wr, 437 op/s 2024-09-19T06:56:06.601 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:06 smithi096 ceph-mon[35004]: pgmap v454: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 111 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 454 op/s 2024-09-19T06:56:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 -- 172.21.15.37:0/2891165845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 msgr2=0x7f9584100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 --2- 172.21.15.37:0/2891165845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584100e30 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f9578009930 tx=0x7f957802a420 comp rx=0 tx=0).stop 2024-09-19T06:56:06.624 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 -- 172.21.15.37:0/2891165845 shutdown_connections 2024-09-19T06:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 --2- 172.21.15.37:0/2891165845 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9584101370 0x7f958410d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 --2- 172.21.15.37:0/2891165845 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584100e30 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 --2- 172.21.15.37:0/2891165845 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f95841069e0 0x7f9584106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 -- 172.21.15.37:0/2891165845 >> 172.21.15.37:0/2891165845 conn(0x7f95840fc6b0 msgr2=0x7f95840fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 -- 172.21.15.37:0/2891165845 shutdown_connections 2024-09-19T06:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.857+0000 7f958a74b640 1 -- 172.21.15.37:0/2891165845 wait complete. 2024-09-19T06:56:06.625 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.858+0000 7f958a74b640 1 Processor -- start 2024-09-19T06:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.858+0000 7f958a74b640 1 -- start start 2024-09-19T06:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f958a74b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f958a74b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9584101370 0x7f9584199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f958a74b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95841069e0 0x7f95841a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f958a74b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f9584110de0 con 0x7f95841009d0 2024-09-19T06:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f958a74b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9584110c60 con 0x7f95841069e0 2024-09-19T06:56:06.626 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f958a74b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f9584110f60 con 0x7f9584101370 2024-09-19T06:56:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42766/0 (socket says 172.21.15.37:42766) 2024-09-19T06:56:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 -- 172.21.15.37:0/156690516 learned_addr learned my addr 172.21.15.37:0/156690516 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f95837fe640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9584101370 0x7f9584199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9588ce8640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95841069e0 0x7f95841a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 -- 172.21.15.37:0/156690516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9584101370 msgr2=0x7f9584199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.627 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9584101370 0x7f9584199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 -- 172.21.15.37:0/156690516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95841069e0 msgr2=0x7f95841a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95841069e0 0x7f95841a4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 -- 172.21.15.37:0/156690516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95841a47d0 con 0x7f95841009d0 2024-09-19T06:56:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9588ce8640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95841069e0 0x7f95841a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f9583fff640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584199210 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f957400d950 tx=0x7f957400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.628 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.859+0000 7f95817fa640 1 -- 172.21.15.37:0/156690516 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9574017070 con 0x7f95841009d0 2024-09-19T06:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.860+0000 7f95817fa640 1 -- 172.21.15.37:0/156690516 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9574009c30 con 0x7f95841009d0 2024-09-19T06:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.860+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95841a49a0 con 0x7f95841009d0 2024-09-19T06:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.860+0000 7f95817fa640 1 -- 172.21.15.37:0/156690516 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957400f6b0 con 0x7f95841009d0 2024-09-19T06:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.860+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95841a4ec0 con 0x7f95841009d0 2024-09-19T06:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.862+0000 7f95817fa640 1 -- 172.21.15.37:0/156690516 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f95740109d0 con 0x7f95841009d0 2024-09-19T06:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.862+0000 7f958d5c5740 1 Processor -- start 2024-09-19T06:56:06.629 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.862+0000 7f958d5c5740 1 -- start start 2024-09-19T06:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.862+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa927d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.862+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa936b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.862+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa9394b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f958d5c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c0aa88c2f0 con 0x55c0aa9282d0 2024-09-19T06:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f958d5c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c0aa834270 con 0x55c0aa9370c0 2024-09-19T06:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f958d5c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0aa8882b0 con 0x55c0aa9279b0 2024-09-19T06:56:06.630 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa936b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa936b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42770/0 (socket says 172.21.15.37:42770) 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 -- 172.21.15.37:0/3658449016 learned_addr learned my addr 172.21.15.37:0/3658449016 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9588ce8640 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa927d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f95817fa640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f956c0a3460 0x7f956c0a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f95837fe640 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa9394b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 -- 172.21.15.37:0/3658449016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 msgr2=0x55c0aa927d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa927d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.631 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 -- 172.21.15.37:0/3658449016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 msgr2=0x55c0aa9394b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa9394b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 -- 172.21.15.37:0/3658449016 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0aa833340 con 0x55c0aa9282d0 2024-09-19T06:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f95837fe640 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa9394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9588ce8640 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa927d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9588ce8640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f956c0a3460 0x7f956c0a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f95817fa640 1 -- 172.21.15.37:0/156690516 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8615+0+0 (secure 0 0 0) 0x7f95740cf9d0 con 0x7f95841009d0 2024-09-19T06:56:06.632 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.863+0000 7f9583fff640 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa936b80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f95740cd030 tx=0x7f9574066590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758630dc23cc7463 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f956a7fc640 1 -- 172.21.15.37:0/3658449016 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957409b790 con 0x55c0aa9282d0 2024-09-19T06:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f9588ce8640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f956c0a3460 0x7f956c0a5920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f957c005fd0 tx=0x7f957c004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f956a7fc640 1 -- 172.21.15.37:0/3658449016 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9574066aa0 con 0x55c0aa9282d0 2024-09-19T06:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f956a7fc640 1 -- 172.21.15.37:0/3658449016 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957409b5b0 con 0x55c0aa9282d0 2024-09-19T06:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f958d5c5740 1 -- 172.21.15.37:0/3658449016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 msgr2=0x55c0aa936b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.633 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa936b80 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f95740cd030 tx=0x7f9574066590 comp rx=0 tx=0).stop 2024-09-19T06:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f958d5c5740 1 -- 172.21.15.37:0/3658449016 shutdown_connections 2024-09-19T06:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa9394b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa936b80 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3658449016 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa927d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.864+0000 7f958d5c5740 1 -- 172.21.15.37:0/3658449016 >> 172.21.15.37:0/3658449016 conn(0x55c0aa924060 msgr2=0x55c0aa924440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 -- 172.21.15.37:0/3658449016 shutdown_connections 2024-09-19T06:56:06.634 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 -- 172.21.15.37:0/3658449016 wait complete. 2024-09-19T06:56:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 -- start start 2024-09-19T06:56:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 Processor -- start 2024-09-19T06:56:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa934000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9282d0 0x55c0aa934540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9370c0 0x55c0aa934a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.635 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55c0aa939bd0 con 0x55c0aa9370c0 2024-09-19T06:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c0aa815290 con 0x55c0aa9282d0 2024-09-19T06:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.865+0000 7f958d5c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55c0aa939d50 con 0x55c0aa9279b0 2024-09-19T06:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f958d5c5740 10 client.0 _ll_register_callbacks cb 0x55c0aa846140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9370c0 0x55c0aa934a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9370c0 0x55c0aa934a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42784/0 (socket says 172.21.15.37:42784) 2024-09-19T06:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 -- 172.21.15.37:0/4155703706 learned_addr learned my addr 172.21.15.37:0/4155703706 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:06.636 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f95837fe640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9282d0 0x55c0aa934540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9583fff640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa934000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 -- 172.21.15.37:0/4155703706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 msgr2=0x55c0aa934000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa934000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 -- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9282d0 msgr2=0x55c0aa934540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9282d0 0x55c0aa934540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 -- 172.21.15.37:0/4155703706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0aa59f210 con 0x55c0aa9370c0 2024-09-19T06:56:06.637 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f95837fe640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9282d0 0x55c0aa934540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f9588ce8640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9370c0 0x55c0aa934a80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f957c012010 tx=0x7f957c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c36a1921c9d749 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f95637fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f95637fe640 1 -- 172.21.15.37:0/4155703706 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f957c022070 con 0x55c0aa9370c0 2024-09-19T06:56:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f95637fe640 1 -- 172.21.15.37:0/4155703706 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8615+0+0 (secure 0 0 0) 0x7f957c010d30 con 0x55c0aa9370c0 2024-09-19T06:56:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.866+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9574066aa0 con 0x55c0aa9370c0 2024-09-19T06:56:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f95637fe640 1 -- 172.21.15.37:0/4155703706 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f957c02e050 con 0x55c0aa9370c0 2024-09-19T06:56:06.638 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 10 client.19434 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c0aa66b620 con 0x55c0aa9370c0 2024-09-19T06:56:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f95637fe640 1 -- 172.21.15.37:0/4155703706 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9574066aa0 con 0x55c0aa9370c0 2024-09-19T06:56:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f95637fe640 1 -- 172.21.15.37:0/4155703706 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f957c01d5e0 con 0x55c0aa9370c0 2024-09-19T06:56:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 10 client.19434 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 10 client.19434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 20 client.19434 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:06.639 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa946fc0 0x55c0aa9493f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 4 client.19434 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c0aa899a20 con 0x55c0aa946fc0 2024-09-19T06:56:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.867+0000 7f9583fff640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa946fc0 0x55c0aa9493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.868+0000 7f95637fe640 10 client.19434 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.868+0000 7f9583fff640 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa946fc0 0x55c0aa9493f0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b664cabe4cfe421 server_cookie=5364c840ad9c4048 in_seq=0 out_seq=0 2024-09-19T06:56:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.868+0000 7f95637fe640 1 -- 172.21.15.37:0/4155703706 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55c0aa899a20 con 0x55c0aa946fc0 2024-09-19T06:56:06.640 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.868+0000 7f95637fe640 10 client.19434 handle_command_reply: tid=0 2024-09-19T06:56:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.868+0000 7f958d5c5740 1 client.19434 shutdown 2024-09-19T06:56:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9370c0 msgr2=0x55c0aa934a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9370c0 0x55c0aa934a80 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f957c012010 tx=0x7f957c004ab0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 shutdown_connections 2024-09-19T06:56:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa946fc0 0x55c0aa9493f0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9370c0 0x55c0aa934a80 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.641 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9282d0 0x55c0aa934540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f95637fe640 0 client.19434 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 --2- 172.21.15.37:0/4155703706 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa934000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 >> 172.21.15.37:0/4155703706 conn(0x55c0aa924060 msgr2=0x55c0aa929dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 shutdown_connections 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 1 -- 172.21.15.37:0/4155703706 wait complete. 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.869+0000 7f958d5c5740 20 client.19434 trim_cache size 0 max 16384 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.883+0000 7f958d5c5740 1 Processor -- start 2024-09-19T06:56:06.642 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.883+0000 7f958d5c5740 1 -- start start 2024-09-19T06:56:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa94ed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa94f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa931240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f958d5c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95841ae0f0 con 0x55c0aa9282d0 2024-09-19T06:56:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f958d5c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95841ab960 con 0x55c0aa9370c0 2024-09-19T06:56:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f958d5c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95841a97d0 con 0x55c0aa9279b0 2024-09-19T06:56:06.643 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa94f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa94f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42794/0 (socket says 172.21.15.37:42794) 2024-09-19T06:56:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 -- 172.21.15.37:0/3119021729 learned_addr learned my addr 172.21.15.37:0/3119021729 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f9583fff640 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa94ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f9588ce8640 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa931240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 -- 172.21.15.37:0/3119021729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 msgr2=0x55c0aa94ed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa94ed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.644 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 -- 172.21.15.37:0/3119021729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 msgr2=0x55c0aa931240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa931240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 -- 172.21.15.37:0/3119021729 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c0aa923e90 con 0x55c0aa9282d0 2024-09-19T06:56:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f95837fe640 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa94f240 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9578009930 tx=0x7f9578005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aed7626cc1c86bc server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.884+0000 7f9569ffb640 1 -- 172.21.15.37:0/3119021729 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578035040 con 0x55c0aa9282d0 2024-09-19T06:56:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f9569ffb640 1 -- 172.21.15.37:0/3119021729 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9578031970 con 0x55c0aa9282d0 2024-09-19T06:56:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f9569ffb640 1 -- 172.21.15.37:0/3119021729 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578004920 con 0x55c0aa9282d0 2024-09-19T06:56:06.645 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 -- 172.21.15.37:0/3119021729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 msgr2=0x55c0aa94f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa94f240 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f9578009930 tx=0x7f9578005e00 comp rx=0 tx=0).stop 2024-09-19T06:56:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 -- 172.21.15.37:0/3119021729 shutdown_connections 2024-09-19T06:56:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9370c0 0x55c0aa931240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa94f240 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 --2- 172.21.15.37:0/3119021729 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9279b0 0x55c0aa94ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 -- 172.21.15.37:0/3119021729 >> 172.21.15.37:0/3119021729 conn(0x55c0aa924060 msgr2=0x55c0aa936080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:06.646 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 -- 172.21.15.37:0/3119021729 shutdown_connections 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 -- 172.21.15.37:0/3119021729 wait complete. 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.885+0000 7f958d5c5740 1 -- start start 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 1 Processor -- start 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9279b0 0x55c0aa93b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa93b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9370c0 0x55c0aa9b0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f95841a9e20 con 0x55c0aa9282d0 2024-09-19T06:56:06.647 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95841ab290 con 0x55c0aa9279b0 2024-09-19T06:56:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f95841aa4f0 con 0x55c0aa9370c0 2024-09-19T06:56:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f958d5c5740 10 client.0 _ll_register_callbacks cb 0x7f957c009970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa93b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa93b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42802/0 (socket says 172.21.15.37:42802) 2024-09-19T06:56:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 -- 172.21.15.37:0/922370079 learned_addr learned my addr 172.21.15.37:0/922370079 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:06.648 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f9583fff640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9279b0 0x55c0aa93b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f9588ce8640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9370c0 0x55c0aa9b0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 -- 172.21.15.37:0/922370079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9370c0 msgr2=0x55c0aa9b0cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9370c0 0x55c0aa9b0cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 -- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9279b0 msgr2=0x55c0aa93b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9279b0 0x55c0aa93b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.649 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f95837fe640 1 -- 172.21.15.37:0/922370079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c0aa9b1210 con 0x55c0aa9282d0 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.886+0000 7f9588ce8640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9370c0 0x55c0aa9b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f95837fe640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa93b9d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9578031b60 tx=0x7f9578009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0fbff6d5cc453b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f95637fe640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f95637fe640 1 -- 172.21.15.37:0/922370079 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9578035050 con 0x55c0aa9282d0 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f95637fe640 1 -- 172.21.15.37:0/922370079 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8615+0+0 (secure 0 0 0) 0x7f95780035f0 con 0x55c0aa9282d0 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9578031970 con 0x55c0aa9282d0 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f95637fe640 1 -- 172.21.15.37:0/922370079 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f9578057050 con 0x55c0aa9282d0 2024-09-19T06:56:06.650 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 10 client.19440 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55c0aa9b2520 con 0x55c0aa9282d0 2024-09-19T06:56:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f95637fe640 1 -- 172.21.15.37:0/922370079 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9578031970 con 0x55c0aa9282d0 2024-09-19T06:56:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f95637fe640 1 -- 172.21.15.37:0/922370079 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f95780458c0 con 0x55c0aa9282d0 2024-09-19T06:56:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 10 client.19440 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 10 client.19440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 20 client.19440 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:06.651 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa9b2c70 0x55c0aa9b5090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 4 client.19440 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-09-19T06:56:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.887+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c0aa893d70 con 0x55c0aa9b2c70 2024-09-19T06:56:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.888+0000 7f9583fff640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa9b2c70 0x55c0aa9b5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.888+0000 7f95637fe640 10 client.19440 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:05.888+0000 7f9583fff640 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa9b2c70 0x55c0aa9b5090 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fed3a27075505d8b server_cookie=fd872f3c7643732e in_seq=0 out_seq=0 2024-09-19T06:56:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f95637fe640 1 -- 172.21.15.37:0/922370079 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c0aa893d70 con 0x55c0aa9b2c70 2024-09-19T06:56:06.652 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f95637fe640 10 client.19440 handle_command_reply: tid=0 2024-09-19T06:56:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 client.19440 shutdown 2024-09-19T06:56:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 msgr2=0x55c0aa93b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa93b9d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f9578031b60 tx=0x7f9578009830 comp rx=0 tx=0).stop 2024-09-19T06:56:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 shutdown_connections 2024-09-19T06:56:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x55c0aa9b2c70 0x55c0aa9b5090 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55c0aa9370c0 0x55c0aa9b0cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.653 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f95637fe640 0 client.19440 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55c0aa9282d0 0x55c0aa93b9d0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 --2- 172.21.15.37:0/922370079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c0aa9279b0 0x55c0aa93b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 >> 172.21.15.37:0/922370079 conn(0x55c0aa924060 msgr2=0x55c0aa9229a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 shutdown_connections 2024-09-19T06:56:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 1 -- 172.21.15.37:0/922370079 wait complete. 2024-09-19T06:56:06.654 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.189+0000 7f958d5c5740 20 client.19440 trim_cache size 0 max 16384 2024-09-19T06:56:06.655 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:{ 2024-09-19T06:56:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "message": "", 2024-09-19T06:56:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout: "return_code": 0 2024-09-19T06:56:06.656 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stdout:} 2024-09-19T06:56:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f956c0a3460 msgr2=0x7f956c0a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f956c0a3460 0x7f956c0a5920 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f957c005fd0 tx=0x7f957c004a40 comp rx=0 tx=0).stop 2024-09-19T06:56:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 msgr2=0x7f9584199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:06.657 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584199210 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f957400d950 tx=0x7f957400de10 comp rx=0 tx=0).stop 2024-09-19T06:56:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 shutdown_connections 2024-09-19T06:56:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95841069e0 0x7f95841a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f9584101370 0x7f9584199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f956c0a3460 0x7f956c0a5920 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 --2- 172.21.15.37:0/156690516 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f95841009d0 0x7f9584199210 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:06.658 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 >> 172.21.15.37:0/156690516 conn(0x7f95840fc6b0 msgr2=0x7f95840fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 shutdown_connections 2024-09-19T06:56:06.659 INFO:teuthology.task.background_exec.ubuntu@smithi037.front.sepia.ceph.com.smithi037.stderr:2024-09-19T06:56:06.190+0000 7f958a74b640 1 -- 172.21.15.37:0/156690516 wait complete. 2024-09-19T06:56:06.659 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:06 smithi037 ceph-mon[31211]: pgmap v454: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 111 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 454 op/s 2024-09-19T06:56:06.918 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:07.853 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.483+0000 7f8a983c2640 1 -- 172.21.15.37:0/2609702920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90102ef0 msgr2=0x7f8a9010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.483+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2609702920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90102ef0 0x7f8a9010da40 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f8a88009ca0 tx=0x7f8a8802a950 comp rx=0 tx=0).stop 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.483+0000 7f8a983c2640 1 -- 172.21.15.37:0/2609702920 shutdown_connections 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.483+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2609702920 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90102ef0 0x7f8a9010da40 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.483+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2609702920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a90102570 0x7f8a901029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.483+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2609702920 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a90100e20 0x7f8a90101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.483+0000 7f8a983c2640 1 -- 172.21.15.37:0/2609702920 >> 172.21.15.37:0/2609702920 conn(0x7f8a900fc7d0 msgr2=0x7f8a900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.484+0000 7f8a983c2640 1 -- 172.21.15.37:0/2609702920 shutdown_connections 2024-09-19T06:56:07.854 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.484+0000 7f8a983c2640 1 -- 172.21.15.37:0/2609702920 wait complete. 2024-09-19T06:56:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.484+0000 7f8a983c2640 1 Processor -- start 2024-09-19T06:56:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.484+0000 7f8a983c2640 1 -- start start 2024-09-19T06:56:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a983c2640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90100e20 0x7f8a900fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a983c2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a90102570 0x7f8a900fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a983c2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a90102ef0 0x7f8a900fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a983c2640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90110fa0 con 0x7f8a90100e20 2024-09-19T06:56:07.855 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a983c2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90110e20 con 0x7f8a90102570 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a983c2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90111120 con 0x7f8a90102ef0 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90100e20 0x7f8a900fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90100e20 0x7f8a900fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:42824/0 (socket says 172.21.15.37:42824) 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 -- 172.21.15.37:0/2504582459 learned_addr learned my addr 172.21.15.37:0/2504582459 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9595d640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a90102570 0x7f8a900fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9695f640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a90102ef0 0x7f8a900fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 -- 172.21.15.37:0/2504582459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a90102ef0 msgr2=0x7f8a900fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:07.856 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a90102ef0 0x7f8a900fd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 -- 172.21.15.37:0/2504582459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a90102570 msgr2=0x7f8a900fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a90102570 0x7f8a900fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 -- 172.21.15.37:0/2504582459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a901a69d0 con 0x7f8a90100e20 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9595d640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a90102570 0x7f8a900fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a9615e640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90100e20 0x7f8a900fcc80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c890 tx=0x7f8a8c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a877fe640 1 -- 172.21.15.37:0/2504582459 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c004220 con 0x7f8a90100e20 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a877fe640 1 -- 172.21.15.37:0/2504582459 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8a8c0043c0 con 0x7f8a90100e20 2024-09-19T06:56:07.857 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a901a6cc0 con 0x7f8a90100e20 2024-09-19T06:56:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.485+0000 7f8a877fe640 1 -- 172.21.15.37:0/2504582459 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0049e0 con 0x7f8a90100e20 2024-09-19T06:56:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.486+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a901a71b0 con 0x7f8a90100e20 2024-09-19T06:56:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.486+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a90101200 con 0x7f8a90100e20 2024-09-19T06:56:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.488+0000 7f8a877fe640 1 -- 172.21.15.37:0/2504582459 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8a8c0079b0 con 0x7f8a90100e20 2024-09-19T06:56:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.489+0000 7f8a877fe640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8a7c0a33b0 0x7f8a7c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.489+0000 7f8a9595d640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8a7c0a33b0 0x7f8a7c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:07.858 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.489+0000 7f8a877fe640 1 -- 172.21.15.37:0/2504582459 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8615+0+0 (secure 0 0 0) 0x7f8a8c0cfe90 con 0x7f8a90100e20 2024-09-19T06:56:07.859 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:56:07.859 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"btime":"2024-09-19T06:55:38:930626+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:55:38.046955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28132":{"gid":28132,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2835776386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2835776386},{"type":"v1","addr":"172.21.15.151:6839","nonce":2835776386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:56:07.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.489+0000 7f8a9595d640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8a7c0a33b0 0x7f8a7c0a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8a900fe710 tx=0x7f8a80008040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:07.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.493+0000 7f8a877fe640 1 -- 172.21.15.37:0/2504582459 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f8a8c014030 con 0x7f8a90100e20 2024-09-19T06:56:07.859 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.707+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8a900fe1b0 con 0x7f8a90100e20 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.708+0000 7f8a877fe640 1 -- 172.21.15.37:0/2504582459 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11907 (secure 0 0 0) 0x7f8a8c09c810 con 0x7f8a90100e20 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 47 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8a7c0a33b0 msgr2=0x7f8a7c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8a7c0a33b0 0x7f8a7c0a5870 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f8a900fe710 tx=0x7f8a80008040 comp rx=0 tx=0).stop 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90100e20 msgr2=0x7f8a900fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90100e20 0x7f8a900fcc80 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c00c890 tx=0x7f8a8c00cd50 comp rx=0 tx=0).stop 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 shutdown_connections 2024-09-19T06:56:07.860 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8a90102ef0 0x7f8a900fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8a90102570 0x7f8a900fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f8a7c0a33b0 0x7f8a7c0a5870 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.710+0000 7f8a983c2640 1 --2- 172.21.15.37:0/2504582459 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8a90100e20 0x7f8a900fcc80 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.711+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 >> 172.21.15.37:0/2504582459 conn(0x7f8a900fc7d0 msgr2=0x7f8a9006a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.711+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 shutdown_connections 2024-09-19T06:56:07.861 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:07.711+0000 7f8a983c2640 1 -- 172.21.15.37:0/2504582459 wait complete. 2024-09-19T06:56:08.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:08 smithi096 ceph-mon[35004]: pgmap v455: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 111 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 454 op/s 2024-09-19T06:56:08.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:08 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2504582459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:08 smithi037 ceph-mon[31211]: pgmap v455: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 111 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 454 op/s 2024-09-19T06:56:08.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:08 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2504582459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:08 smithi151 ceph-mon[35236]: pgmap v455: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 111 MiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 3.5 MiB/s wr, 454 op/s 2024-09-19T06:56:08.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:08 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2504582459' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:08.826 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24497 quiesce db --cancel --all' 2024-09-19T06:56:08.826 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24497 quiesce db --cancel --all 2024-09-19T06:56:09.085 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:10 smithi096 ceph-mon[35004]: pgmap v456: 97 pgs: 97 active+clean; 98 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.9 MiB/s wr, 578 op/s 2024-09-19T06:56:10.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:10 smithi096 ceph-mon[35004]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T06:56:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:10 smithi037 ceph-mon[31211]: pgmap v456: 97 pgs: 97 active+clean; 98 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.9 MiB/s wr, 578 op/s 2024-09-19T06:56:10.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:10 smithi037 ceph-mon[31211]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T06:56:10.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:10 smithi151 ceph-mon[35236]: pgmap v456: 97 pgs: 97 active+clean; 98 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 876 KiB/s rd, 2.9 MiB/s wr, 578 op/s 2024-09-19T06:56:10.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:10 smithi151 ceph-mon[35236]: osdmap e107: 12 total, 12 up, 12 in 2024-09-19T06:56:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 -- 172.21.15.37:0/1423209764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8100a60 msgr2=0x7f41a8100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 --2- 172.21.15.37:0/1423209764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8100a60 0x7f41a8100ec0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f4198009930 tx=0x7f419802a420 comp rx=0 tx=0).stop 2024-09-19T06:56:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 -- 172.21.15.37:0/1423209764 shutdown_connections 2024-09-19T06:56:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 --2- 172.21.15.37:0/1423209764 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41a8101400 0x7f41a810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 --2- 172.21.15.37:0/1423209764 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8100a60 0x7f41a8100ec0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 --2- 172.21.15.37:0/1423209764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a8106a60 0x7f41a8106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 -- 172.21.15.37:0/1423209764 >> 172.21.15.37:0/1423209764 conn(0x7f41a80fc780 msgr2=0x7f41a80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 -- 172.21.15.37:0/1423209764 shutdown_connections 2024-09-19T06:56:10.536 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.526+0000 7f41af300640 1 -- 172.21.15.37:0/1423209764 wait complete. 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 Processor -- start 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 -- start start 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a8100a60 0x7f41a819a660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8101400 0x7f41a819aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41a8106a60 0x7f41a819ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8110e20 con 0x7f41a8101400 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8110ca0 con 0x7f41a8106a60 2024-09-19T06:56:10.537 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41af300640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41a8110fa0 con 0x7f41a8100a60 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ac89b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8101400 0x7f41a819aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ac89b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8101400 0x7f41a819aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60108/0 (socket says 172.21.15.37:60108) 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ac89b640 1 -- 172.21.15.37:0/3125593758 learned_addr learned my addr 172.21.15.37:0/3125593758 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ad89d640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41a8106a60 0x7f41a819ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ad09c640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a8100a60 0x7f41a819a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ac89b640 1 -- 172.21.15.37:0/3125593758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a8100a60 msgr2=0x7f41a819a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ac89b640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a8100a60 0x7f41a819a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.538 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.527+0000 7f41ac89b640 1 -- 172.21.15.37:0/3125593758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41a8106a60 msgr2=0x7f41a819ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f41ac89b640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41a8106a60 0x7f41a819ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f41ac89b640 1 -- 172.21.15.37:0/3125593758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41a819f610 con 0x7f41a8101400 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f41ac89b640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8101400 0x7f41a819aba0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4198009230 tx=0x7f4198004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f419e7fc640 1 -- 172.21.15.37:0/3125593758 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f419803f030 con 0x7f41a8101400 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41a819f8a0 con 0x7f41a8101400 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f419e7fc640 1 -- 172.21.15.37:0/3125593758 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4198003ad0 con 0x7f41a8101400 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f419e7fc640 1 -- 172.21.15.37:0/3125593758 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198035070 con 0x7f41a8101400 2024-09-19T06:56:10.539 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.528+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41a81a71e0 con 0x7f41a8101400 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.530+0000 7f419e7fc640 1 -- 172.21.15.37:0/3125593758 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4198042030 con 0x7f41a8101400 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41b16d9740 1 Processor -- start 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41b16d9740 1 -- start start 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e2320 0x5605973ec6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973eefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41b16d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x560597346200 con 0x5605973e1a00 2024-09-19T06:56:10.540 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41b16d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5605972ecdc0 con 0x5605973e2320 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41b16d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5605973421c0 con 0x5605973ecbe0 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.531+0000 7f41ad89d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973e1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60110/0 (socket says 172.21.15.37:60110) 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 -- 172.21.15.37:0/704879420 learned_addr learned my addr 172.21.15.37:0/704879420 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ac89b640 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973eefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad09c640 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e2320 0x5605973ec6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f419e7fc640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f41940a33b0 0x7f41940a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.541 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f41940a33b0 0x7f41940a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f419e7fc640 1 -- 172.21.15.37:0/3125593758 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f41980f0280 con 0x7f41a8101400 2024-09-19T06:56:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 -- 172.21.15.37:0/704879420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 msgr2=0x5605973eefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973eefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 -- 172.21.15.37:0/704879420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e2320 msgr2=0x5605973ec6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e2320 0x5605973ec6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 -- 172.21.15.37:0/704879420 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605970b7620 con 0x5605973e1a00 2024-09-19T06:56:10.542 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad09c640 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e2320 0x5605973ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ac89b640 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41ad89d640 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973e1de0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005f40 tx=0x7f41a0014f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e8b42095716b70 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41937fe640 1 -- 172.21.15.37:0/704879420 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0012d90 con 0x5605973e1a00 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41937fe640 1 -- 172.21.15.37:0/704879420 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0023070 con 0x5605973e1a00 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.532+0000 7f41937fe640 1 -- 172.21.15.37:0/704879420 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a00056f0 con 0x5605973e1a00 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 -- 172.21.15.37:0/704879420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 msgr2=0x5605973e1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41ad89d640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f41940a33b0 0x7f41940a5870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f41a0022000 tx=0x7f41a0020000 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:10.543 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973e1de0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005f40 tx=0x7f41a0014f70 comp rx=0 tx=0).stop 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 -- 172.21.15.37:0/704879420 shutdown_connections 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973eefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e2320 0x5605973ec6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 --2- 172.21.15.37:0/704879420 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973e1de0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 -- 172.21.15.37:0/704879420 >> 172.21.15.37:0/704879420 conn(0x5605973ddf90 msgr2=0x5605973de370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 -- 172.21.15.37:0/704879420 shutdown_connections 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.533+0000 7f41b16d9740 1 -- 172.21.15.37:0/704879420 wait complete. 2024-09-19T06:56:10.544 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 -- start start 2024-09-19T06:56:10.545 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 Processor -- start 2024-09-19T06:56:10.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973ec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e2320 0x5605973e9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.547 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973e9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5605973ef7b0 con 0x5605973e2320 2024-09-19T06:56:10.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5605972e0c50 con 0x5605973e1a00 2024-09-19T06:56:10.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5605973ef930 con 0x5605973ecbe0 2024-09-19T06:56:10.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41b16d9740 10 client.0 _ll_register_callbacks cb 0x5605972f2ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:10.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41ac89b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e2320 0x5605973e9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41ad09c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.548 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41ac89b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e2320 0x5605973e9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60122/0 (socket says 172.21.15.37:60122) 2024-09-19T06:56:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.534+0000 7f41ac89b640 1 -- 172.21.15.37:0/3177677814 learned_addr learned my addr 172.21.15.37:0/3177677814 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ad89d640 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973e9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ac89b640 1 -- 172.21.15.37:0/3177677814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 msgr2=0x5605973e9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ac89b640 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973e9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ac89b640 1 -- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 msgr2=0x5605973ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ac89b640 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.549 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ac89b640 1 -- 172.21.15.37:0/3177677814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605973ea480 con 0x5605973e2320 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ad09c640 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41ac89b640 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e2320 0x5605973e9740 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f41980096c0 tx=0x7f41980bc2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9130e895ff2a5dd5 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f4190ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f4190ff9640 1 -- 172.21.15.37:0/3177677814 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4198049070 con 0x5605973e2320 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f4190ff9640 1 -- 172.21.15.37:0/3177677814 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f4198003c80 con 0x5605973e2320 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a0023070 con 0x5605973e2320 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f4190ff9640 1 -- 172.21.15.37:0/3177677814 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f41980bc4d0 con 0x5605973e2320 2024-09-19T06:56:10.550 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41b16d9740 10 client.19458 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5605973fb2a0 con 0x5605973e2320 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.535+0000 7f4190ff9640 1 -- 172.21.15.37:0/3177677814 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a0023070 con 0x5605973e2320 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f4190ff9640 1 -- 172.21.15.37:0/3177677814 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f41980d52e0 con 0x5605973e2320 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f41b16d9740 10 client.19458 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f41b16d9740 10 client.19458 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f41b16d9740 20 client.19458 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f41b16d9740 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605973fc700 0x5605973feba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.551 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f41b16d9740 4 client.19458 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560597353770 con 0x5605973fc700 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.536+0000 7f41ad09c640 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605973fc700 0x5605973feba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.537+0000 7f41ad09c640 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605973fc700 0x5605973feba0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f15698011318b6 server_cookie=43169803745306ed in_seq=0 out_seq=0 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.537+0000 7f4190ff9640 10 client.19458 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f4190ff9640 1 -- 172.21.15.37:0/3177677814 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560597353770 con 0x5605973fc700 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f4190ff9640 10 client.19458 handle_command_reply: tid=0 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 client.19458 shutdown 2024-09-19T06:56:10.552 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e2320 msgr2=0x5605973e9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e2320 0x5605973e9740 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f41980096c0 tx=0x7f41980bc2a0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 shutdown_connections 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5605973fc700 0x5605973feba0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973ecbe0 0x5605973e9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e2320 0x5605973e9740 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 --2- 172.21.15.37:0/3177677814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 >> 172.21.15.37:0/3177677814 conn(0x5605973ddf90 msgr2=0x5605973dd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:10.553 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f4190ff9640 0 client.19458 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 shutdown_connections 2024-09-19T06:56:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 1 -- 172.21.15.37:0/3177677814 wait complete. 2024-09-19T06:56:10.554 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.540+0000 7f41b16d9740 20 client.19458 trim_cache size 0 max 16384 2024-09-19T06:56:10.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 Processor -- start 2024-09-19T06:56:10.559 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 -- start start 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973f8a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973e4d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973ecbe0 0x5605974044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81b2490 con 0x5605973e1a00 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81afd00 con 0x5605973ecbe0 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41b16d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81adb70 con 0x5605973e2320 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41ad09c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973f8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.560 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41ad09c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973f8a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60130/0 (socket says 172.21.15.37:60130) 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41ad09c640 1 -- 172.21.15.37:0/1040847005 learned_addr learned my addr 172.21.15.37:0/1040847005 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.557+0000 7f41ac89b640 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973e4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f41ad09c640 1 -- 172.21.15.37:0/1040847005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 msgr2=0x5605973e4d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f41ad09c640 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973e4d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f41ad09c640 1 -- 172.21.15.37:0/1040847005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973ecbe0 msgr2=0x5605974044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f41ad09c640 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973ecbe0 0x5605974044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f41ad09c640 1 -- 172.21.15.37:0/1040847005 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5605973e86f0 con 0x5605973e1a00 2024-09-19T06:56:10.561 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f41ad09c640 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973f8a10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f41a4005f80 tx=0x7f41a400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a761ae3e5e68d0cd server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f4192ffd640 1 -- 172.21.15.37:0/1040847005 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a4003ca0 con 0x5605973e1a00 2024-09-19T06:56:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f4192ffd640 1 -- 172.21.15.37:0/1040847005 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4003e40 con 0x5605973e1a00 2024-09-19T06:56:10.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.558+0000 7f4192ffd640 1 -- 172.21.15.37:0/1040847005 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a401c840 con 0x5605973e1a00 2024-09-19T06:56:10.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 -- 172.21.15.37:0/1040847005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 msgr2=0x5605973f8a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973f8a10 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f41a4005f80 tx=0x7f41a400dd40 comp rx=0 tx=0).stop 2024-09-19T06:56:10.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 -- 172.21.15.37:0/1040847005 shutdown_connections 2024-09-19T06:56:10.565 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973ecbe0 0x5605974044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973e4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 --2- 172.21.15.37:0/1040847005 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973e1a00 0x5605973f8a10 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 -- 172.21.15.37:0/1040847005 >> 172.21.15.37:0/1040847005 conn(0x5605973ddf90 msgr2=0x5605973e4640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 -- 172.21.15.37:0/1040847005 shutdown_connections 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 -- 172.21.15.37:0/1040847005 wait complete. 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 -- start start 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.559+0000 7f41b16d9740 1 Processor -- start 2024-09-19T06:56:10.566 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973f0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41b16d9740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973ecbe0 0x5605973f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41b16d9740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81ae1c0 con 0x5605973ecbe0 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41b16d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81af630 con 0x5605973e1a00 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41b16d9740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f41a81ae890 con 0x5605973e2320 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41b16d9740 10 client.0 _ll_register_callbacks cb 0x7f41980b8770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41ac89b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41ad89d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973ecbe0 0x5605973f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.567 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41ac89b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973f0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.37:57068/0 (socket says 172.21.15.37:57068) 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41ac89b640 1 -- 172.21.15.37:0/485281620 learned_addr learned my addr 172.21.15.37:0/485281620 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.560+0000 7f41ad09c640 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41ad89d640 1 -- 172.21.15.37:0/485281620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 msgr2=0x5605973f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41ad89d640 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41ad89d640 1 -- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 msgr2=0x5605973f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41ad89d640 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41ad89d640 1 -- 172.21.15.37:0/485281620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5605973f0e80 con 0x5605973ecbe0 2024-09-19T06:56:10.568 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41ad89d640 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973ecbe0 0x5605973f08c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005fd0 tx=0x7f41a0009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e557f02e4b56b4ae server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f4190ff9640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f4190ff9640 1 -- 172.21.15.37:0/485281620 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41a0005990 con 0x5605973ecbe0 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f4190ff9640 1 -- 172.21.15.37:0/485281620 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f41a001ec50 con 0x5605973ecbe0 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f41a4003e40 con 0x5605973ecbe0 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f4190ff9640 1 -- 172.21.15.37:0/485281620 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f41a0014050 con 0x5605973ecbe0 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41b16d9740 10 client.19464 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.561+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5605974671d0 con 0x5605973ecbe0 2024-09-19T06:56:10.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f4190ff9640 1 -- 172.21.15.37:0/485281620 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f41a4003e40 con 0x5605973ecbe0 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f4190ff9640 1 -- 172.21.15.37:0/485281620 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f41a000f040 con 0x5605973ecbe0 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f41b16d9740 10 client.19464 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f41b16d9740 10 client.19464 resolve_mds: validated gid 24497 aka daemon mds.b 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f41b16d9740 20 client.19464 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f41b16d9740 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560597467730 0x560597469b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f41b16d9740 4 client.19464 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x56059734dc80 con 0x560597467730 2024-09-19T06:56:10.570 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.562+0000 7f41ad09c640 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560597467730 0x560597469b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41ad09c640 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560597467730 0x560597469b50 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52888050351f6261 server_cookie=eaf0b0db94173513 in_seq=0 out_seq=0 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f4190ff9640 10 client.19464 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f4190ff9640 1 -- 172.21.15.37:0/485281620 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x56059734dc80 con 0x560597467730 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f4190ff9640 10 client.19464 handle_command_reply: tid=0 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 client.19464 shutdown 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973ecbe0 msgr2=0x5605973f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973ecbe0 0x5605973f08c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f41a0005fd0 tx=0x7f41a0009b00 comp rx=0 tx=0).stop 2024-09-19T06:56:10.571 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 shutdown_connections 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x560597467730 0x560597469b50 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5605973ecbe0 0x5605973f08c0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5605973e2320 0x5605973f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 --2- 172.21.15.37:0/485281620 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5605973e1a00 0x5605973f5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 >> 172.21.15.37:0/485281620 conn(0x5605973ddf90 msgr2=0x5605973e4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.563+0000 7f4190ff9640 0 client.19464 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 shutdown_connections 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41b16d9740 1 -- 172.21.15.37:0/485281620 wait complete. 2024-09-19T06:56:10.572 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41b16d9740 20 client.19464 trim_cache size 0 max 16384 2024-09-19T06:56:10.573 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:56:10.573 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 47, 2024-09-19T06:56:10.573 INFO:teuthology.orchestra.run.smithi037.stdout: "leader": 24497, 2024-09-19T06:56:10.573 INFO:teuthology.orchestra.run.smithi037.stdout: "set_version": 48, 2024-09-19T06:56:10.574 INFO:teuthology.orchestra.run.smithi037.stdout: "sets": {} 2024-09-19T06:56:10.574 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:56:10.574 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f41940a33b0 msgr2=0x7f41940a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f41940a33b0 0x7f41940a5870 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f41a0022000 tx=0x7f41a0020000 comp rx=0 tx=0).stop 2024-09-19T06:56:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8101400 msgr2=0x7f41a819aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8101400 0x7f41a819aba0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f4198009230 tx=0x7f4198004270 comp rx=0 tx=0).stop 2024-09-19T06:56:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 shutdown_connections 2024-09-19T06:56:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f41a8106a60 0x7f41a819ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.575 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f41940a33b0 0x7f41940a5870 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f41a8101400 0x7f41a819aba0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 --2- 172.21.15.37:0/3125593758 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f41a8100a60 0x7f41a819a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.564+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 >> 172.21.15.37:0/3125593758 conn(0x7f41a80fc780 msgr2=0x7f41a810c720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.565+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 shutdown_connections 2024-09-19T06:56:10.576 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:10.565+0000 7f41af300640 1 -- 172.21.15.37:0/3125593758 wait complete. 2024-09-19T06:56:11.600 INFO:tasks.quiescer:done joining Quiescers 2024-09-19T06:56:11.601 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-09-19T06:56:11.616 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:56:11.626 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-19T06:56:11.635 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-19T06:56:11.635 INFO:teuthology.task.exec:Executing custom commands... 2024-09-19T06:56:11.636 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:56:11.636 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-19T06:56:11.700 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-19T06:56:11.770 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-19T06:56:11.919 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 -- 172.21.15.37:0/2495286850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3801069c0 msgr2=0x7fb380106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 --2- 172.21.15.37:0/2495286850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3801069c0 0x7fb380106da0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fb370009cd0 tx=0x7fb37002a8e0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 -- 172.21.15.37:0/2495286850 shutdown_connections 2024-09-19T06:56:11.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 --2- 172.21.15.37:0/2495286850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb380101350 0x7fb38010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 --2- 172.21.15.37:0/2495286850 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3801009b0 0x7fb380100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.920 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 --2- 172.21.15.37:0/2495286850 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb3801069c0 0x7fb380106da0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 -- 172.21.15.37:0/2495286850 >> 172.21.15.37:0/2495286850 conn(0x7fb380077990 msgr2=0x7fb3800fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 -- 172.21.15.37:0/2495286850 shutdown_connections 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 -- 172.21.15.37:0/2495286850 wait complete. 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.916+0000 7fb3861c4640 1 Processor -- start 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb3861c4640 1 -- start start 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb3861c4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3801009b0 0x7fb380199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb3861c4640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb380101350 0x7fb380199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.921 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb3861c4640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3801069c0 0x7fb3801a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb3861c4640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fb380110dc0 con 0x7fb380101350 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb3861c4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb380110c40 con 0x7fb3801009b0 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb3861c4640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fb380110f40 con 0x7fb3801069c0 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb380101350 0x7fb380199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb380101350 0x7fb380199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60156/0 (socket says 172.21.15.37:60156) 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 -- 172.21.15.37:0/1566906321 learned_addr learned my addr 172.21.15.37:0/1566906321 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37f7fe640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3801009b0 0x7fb380199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.922 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37ffff640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3801069c0 0x7fb3801a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 -- 172.21.15.37:0/1566906321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3801069c0 msgr2=0x7fb3801a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3801069c0 0x7fb3801a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 -- 172.21.15.37:0/1566906321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3801009b0 msgr2=0x7fb380199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3801009b0 0x7fb380199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.917+0000 7fb37effd640 1 -- 172.21.15.37:0/1566906321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3801a4700 con 0x7fb380101350 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.918+0000 7fb37effd640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb380101350 0x7fb380199680 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb37400c920 tx=0x7fb37400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.918+0000 7fb37cff9640 1 -- 172.21.15.37:0/1566906321 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374007c70 con 0x7fb380101350 2024-09-19T06:56:11.923 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.918+0000 7fb37cff9640 1 -- 172.21.15.37:0/1566906321 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb374007e10 con 0x7fb380101350 2024-09-19T06:56:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.918+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3801a48d0 con 0x7fb380101350 2024-09-19T06:56:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.918+0000 7fb37cff9640 1 -- 172.21.15.37:0/1566906321 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb374018890 con 0x7fb380101350 2024-09-19T06:56:11.924 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.918+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3801a4df0 con 0x7fb380101350 2024-09-19T06:56:11.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.919+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb380100e10 con 0x7fb380101350 2024-09-19T06:56:11.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.920+0000 7fb37cff9640 1 -- 172.21.15.37:0/1566906321 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fb374018a50 con 0x7fb380101350 2024-09-19T06:56:11.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.921+0000 7fb37cff9640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3680a3460 0x7fb3680a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:11.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.921+0000 7fb37f7fe640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3680a3460 0x7fb3680a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:11.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.921+0000 7fb37cff9640 1 -- 172.21.15.37:0/1566906321 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fb3740d0130 con 0x7fb380101350 2024-09-19T06:56:11.925 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.922+0000 7fb37f7fe640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3680a3460 0x7fb3680a5920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb370006010 tx=0x7fb370009900 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:11.926 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:11.925+0000 7fb37cff9640 1 -- 172.21.15.37:0/1566906321 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fb374014030 con 0x7fb380101350 2024-09-19T06:56:12.126 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.125+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb380103ef0 con 0x7fb3680a3460 2024-09-19T06:56:12.135 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.133+0000 7fb37cff9640 1 -- 172.21.15.37:0/1566906321 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7fb380103ef0 con 0x7fb3680a3460 2024-09-19T06:56:12.135 INFO:teuthology.orchestra.run.smithi037.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-19T00:00:00", "created": "2024-09-19T06:41:56", "first": "2024-09-19T06:42:00", "last": "2024-09-19T06:56:00", "last_pruned": "2024-09-19T06:56:00", "created_count": 15, "pruned_count": 9, "active": true} 2024-09-19T06:56:12.137 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3680a3460 msgr2=0x7fb3680a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3680a3460 0x7fb3680a5920 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb370006010 tx=0x7fb370009900 comp rx=0 tx=0).stop 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb380101350 msgr2=0x7fb380199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb380101350 0x7fb380199680 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fb37400c920 tx=0x7fb37400cde0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 shutdown_connections 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb3801069c0 0x7fb3801a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fb3680a3460 0x7fb3680a5920 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fb380101350 0x7fb380199680 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.138 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 --2- 172.21.15.37:0/1566906321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb3801009b0 0x7fb380199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 >> 172.21.15.37:0/1566906321 conn(0x7fb380077990 msgr2=0x7fb3801037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 shutdown_connections 2024-09-19T06:56:12.139 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.136+0000 7fb3861c4640 1 -- 172.21.15.37:0/1566906321 wait complete. 2024-09-19T06:56:12.147 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:12 smithi037 ceph-mon[31211]: pgmap v458: 97 pgs: 97 active+clean; 98 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.3 MiB/s wr, 654 op/s 2024-09-19T06:56:12.149 DEBUG:teuthology.orchestra.run.smithi037:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-19T06:56:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.252+0000 7f0e5725e640 1 -- 172.21.15.37:0/3639864933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50101ae0 msgr2=0x7f0e50101f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.256 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.252+0000 7f0e5725e640 1 --2- 172.21.15.37:0/3639864933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50101ae0 0x7f0e50101f40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009930 tx=0x7f0e4402a420 comp rx=0 tx=0).stop 2024-09-19T06:56:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- 172.21.15.37:0/3639864933 shutdown_connections 2024-09-19T06:56:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 --2- 172.21.15.37:0/3639864933 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e50102480 0x7f0e5010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 --2- 172.21.15.37:0/3639864933 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50101ae0 0x7f0e50101f40 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 --2- 172.21.15.37:0/3639864933 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e50107b80 0x7f0e50107f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- 172.21.15.37:0/3639864933 >> 172.21.15.37:0/3639864933 conn(0x7f0e500fd7e0 msgr2=0x7f0e500ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- 172.21.15.37:0/3639864933 shutdown_connections 2024-09-19T06:56:12.257 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- 172.21.15.37:0/3639864933 wait complete. 2024-09-19T06:56:12.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 Processor -- start 2024-09-19T06:56:12.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- start start 2024-09-19T06:56:12.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e50101ae0 0x7f0e500744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50102480 0x7f0e50074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e50107b80 0x7f0e50074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50111f30 con 0x7f0e50102480 2024-09-19T06:56:12.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50111db0 con 0x7f0e50101ae0 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e5725e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0e501120b0 con 0x7f0e50107b80 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50102480 0x7f0e50074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.253+0000 7f0e4ffff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50102480 0x7f0e50074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:60170/0 (socket says 172.21.15.37:60170) 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4ffff640 1 -- 172.21.15.37:0/240063335 learned_addr learned my addr 172.21.15.37:0/240063335 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e54ffa640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e50101ae0 0x7f0e500744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e557fb640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e50107b80 0x7f0e50074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4ffff640 1 -- 172.21.15.37:0/240063335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e50107b80 msgr2=0x7f0e50074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.259 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4ffff640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e50107b80 0x7f0e50074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4ffff640 1 -- 172.21.15.37:0/240063335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e50101ae0 msgr2=0x7f0e500744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4ffff640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e50101ae0 0x7f0e500744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4ffff640 1 -- 172.21.15.37:0/240063335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e501a68a0 con 0x7f0e50102480 2024-09-19T06:56:12.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4ffff640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50102480 0x7f0e50074a30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009230 tx=0x7f0e44004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4dffb640 1 -- 172.21.15.37:0/240063335 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e4403f030 con 0x7f0e50102480 2024-09-19T06:56:12.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4dffb640 1 -- 172.21.15.37:0/240063335 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e44003ad0 con 0x7f0e50102480 2024-09-19T06:56:12.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e4dffb640 1 -- 172.21.15.37:0/240063335 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e44035070 con 0x7f0e50102480 2024-09-19T06:56:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e501a6b30 con 0x7f0e50102480 2024-09-19T06:56:12.261 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.254+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e501a7020 con 0x7f0e50102480 2024-09-19T06:56:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.255+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e50101f40 con 0x7f0e50102480 2024-09-19T06:56:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.256+0000 7f0e4dffb640 1 -- 172.21.15.37:0/240063335 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0e44042030 con 0x7f0e50102480 2024-09-19T06:56:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.257+0000 7f0e4dffb640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0e380a3460 0x7f0e380a5920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.258+0000 7f0e54ffa640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0e380a3460 0x7f0e380a5920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.258+0000 7f0e4dffb640 1 -- 172.21.15.37:0/240063335 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f0e440f0250 con 0x7f0e50102480 2024-09-19T06:56:12.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.258+0000 7f0e54ffa640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0e380a3460 0x7f0e380a5920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e400058c0 tx=0x7f0e4000a250 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:12.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.262+0000 7f0e4dffb640 1 -- 172.21.15.37:0/240063335 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f0e440bcc00 con 0x7f0e50102480 2024-09-19T06:56:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:12 smithi096 ceph-mon[35004]: pgmap v458: 97 pgs: 97 active+clean; 98 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.3 MiB/s wr, 654 op/s 2024-09-19T06:56:12.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:12 smithi151 ceph-mon[35236]: pgmap v458: 97 pgs: 97 active+clean; 98 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 993 KiB/s rd, 3.3 MiB/s wr, 654 op/s 2024-09-19T06:56:12.462 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.461+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 --> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0e500fe640 con 0x7f0e380a3460 2024-09-19T06:56:12.471 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.469+0000 7f0e4dffb640 1 -- 172.21.15.37:0/240063335 <== mgr.15156 v2:172.21.15.37:6800/3483159058 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0e500fe640 con 0x7f0e380a3460 2024-09-19T06:56:12.471 INFO:teuthology.orchestra.run.smithi037.stdout:/ 1m 6m3h 2024-09-19T06:56:12.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0e380a3460 msgr2=0x7f0e380a5920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.473 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0e380a3460 0x7f0e380a5920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0e400058c0 tx=0x7f0e4000a250 comp rx=0 tx=0).stop 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50102480 msgr2=0x7f0e50074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50102480 0x7f0e50074a30 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f0e44009230 tx=0x7f0e44004270 comp rx=0 tx=0).stop 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 shutdown_connections 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0e50107b80 0x7f0e50074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f0e380a3460 0x7f0e380a5920 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0e50102480 0x7f0e50074a30 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 --2- 172.21.15.37:0/240063335 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e50101ae0 0x7f0e500744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:12.474 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 >> 172.21.15.37:0/240063335 conn(0x7f0e500fd7e0 msgr2=0x7f0e500fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:12.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 shutdown_connections 2024-09-19T06:56:12.475 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:12.472+0000 7f0e5725e640 1 -- 172.21.15.37:0/240063335 wait complete. 2024-09-19T06:56:12.484 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-19T06:56:12.495 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-19T06:56:13.273 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:13 smithi096 ceph-mon[35004]: from='client.19470 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:56:13.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:13 smithi096 ceph-mon[35004]: from='client.19476 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:56:13.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:13 smithi037 ceph-mon[31211]: from='client.19470 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:56:13.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:13 smithi037 ceph-mon[31211]: from='client.19476 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:56:13.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:13 smithi151 ceph-mon[35236]: from='client.19470 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:56:13.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:13 smithi151 ceph-mon[35236]: from='client.19476 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-19T06:56:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:14 smithi096 ceph-mon[35004]: pgmap v459: 97 pgs: 97 active+clean; 275 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 19 MiB/s wr, 313 op/s 2024-09-19T06:56:14.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:14 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:56:14.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:14 smithi037 ceph-mon[31211]: pgmap v459: 97 pgs: 97 active+clean; 275 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 19 MiB/s wr, 313 op/s 2024-09-19T06:56:14.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:14 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:56:14.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:14 smithi151 ceph-mon[35236]: pgmap v459: 97 pgs: 97 active+clean; 275 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 471 KiB/s rd, 19 MiB/s wr, 313 op/s 2024-09-19T06:56:14.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:14 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-19T06:56:15.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:15 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:56:15.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:15 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:56:15.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:15 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:56:15.274 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:15 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:56:15.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:15 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:56:15.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:15 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:56:15.362 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:15 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-19T06:56:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:15 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-19T06:56:15.363 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:15 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' 2024-09-19T06:56:16.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:16 smithi096 ceph-mon[35004]: pgmap v460: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-19T06:56:16.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:16 smithi037 ceph-mon[31211]: pgmap v460: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-19T06:56:16.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:16 smithi151 ceph-mon[35236]: pgmap v460: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-19T06:56:18.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:18 smithi096 ceph-mon[35004]: pgmap v461: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-19T06:56:18.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:18 smithi037 ceph-mon[31211]: pgmap v461: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-19T06:56:18.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:18 smithi151 ceph-mon[35236]: pgmap v461: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 437 KiB/s rd, 19 MiB/s wr, 300 op/s 2024-09-19T06:56:20.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:20 smithi096 ceph-mon[35004]: pgmap v462: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 20 MiB/s wr, 121 op/s 2024-09-19T06:56:20.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:20 smithi037 ceph-mon[31211]: pgmap v462: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 20 MiB/s wr, 121 op/s 2024-09-19T06:56:20.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:20 smithi151 ceph-mon[35236]: pgmap v462: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 419 KiB/s rd, 20 MiB/s wr, 121 op/s 2024-09-19T06:56:22.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:22 smithi096 ceph-mon[35004]: pgmap v463: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 19 MiB/s wr, 111 op/s 2024-09-19T06:56:22.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:22 smithi037 ceph-mon[31211]: pgmap v463: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 19 MiB/s wr, 111 op/s 2024-09-19T06:56:22.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:22 smithi151 ceph-mon[35236]: pgmap v463: 97 pgs: 97 active+clean; 281 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 385 KiB/s rd, 19 MiB/s wr, 111 op/s 2024-09-19T06:56:24.242 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-19T06:56:24.505 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:24.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:24 smithi096 ceph-mon[35004]: pgmap v464: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 20 MiB/s wr, 181 op/s 2024-09-19T06:56:24.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:24 smithi037 ceph-mon[31211]: pgmap v464: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 20 MiB/s wr, 181 op/s 2024-09-19T06:56:24.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:24 smithi151 ceph-mon[35236]: pgmap v464: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 458 KiB/s rd, 20 MiB/s wr, 181 op/s 2024-09-19T06:56:24.962 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 -- 172.21.15.37:0/969965019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0528102d30 msgr2=0x7f0528103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:24.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 --2- 172.21.15.37:0/969965019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0528102d30 0x7f0528103110 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f0520009930 tx=0x7f052002a420 comp rx=0 tx=0).stop 2024-09-19T06:56:24.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 -- 172.21.15.37:0/969965019 shutdown_connections 2024-09-19T06:56:24.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 --2- 172.21.15.37:0/969965019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05281082a0 0x7f052810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 --2- 172.21.15.37:0/969965019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05281036e0 0x7f0528107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 --2- 172.21.15.37:0/969965019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0528102d30 0x7f0528103110 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.963 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 -- 172.21.15.37:0/969965019 >> 172.21.15.37:0/969965019 conn(0x7f05280fc7d0 msgr2=0x7f05280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 -- 172.21.15.37:0/969965019 shutdown_connections 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.954+0000 7f052da61640 1 -- 172.21.15.37:0/969965019 wait complete. 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 Processor -- start 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 -- start start 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0528102d30 0x7f05281985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05281036e0 0x7f0528198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05281082a0 0x7f052819ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.964 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05281120a0 con 0x7f05281036e0 2024-09-19T06:56:24.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0528111f20 con 0x7f05281082a0 2024-09-19T06:56:24.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f052da61640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0528112220 con 0x7f0528102d30 2024-09-19T06:56:24.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05281036e0 0x7f0528198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05281036e0 0x7f0528198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44258/0 (socket says 172.21.15.37:44258) 2024-09-19T06:56:24.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f05267fc640 1 -- 172.21.15.37:0/4174281301 learned_addr learned my addr 172.21.15.37:0/4174281301 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:24.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f05277fe640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05281082a0 0x7f052819ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.965 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.955+0000 7f0526ffd640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0528102d30 0x7f05281985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f05267fc640 1 -- 172.21.15.37:0/4174281301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0528102d30 msgr2=0x7f05281985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f05267fc640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0528102d30 0x7f05281985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f05267fc640 1 -- 172.21.15.37:0/4174281301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05281082a0 msgr2=0x7f052819ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f05267fc640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05281082a0 0x7f052819ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f05267fc640 1 -- 172.21.15.37:0/4174281301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f052819d590 con 0x7f05281036e0 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f0526ffd640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0528102d30 0x7f05281985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f05267fc640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05281036e0 0x7f0528198b20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f051000e9c0 tx=0x7f051000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:24.966 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f052ca5f640 1 -- 172.21.15.37:0/4174281301 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051000ccc0 con 0x7f05281036e0 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f052ca5f640 1 -- 172.21.15.37:0/4174281301 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0510005500 con 0x7f05281036e0 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f052819d880 con 0x7f05281036e0 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f052ca5f640 1 -- 172.21.15.37:0/4174281301 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510010690 con 0x7f05281036e0 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.956+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05281a5160 con 0x7f05281036e0 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.958+0000 7f052ca5f640 1 -- 172.21.15.37:0/4174281301 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0510004e80 con 0x7f05281036e0 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 Processor -- start 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 -- start start 2024-09-19T06:56:24.967 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9080550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9082e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562db8fd72e0 con 0x562db9080a90 2024-09-19T06:56:24.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562db8f7ece0 con 0x562db9071c80 2024-09-19T06:56:24.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f052fe3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562db8fd32a0 con 0x562db9071360 2024-09-19T06:56:24.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9082e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.968 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9082e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44272/0 (socket says 172.21.15.37:44272) 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f05267fc640 1 -- 172.21.15.37:0/3361456609 learned_addr learned my addr 172.21.15.37:0/3361456609 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f0526ffd640 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9080550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f05277fe640 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.959+0000 7f0526ffd640 1 -- 172.21.15.37:0/3361456609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 msgr2=0x562db9071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f0526ffd640 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f0526ffd640 1 -- 172.21.15.37:0/3361456609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 msgr2=0x562db9082e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f0526ffd640 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9082e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.969 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f0526ffd640 1 -- 172.21.15.37:0/3361456609 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db8f63440 con 0x562db9071c80 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f05277fe640 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f052ca5f640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05140a33b0 0x7f05140a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f0526ffd640 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9080550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0520009900 tx=0x7f0520005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7cac0d70949e7dcb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f05277fe640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05140a33b0 0x7f05140a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f051dffb640 1 -- 172.21.15.37:0/3361456609 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f052003f070 con 0x562db9071c80 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f052ca5f640 1 -- 172.21.15.37:0/4174281301 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f05100d0130 con 0x7f05281036e0 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f051dffb640 1 -- 172.21.15.37:0/3361456609 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0520004030 con 0x562db9071c80 2024-09-19T06:56:24.970 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.960+0000 7f051dffb640 1 -- 172.21.15.37:0/3361456609 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0520035070 con 0x562db9071c80 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 -- 172.21.15.37:0/3361456609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 msgr2=0x562db9080550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f05277fe640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05140a33b0 0x7f05140a5870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0518005fd0 tx=0x7f0518009500 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9080550 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f0520009900 tx=0x7f0520005e60 comp rx=0 tx=0).stop 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 -- 172.21.15.37:0/3361456609 shutdown_connections 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9082e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9080550 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 --2- 172.21.15.37:0/3361456609 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.971 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 -- 172.21.15.37:0/3361456609 >> 172.21.15.37:0/3361456609 conn(0x562db906da30 msgr2=0x562db906de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:24.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 -- 172.21.15.37:0/3361456609 shutdown_connections 2024-09-19T06:56:24.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 -- 172.21.15.37:0/3361456609 wait complete. 2024-09-19T06:56:24.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.961+0000 7f052fe3a740 1 -- start start 2024-09-19T06:56:24.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 1 Processor -- start 2024-09-19T06:56:24.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db90804d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db907d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9080a90 0x562db907df10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x562db9083600 con 0x562db9071c80 2024-09-19T06:56:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562db8f92050 con 0x562db9080a90 2024-09-19T06:56:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x562db9083780 con 0x562db9071360 2024-09-19T06:56:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f052fe3a740 10 client.0 _ll_register_callbacks cb 0x562db8fd2160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db907d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db907d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44286/0 (socket says 172.21.15.37:44286) 2024-09-19T06:56:24.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f05267fc640 1 -- 172.21.15.37:0/92827668 learned_addr learned my addr 172.21.15.37:0/92827668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.962+0000 7f05277fe640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9080a90 0x562db907df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f0526ffd640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db90804d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f05267fc640 1 -- 172.21.15.37:0/92827668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 msgr2=0x562db90804d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f05267fc640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db90804d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f05267fc640 1 -- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9080a90 msgr2=0x562db907df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f05267fc640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9080a90 0x562db907df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f05267fc640 1 -- 172.21.15.37:0/92827668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db8d483c0 con 0x562db9071c80 2024-09-19T06:56:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f05277fe640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9080a90 0x562db907df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f0526ffd640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db90804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f05267fc640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db907d780 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0510098e50 tx=0x7f05100096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d3d85002f0d07c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f0506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f0506ffd640 1 -- 172.21.15.37:0/92827668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051009bfc0 con 0x562db9071c80 2024-09-19T06:56:24.978 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0520004030 con 0x562db9071c80 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f0506ffd640 1 -- 172.21.15.37:0/92827668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f0510005170 con 0x562db9071c80 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f0506ffd640 1 -- 172.21.15.37:0/92827668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f051000ce60 con 0x562db9071c80 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f052fe3a740 10 client.19488 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.963+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562db908f140 con 0x562db9071c80 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f0506ffd640 1 -- 172.21.15.37:0/92827668 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0520004030 con 0x562db9071c80 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f0506ffd640 1 -- 172.21.15.37:0/92827668 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f05100a4c60 con 0x562db9071c80 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f052fe3a740 10 client.19488 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:24.979 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f052fe3a740 10 client.19488 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f052fe3a740 20 client.19488 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f052fe3a740 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db9090550 0x562db90929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f052fe3a740 4 client.19488 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562db8fe4850 con 0x562db9090550 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f0526ffd640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db9090550 0x562db90929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f0526ffd640 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db9090550 0x562db90929c0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92a571500cd11317 server_cookie=38e6c82f8b26e9e5 in_seq=0 out_seq=0 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.964+0000 7f0506ffd640 10 client.19488 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:24.980 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.965+0000 7f0506ffd640 1 -- 172.21.15.37:0/92827668 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562db8fe4850 con 0x562db9090550 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.965+0000 7f0506ffd640 10 client.19488 handle_command_reply: tid=0 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.965+0000 7f052fe3a740 1 client.19488 shutdown 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 msgr2=0x562db907d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db907d780 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f0510098e50 tx=0x7f05100096c0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 shutdown_connections 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db9090550 0x562db90929c0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9080a90 0x562db907df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.981 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db907d780 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 --2- 172.21.15.37:0/92827668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db90804d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f0506ffd640 0 client.19488 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 >> 172.21.15.37:0/92827668 conn(0x562db906da30 msgr2=0x562db9073790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 shutdown_connections 2024-09-19T06:56:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 1 -- 172.21.15.37:0/92827668 wait complete. 2024-09-19T06:56:24.982 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.966+0000 7f052fe3a740 20 client.19488 trim_cache size 0 max 16384 2024-09-19T06:56:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 Processor -- start 2024-09-19T06:56:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 -- start start 2024-09-19T06:56:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071360 0x562db907d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db9078c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9080a90 0x562db9098290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05281ae000 con 0x562db9071c80 2024-09-19T06:56:24.986 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05281ab870 con 0x562db9071360 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f052fe3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05281a96e0 con 0x562db9080a90 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db9078c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f05267fc640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db9078c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44298/0 (socket says 172.21.15.37:44298) 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.983+0000 7f05267fc640 1 -- 172.21.15.37:0/498534592 learned_addr learned my addr 172.21.15.37:0/498534592 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f05277fe640 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9080a90 0x562db9098290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f0526ffd640 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071360 0x562db907d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f05267fc640 1 -- 172.21.15.37:0/498534592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9080a90 msgr2=0x562db9098290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.987 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f05267fc640 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9080a90 0x562db9098290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f05267fc640 1 -- 172.21.15.37:0/498534592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071360 msgr2=0x562db907d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f05267fc640 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071360 0x562db907d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f05267fc640 1 -- 172.21.15.37:0/498534592 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562db9071ab0 con 0x562db9071c80 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f0526ffd640 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071360 0x562db907d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f05267fc640 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db9078c00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f05100048f0 tx=0x7f0510098120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c98385ec8d957db server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f051d7fa640 1 -- 172.21.15.37:0/498534592 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051009bde0 con 0x562db9071c80 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f051d7fa640 1 -- 172.21.15.37:0/498534592 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05100a50f0 con 0x562db9071c80 2024-09-19T06:56:24.988 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.984+0000 7f051d7fa640 1 -- 172.21.15.37:0/498534592 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0510098af0 con 0x562db9071c80 2024-09-19T06:56:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 -- 172.21.15.37:0/498534592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 msgr2=0x562db9078c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db9078c00 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f05100048f0 tx=0x7f0510098120 comp rx=0 tx=0).stop 2024-09-19T06:56:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 -- 172.21.15.37:0/498534592 shutdown_connections 2024-09-19T06:56:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9080a90 0x562db9098290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.992 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9071c80 0x562db9078c00 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 --2- 172.21.15.37:0/498534592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071360 0x562db907d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 -- 172.21.15.37:0/498534592 >> 172.21.15.37:0/498534592 conn(0x562db906da30 msgr2=0x562db90784f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 -- 172.21.15.37:0/498534592 shutdown_connections 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 -- 172.21.15.37:0/498534592 wait complete. 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 -- start start 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.985+0000 7f052fe3a740 1 Processor -- start 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9088f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.993 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9084250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f052fe3a740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9084790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f052fe3a740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05281a9d30 con 0x562db9080a90 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f052fe3a740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05281ab1a0 con 0x562db9071c80 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f052fe3a740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05281aa400 con 0x562db9071360 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f052fe3a740 10 client.0 _ll_register_callbacks cb 0x7f05100056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9084790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9084790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44310/0 (socket says 172.21.15.37:44310) 2024-09-19T06:56:24.994 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 -- 172.21.15.37:0/1880125363 learned_addr learned my addr 172.21.15.37:0/1880125363 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f0526ffd640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9088f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05267fc640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9084250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 -- 172.21.15.37:0/1880125363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 msgr2=0x562db9088f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9088f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 -- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 msgr2=0x562db9084250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9084250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 -- 172.21.15.37:0/1880125363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562db9084d50 con 0x562db9080a90 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f0526ffd640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9088f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:24.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05267fc640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9084250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f05277fe640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9084790 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f05180042c0 tx=0x7f0518004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573b763424ae68c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.986+0000 7f0506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f0506ffd640 1 -- 172.21.15.37:0/1880125363 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f051800fc10 con 0x562db9080a90 2024-09-19T06:56:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f0506ffd640 1 -- 172.21.15.37:0/1880125363 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f051800fdb0 con 0x562db9080a90 2024-09-19T06:56:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562db8fded60 con 0x562db9080a90 2024-09-19T06:56:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f0506ffd640 1 -- 172.21.15.37:0/1880125363 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f0518004b20 con 0x562db9080a90 2024-09-19T06:56:24.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 10 client.19494 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x562db90fb0a0 con 0x562db9080a90 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f0506ffd640 1 -- 172.21.15.37:0/1880125363 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x562db8fded60 con 0x562db9080a90 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f0506ffd640 1 -- 172.21.15.37:0/1880125363 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f051801f430 con 0x562db9080a90 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 10 client.19494 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 10 client.19494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 20 client.19494 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db90fb880 0x562db90fdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:24.997 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 4 client.19494 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.987+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562db8f5b210 con 0x562db90fb880 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f0526ffd640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db90fb880 0x562db90fdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f0526ffd640 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db90fb880 0x562db90fdca0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd2af67eeac6016 server_cookie=3fca8408121b48b1 in_seq=0 out_seq=0 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f0506ffd640 10 client.19494 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f0506ffd640 1 -- 172.21.15.37:0/1880125363 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562db8f5b210 con 0x562db90fb880 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f0506ffd640 10 client.19494 handle_command_reply: tid=0 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 client.19494 shutdown 2024-09-19T06:56:24.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 msgr2=0x562db9084790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9084790 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f05180042c0 tx=0x7f0518004a40 comp rx=0 tx=0).stop 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 shutdown_connections 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x562db90fb880 0x562db90fdca0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x562db9080a90 0x562db9084790 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f0506ffd640 0 client.19494 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562db9071c80 0x562db9084250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 --2- 172.21.15.37:0/1880125363 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x562db9071360 0x562db9088f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 >> 172.21.15.37:0/1880125363 conn(0x562db906da30 msgr2=0x562db907efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:24.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 shutdown_connections 2024-09-19T06:56:25.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.988+0000 7f052fe3a740 1 -- 172.21.15.37:0/1880125363 wait complete. 2024-09-19T06:56:25.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052fe3a740 20 client.19494 trim_cache size 0 max 16384 2024-09-19T06:56:25.000 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:56:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "status": "no active scrubs running", 2024-09-19T06:56:25.000 INFO:teuthology.orchestra.run.smithi037.stdout: "scrubs": {} 2024-09-19T06:56:25.001 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:56:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05140a33b0 msgr2=0x7f05140a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05140a33b0 0x7f05140a5870 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f0518005fd0 tx=0x7f0518009500 comp rx=0 tx=0).stop 2024-09-19T06:56:25.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05281036e0 msgr2=0x7f0528198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05281036e0 0x7f0528198b20 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f051000e9c0 tx=0x7f051000ee80 comp rx=0 tx=0).stop 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 shutdown_connections 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05281082a0 0x7f052819ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f05140a33b0 0x7f05140a5870 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05281036e0 0x7f0528198b20 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 --2- 172.21.15.37:0/4174281301 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0528102d30 0x7f05281985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 >> 172.21.15.37:0/4174281301 conn(0x7f05280fc7d0 msgr2=0x7f0528105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:25.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 shutdown_connections 2024-09-19T06:56:25.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:24.989+0000 7f052da61640 1 -- 172.21.15.37:0/4174281301 wait complete. 2024-09-19T06:56:25.774 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-19T06:56:25.775 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:56:26.035 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:26.429 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:26 smithi037 ceph-mon[31211]: pgmap v465: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.8 MiB/s wr, 121 op/s 2024-09-19T06:56:26.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:26 smithi096 ceph-mon[35004]: pgmap v465: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.8 MiB/s wr, 121 op/s 2024-09-19T06:56:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.511+0000 7f077d5a1640 1 -- 172.21.15.37:0/2583945788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101080 msgr2=0x7f07781014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.511+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2583945788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101080 0x7f07781014c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f076c009900 tx=0x7f076c02a410 comp rx=0 tx=0).stop 2024-09-19T06:56:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.511+0000 7f077d5a1640 1 -- 172.21.15.37:0/2583945788 shutdown_connections 2024-09-19T06:56:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.511+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2583945788 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0778101a00 0x7f077810d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.511+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2583945788 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101080 0x7f07781014c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.511+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2583945788 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f07781059e0 0x7f0778105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.530 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.511+0000 7f077d5a1640 1 -- 172.21.15.37:0/2583945788 >> 172.21.15.37:0/2583945788 conn(0x7f0778077010 msgr2=0x7f07780fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.512+0000 7f077d5a1640 1 -- 172.21.15.37:0/2583945788 shutdown_connections 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.512+0000 7f077d5a1640 1 -- 172.21.15.37:0/2583945788 wait complete. 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.512+0000 7f077d5a1640 1 Processor -- start 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.512+0000 7f077d5a1640 1 -- start start 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.512+0000 7f077d5a1640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101080 0x7f07781992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.512+0000 7f077d5a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101a00 0x7f0778199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.512+0000 7f077d5a1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781059e0 0x7f0778199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:26.531 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f077d5a1640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f0778110ec0 con 0x7f0778101a00 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f077d5a1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0778110d40 con 0x7f07781059e0 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f077d5a1640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f0778111040 con 0x7f0778101080 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101a00 0x7f0778199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101a00 0x7f0778199830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44330/0 (socket says 172.21.15.37:44330) 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 -- 172.21.15.37:0/2930414271 learned_addr learned my addr 172.21.15.37:0/2930414271 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f0777fff640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101080 0x7f07781992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f077cda0640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781059e0 0x7f0778199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:26.532 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 -- 172.21.15.37:0/2930414271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101080 msgr2=0x7f07781992f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101080 0x7f07781992f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 -- 172.21.15.37:0/2930414271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781059e0 msgr2=0x7f0778199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781059e0 0x7f0778199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 -- 172.21.15.37:0/2930414271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07781a46f0 con 0x7f0778101a00 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07777fe640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101a00 0x7f0778199830 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f076c006010 tx=0x7f076c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07757fa640 1 -- 172.21.15.37:0/2930414271 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c03f070 con 0x7f0778101a00 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07781a4980 con 0x7f0778101a00 2024-09-19T06:56:26.533 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07757fa640 1 -- 172.21.15.37:0/2930414271 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f076c031c30 con 0x7f0778101a00 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.513+0000 7f07757fa640 1 -- 172.21.15.37:0/2930414271 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c035030 con 0x7f0778101a00 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.514+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781a4eb0 con 0x7f0778101a00 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.514+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0778102090 con 0x7f0778101a00 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.516+0000 7f07757fa640 1 -- 172.21.15.37:0/2930414271 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f076c042070 con 0x7f0778101a00 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.516+0000 7f07757fa640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07600a3430 0x7f07600a58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.516+0000 7f0777fff640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07600a3430 0x7f07600a58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.516+0000 7f07757fa640 1 -- 172.21.15.37:0/2930414271 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f076c0f05e0 con 0x7f0778101a00 2024-09-19T06:56:26.534 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.517+0000 7f0777fff640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07600a3430 0x7f07600a58f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0768005fd0 tx=0x7f076800b510 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:26.535 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.520+0000 7f07757fa640 1 -- 172.21.15.37:0/2930414271 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f076c031de0 con 0x7f0778101a00 2024-09-19T06:56:26.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:26 smithi151 ceph-mon[35236]: pgmap v465: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 4.8 MiB/s wr, 121 op/s 2024-09-19T06:56:26.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.731+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0778100590 con 0x7f0778101a00 2024-09-19T06:56:26.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.731+0000 7f07757fa640 1 -- 172.21.15.37:0/2930414271 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11907 (secure 0 0 0) 0x7f0778100590 con 0x7f0778101a00 2024-09-19T06:56:26.733 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:56:26.734 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"btime":"2024-09-19T06:55:38:930626+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:55:38.046955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28132":{"gid":28132,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2835776386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2835776386},{"type":"v1","addr":"172.21.15.151:6839","nonce":2835776386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:56:26.734 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 47 2024-09-19T06:56:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.733+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07600a3430 msgr2=0x7f07600a58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:26.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.733+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07600a3430 0x7f07600a58f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f0768005fd0 tx=0x7f076800b510 comp rx=0 tx=0).stop 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.733+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101a00 msgr2=0x7f0778199830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101a00 0x7f0778199830 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f076c006010 tx=0x7f076c003c10 comp rx=0 tx=0).stop 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 shutdown_connections 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f07781059e0 0x7f0778199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f07600a3430 0x7f07600a58f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f0778101a00 0x7f0778199830 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 --2- 172.21.15.37:0/2930414271 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0778101080 0x7f07781992f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:26.736 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 >> 172.21.15.37:0/2930414271 conn(0x7f0778077010 msgr2=0x7f07781038f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 shutdown_connections 2024-09-19T06:56:26.737 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:26.734+0000 7f077d5a1640 1 -- 172.21.15.37:0/2930414271 wait complete. 2024-09-19T06:56:27.418 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:27 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/2930414271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:27.452 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-19T06:56:27.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:27 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/2930414271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:27.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:27 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/2930414271' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:56:27.711 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 -- 172.21.15.37:0/1973575179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070100a80 msgr2=0x7f4070100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 --2- 172.21.15.37:0/1973575179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070100a80 0x7f4070100ee0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f4060009930 tx=0x7f406002a420 comp rx=0 tx=0).stop 2024-09-19T06:56:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 -- 172.21.15.37:0/1973575179 shutdown_connections 2024-09-19T06:56:28.217 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 --2- 172.21.15.37:0/1973575179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070101420 0x7f407010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 --2- 172.21.15.37:0/1973575179 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070100a80 0x7f4070100ee0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 --2- 172.21.15.37:0/1973575179 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4070106a80 0x7f4070106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 -- 172.21.15.37:0/1973575179 >> 172.21.15.37:0/1973575179 conn(0x7f40700fc7a0 msgr2=0x7f40700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 -- 172.21.15.37:0/1973575179 shutdown_connections 2024-09-19T06:56:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.206+0000 7f4078505640 1 -- 172.21.15.37:0/1973575179 wait complete. 2024-09-19T06:56:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 Processor -- start 2024-09-19T06:56:28.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 -- start start 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100a80 0x7f40701984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070101420 0x7f40701989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4070106a80 0x7f407019cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4070110e80 con 0x7f4070101420 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4070110d00 con 0x7f4070100a80 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4078505640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f4070111000 con 0x7f4070106a80 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070101420 0x7f40701989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.219 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070101420 0x7f40701989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44346/0 (socket says 172.21.15.37:44346) 2024-09-19T06:56:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 -- 172.21.15.37:0/3183249473 learned_addr learned my addr 172.21.15.37:0/3183249473 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f40762a1640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100a80 0x7f40701984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4076aa2640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4070106a80 0x7f407019cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 -- 172.21.15.37:0/3183249473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4070106a80 msgr2=0x7f407019cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4070106a80 0x7f407019cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 -- 172.21.15.37:0/3183249473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100a80 msgr2=0x7f40701984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.220 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100a80 0x7f40701984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4075aa0640 1 -- 172.21.15.37:0/3183249473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f407019d460 con 0x7f4070101420 2024-09-19T06:56:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f40762a1640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100a80 0x7f40701984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.207+0000 7f4076aa2640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4070106a80 0x7f407019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.208+0000 7f4075aa0640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070101420 0x7f40701989f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f4060009230 tx=0x7f4060004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.208+0000 7f40677fe640 1 -- 172.21.15.37:0/3183249473 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406003f030 con 0x7f4070101420 2024-09-19T06:56:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.208+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f407019d6f0 con 0x7f4070101420 2024-09-19T06:56:28.221 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.208+0000 7f40677fe640 1 -- 172.21.15.37:0/3183249473 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4060003ad0 con 0x7f4070101420 2024-09-19T06:56:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.208+0000 7f40677fe640 1 -- 172.21.15.37:0/3183249473 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060035070 con 0x7f4070101420 2024-09-19T06:56:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.208+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40701a5030 con 0x7f4070101420 2024-09-19T06:56:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.210+0000 7f40677fe640 1 -- 172.21.15.37:0/3183249473 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4060042030 con 0x7f4070101420 2024-09-19T06:56:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.210+0000 7f407a8de740 1 Processor -- start 2024-09-19T06:56:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f407a8de740 1 -- start start 2024-09-19T06:56:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f407a8de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1b2090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.222 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f407a8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f407a8de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1c1780 0x55d3dc1c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f407a8de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d3dc1182f0 con 0x55d3dc1c1780 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f407a8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d3dc0bfcf0 con 0x55d3dc1b25d0 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f407a8de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3dc1142b0 con 0x55d3dc1b3b40 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1c1780 0x55d3dc1c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1c1780 0x55d3dc1c3b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44354/0 (socket says 172.21.15.37:44354) 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 -- 172.21.15.37:0/1687192198 learned_addr learned my addr 172.21.15.37:0/1687192198 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:28.223 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f40762a1640 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4076aa2640 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f40677fe640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f405c0a33b0 0x7f405c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 -- 172.21.15.37:0/1687192198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b3b40 msgr2=0x55d3dc1b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 -- 172.21.15.37:0/1687192198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 msgr2=0x55d3dc1b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.224 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.211+0000 7f4075aa0640 1 -- 172.21.15.37:0/1687192198 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3dc0a43a0 con 0x55d3dc1c1780 2024-09-19T06:56:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.212+0000 7f4076aa2640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f405c0a33b0 0x7f405c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.212+0000 7f40677fe640 1 -- 172.21.15.37:0/3183249473 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f40600f02f0 con 0x7f4070101420 2024-09-19T06:56:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.212+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1c1780 0x55d3dc1c3b70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f40600ed980 tx=0x7f4060086ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cec897426b71af8 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.212+0000 7f4064ff9640 1 -- 172.21.15.37:0/1687192198 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600bbf00 con 0x55d3dc1c1780 2024-09-19T06:56:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.212+0000 7f4076aa2640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f405c0a33b0 0x7f405c0a5870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4068005fd0 tx=0x7f4068004a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:28.225 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.212+0000 7f4064ff9640 1 -- 172.21.15.37:0/1687192198 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40600873f0 con 0x55d3dc1c1780 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.212+0000 7f4064ff9640 1 -- 172.21.15.37:0/1687192198 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40600bcca0 con 0x55d3dc1c1780 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 -- 172.21.15.37:0/1687192198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1c1780 msgr2=0x55d3dc1c3b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1c1780 0x55d3dc1c3b70 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f40600ed980 tx=0x7f4060086ee0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 -- 172.21.15.37:0/1687192198 shutdown_connections 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1c1780 0x55d3dc1c3b70 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 --2- 172.21.15.37:0/1687192198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.226 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 -- 172.21.15.37:0/1687192198 >> 172.21.15.37:0/1687192198 conn(0x55d3dc17b310 msgr2=0x55d3dc1aec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 -- 172.21.15.37:0/1687192198 shutdown_connections 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 -- 172.21.15.37:0/1687192198 wait complete. 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 -- start start 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.213+0000 7f407a8de740 1 Processor -- start 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f407a8de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f407a8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f407a8de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1be980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.227 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f407a8de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55d3dc1c4240 con 0x55d3dc1b25d0 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f407a8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d3dc0d3060 con 0x55d3dc1b3b40 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f407a8de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55d3dc1c43c0 con 0x55d3dc1c1780 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f407a8de740 10 client.0 _ll_register_callbacks cb 0x55d3dbe268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44368/0 (socket says 172.21.15.37:44368) 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 -- 172.21.15.37:0/3857442816 learned_addr learned my addr 172.21.15.37:0/3857442816 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f4076aa2640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1be980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f4075aa0640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 -- 172.21.15.37:0/3857442816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 msgr2=0x55d3dc1be980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1be980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 -- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b3b40 msgr2=0x55d3dc1be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1be3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 -- 172.21.15.37:0/3857442816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3dbe893c0 con 0x55d3dc1b25d0 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f4075aa0640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f40762a1640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c1130 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f406c00bf30 tx=0x7f406c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=758010d9ffa62dc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:28.229 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.214+0000 7f4059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f4059ffb640 1 -- 172.21.15.37:0/3857442816 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406c0035f0 con 0x55d3dc1b25d0 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f4059ffb640 1 -- 172.21.15.37:0/3857442816 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f406c003790 con 0x55d3dc1b25d0 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40600873f0 con 0x55d3dc1b25d0 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f4059ffb640 1 -- 172.21.15.37:0/3857442816 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f406c003990 con 0x55d3dc1b25d0 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 10 client.19512 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d3dc1cfbe0 con 0x55d3dc1b25d0 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f4059ffb640 1 -- 172.21.15.37:0/3857442816 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40600873f0 con 0x55d3dc1b25d0 2024-09-19T06:56:28.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f4059ffb640 1 -- 172.21.15.37:0/3857442816 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f406c019070 con 0x55d3dc1b25d0 2024-09-19T06:56:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 10 client.19512 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 10 client.19512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:56:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 20 client.19512 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc1d10f0 0x55d3dc1d3540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 4 client.19512 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.215+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3dc125860 con 0x55d3dc1d10f0 2024-09-19T06:56:28.231 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.216+0000 7f4075aa0640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc1d10f0 0x55d3dc1d3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.216+0000 7f4059ffb640 10 client.19512 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:56:28.236 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.216+0000 7f4075aa0640 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc1d10f0 0x55d3dc1d3540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c539f151a946bd6 server_cookie=8340aa0587b44844 in_seq=0 out_seq=0 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f4059ffb640 1 -- 172.21.15.37:0/3857442816 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d3dc125860 con 0x55d3dc1d10f0 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f4059ffb640 10 client.19512 handle_command_reply: tid=0 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 client.19512 shutdown 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b25d0 msgr2=0x55d3dc1c1130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c1130 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f406c00bf30 tx=0x7f406c0128e0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 shutdown_connections 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc1d10f0 0x55d3dc1d3540 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.237 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1be980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 --2- 172.21.15.37:0/3857442816 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c1130 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 >> 172.21.15.37:0/3857442816 conn(0x55d3dc17b310 msgr2=0x55d3dc1b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f4059ffb640 0 client.19512 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:56:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 shutdown_connections 2024-09-19T06:56:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 1 -- 172.21.15.37:0/3857442816 wait complete. 2024-09-19T06:56:28.238 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.217+0000 7f407a8de740 20 client.19512 trim_cache size 0 max 16384 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.230+0000 7f407a8de740 1 Processor -- start 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.230+0000 7f407a8de740 1 -- start start 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f407a8de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1aee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f407a8de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f407a8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1c1780 0x55d3dc1cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f407a8de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40701ae190 con 0x55d3dc1b3b40 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f407a8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40701aba00 con 0x55d3dc1c1780 2024-09-19T06:56:28.239 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f407a8de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40701a9870 con 0x55d3dc1b25d0 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1bdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44382/0 (socket says 172.21.15.37:44382) 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 -- 172.21.15.37:0/1691688186 learned_addr learned my addr 172.21.15.37:0/1691688186 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f40762a1640 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 -- 172.21.15.37:0/1691688186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b25d0 msgr2=0x55d3dc1aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4076aa2640 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1c1780 0x55d3dc1cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 -- 172.21.15.37:0/1691688186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1c1780 msgr2=0x55d3dc1cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.240 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1c1780 0x55d3dc1cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 -- 172.21.15.37:0/1691688186 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3dc1bd390 con 0x55d3dc1b3b40 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f40762a1640 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1bdca0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f40600b8280 tx=0x7f4060005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c305afa5c889cedf server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.231+0000 7f405bfff640 1 -- 172.21.15.37:0/1691688186 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060049040 con 0x55d3dc1b3b40 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f405bfff640 1 -- 172.21.15.37:0/1691688186 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40600bc5a0 con 0x55d3dc1b3b40 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f405bfff640 1 -- 172.21.15.37:0/1691688186 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406003b460 con 0x55d3dc1b3b40 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 -- 172.21.15.37:0/1691688186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 msgr2=0x55d3dc1bdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.241 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1bdca0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f40600b8280 tx=0x7f4060005c90 comp rx=0 tx=0).stop 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 -- 172.21.15.37:0/1691688186 shutdown_connections 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1c1780 0x55d3dc1cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1bdca0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 --2- 172.21.15.37:0/1691688186 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 -- 172.21.15.37:0/1691688186 >> 172.21.15.37:0/1691688186 conn(0x55d3dc17b310 msgr2=0x55d3dc1ae700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 -- 172.21.15.37:0/1691688186 shutdown_connections 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.232+0000 7f407a8de740 1 -- 172.21.15.37:0/1691688186 wait complete. 2024-09-19T06:56:28.242 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 -- start start 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 Processor -- start 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1baf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1bb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f40701a9ec0 con 0x55d3dc1b3b40 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f40701ab330 con 0x55d3dc1b25d0 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f40701aa590 con 0x55d3dc1c1780 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f407a8de740 10 client.0 _ll_register_callbacks cb 0x7f406c00bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:28.243 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1baf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f4075aa0640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1baf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44388/0 (socket says 172.21.15.37:44388) 2024-09-19T06:56:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f4075aa0640 1 -- 172.21.15.37:0/1478871546 learned_addr learned my addr 172.21.15.37:0/1478871546 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f4076aa2640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1bb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f40762a1640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.233+0000 7f4075aa0640 1 -- 172.21.15.37:0/1478871546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 msgr2=0x55d3dc1bb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1bb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.244 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4075aa0640 1 -- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 msgr2=0x55d3dc1c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4075aa0640 1 -- 172.21.15.37:0/1478871546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3dc1bba00 con 0x55d3dc1b3b40 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f40762a1640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4075aa0640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1baf00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f40600b8f70 tx=0x7f40600f0630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1ba93113933ef97 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4059ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4059ffb640 1 -- 172.21.15.37:0/1478871546 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4060049040 con 0x55d3dc1b3b40 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4059ffb640 1 -- 172.21.15.37:0/1478871546 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f40600d0780 con 0x55d3dc1b3b40 2024-09-19T06:56:28.245 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40600bc5a0 con 0x55d3dc1b3b40 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4059ffb640 1 -- 172.21.15.37:0/1478871546 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f40600c0950 con 0x55d3dc1b3b40 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 10 client.19518 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55d3dc23bb20 con 0x55d3dc1b3b40 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4059ffb640 1 -- 172.21.15.37:0/1478871546 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f40600bc5a0 con 0x55d3dc1b3b40 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f4059ffb640 1 -- 172.21.15.37:0/1478871546 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f40600edc90 con 0x55d3dc1b3b40 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 10 client.19518 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 10 client.19518 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 20 client.19518 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:28.246 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc23be00 0x55d3dc23e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 4 client.19518 mds_command: new command op to 24449 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.234+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 --> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3dc11fd70 con 0x55d3dc23be00 2024-09-19T06:56:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f40762a1640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc23be00 0x55d3dc23e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f40762a1640 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc23be00 0x55d3dc23e1f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b4a994e500542e7 server_cookie=56091cd18862e7e8 in_seq=0 out_seq=0 2024-09-19T06:56:28.247 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f4059ffb640 10 client.19518 ms_handle_connect on v2:172.21.15.96:6834/946199433 2024-09-19T06:56:28.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f4059ffb640 1 -- 172.21.15.37:0/1478871546 <== mds.2 v2:172.21.15.96:6834/946199433 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3dc11fd70 con 0x55d3dc23be00 2024-09-19T06:56:28.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f4059ffb640 10 client.19518 handle_command_reply: tid=0 2024-09-19T06:56:28.248 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 client.19518 shutdown 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 msgr2=0x55d3dc1baf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1baf00 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f40600b8f70 tx=0x7f40600f0630 comp rx=0 tx=0).stop 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 shutdown_connections 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:6834/946199433,v1:172.21.15.96:6835/946199433] conn(0x55d3dc23be00 0x55d3dc23e1f0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55d3dc1c1780 0x55d3dc1bb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55d3dc1b3b40 0x55d3dc1baf00 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 --2- 172.21.15.37:0/1478871546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d3dc1b25d0 0x55d3dc1c9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.249 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f4059ffb640 0 client.19518 ms_handle_reset on v2:172.21.15.96:6834/946199433 2024-09-19T06:56:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.235+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 >> 172.21.15.37:0/1478871546 conn(0x55d3dc17b310 msgr2=0x55d3dc1ad310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 shutdown_connections 2024-09-19T06:56:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f407a8de740 1 -- 172.21.15.37:0/1478871546 wait complete. 2024-09-19T06:56:28.250 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f407a8de740 20 client.19518 trim_cache size 0 max 16384 2024-09-19T06:56:28.250 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:56:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f405c0a33b0 msgr2=0x7f405c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f405c0a33b0 0x7f405c0a5870 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4068005fd0 tx=0x7f4068004a40 comp rx=0 tx=0).stop 2024-09-19T06:56:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070101420 msgr2=0x7f40701989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:28.251 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070101420 0x7f40701989f0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f4060009230 tx=0x7f4060004270 comp rx=0 tx=0).stop 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 shutdown_connections 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4070106a80 0x7f407019cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f405c0a33b0 0x7f405c0a5870 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4070101420 0x7f40701989f0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 --2- 172.21.15.37:0/3183249473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4070100a80 0x7f40701984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 >> 172.21.15.37:0/3183249473 conn(0x7f40700fc7a0 msgr2=0x7f407006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 shutdown_connections 2024-09-19T06:56:28.252 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:28.236+0000 7f4078505640 1 -- 172.21.15.37:0/3183249473 wait complete. 2024-09-19T06:56:28.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:28 smithi096 ceph-mon[35004]: pgmap v466: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.3 MiB/s wr, 119 op/s 2024-09-19T06:56:28.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:28 smithi037 ceph-mon[31211]: pgmap v466: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.3 MiB/s wr, 119 op/s 2024-09-19T06:56:28.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:28 smithi151 ceph-mon[35236]: pgmap v466: 97 pgs: 97 active+clean; 285 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 4.3 MiB/s wr, 119 op/s 2024-09-19T06:56:28.999 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-19T06:56:29.254 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:29.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 -- 172.21.15.37:0/662924110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f010c170 msgr2=0x7fa9f010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 --2- 172.21.15.37:0/662924110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f010c170 0x7fa9f010e610 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009ca0 tx=0x7fa9e802a950 comp rx=0 tx=0).stop 2024-09-19T06:56:29.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 -- 172.21.15.37:0/662924110 shutdown_connections 2024-09-19T06:56:29.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 --2- 172.21.15.37:0/662924110 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f010c170 0x7fa9f010e610 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.701 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 --2- 172.21.15.37:0/662924110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9f0103660 0x7fa9f010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 --2- 172.21.15.37:0/662924110 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f0102d40 0x7fa9f0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 -- 172.21.15.37:0/662924110 >> 172.21.15.37:0/662924110 conn(0x7fa9f00fc7a0 msgr2=0x7fa9f00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.690+0000 7fa9f831c640 1 -- 172.21.15.37:0/662924110 shutdown_connections 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.691+0000 7fa9f831c640 1 -- 172.21.15.37:0/662924110 wait complete. 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.691+0000 7fa9f831c640 1 Processor -- start 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.691+0000 7fa9f831c640 1 -- start start 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.691+0000 7fa9f831c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f0102d40 0x7fa9f019a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.702 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.691+0000 7fa9f831c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f0103660 0x7fa9f019ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f831c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9f010c170 0x7fa9f019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f831c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0111fd0 con 0x7fa9f0103660 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f831c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0111e50 con 0x7fa9f010c170 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f831c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f0112150 con 0x7fa9f0102d40 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f0103660 0x7fa9f019ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f0103660 0x7fa9f019ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58538/0 (socket says 172.21.15.37:58538) 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 -- 172.21.15.37:0/3136592472 learned_addr learned my addr 172.21.15.37:0/3136592472 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f0102d40 0x7fa9f019a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.703 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9f010c170 0x7fa9f019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 -- 172.21.15.37:0/3136592472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f0102d40 msgr2=0x7fa9f019a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f0102d40 0x7fa9f019a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 -- 172.21.15.37:0/3136592472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9f010c170 msgr2=0x7fa9f019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9f010c170 0x7fa9f019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 -- 172.21.15.37:0/3136592472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f019f670 con 0x7fa9f0103660 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9f010c170 0x7fa9f019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f0103660 0x7fa9f019ac00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000b3f0 tx=0x7fa9e000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9e77fe640 1 -- 172.21.15.37:0/3136592472 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0015020 con 0x7fa9f0103660 2024-09-19T06:56:29.704 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9e77fe640 1 -- 172.21.15.37:0/3136592472 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e0011d10 con 0x7fa9f0103660 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f019f960 con 0x7fa9f0103660 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.692+0000 7fa9e77fe640 1 -- 172.21.15.37:0/3136592472 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e000f2e0 con 0x7fa9f0103660 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.693+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f01a7240 con 0x7fa9f0103660 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9e77fe640 1 -- 172.21.15.37:0/3136592472 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fa9e0018020 con 0x7fa9f0103660 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 Processor -- start 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 -- start start 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b23ff090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.705 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b23ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b240a370 0x55a9b240c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a9b23652f0 con 0x55a9b240a370 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a9b230ccf0 con 0x55a9b2400b40 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.695+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a9b23612b0 con 0x55a9b23ff5d0 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b240a370 0x55a9b240c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b240a370 0x55a9b240c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58550/0 (socket says 172.21.15.37:58550) 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 -- 172.21.15.37:0/858409047 learned_addr learned my addr 172.21.15.37:0/858409047 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:29.706 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9e77fe640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9dc0a33b0 0x7fa9dc0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b23ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b23ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9dc0a33b0 0x7fa9dc0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 -- 172.21.15.37:0/858409047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 msgr2=0x55a9b23ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b23ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 -- 172.21.15.37:0/858409047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 msgr2=0x55a9b23ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.707 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b23ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 -- 172.21.15.37:0/858409047 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b22f13a0 con 0x55a9b240a370 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9e77fe640 1 -- 172.21.15.37:0/3136592472 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fa9e00cf550 con 0x7fa9f0103660 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b240a370 0x55a9b240c760 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa9e009b610 tx=0x7fa9e00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d5a91ab94b9e1f4 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9e4dea640 1 -- 172.21.15.37:0/858409047 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e009b270 con 0x55a9b240a370 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9dc0a33b0 0x7fa9dc0a5870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009ca0 tx=0x7fa9e802a950 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9e4dea640 1 -- 172.21.15.37:0/858409047 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e0010eb0 con 0x55a9b240a370 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.696+0000 7fa9e4dea640 1 -- 172.21.15.37:0/858409047 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0009200 con 0x55a9b240a370 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/858409047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b240a370 msgr2=0x55a9b240c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.708 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b240a370 0x55a9b240c760 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fa9e009b610 tx=0x7fa9e00047d0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/858409047 shutdown_connections 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b240a370 0x55a9b240c760 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b23ffa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/858409047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b23ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/858409047 >> 172.21.15.37:0/858409047 conn(0x55a9b23c8310 msgr2=0x55a9b23fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/858409047 shutdown_connections 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.697+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/858409047 wait complete. 2024-09-19T06:56:29.709 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 -- start start 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 Processor -- start 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b2409d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b2400b40 0x55a9b2407000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b240a370 0x55a9b24075d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x55a9b240cfb0 con 0x55a9b2400b40 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a9b2320060 con 0x55a9b240a370 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x55a9b240d130 con 0x55a9b23ff5d0 2024-09-19T06:56:29.710 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9fa6f5740 10 client.0 _ll_register_callbacks cb 0x55a9b20738c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9f58b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b2400b40 0x55a9b2407000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9f58b7640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b2400b40 0x55a9b2407000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58564/0 (socket says 172.21.15.37:58564) 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9f58b7640 1 -- 172.21.15.37:0/1670658744 learned_addr learned my addr 172.21.15.37:0/1670658744 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b2409d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b240a370 0x55a9b24075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.698+0000 7fa9f58b7640 1 -- 172.21.15.37:0/1670658744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 msgr2=0x55a9b2409d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b2409d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.711 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9f58b7640 1 -- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b240a370 msgr2=0x55a9b24075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b240a370 0x55a9b24075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9f58b7640 1 -- 172.21.15.37:0/1670658744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b20d63c0 con 0x55a9b2400b40 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b240a370 0x55a9b24075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b2400b40 0x55a9b2407000 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00982e0 tx=0x7fa9e0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24c3a1d8bc8a91b server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/1670658744 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0050ce0 con 0x55a9b2400b40 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/1670658744 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fa9e0097f80 con 0x55a9b2400b40 2024-09-19T06:56:29.712 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55a9b2372860 con 0x55a9b2400b40 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/1670658744 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa9e0009200 con 0x55a9b2400b40 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9fa6f5740 10 client.19530 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a9b240cfb0 con 0x55a9b2400b40 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.699+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/1670658744 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x55a9b2372860 con 0x55a9b2400b40 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/1670658744 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fa9e0097030 con 0x55a9b2400b40 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9fa6f5740 10 client.19530 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9fa6f5740 10 client.19530 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9fa6f5740 20 client.19530 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:29.713 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2419bb0 0x55a9b241bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9fa6f5740 4 client.19530 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:29.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b236cd70 con 0x55a9b2419bb0 2024-09-19T06:56:29.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.700+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2419bb0 0x55a9b241bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2419bb0 0x55a9b241bfa0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eaaa1d766243ea99 server_cookie=19baf2bb1a4078bf in_seq=0 out_seq=0 2024-09-19T06:56:29.714 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9d9ffb640 10 client.19530 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:56:29.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/1670658744 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55a9b236cd70 con 0x55a9b2419bb0 2024-09-19T06:56:29.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9d9ffb640 10 client.19530 handle_command_reply: tid=0 2024-09-19T06:56:29.716 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 client.19530 shutdown 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b2400b40 msgr2=0x55a9b2407000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b2400b40 0x55a9b2407000 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00982e0 tx=0x7fa9e0010740 comp rx=0 tx=0).stop 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 shutdown_connections 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2419bb0 0x55a9b241bfa0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b240a370 0x55a9b24075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b2400b40 0x55a9b2407000 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9d9ffb640 0 client.19530 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:56:29.717 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/1670658744 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b23ff5d0 0x55a9b2409d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.701+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 >> 172.21.15.37:0/1670658744 conn(0x55a9b23c8310 msgr2=0x55a9b2401990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:29.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.702+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 shutdown_connections 2024-09-19T06:56:29.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.702+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/1670658744 wait complete. 2024-09-19T06:56:29.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.702+0000 7fa9fa6f5740 20 client.19530 trim_cache size 0 max 16384 2024-09-19T06:56:29.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 Processor -- start 2024-09-19T06:56:29.718 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 -- start start 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2416250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b24216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 0x55a9b2421c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01b25d0 con 0x55a9b23ff5d0 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01afe40 con 0x55a9b2400b40 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01adcb0 con 0x55a9b240a370 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2416250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2416250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58568/0 (socket says 172.21.15.37:58568) 2024-09-19T06:56:29.719 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 -- 172.21.15.37:0/2681846912 learned_addr learned my addr 172.21.15.37:0/2681846912 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 -- 172.21.15.37:0/2681846912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 msgr2=0x55a9b2421c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/2681846912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 0x55a9b2421c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 -- 172.21.15.37:0/2681846912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 msgr2=0x55a9b24216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/2681846912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b24216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 -- 172.21.15.37:0/2681846912 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b2407940 con 0x55a9b23ff5d0 2024-09-19T06:56:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.715+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/2681846912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2416250 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec004700 tx=0x7fa9ec004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b70ccf7d77e3fbbb server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:29.720 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9dbfff640 1 -- 172.21.15.37:0/2681846912 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec003d20 con 0x55a9b23ff5d0 2024-09-19T06:56:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9dbfff640 1 -- 172.21.15.37:0/2681846912 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ec01e900 con 0x55a9b23ff5d0 2024-09-19T06:56:29.721 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9dbfff640 1 -- 172.21.15.37:0/2681846912 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec012c00 con 0x55a9b23ff5d0 2024-09-19T06:56:29.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/2681846912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 msgr2=0x55a9b2416250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.724 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/2681846912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2416250 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec004700 tx=0x7fa9ec004410 comp rx=0 tx=0).stop 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/2681846912 shutdown_connections 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/2681846912 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 0x55a9b2421c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/2681846912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b24216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/2681846912 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2416250 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/2681846912 >> 172.21.15.37:0/2681846912 conn(0x55a9b23c8310 msgr2=0x55a9b23fad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/2681846912 shutdown_connections 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.716+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/2681846912 wait complete. 2024-09-19T06:56:29.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 -- start start 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 Processor -- start 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2412bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b240def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 0x55a9b240e470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01ae300 con 0x55a9b23ff5d0 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01af770 con 0x55a9b2400b40 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f01ae9d0 con 0x55a9b240a370 2024-09-19T06:56:29.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9fa6f5740 10 client.0 _ll_register_callbacks cb 0x7fa9e00974e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9f60b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2412bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9f60b8640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2412bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58572/0 (socket says 172.21.15.37:58572) 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9f60b8640 1 -- 172.21.15.37:0/3619444307 learned_addr learned my addr 172.21.15.37:0/3619444307 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b240def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.717+0000 7fa9f68b9640 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 0x55a9b240e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9f60b8640 1 -- 172.21.15.37:0/3619444307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 msgr2=0x55a9b240e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 0x55a9b240e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9f60b8640 1 -- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 msgr2=0x55a9b240def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b240def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9f60b8640 1 -- 172.21.15.37:0/3619444307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b240eca0 con 0x55a9b23ff5d0 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9f60b8640 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2412bc0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009b10 tx=0x7fa9ec0128a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d3ff0572c9f16c server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/3619444307 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec01e620 con 0x55a9b23ff5d0 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/3619444307 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fa9ec003b20 con 0x55a9b23ff5d0 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9ec01e900 con 0x55a9b23ff5d0 2024-09-19T06:56:29.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/3619444307 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7fa9ec01ecc0 con 0x55a9b23ff5d0 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9fa6f5740 10 client.19536 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x55a9b2484750 con 0x55a9b23ff5d0 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/3619444307 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9ec01e900 con 0x55a9b23ff5d0 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.718+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/3619444307 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7fa9ec030070 con 0x55a9b23ff5d0 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 10 client.19536 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 10 client.19536 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 20 client.19536 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:29.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2484db0 0x55a9b24871d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:29.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 4 client.19536 mds_command: new command op to 24470 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:29.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 --> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9b22e9170 con 0x55a9b2484db0 2024-09-19T06:56:29.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2484db0 0x55a9b24871d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:29.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9f58b7640 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2484db0 0x55a9b24871d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e2e12a2fcdb03b16 server_cookie=263336ef3fead8a in_seq=0 out_seq=0 2024-09-19T06:56:29.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9d9ffb640 10 client.19536 ms_handle_connect on v2:172.21.15.96:6836/3655112770 2024-09-19T06:56:29.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9d9ffb640 1 -- 172.21.15.37:0/3619444307 <== mds.1 v2:172.21.15.96:6836/3655112770 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9b22e9170 con 0x55a9b2484db0 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9d9ffb640 10 client.19536 handle_command_reply: tid=0 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 1 client.19536 shutdown 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.719+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 msgr2=0x55a9b2412bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2412bc0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009b10 tx=0x7fa9ec0128a0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 shutdown_connections 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:6836/3655112770,v1:172.21.15.96:6837/3655112770] conn(0x55a9b2484db0 0x55a9b24871d0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x55a9b240a370 0x55a9b240e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a9b2400b40 0x55a9b240def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9d9ffb640 0 client.19536 ms_handle_reset on v2:172.21.15.96:6836/3655112770 2024-09-19T06:56:29.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 --2- 172.21.15.37:0/3619444307 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x55a9b23ff5d0 0x55a9b2412bc0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 >> 172.21.15.37:0/3619444307 conn(0x55a9b23c8310 msgr2=0x55a9b2401030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:29.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 shutdown_connections 2024-09-19T06:56:29.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 1 -- 172.21.15.37:0/3619444307 wait complete. 2024-09-19T06:56:29.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9fa6f5740 20 client.19536 trim_cache size 0 max 16384 2024-09-19T06:56:29.733 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:56:29.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9dc0a33b0 msgr2=0x7fa9dc0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.733 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9dc0a33b0 0x7fa9dc0a5870 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa9e8009ca0 tx=0x7fa9e802a950 comp rx=0 tx=0).stop 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f0103660 msgr2=0x7fa9f019ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f0103660 0x7fa9f019ac00 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fa9e000b3f0 tx=0x7fa9e000b8b0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 shutdown_connections 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9f010c170 0x7fa9f019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fa9dc0a33b0 0x7fa9dc0a5870 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fa9f0103660 0x7fa9f019ac00 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 --2- 172.21.15.37:0/3136592472 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa9f0102d40 0x7fa9f019a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:29.734 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 >> 172.21.15.37:0/3136592472 conn(0x7fa9f00fc7a0 msgr2=0x7fa9f00fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 shutdown_connections 2024-09-19T06:56:29.735 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:29.720+0000 7fa9f831c640 1 -- 172.21.15.37:0/3136592472 wait complete. 2024-09-19T06:56:30.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:30 smithi037 ceph-mon[31211]: pgmap v467: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.2 MiB/s wr, 180 op/s 2024-09-19T06:56:30.484 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-19T06:56:30.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:30 smithi096 ceph-mon[35004]: pgmap v467: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.2 MiB/s wr, 180 op/s 2024-09-19T06:56:30.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:30 smithi151 ceph-mon[35236]: pgmap v467: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 303 KiB/s rd, 5.2 MiB/s wr, 180 op/s 2024-09-19T06:56:30.738 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:31.218 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 -- 172.21.15.37:0/150428374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8100a60 msgr2=0x7f5bc8100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.345 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 --2- 172.21.15.37:0/150428374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8100a60 0x7f5bc8100ec0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009930 tx=0x7f5bb802a420 comp rx=0 tx=0).stop 2024-09-19T06:56:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 -- 172.21.15.37:0/150428374 shutdown_connections 2024-09-19T06:56:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 --2- 172.21.15.37:0/150428374 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc8101400 0x7f5bc810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 --2- 172.21.15.37:0/150428374 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8100a60 0x7f5bc8100ec0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 --2- 172.21.15.37:0/150428374 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8106a60 0x7f5bc8106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 -- 172.21.15.37:0/150428374 >> 172.21.15.37:0/150428374 conn(0x7f5bc80fc780 msgr2=0x7f5bc80feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 -- 172.21.15.37:0/150428374 shutdown_connections 2024-09-19T06:56:31.346 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 -- 172.21.15.37:0/150428374 wait complete. 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 Processor -- start 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.211+0000 7f5bd0196640 1 -- start start 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bd0196640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100a60 0x7f5bc80fcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bd0196640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc8101400 0x7f5bc80fd300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bd0196640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8106a60 0x7f5bc80fd840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bd0196640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8110e60 con 0x7f5bc8106a60 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bd0196640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8110ce0 con 0x7f5bc8100a60 2024-09-19T06:56:31.347 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bd0196640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc8110fe0 con 0x7f5bc8101400 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8106a60 0x7f5bc80fd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8106a60 0x7f5bc80fd840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58594/0 (socket says 172.21.15.37:58594) 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 -- 172.21.15.37:0/1968624490 learned_addr learned my addr 172.21.15.37:0/1968624490 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100a60 0x7f5bc80fcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bcd731640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc8101400 0x7f5bc80fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 -- 172.21.15.37:0/1968624490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc8101400 msgr2=0x7f5bc80fd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc8101400 0x7f5bc80fd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.348 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 -- 172.21.15.37:0/1968624490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100a60 msgr2=0x7f5bc80fcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100a60 0x7f5bc80fcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 -- 172.21.15.37:0/1968624490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bc81a6850 con 0x7f5bc8106a60 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bcd731640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc8101400 0x7f5bc80fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bce733640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8106a60 0x7f5bc80fd840 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000eaa0 tx=0x7f5bc000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.212+0000 7f5bbeffd640 1 -- 172.21.15.37:0/1968624490 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0004d20 con 0x7f5bc8106a60 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.213+0000 7f5bbeffd640 1 -- 172.21.15.37:0/1968624490 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bc0005020 con 0x7f5bc8106a60 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.213+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bc81a6b40 con 0x7f5bc8106a60 2024-09-19T06:56:31.349 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.213+0000 7f5bbeffd640 1 -- 172.21.15.37:0/1968624490 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00116d0 con 0x7f5bc8106a60 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.213+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bc81a7050 con 0x7f5bc8106a60 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bbeffd640 1 -- 172.21.15.37:0/1968624490 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f5bc00051d0 con 0x7f5bc8106a60 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bd256f740 1 Processor -- start 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bd256f740 1 -- start start 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d1090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82de760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.350 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bd256f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640d82372f0 con 0x5640d82d2b40 2024-09-19T06:56:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.215+0000 7f5bd256f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640d81decf0 con 0x5640d82dc370 2024-09-19T06:56:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bd256f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5640d82332b0 con 0x5640d82d15d0 2024-09-19T06:56:31.351 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bbeffd640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5bb40a33b0 0x7f5bb40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58606/0 (socket says 172.21.15.37:58606) 2024-09-19T06:56:31.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 -- 172.21.15.37:0/148331897 learned_addr learned my addr 172.21.15.37:0/148331897 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:31.359 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82de760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bce733640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5bb40a33b0 0x7f5bb40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bbeffd640 1 -- 172.21.15.37:0/1968624490 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f5bc00cff80 con 0x7f5bc8106a60 2024-09-19T06:56:31.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 -- 172.21.15.37:0/148331897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 msgr2=0x5640d82d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 -- 172.21.15.37:0/148331897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 msgr2=0x5640d82de760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.360 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82de760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 -- 172.21.15.37:0/148331897 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d81c33a0 con 0x5640d82d2b40 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bcd731640 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d1090 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x5640d82d2070 tx=0x7f5bb8031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f0a5f5927cb7266 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bce733640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5bb40a33b0 0x7f5bb40a5870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cd750 tx=0x7f5bc0000f40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.216+0000 7f5bb3fff640 1 -- 172.21.15.37:0/148331897 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8035050 con 0x5640d82d2b40 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bb3fff640 1 -- 172.21.15.37:0/148331897 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb8031950 con 0x5640d82d2b40 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bb3fff640 1 -- 172.21.15.37:0/148331897 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb80048e0 con 0x5640d82d2b40 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 -- 172.21.15.37:0/148331897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 msgr2=0x5640d82d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.361 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d1090 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x5640d82d2070 tx=0x7f5bb8031480 comp rx=0 tx=0).stop 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 -- 172.21.15.37:0/148331897 shutdown_connections 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82de760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 --2- 172.21.15.37:0/148331897 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d1090 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 -- 172.21.15.37:0/148331897 >> 172.21.15.37:0/148331897 conn(0x5640d829a310 msgr2=0x5640d82cdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.217+0000 7f5bd256f740 1 -- 172.21.15.37:0/148331897 shutdown_connections 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 -- 172.21.15.37:0/148331897 wait complete. 2024-09-19T06:56:31.362 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 -- start start 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 Processor -- start 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82dbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82d9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x5640d82df010 con 0x5640d82d2b40 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5640d81f2060 con 0x5640d82dc370 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x5640d82df190 con 0x5640d82d15d0 2024-09-19T06:56:31.363 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.218+0000 7f5bd256f740 10 client.0 _ll_register_callbacks cb 0x5640d7f458c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d8fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58612/0 (socket says 172.21.15.37:58612) 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 -- 172.21.15.37:0/99769668 learned_addr learned my addr 172.21.15.37:0/99769668 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcd731640 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82dbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcd731640 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82d9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 -- 172.21.15.37:0/99769668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 msgr2=0x5640d82dbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82dbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.364 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 -- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 msgr2=0x5640d82d9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82d9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 -- 172.21.15.37:0/99769668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d7fa83c0 con 0x5640d82d2b40 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d8fd0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4012750 tx=0x7f5bc4012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b0d5e72c96c88a server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bb17fa640 1 -- 172.21.15.37:0/99769668 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bc401f060 con 0x5640d82d2b40 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bb17fa640 1 -- 172.21.15.37:0/99769668 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f5bc4010dc0 con 0x5640d82d2b40 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb8031950 con 0x5640d82d2b40 2024-09-19T06:56:31.365 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bb17fa640 1 -- 172.21.15.37:0/99769668 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5bc4012e60 con 0x5640d82d2b40 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.219+0000 7f5bd256f740 10 client.19548 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5640d82eaaf0 con 0x5640d82d2b40 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bb17fa640 1 -- 172.21.15.37:0/99769668 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb8031950 con 0x5640d82d2b40 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bb17fa640 1 -- 172.21.15.37:0/99769668 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f5bc40226e0 con 0x5640d82d2b40 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bd256f740 10 client.19548 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bd256f740 10 client.19548 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bd256f740 20 client.19548 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:31.366 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bd256f740 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d82ec050 0x5640d82ee470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bd256f740 4 client.19548 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640d8244860 con 0x5640d82ec050 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.220+0000 7f5bcd731640 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d82ec050 0x5640d82ee470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.221+0000 7f5bb17fa640 10 client.19548 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.221+0000 7f5bcd731640 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d82ec050 0x5640d82ee470 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96be7d850d4dbd9d server_cookie=c65e3abb8f4e48aa in_seq=0 out_seq=0 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.221+0000 7f5bb17fa640 1 -- 172.21.15.37:0/99769668 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5640d8244860 con 0x5640d82ec050 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.221+0000 7f5bb17fa640 10 client.19548 handle_command_reply: tid=0 2024-09-19T06:56:31.367 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 client.19548 shutdown 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 msgr2=0x5640d82d8fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d8fd0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4012750 tx=0x7f5bc4012c10 comp rx=0 tx=0).stop 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 shutdown_connections 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d82ec050 0x5640d82ee470 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82dc370 0x5640d82d9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d2b40 0x5640d82d8fd0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 --2- 172.21.15.37:0/99769668 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82d15d0 0x5640d82dbd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.368 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bb17fa640 0 client.19548 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 >> 172.21.15.37:0/99769668 conn(0x5640d829a310 msgr2=0x5640d82d3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 shutdown_connections 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 1 -- 172.21.15.37:0/99769668 wait complete. 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.222+0000 7f5bd256f740 20 client.19548 trim_cache size 0 max 16384 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 Processor -- start 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 -- start start 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.369 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82d7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 0x5640d82f3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81b2350 con 0x5640d82d15d0 2024-09-19T06:56:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81afbc0 con 0x5640d82d2b40 2024-09-19T06:56:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bd256f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ada30 con 0x5640d82dc370 2024-09-19T06:56:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bcd731640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bcd731640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58616/0 (socket says 172.21.15.37:58616) 2024-09-19T06:56:31.370 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bcdf32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82d7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.235+0000 7f5bcd731640 1 -- 172.21.15.37:0/813523019 learned_addr learned my addr 172.21.15.37:0/813523019 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcd731640 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 0x5640d82f3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcd731640 1 -- 172.21.15.37:0/813523019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 msgr2=0x5640d82f3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcd731640 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 0x5640d82f3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcd731640 1 -- 172.21.15.37:0/813523019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 msgr2=0x5640d82d7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcd731640 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82d7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcd731640 1 -- 172.21.15.37:0/813523019 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640d82cc310 con 0x5640d82d15d0 2024-09-19T06:56:31.371 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bcd731640 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e82e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005460 tx=0x7f5bb802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bed0dac208700ba server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bb37fe640 1 -- 172.21.15.37:0/813523019 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8035050 con 0x5640d82d15d0 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bb37fe640 1 -- 172.21.15.37:0/813523019 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb804e070 con 0x5640d82d15d0 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.236+0000 7f5bb37fe640 1 -- 172.21.15.37:0/813523019 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8003950 con 0x5640d82d15d0 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 -- 172.21.15.37:0/813523019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 msgr2=0x5640d82e82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e82e0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8005460 tx=0x7f5bb802a420 comp rx=0 tx=0).stop 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 -- 172.21.15.37:0/813523019 shutdown_connections 2024-09-19T06:56:31.372 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 0x5640d82f3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82d7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 --2- 172.21.15.37:0/813523019 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e82e0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 -- 172.21.15.37:0/813523019 >> 172.21.15.37:0/813523019 conn(0x5640d829a310 msgr2=0x5640d82d78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 -- 172.21.15.37:0/813523019 shutdown_connections 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 -- 172.21.15.37:0/813523019 wait complete. 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 -- start start 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.237+0000 7f5bd256f740 1 Processor -- start 2024-09-19T06:56:31.373 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bd256f740 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 0x5640d82e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bd256f740 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ae080 con 0x5640d82d15d0 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bd256f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81af4f0 con 0x5640d82d2b40 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bd256f740 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f5bc81ae750 con 0x5640d82dc370 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bd256f740 10 client.0 _ll_register_callbacks cb 0x7f5bc400a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.374 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58622/0 (socket says 172.21.15.37:58622) 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 -- 172.21.15.37:0/3736624703 learned_addr learned my addr 172.21.15.37:0/3736624703 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 -- 172.21.15.37:0/3736624703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 msgr2=0x5640d82e01d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 0x5640d82e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 -- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 msgr2=0x5640d82dfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82dfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 -- 172.21.15.37:0/3736624703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640d82d4120 con 0x5640d82d15d0 2024-09-19T06:56:31.375 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bcd731640 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e4960 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8054040 tx=0x7f5bb8009850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e65abf940647db34 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.238+0000 7f5bb17fa640 10 client.0 ms_handle_connect on v2:172.21.15.37:3300/0 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3736624703 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8035070 con 0x5640d82d15d0 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3736624703 <== mon.0 v2:172.21.15.37:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f5bb8004030 con 0x5640d82d15d0 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5bb804e070 con 0x5640d82d15d0 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3736624703 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) ==== 24+0+0 (secure 0 0 0) 0x7f5bb80316f0 con 0x5640d82d15d0 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 10 client.19554 fetch_fsmap learned FSMap version 47 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) -- 0x5640d8356930 con 0x5640d82d15d0 2024-09-19T06:56:31.376 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3736624703 <== mon.0 v2:172.21.15.37:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bb804e070 con 0x5640d82d15d0 2024-09-19T06:56:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3736624703 <== mon.0 v2:172.21.15.37:3300/0 5 ==== fsmap(e 47) ==== 8358+0+0 (secure 0 0 0) 0x7f5bb803a480 con 0x5640d82d15d0 2024-09-19T06:56:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 10 client.19554 fetch_fsmap finished waiting for FSMap version 47 2024-09-19T06:56:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 10 client.19554 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-19T06:56:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 20 client.19554 populate_metadata read hostname 'smithi037' 2024-09-19T06:56:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d8356c60 0x5640d8359050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 4 client.19554 mds_command: new command op to 24497 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-19T06:56:31.377 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 --> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640d823ed70 con 0x5640d8356c60 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.239+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d8356c60 0x5640d8359050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bcdf32640 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d8356c60 0x5640d8359050 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea3e29b98566ab6 server_cookie=7ad7f5d18539094f in_seq=0 out_seq=0 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bb17fa640 10 client.19554 ms_handle_connect on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bb17fa640 1 -- 172.21.15.37:0/3736624703 <== mds.0 v2:172.21.15.96:6838/1066552330 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640d823ed70 con 0x5640d8356c60 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bb17fa640 10 client.19554 handle_command_reply: tid=0 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 client.19554 shutdown 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 msgr2=0x5640d82e4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e4960 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8054040 tx=0x7f5bb8009850 comp rx=0 tx=0).stop 2024-09-19T06:56:31.378 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 shutdown_connections 2024-09-19T06:56:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:6838/1066552330,v1:172.21.15.96:6839/1066552330] conn(0x5640d8356c60 0x5640d8359050 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x5640d82dc370 0x5640d82e01d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bb17fa640 0 client.19554 ms_handle_reset on v2:172.21.15.96:6838/1066552330 2024-09-19T06:56:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5640d82d2b40 0x5640d82dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 --2- 172.21.15.37:0/3736624703 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x5640d82d15d0 0x5640d82e4960 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.379 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 >> 172.21.15.37:0/3736624703 conn(0x5640d829a310 msgr2=0x5640d82d80c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 shutdown_connections 2024-09-19T06:56:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.240+0000 7f5bd256f740 1 -- 172.21.15.37:0/3736624703 wait complete. 2024-09-19T06:56:31.380 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd256f740 20 client.19554 trim_cache size 0 max 16384 2024-09-19T06:56:31.381 INFO:teuthology.orchestra.run.smithi037.stdout:[] 2024-09-19T06:56:31.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5bb40a33b0 msgr2=0x7f5bb40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5bb40a33b0 0x7f5bb40a5870 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5bc00cd750 tx=0x7f5bc0000f40 comp rx=0 tx=0).stop 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8106a60 msgr2=0x7f5bc80fd840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8106a60 0x7f5bc80fd840 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000eaa0 tx=0x7f5bc000eea0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 shutdown_connections 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f5bb40a33b0 0x7f5bb40a5870 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f5bc8106a60 0x7f5bc80fd840 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc8101400 0x7f5bc80fd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 --2- 172.21.15.37:0/1968624490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5bc8100a60 0x7f5bc80fcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:31.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 >> 172.21.15.37:0/1968624490 conn(0x7f5bc80fc780 msgr2=0x7f5bc806a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:31.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 shutdown_connections 2024-09-19T06:56:31.383 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:31.241+0000 7f5bd0196640 1 -- 172.21.15.37:0/1968624490 wait complete. 2024-09-19T06:56:31.944 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_6fad5dd73c93545e52f57f681e140a9c3529c11e/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-19T06:56:31.950 INFO:tasks.fwd_scrub:done joining 2024-09-19T06:56:31.950 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-19T06:56:31.960 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-19T06:56:32.068 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 -- 172.21.15.37:0/2698781249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8680100d90 msgr2=0x7f8680104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 --2- 172.21.15.37:0/2698781249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8680100d90 0x7f8680104d80 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f8674009900 tx=0x7f867402a410 comp rx=0 tx=0).stop 2024-09-19T06:56:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 -- 172.21.15.37:0/2698781249 shutdown_connections 2024-09-19T06:56:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 --2- 172.21.15.37:0/2698781249 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86801054e0 0x7f868010d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 --2- 172.21.15.37:0/2698781249 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f8680100d90 0x7f8680104d80 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 --2- 172.21.15.37:0/2698781249 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86801003e0 0x7f86801007c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 -- 172.21.15.37:0/2698781249 >> 172.21.15.37:0/2698781249 conn(0x7f86800771a0 msgr2=0x7f86800fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:32.069 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.067+0000 7f8688366640 1 -- 172.21.15.37:0/2698781249 shutdown_connections 2024-09-19T06:56:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 -- 172.21.15.37:0/2698781249 wait complete. 2024-09-19T06:56:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 Processor -- start 2024-09-19T06:56:32.070 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 -- start start 2024-09-19T06:56:32.072 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86801003e0 0x7f8680197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86801054e0 0x7f86801977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f868019bb50 0x7f868019c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f8680110da0 con 0x7f86801054e0 2024-09-19T06:56:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8680110c20 con 0x7f868019bb50 2024-09-19T06:56:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8688366640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f8680110f20 con 0x7f86801003e0 2024-09-19T06:56:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.068+0000 7f8685901640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86801054e0 0x7f86801977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.073 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86801054e0 0x7f86801977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58642/0 (socket says 172.21.15.37:58642) 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 -- 172.21.15.37:0/1446005001 learned_addr learned my addr 172.21.15.37:0/1446005001 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8686102640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86801003e0 0x7f8680197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 -- 172.21.15.37:0/1446005001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86801003e0 msgr2=0x7f8680197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86801003e0 0x7f8680197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 -- 172.21.15.37:0/1446005001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f868019bb50 msgr2=0x7f868019c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8686903640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f868019bb50 0x7f868019c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f868019bb50 0x7f868019c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.074 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 -- 172.21.15.37:0/1446005001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f868019c640 con 0x7f86801054e0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8685901640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86801054e0 0x7f86801977d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f8674005940 tx=0x7f8674004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f86737fe640 1 -- 172.21.15.37:0/1446005001 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f867403f030 con 0x7f86801054e0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f86737fe640 1 -- 172.21.15.37:0/1446005001 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8674003ad0 con 0x7f86801054e0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f868019c8d0 con 0x7f86801054e0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f86737fe640 1 -- 172.21.15.37:0/1446005001 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8674035070 con 0x7f86801054e0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.069+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86801a40f0 con 0x7f86801054e0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.070+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86801007c0 con 0x7f86801054e0 2024-09-19T06:56:32.075 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.071+0000 7f86737fe640 1 -- 172.21.15.37:0/1446005001 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f8674042030 con 0x7f86801054e0 2024-09-19T06:56:32.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.072+0000 7f86737fe640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f866c0a3440 0x7f866c0a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.072+0000 7f8686102640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f866c0a3440 0x7f866c0a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.072+0000 7f86737fe640 1 -- 172.21.15.37:0/1446005001 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f86740f02c0 con 0x7f86801054e0 2024-09-19T06:56:32.076 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.073+0000 7f8686102640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f866c0a3440 0x7f866c0a5900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f867c007350 tx=0x7f867c0072e0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:32.077 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.076+0000 7f86737fe640 1 -- 172.21.15.37:0/1446005001 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f86740bcc10 con 0x7f86801054e0 2024-09-19T06:56:32.258 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.258+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8680100d90 con 0x7f86801054e0 2024-09-19T06:56:32.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.259+0000 7f86737fe640 1 -- 172.21.15.37:0/1446005001 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+23264 (secure 0 0 0) 0x7f86740bc510 con 0x7f86801054e0 2024-09-19T06:56:32.260 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:56:32.260 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":107,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:56:09.906178+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3406854118":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/1436144832":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6801/871019432":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1644258429":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/711456420":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6800/871019432":"2024-09-20T06:41:46.581265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:56:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f866c0a3440 msgr2=0x7f866c0a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f866c0a3440 0x7f866c0a5900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f867c007350 tx=0x7f867c0072e0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86801054e0 msgr2=0x7f86801977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86801054e0 0x7f86801977d0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f8674005940 tx=0x7f8674004270 comp rx=0 tx=0).stop 2024-09-19T06:56:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 shutdown_connections 2024-09-19T06:56:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f868019bb50 0x7f868019c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.262 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f866c0a3440 0x7f866c0a5900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f86801054e0 0x7f86801977d0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 --2- 172.21.15.37:0/1446005001 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f86801003e0 0x7f8680197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 >> 172.21.15.37:0/1446005001 conn(0x7f86800771a0 msgr2=0x7f86801036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 shutdown_connections 2024-09-19T06:56:32.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.261+0000 7f8688366640 1 -- 172.21.15.37:0/1446005001 wait complete. 2024-09-19T06:56:32.273 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-19T06:35:14.622171+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-19T06:39:49.247361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-19T06:39:51.369694+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 25, 'snap_epoch': 106, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-19T06:56:32.274 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-19T06:56:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 -- 172.21.15.37:0/229171120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 msgr2=0x7f6ddc10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/229171120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc10d850 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4009930 tx=0x7f6dd402a420 comp rx=0 tx=0).stop 2024-09-19T06:56:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 -- 172.21.15.37:0/229171120 shutdown_connections 2024-09-19T06:56:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/229171120 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc10d850 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/229171120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ddc100980 0x7f6ddc100de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.381 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/229171120 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ddc106990 0x7f6ddc106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 -- 172.21.15.37:0/229171120 >> 172.21.15.37:0/229171120 conn(0x7f6ddc0779d0 msgr2=0x7f6ddc0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 -- 172.21.15.37:0/229171120 shutdown_connections 2024-09-19T06:56:32.382 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.380+0000 7f6de0a2c640 1 -- 172.21.15.37:0/229171120 wait complete. 2024-09-19T06:56:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 Processor -- start 2024-09-19T06:56:32.384 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 -- start start 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ddc100980 0x7f6ddc102550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc102a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ddc106990 0x7f6ddc102fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc110dd0 con 0x7f6ddc101320 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc110c50 con 0x7f6ddc106990 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6de0a2c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f6ddc110f50 con 0x7f6ddc100980 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc102a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.385 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc102a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58662/0 (socket says 172.21.15.37:58662) 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 -- 172.21.15.37:0/3599524847 learned_addr learned my addr 172.21.15.37:0/3599524847 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 -- 172.21.15.37:0/3599524847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ddc100980 msgr2=0x7f6ddc102550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dda59c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ddc100980 0x7f6ddc102550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ddc100980 0x7f6ddc102550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 -- 172.21.15.37:0/3599524847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ddc106990 msgr2=0x7f6ddc102fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ddc106990 0x7f6ddc102fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 -- 172.21.15.37:0/3599524847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ddc1a67f0 con 0x7f6ddc101320 2024-09-19T06:56:32.386 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.381+0000 7f6dd9d9b640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc102a90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000d950 tx=0x7f6dd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.382+0000 7f6dcb7fe640 1 -- 172.21.15.37:0/3599524847 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd0017030 con 0x7f6ddc101320 2024-09-19T06:56:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.382+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ddc1a6ae0 con 0x7f6ddc101320 2024-09-19T06:56:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.382+0000 7f6dcb7fe640 1 -- 172.21.15.37:0/3599524847 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6dd0004020 con 0x7f6ddc101320 2024-09-19T06:56:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.382+0000 7f6dcb7fe640 1 -- 172.21.15.37:0/3599524847 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dd00047a0 con 0x7f6ddc101320 2024-09-19T06:56:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.382+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ddc1a7030 con 0x7f6ddc101320 2024-09-19T06:56:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.383+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ddc100de0 con 0x7f6ddc101320 2024-09-19T06:56:32.387 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.384+0000 7f6dcb7fe640 1 -- 172.21.15.37:0/3599524847 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f6dd00041d0 con 0x7f6ddc101320 2024-09-19T06:56:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.385+0000 7f6dcb7fe640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6dc40a33c0 0x7f6dc40a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.385+0000 7f6dda59c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6dc40a33c0 0x7f6dc40a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.385+0000 7f6dcb7fe640 1 -- 172.21.15.37:0/3599524847 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f6dd00cf7e0 con 0x7f6ddc101320 2024-09-19T06:56:32.388 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.385+0000 7f6dda59c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6dc40a33c0 0x7f6dc40a5880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005fd0 tx=0x7f6dcc008230 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:32.389 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.389+0000 7f6dcb7fe640 1 -- 172.21.15.37:0/3599524847 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f6dd009c190 con 0x7f6ddc101320 2024-09-19T06:56:32.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:32 smithi096 ceph-mon[35004]: pgmap v468: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 3.6 MiB/s wr, 142 op/s 2024-09-19T06:56:32.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:32 smithi037 ceph-mon[31211]: pgmap v468: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 3.6 MiB/s wr, 142 op/s 2024-09-19T06:56:32.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.587+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6ddc1a7360 con 0x7f6ddc101320 2024-09-19T06:56:32.588 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.587+0000 7f6dcb7fe640 1 -- 172.21.15.37:0/3599524847 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) ==== 93+0+10 (secure 0 0 0) 0x7f6dd009ba90 con 0x7f6ddc101320 2024-09-19T06:56:32.589 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 1 2024-09-19T06:56:32.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6dc40a33c0 msgr2=0x7f6dc40a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.590 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6dc40a33c0 0x7f6dc40a5880 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc005fd0 tx=0x7f6dcc008230 comp rx=0 tx=0).stop 2024-09-19T06:56:32.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 msgr2=0x7f6ddc102a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc102a90 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f6dd000d950 tx=0x7f6dd000de10 comp rx=0 tx=0).stop 2024-09-19T06:56:32.591 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 shutdown_connections 2024-09-19T06:56:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6ddc106990 0x7f6ddc102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f6dc40a33c0 0x7f6dc40a5880 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f6ddc101320 0x7f6ddc102a90 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 --2- 172.21.15.37:0/3599524847 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6ddc100980 0x7f6ddc102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.590+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 >> 172.21.15.37:0/3599524847 conn(0x7f6ddc0779d0 msgr2=0x7f6ddc0693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.591+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 shutdown_connections 2024-09-19T06:56:32.592 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.591+0000 7f6de0a2c640 1 -- 172.21.15.37:0/3599524847 wait complete. 2024-09-19T06:56:32.610 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-19T06:56:32.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:32 smithi151 ceph-mon[35236]: pgmap v468: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 3.6 MiB/s wr, 142 op/s 2024-09-19T06:56:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 -- 172.21.15.37:0/1747737349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a010c0d0 msgr2=0x7f05a010e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 --2- 172.21.15.37:0/1747737349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a010c0d0 0x7f05a010e550 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f0598009ca0 tx=0x7f059802a950 comp rx=0 tx=0).stop 2024-09-19T06:56:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 -- 172.21.15.37:0/1747737349 shutdown_connections 2024-09-19T06:56:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 --2- 172.21.15.37:0/1747737349 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a010c0d0 0x7f05a010e550 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.725 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 --2- 172.21.15.37:0/1747737349 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a01035a0 0x7f05a010ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 --2- 172.21.15.37:0/1747737349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05a0102c80 0x7f05a0103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 -- 172.21.15.37:0/1747737349 >> 172.21.15.37:0/1747737349 conn(0x7f05a00fc690 msgr2=0x7f05a00feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:32.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 -- 172.21.15.37:0/1747737349 shutdown_connections 2024-09-19T06:56:32.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.722+0000 7f05a834b640 1 -- 172.21.15.37:0/1747737349 wait complete. 2024-09-19T06:56:32.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 Processor -- start 2024-09-19T06:56:32.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 -- start start 2024-09-19T06:56:32.726 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05a0102c80 0x7f05a01008d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a01035a0 0x7f05a0100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a010c0d0 0x7f05a0101350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0111ec0 con 0x7f05a01035a0 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0111d40 con 0x7f05a0102c80 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a834b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f05a0112040 con 0x7f05a010c0d0 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a58e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a01035a0 0x7f05a0100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a58e6640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a01035a0 0x7f05a0100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58684/0 (socket says 172.21.15.37:58684) 2024-09-19T06:56:32.727 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a58e6640 1 -- 172.21.15.37:0/793191126 learned_addr learned my addr 172.21.15.37:0/793191126 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a60e7640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05a0102c80 0x7f05a01008d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.723+0000 7f05a68e8640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a010c0d0 0x7f05a0101350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a58e6640 1 -- 172.21.15.37:0/793191126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a010c0d0 msgr2=0x7f05a0101350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.728 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a58e6640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a010c0d0 0x7f05a0101350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a58e6640 1 -- 172.21.15.37:0/793191126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05a0102c80 msgr2=0x7f05a01008d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a58e6640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05a0102c80 0x7f05a01008d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a58e6640 1 -- 172.21.15.37:0/793191126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a01a68b0 con 0x7f05a01035a0 2024-09-19T06:56:32.729 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a58e6640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a01035a0 0x7f05a0100e10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f059000d740 tx=0x7f059000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05977fe640 1 -- 172.21.15.37:0/793191126 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0590017070 con 0x7f05a01035a0 2024-09-19T06:56:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05977fe640 1 -- 172.21.15.37:0/793191126 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0590010460 con 0x7f05a01035a0 2024-09-19T06:56:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a01a6ba0 con 0x7f05a01035a0 2024-09-19T06:56:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05977fe640 1 -- 172.21.15.37:0/793191126 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059000f620 con 0x7f05a01035a0 2024-09-19T06:56:32.730 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.724+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a01a70f0 con 0x7f05a01035a0 2024-09-19T06:56:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.725+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a0103060 con 0x7f05a01035a0 2024-09-19T06:56:32.731 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.726+0000 7f05977fe640 1 -- 172.21.15.37:0/793191126 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f0590010610 con 0x7f05a01035a0 2024-09-19T06:56:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.727+0000 7f05977fe640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f058c0a33c0 0x7f058c0a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.728+0000 7f05a60e7640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f058c0a33c0 0x7f058c0a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.728+0000 7f05977fe640 1 -- 172.21.15.37:0/793191126 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f05900cf7c0 con 0x7f05a01035a0 2024-09-19T06:56:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.728+0000 7f05a60e7640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f058c0a33c0 0x7f058c0a5880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f059c00abe0 tx=0x7f059c00a2d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:32.732 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.731+0000 7f05977fe640 1 -- 172.21.15.37:0/793191126 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f059009c170 con 0x7f05a01035a0 2024-09-19T06:56:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.907+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f05a01a72c0 con 0x7f05a01035a0 2024-09-19T06:56:32.908 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.907+0000 7f05977fe640 1 -- 172.21.15.37:0/793191126 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) ==== 104+0+11 (secure 0 0 0) 0x7f059009ba70 con 0x7f05a01035a0 2024-09-19T06:56:32.908 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 32 2024-09-19T06:56:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f058c0a33c0 msgr2=0x7f058c0a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f058c0a33c0 0x7f058c0a5880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f059c00abe0 tx=0x7f059c00a2d0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a01035a0 msgr2=0x7f05a0100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:32.910 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a01035a0 0x7f05a0100e10 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f059000d740 tx=0x7f059000db40 comp rx=0 tx=0).stop 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 shutdown_connections 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f05a010c0d0 0x7f05a0101350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f058c0a33c0 0x7f058c0a5880 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f05a01035a0 0x7f05a0100e10 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 --2- 172.21.15.37:0/793191126 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f05a0102c80 0x7f05a01008d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 >> 172.21.15.37:0/793191126 conn(0x7f05a00fc690 msgr2=0x7f05a00fdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 shutdown_connections 2024-09-19T06:56:32.911 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:32.909+0000 7f05a834b640 1 -- 172.21.15.37:0/793191126 wait complete. 2024-09-19T06:56:32.921 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-19T06:56:33.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.027+0000 7f5771aa5640 1 -- 172.21.15.37:0/2191508311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c1069c0 msgr2=0x7f576c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.027+0000 7f5771aa5640 1 --2- 172.21.15.37:0/2191508311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c1069c0 0x7f576c106da0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f575c009930 tx=0x7f575c02a420 comp rx=0 tx=0).stop 2024-09-19T06:56:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 -- 172.21.15.37:0/2191508311 shutdown_connections 2024-09-19T06:56:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 --2- 172.21.15.37:0/2191508311 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576c101350 0x7f576c10d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 --2- 172.21.15.37:0/2191508311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576c1009b0 0x7f576c100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 --2- 172.21.15.37:0/2191508311 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c1069c0 0x7f576c106da0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 -- 172.21.15.37:0/2191508311 >> 172.21.15.37:0/2191508311 conn(0x7f576c0fc690 msgr2=0x7f576c0feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 -- 172.21.15.37:0/2191508311 shutdown_connections 2024-09-19T06:56:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 -- 172.21.15.37:0/2191508311 wait complete. 2024-09-19T06:56:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 Processor -- start 2024-09-19T06:56:33.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 -- start start 2024-09-19T06:56:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576c1009b0 0x7f576c102580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c101350 0x7f576c102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.028+0000 7f5771aa5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576c1069c0 0x7f576c103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5771aa5640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f576c110e00 con 0x7f576c101350 2024-09-19T06:56:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5771aa5640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f576c110c80 con 0x7f576c1069c0 2024-09-19T06:56:33.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5771aa5640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f576c110f80 con 0x7f576c1009b0 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c101350 0x7f576c102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c101350 0x7f576c102ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58696/0 (socket says 172.21.15.37:58696) 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 -- 172.21.15.37:0/4064362234 learned_addr learned my addr 172.21.15.37:0/4064362234 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f576bfff640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576c1069c0 0x7f576c103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f576b7fe640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576c1009b0 0x7f576c102580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 -- 172.21.15.37:0/4064362234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576c1009b0 msgr2=0x7f576c102580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576c1009b0 0x7f576c102580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 -- 172.21.15.37:0/4064362234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576c1069c0 msgr2=0x7f576c103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576c1069c0 0x7f576c103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 -- 172.21.15.37:0/4064362234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f576c1a6940 con 0x7f576c101350 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5763fff640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c101350 0x7f576c102ac0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f575400d950 tx=0x7f575400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f57697fa640 1 -- 172.21.15.37:0/4064362234 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5754017070 con 0x7f576c101350 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f576c1a6c30 con 0x7f576c101350 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f57697fa640 1 -- 172.21.15.37:0/4064362234 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5754009c30 con 0x7f576c101350 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f57697fa640 1 -- 172.21.15.37:0/4064362234 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f575400f6b0 con 0x7f576c101350 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.029+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f576c1a71d0 con 0x7f576c101350 2024-09-19T06:56:33.034 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.030+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f576c100e10 con 0x7f576c101350 2024-09-19T06:56:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.032+0000 7f57697fa640 1 -- 172.21.15.37:0/4064362234 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f57540109d0 con 0x7f576c101350 2024-09-19T06:56:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.032+0000 7f57697fa640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f57580a33c0 0x7f57580a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.035 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.032+0000 7f57697fa640 1 -- 172.21.15.37:0/4064362234 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f57540cfa10 con 0x7f576c101350 2024-09-19T06:56:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.032+0000 7f576b7fe640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f57580a33c0 0x7f57580a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.036 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.033+0000 7f576b7fe640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f57580a33c0 0x7f57580a5880 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f575c0096c0 tx=0x7f575c000bc0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.037 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.036+0000 7f57697fa640 1 -- 172.21.15.37:0/4064362234 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f5754009de0 con 0x7f576c101350 2024-09-19T06:56:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.212+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f576c1a73a0 con 0x7f576c101350 2024-09-19T06:56:33.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.212+0000 7f57697fa640 1 -- 172.21.15.37:0/4064362234 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) ==== 100+0+11 (secure 0 0 0) 0x7f575409c3c0 con 0x7f576c101350 2024-09-19T06:56:33.213 INFO:teuthology.orchestra.run.smithi037.stdout:pg_num: 64 2024-09-19T06:56:33.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f57580a33c0 msgr2=0x7f57580a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f57580a33c0 0x7f57580a5880 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f575c0096c0 tx=0x7f575c000bc0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c101350 msgr2=0x7f576c102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.215 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c101350 0x7f576c102ac0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f575400d950 tx=0x7f575400de10 comp rx=0 tx=0).stop 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 shutdown_connections 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f576c1069c0 0x7f576c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f57580a33c0 0x7f57580a5880 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f576c101350 0x7f576c102ac0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 --2- 172.21.15.37:0/4064362234 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f576c1009b0 0x7f576c102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 >> 172.21.15.37:0/4064362234 conn(0x7f576c0fc690 msgr2=0x7f576c06a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 shutdown_connections 2024-09-19T06:56:33.216 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.214+0000 7f5771aa5640 1 -- 172.21.15.37:0/4064362234 wait complete. 2024-09-19T06:56:33.226 DEBUG:teuthology.orchestra.run.smithi037:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-19T06:56:33.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 -- 172.21.15.37:0/2423296003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c1080d0 msgr2=0x7f7d5c10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.331 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 --2- 172.21.15.37:0/2423296003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c1080d0 0x7f7d5c10e9d0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f7d54009ca0 tx=0x7f7d5402a8c0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 -- 172.21.15.37:0/2423296003 shutdown_connections 2024-09-19T06:56:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 --2- 172.21.15.37:0/2423296003 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c1080d0 0x7f7d5c10e9d0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 --2- 172.21.15.37:0/2423296003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d5c101230 0x7f7d5c107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 --2- 172.21.15.37:0/2423296003 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5c100880 0x7f7d5c100c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 -- 172.21.15.37:0/2423296003 >> 172.21.15.37:0/2423296003 conn(0x7f7d5c075f60 msgr2=0x7f7d5c076370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 -- 172.21.15.37:0/2423296003 shutdown_connections 2024-09-19T06:56:33.332 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 -- 172.21.15.37:0/2423296003 wait complete. 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 Processor -- start 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.329+0000 7f7d62f98640 1 -- start start 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d62f98640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100880 0x7f7d5c198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d62f98640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5c1080d0 0x7f7d5c1988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d62f98640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d5c19cc50 0x7f7d5c19d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d62f98640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c111ee0 con 0x7f7d5c100880 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d62f98640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c111d60 con 0x7f7d5c19cc50 2024-09-19T06:56:33.333 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d62f98640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f7d5c112060 con 0x7f7d5c1080d0 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100880 0x7f7d5c198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100880 0x7f7d5c198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:58720/0 (socket says 172.21.15.37:58720) 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 -- 172.21.15.37:0/1689363924 learned_addr learned my addr 172.21.15.37:0/1689363924 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d61535640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d5c19cc50 0x7f7d5c19d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d5bfff640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5c1080d0 0x7f7d5c1988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 -- 172.21.15.37:0/1689363924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5c1080d0 msgr2=0x7f7d5c1988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5c1080d0 0x7f7d5c1988d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.334 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 -- 172.21.15.37:0/1689363924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d5c19cc50 msgr2=0x7f7d5c19d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d5c19cc50 0x7f7d5c19d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 -- 172.21.15.37:0/1689363924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d5c19d740 con 0x7f7d5c100880 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d60d34640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100880 0x7f7d5c198390 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e9f0 tx=0x7f7d4c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d59ffb640 1 -- 172.21.15.37:0/1689363924 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00cd80 con 0x7f7d5c100880 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d5c19da30 con 0x7f7d5c100880 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d59ffb640 1 -- 172.21.15.37:0/1689363924 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7d4c005500 con 0x7f7d5c100880 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.330+0000 7f7d59ffb640 1 -- 172.21.15.37:0/1689363924 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c010690 con 0x7f7d5c100880 2024-09-19T06:56:33.335 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.331+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d5c1a51d0 con 0x7f7d5c100880 2024-09-19T06:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.332+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d5c100c60 con 0x7f7d5c100880 2024-09-19T06:56:33.336 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.333+0000 7f7d59ffb640 1 -- 172.21.15.37:0/1689363924 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f7d4c0046c0 con 0x7f7d5c100880 2024-09-19T06:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.333+0000 7f7d59ffb640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d440a33c0 0x7f7d440a5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.334+0000 7f7d59ffb640 1 -- 172.21.15.37:0/1689363924 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f7d4c0d0170 con 0x7f7d5c100880 2024-09-19T06:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.334+0000 7f7d5bfff640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d440a33c0 0x7f7d440a5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:33.337 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.334+0000 7f7d5bfff640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d440a33c0 0x7f7d440a5880 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c1000c0 tx=0x7f7d5000b410 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:33.338 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.337+0000 7f7d59ffb640 1 -- 172.21.15.37:0/1689363924 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f7d4c09cb20 con 0x7f7d5c100880 2024-09-19T06:56:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:33 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1446005001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:56:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:33 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3599524847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:56:33.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:33 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/793191126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:56:33.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:33 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1446005001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:56:33.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:33 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3599524847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:56:33.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:33 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/793191126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:56:33.562 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.561+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f7d5c1996e0 con 0x7f7d5c100880 2024-09-19T06:56:33.569 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.569+0000 7f7d59ffb640 1 -- 172.21.15.37:0/1689363924 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+742873 (secure 0 0 0) 0x7f7d4c09c420 con 0x7f7d5c100880 2024-09-19T06:56:33.579 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:56:33.579 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:56:33.580 INFO:teuthology.orchestra.run.smithi037.stdout: "epoch": 26, 2024-09-19T06:56:33.580 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.580 INFO:teuthology.orchestra.run.smithi037.stdout: "active_gid": 15156, 2024-09-19T06:56:33.580 INFO:teuthology.orchestra.run.smithi037.stdout: "active_name": "x", 2024-09-19T06:56:33.580 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addrs": { 2024-09-19T06:56:33.580 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-09-19T06:56:33.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6800", 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3483159058 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v1", 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:6801", 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3483159058 2024-09-19T06:56:33.581 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: "active_addr": "172.21.15.37:6801/3483159058", 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: "active_change": "2024-09-19T06:41:46.581493+0000", 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: "active_mgr_features": 4540701547738038271, 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: "available": true, 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: "standbys": [ 2024-09-19T06:56:33.582 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24784, 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "z", 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.583 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.584 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.585 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.586 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.587 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.587 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.587 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.587 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.587 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.587 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.590 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.591 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.591 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.591 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.591 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.593 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.593 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.595 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-09-19T06:56:33.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-09-19T06:56:33.595 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-09-19T06:56:33.596 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.597 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-09-19T06:56:33.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-09-19T06:56:33.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.597 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-09-19T06:56:33.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.598 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-09-19T06:56:33.598 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.598 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.599 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-09-19T06:56:33.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.601 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-09-19T06:56:33.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-09-19T06:56:33.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.602 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.602 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-09-19T06:56:33.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.603 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.603 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.603 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.603 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-09-19T06:56:33.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-09-19T06:56:33.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.604 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-09-19T06:56:33.604 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.604 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.604 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-09-19T06:56:33.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.605 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.605 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.605 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-09-19T06:56:33.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T06:56:33.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.609 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-09-19T06:56:33.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-09-19T06:56:33.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.610 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.611 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-09-19T06:56:33.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:56:33.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-09-19T06:56:33.617 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.618 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-09-19T06:56:33.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-09-19T06:56:33.621 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-09-19T06:56:33.621 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-09-19T06:56:33.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T06:56:33.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.623 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T06:56:33.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:56:33.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.623 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-09-19T06:56:33.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.626 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.626 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.629 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.629 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.629 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.630 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.630 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.630 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.630 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.630 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.631 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.632 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.632 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.632 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.632 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.633 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T06:56:33.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.637 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.637 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.637 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.638 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.639 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-09-19T06:56:33.639 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-09-19T06:56:33.639 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-09-19T06:56:33.639 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-09-19T06:56:33.639 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-09-19T06:56:33.639 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.640 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-09-19T06:56:33.640 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.640 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.640 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-09-19T06:56:33.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-09-19T06:56:33.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.641 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.641 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.641 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.641 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T06:56:33.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.642 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.642 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.642 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.642 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:33.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:33.643 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.643 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.643 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.643 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:33.644 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.644 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.644 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.644 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.645 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T06:56:33.646 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:33.647 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.648 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-09-19T06:56:33.649 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T06:56:33.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-09-19T06:56:33.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T06:56:33.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.657 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-09-19T06:56:33.660 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-09-19T06:56:33.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.665 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.665 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.665 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-09-19T06:56:33.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-09-19T06:56:33.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T06:56:33.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-09-19T06:56:33.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.670 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-09-19T06:56:33.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-09-19T06:56:33.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T06:56:33.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.672 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-09-19T06:56:33.672 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.672 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.673 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-09-19T06:56:33.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-09-19T06:56:33.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.673 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T06:56:33.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:33.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.675 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-09-19T06:56:33.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-09-19T06:56:33.676 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T06:56:33.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.677 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.677 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-09-19T06:56:33.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.678 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-09-19T06:56:33.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-09-19T06:56:33.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.679 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.679 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T06:56:33.679 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-09-19T06:56:33.685 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-09-19T06:56:33.686 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-09-19T06:56:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-09-19T06:56:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T06:56:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-09-19T06:56:33.692 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-09-19T06:56:33.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T06:56:33.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-09-19T06:56:33.695 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.696 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T06:56:33.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.701 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-09-19T06:56:33.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T06:56:33.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-09-19T06:56:33.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.707 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.708 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-09-19T06:56:33.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-09-19T06:56:33.708 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.708 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.708 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.709 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.710 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-09-19T06:56:33.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-09-19T06:56:33.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.710 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.712 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-09-19T06:56:33.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-09-19T06:56:33.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.712 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:33.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-09-19T06:56:33.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.714 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.714 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-09-19T06:56:33.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-09-19T06:56:33.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.715 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.715 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.715 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T06:56:33.716 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.716 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.716 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.716 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.716 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.717 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.718 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.719 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T06:56:33.723 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.725 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.726 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-09-19T06:56:33.730 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-09-19T06:56:33.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-09-19T06:56:33.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:33.732 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-09-19T06:56:33.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-09-19T06:56:33.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.738 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.738 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-09-19T06:56:33.738 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-09-19T06:56:33.738 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.738 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T06:56:33.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.739 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-09-19T06:56:33.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-09-19T06:56:33.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T06:56:33.740 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.741 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-09-19T06:56:33.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-09-19T06:56:33.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T06:56:33.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.743 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-09-19T06:56:33.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-09-19T06:56:33.744 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.744 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.745 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T06:56:33.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.745 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.746 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-09-19T06:56:33.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-09-19T06:56:33.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T06:56:33.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.748 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-09-19T06:56:33.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-09-19T06:56:33.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.749 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T06:56:33.750 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.750 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.750 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.750 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.751 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-09-19T06:56:33.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-09-19T06:56:33.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.751 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.751 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.751 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.751 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.752 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.756 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-09-19T06:56:33.757 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.758 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-09-19T06:56:33.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T06:56:33.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.763 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.764 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.765 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.770 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.771 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.771 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.771 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.771 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.772 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.772 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.772 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.772 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.772 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.772 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-09-19T06:56:33.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.779 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.780 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T06:56:33.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T06:56:33.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:33.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.781 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T06:56:33.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T06:56:33.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.783 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.784 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T06:56:33.784 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T06:56:33.784 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.785 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.785 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.785 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.785 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.786 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.786 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.786 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-09-19T06:56:33.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T06:56:33.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.787 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.787 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.788 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.788 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T06:56:33.788 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.789 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.790 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.791 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.791 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.791 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.791 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.791 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.792 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T06:56:33.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T06:56:33.793 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.793 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.793 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.793 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.793 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.793 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T06:56:33.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.798 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.799 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.800 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T06:56:33.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.803 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.804 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.805 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T06:56:33.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.809 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.810 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.811 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.811 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.811 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.813 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.813 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.814 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.815 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T06:56:33.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T06:56:33.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.817 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.818 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.819 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T06:56:33.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T06:56:33.821 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.821 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.821 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.822 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T06:56:33.823 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T06:56:33.823 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.823 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.823 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.823 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.824 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.824 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.825 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.825 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.825 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.825 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T06:56:33.825 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T06:56:33.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.826 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.826 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.826 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.826 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.826 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.827 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.829 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.829 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T06:56:33.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T06:56:33.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.832 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T06:56:33.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.838 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T06:56:33.839 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.842 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.843 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.844 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-09-19T06:56:33.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.845 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T06:56:33.846 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.847 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:33.849 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.849 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.849 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.851 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.852 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.852 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.852 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.852 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T06:56:33.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T06:56:33.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.853 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:33.854 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T06:56:33.855 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.856 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.857 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.857 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T06:56:33.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T06:56:33.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:33.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.859 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-09-19T06:56:33.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-09-19T06:56:33.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.861 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.861 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.861 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.861 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.863 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-09-19T06:56:33.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-09-19T06:56:33.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-09-19T06:56:33.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.866 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.867 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.868 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.869 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.871 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.872 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.873 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.874 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.875 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.876 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.877 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-09-19T06:56:33.878 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.879 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.881 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:33.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:33.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-09-19T06:56:33.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-09-19T06:56:33.883 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.885 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.885 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.885 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-09-19T06:56:33.885 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.886 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.887 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.887 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.887 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.887 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.887 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-09-19T06:56:33.888 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-09-19T06:56:33.888 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.888 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.888 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.888 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-09-19T06:56:33.888 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.888 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.889 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.889 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-09-19T06:56:33.889 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-09-19T06:56:33.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.892 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-09-19T06:56:33.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-09-19T06:56:33.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.893 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.893 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.894 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.894 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.894 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.894 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.894 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.895 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-09-19T06:56:33.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-09-19T06:56:33.896 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.896 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.896 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.896 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.897 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.899 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.900 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.903 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.904 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.905 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-09-19T06:56:33.906 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-09-19T06:56:33.907 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T06:56:33.908 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.909 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.910 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:33.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-09-19T06:56:33.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-09-19T06:56:33.916 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.916 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.916 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-09-19T06:56:33.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.917 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.918 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.918 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-09-19T06:56:33.918 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.918 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.918 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.918 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.920 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.922 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.922 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.922 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.922 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.922 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.923 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.923 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.923 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.923 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-09-19T06:56:33.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.929 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-09-19T06:56:33.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-09-19T06:56:33.930 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.930 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.930 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.930 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-09-19T06:56:33.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.931 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.932 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:33.933 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:33.933 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.933 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:33.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-09-19T06:56:33.935 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T06:56:33.936 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.937 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:33.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.943 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.947 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.948 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.949 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-09-19T06:56:33.950 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-09-19T06:56:33.951 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.952 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.953 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.954 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-09-19T06:56:33.954 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-09-19T06:56:33.954 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.954 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.954 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.955 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.956 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.956 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-09-19T06:56:33.956 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-09-19T06:56:33.956 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.956 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.957 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.957 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:33.957 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:33.957 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-09-19T06:56:33.957 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T06:56:33.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.958 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-09-19T06:56:33.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-09-19T06:56:33.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.959 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-09-19T06:56:33.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.960 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.960 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.960 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.960 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-09-19T06:56:33.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-09-19T06:56:33.961 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.961 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.961 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.961 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-09-19T06:56:33.961 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.961 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:56:33.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.962 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.963 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.963 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.963 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-09-19T06:56:33.963 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.963 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.963 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.963 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.965 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.966 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.967 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.968 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.969 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.970 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.971 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.972 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.972 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.972 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.972 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.973 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.973 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.973 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-09-19T06:56:33.973 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.975 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.976 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.977 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.977 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.977 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.977 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.977 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.977 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.979 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.984 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-09-19T06:56:33.985 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.986 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.987 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.990 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.990 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.990 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.990 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.990 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:33.992 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:33.993 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:33.993 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:33.993 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:33.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.994 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:33.995 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.995 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.995 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.995 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:33.995 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:33.995 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.995 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.996 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:33.996 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.996 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:33.996 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-09-19T06:56:33.996 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.997 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-09-19T06:56:33.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:33.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:33.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:33.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-09-19T06:56:33.998 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T06:56:33.998 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:33.998 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:33.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:33.999 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:33.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:33.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:33.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:33.999 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:33.999 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.000 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.000 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.000 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.001 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.001 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.001 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.001 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.001 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.002 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.002 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.002 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.002 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.003 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.003 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.003 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.004 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.005 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.005 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.008 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.008 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.008 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.008 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.009 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-09-19T06:56:34.009 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-09-19T06:56:34.009 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.009 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.010 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.011 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.011 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-09-19T06:56:34.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-09-19T06:56:34.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-09-19T06:56:34.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.015 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.016 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.017 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.018 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.019 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.020 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.021 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.023 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.024 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.026 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.028 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.029 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.031 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.031 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.032 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.033 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.033 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.033 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.033 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.033 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.033 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.033 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.034 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.034 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.034 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.035 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.035 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.035 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.035 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.035 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.035 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.036 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.037 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-09-19T06:56:34.037 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.037 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.037 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.037 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.037 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.038 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.038 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.038 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.038 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.039 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.040 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.040 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.042 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.043 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.043 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.043 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.043 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.044 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.048 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.048 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.048 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.048 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-09-19T06:56:34.048 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.048 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.049 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.051 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.052 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.053 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.054 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.055 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.056 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-09-19T06:56:34.058 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-09-19T06:56:34.059 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.060 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.062 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.063 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.064 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.065 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.065 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.066 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.067 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.067 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.067 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.067 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.067 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.067 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.070 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-09-19T06:56:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.072 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.073 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.073 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.073 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.073 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.073 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.073 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.074 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.075 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.076 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.076 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.076 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.076 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.077 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.077 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.077 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.078 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.078 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.078 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.078 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.078 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.079 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.080 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.081 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.082 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.083 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.083 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.083 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.083 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.083 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.085 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.086 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.087 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.088 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.089 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.090 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.091 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-09-19T06:56:34.092 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.093 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-09-19T06:56:34.094 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.095 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.096 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.097 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.098 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.101 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.102 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.104 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.104 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.104 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.105 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.106 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.106 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-09-19T06:56:34.106 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-09-19T06:56:34.106 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.106 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.106 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.108 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:34.108 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:34.108 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.108 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.108 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.108 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:34.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.109 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-09-19T06:56:34.109 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.109 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.109 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.110 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.110 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.110 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.110 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-09-19T06:56:34.110 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.110 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.111 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.112 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.112 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.113 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-09-19T06:56:34.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-09-19T06:56:34.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.114 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T06:56:34.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T06:56:34.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.115 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.115 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.115 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.116 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.116 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.116 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.116 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.116 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.120 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.126 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-09-19T06:56:34.127 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.128 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-09-19T06:56:34.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.134 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-09-19T06:56:34.135 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-09-19T06:56:34.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-09-19T06:56:34.138 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.139 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.140 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.141 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.141 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.141 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.141 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.141 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.141 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.141 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.142 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.142 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.142 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.144 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.144 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.145 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.145 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.145 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.145 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.145 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.146 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.146 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.146 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.146 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.148 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.148 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.148 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.148 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.148 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.149 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.149 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.149 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-09-19T06:56:34.149 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-09-19T06:56:34.149 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.150 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.151 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.152 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.153 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.154 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.158 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.158 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.158 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.158 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.158 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.159 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.159 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.159 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.159 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.159 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.160 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.160 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.160 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.161 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.161 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.161 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.161 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.161 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.161 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.165 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.166 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.167 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:34.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.172 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.173 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.174 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.174 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.174 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.174 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.175 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.178 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.183 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T06:56:34.183 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.183 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.183 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.183 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.184 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-09-19T06:56:34.184 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.184 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.184 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.185 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.185 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.185 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.185 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.185 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.186 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.187 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.187 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.187 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.187 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.187 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.188 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.189 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.189 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.189 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.190 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.190 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.192 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.193 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.197 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.198 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.199 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.200 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.201 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.202 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.203 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.204 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.205 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-09-19T06:56:34.206 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.207 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.209 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.214 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.214 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-09-19T06:56:34.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-09-19T06:56:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-09-19T06:56:34.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-09-19T06:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.220 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.220 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.220 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.220 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.220 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.220 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-09-19T06:56:34.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-09-19T06:56:34.221 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.221 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.221 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.221 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.221 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:34.221 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-09-19T06:56:34.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.222 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.222 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.222 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.222 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.222 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.222 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-09-19T06:56:34.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-09-19T06:56:34.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.223 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.223 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.223 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.223 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.223 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.224 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.224 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.224 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.225 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-09-19T06:56:34.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-09-19T06:56:34.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.225 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.227 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.227 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.227 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-09-19T06:56:34.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-09-19T06:56:34.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.229 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.230 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-09-19T06:56:34.231 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-09-19T06:56:34.232 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.233 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.234 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.238 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-09-19T06:56:34.241 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.242 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.243 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.246 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.246 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.246 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.246 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.247 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.248 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.248 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.248 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.248 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.248 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.248 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.248 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.249 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.249 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.249 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.249 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.250 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.251 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.251 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-09-19T06:56:34.251 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.251 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.253 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.254 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.255 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.256 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.257 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.260 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.260 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.260 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.261 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.261 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.261 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.261 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.261 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-09-19T06:56:34.261 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.262 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.262 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.262 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-09-19T06:56:34.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-09-19T06:56:34.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.263 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.264 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.264 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:34.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:34.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.265 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.265 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-09-19T06:56:34.265 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.265 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.265 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.267 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.267 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.267 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.267 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.267 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.268 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.268 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.268 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.268 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.268 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.273 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.274 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.275 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-09-19T06:56:34.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T06:56:34.278 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.279 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.280 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.281 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.285 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-09-19T06:56:34.286 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-09-19T06:56:34.286 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.286 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.286 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.286 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.286 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.287 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.287 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.287 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.287 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.287 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-09-19T06:56:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-09-19T06:56:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T06:56:34.288 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-09-19T06:56:34.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-09-19T06:56:34.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.291 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.292 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:34.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:34.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.292 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-09-19T06:56:34.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-09-19T06:56:34.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.293 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.294 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-09-19T06:56:34.295 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.296 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.297 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-09-19T06:56:34.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-09-19T06:56:34.298 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.299 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.299 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.299 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.300 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.300 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.300 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.300 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.301 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.301 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.301 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.301 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.302 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.302 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.302 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.302 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.302 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.303 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.303 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.303 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.303 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.303 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.306 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.307 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.308 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.309 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.310 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.313 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.314 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.315 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.316 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.317 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.319 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.320 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.320 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.320 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.320 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.320 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.321 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.321 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.321 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.321 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.321 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.321 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.322 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.323 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.323 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.323 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-09-19T06:56:34.323 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.323 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.323 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.324 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.325 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.325 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.325 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.325 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.325 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.325 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.329 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.330 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.331 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.332 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-09-19T06:56:34.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T06:56:34.333 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.334 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-09-19T06:56:34.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-09-19T06:56:34.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.335 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.335 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.335 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-09-19T06:56:34.336 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.336 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.336 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.336 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-09-19T06:56:34.337 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-09-19T06:56:34.337 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.337 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.337 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.337 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.338 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.339 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.340 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.341 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.342 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.343 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.344 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.347 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.348 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.349 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.350 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-09-19T06:56:34.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.357 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-09-19T06:56:34.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-09-19T06:56:34.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.358 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T06:56:34.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.359 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:34.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.360 INFO:teuthology.orchestra.run.smithi037.stdout: "gid": 24833, 2024-09-19T06:56:34.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "y", 2024-09-19T06:56:34.360 INFO:teuthology.orchestra.run.smithi037.stdout: "mgr_features": 4540701547738038271, 2024-09-19T06:56:34.360 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-09-19T06:56:34.360 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-09-19T06:56:34.361 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.361 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.361 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.361 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:34.361 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:34.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.362 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:34.362 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.362 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.362 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T06:56:34.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.363 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.363 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.364 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.365 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.365 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.365 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.365 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.365 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.365 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.366 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.366 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.366 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.366 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.366 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.367 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.367 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.367 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.369 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.370 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.370 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.370 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.370 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.370 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.370 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.371 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.371 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.371 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.371 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.371 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.371 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.375 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.376 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.377 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.378 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.379 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.380 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.381 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.382 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.383 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.384 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-09-19T06:56:34.385 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.386 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-09-19T06:56:34.387 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.388 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.389 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.390 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.391 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:34.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-09-19T06:56:34.392 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-09-19T06:56:34.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.392 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:34.393 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-09-19T06:56:34.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:34.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-09-19T06:56:34.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T06:56:34.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.396 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T06:56:34.396 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T06:56:34.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T06:56:34.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.399 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-09-19T06:56:34.399 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-09-19T06:56:34.399 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-09-19T06:56:34.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.400 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.400 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.400 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T06:56:34.400 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:56:34.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.401 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.401 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-09-19T06:56:34.401 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-09-19T06:56:34.401 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:34.401 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.401 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.402 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.403 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.404 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.406 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.406 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.406 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.407 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.409 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.409 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.409 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.409 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.409 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.409 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.410 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.411 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-09-19T06:56:34.412 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.413 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.414 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-09-19T06:56:34.415 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-09-19T06:56:34.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.417 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.418 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.419 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:34.420 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-09-19T06:56:34.421 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-09-19T06:56:34.423 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.424 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.425 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.426 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-09-19T06:56:34.427 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-09-19T06:56:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.428 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.429 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.430 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.431 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-09-19T06:56:34.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-09-19T06:56:34.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T06:56:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-09-19T06:56:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T06:56:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-09-19T06:56:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T06:56:34.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.436 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-09-19T06:56:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-09-19T06:56:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-09-19T06:56:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T06:56:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.437 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-09-19T06:56:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-09-19T06:56:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.438 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T06:56:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.439 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-09-19T06:56:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-09-19T06:56:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.440 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T06:56:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.441 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.442 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-09-19T06:56:34.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.448 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.449 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.450 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-09-19T06:56:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-09-19T06:56:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T06:56:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-09-19T06:56:34.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-09-19T06:56:34.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-09-19T06:56:34.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-09-19T06:56:34.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.463 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.464 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-09-19T06:56:34.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-09-19T06:56:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T06:56:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-09-19T06:56:34.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T06:56:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.469 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.470 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T06:56:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-09-19T06:56:34.471 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-09-19T06:56:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-09-19T06:56:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.472 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.473 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T06:56:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-09-19T06:56:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-09-19T06:56:34.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T06:56:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-09-19T06:56:34.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.478 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.479 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-09-19T06:56:34.480 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-09-19T06:56:34.481 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.482 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.483 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.484 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-09-19T06:56:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-09-19T06:56:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T06:56:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T06:56:34.486 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-09-19T06:56:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-09-19T06:56:34.487 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-09-19T06:56:34.488 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.489 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.490 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.491 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.493 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.494 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.496 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T06:56:34.497 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.498 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.499 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.501 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.502 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T06:56:34.503 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.504 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-09-19T06:56:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-09-19T06:56:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T06:56:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-09-19T06:56:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.510 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.511 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-09-19T06:56:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-09-19T06:56:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T06:56:34.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-09-19T06:56:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-09-19T06:56:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.513 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.514 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-09-19T06:56:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-09-19T06:56:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.515 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T06:56:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T06:56:34.516 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-09-19T06:56:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-09-19T06:56:34.517 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-09-19T06:56:34.519 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.520 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.521 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.522 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-09-19T06:56:34.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-09-19T06:56:34.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-09-19T06:56:34.527 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.528 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.529 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.530 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T06:56:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-09-19T06:56:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-09-19T06:56:34.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T06:56:34.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.534 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.535 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.536 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.537 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.538 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.539 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.542 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.543 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.544 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.545 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.546 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-09-19T06:56:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-09-19T06:56:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.549 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-09-19T06:56:34.549 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.549 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T06:56:34.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.550 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.550 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.550 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.550 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-09-19T06:56:34.550 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-09-19T06:56:34.550 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-09-19T06:56:34.551 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.551 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.551 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.551 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T06:56:34.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.552 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.552 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.553 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.554 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.555 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.556 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.556 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.556 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.556 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.556 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T06:56:34.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.559 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.560 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T06:56:34.561 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.562 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.564 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.565 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T06:56:34.566 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.567 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.568 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.569 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.571 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.572 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.573 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.574 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.576 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.576 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.576 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.576 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.576 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T06:56:34.576 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T06:56:34.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.578 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.578 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T06:56:34.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T06:56:34.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.580 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.580 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.580 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-09-19T06:56:34.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-09-19T06:56:34.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.583 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.583 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:34.583 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.583 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.584 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T06:56:34.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T06:56:34.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.586 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T06:56:34.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T06:56:34.588 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.588 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.588 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.588 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.589 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.590 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T06:56:34.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T06:56:34.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.591 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T06:56:34.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T06:56:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.593 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T06:56:34.594 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.599 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.600 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.601 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.602 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.603 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.604 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.605 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.605 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.605 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.605 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.606 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T06:56:34.607 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.609 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.610 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.610 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.610 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.610 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.610 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.610 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.610 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.611 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.612 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T06:56:34.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T06:56:34.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-09-19T06:56:34.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.616 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T06:56:34.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T06:56:34.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.617 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.617 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-09-19T06:56:34.617 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.618 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T06:56:34.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T06:56:34.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.619 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.620 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T06:56:34.621 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T06:56:34.621 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.621 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.621 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.621 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.622 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.623 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T06:56:34.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T06:56:34.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.623 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.623 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.623 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.623 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.624 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.624 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.625 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T06:56:34.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T06:56:34.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.626 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.627 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.627 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.627 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.627 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.627 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T06:56:34.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.632 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.633 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-09-19T06:56:34.633 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-09-19T06:56:34.633 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.633 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.633 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.633 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.633 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.634 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.636 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.636 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.636 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.636 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.636 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-09-19T06:56:34.636 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-09-19T06:56:34.636 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-09-19T06:56:34.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.639 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-09-19T06:56:34.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.641 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.642 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.649 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.649 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.649 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.649 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.649 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.653 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.654 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:34.655 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.656 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.657 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.658 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.659 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.659 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-09-19T06:56:34.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-09-19T06:56:34.659 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.659 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.659 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-09-19T06:56:34.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.660 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.660 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.661 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-09-19T06:56:34.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-09-19T06:56:34.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-09-19T06:56:34.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.662 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-09-19T06:56:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-09-19T06:56:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-09-19T06:56:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.664 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-09-19T06:56:34.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-09-19T06:56:34.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-09-19T06:56:34.665 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.666 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-09-19T06:56:34.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.667 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.671 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.672 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.673 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.674 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.675 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.676 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.677 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.678 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-09-19T06:56:34.679 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T06:56:34.680 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.681 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.682 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.683 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-09-19T06:56:34.684 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T06:56:34.685 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.688 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.688 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:34.688 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:34.688 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.688 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:34.689 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.689 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.689 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.689 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T06:56:34.689 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.690 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.690 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.690 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.690 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-09-19T06:56:34.690 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-09-19T06:56:34.690 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.691 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.691 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.691 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-09-19T06:56:34.691 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.691 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.691 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T06:56:34.692 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.692 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.692 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.692 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.692 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.692 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.692 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.694 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.697 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.697 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.697 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.697 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.698 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.698 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.698 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.698 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.698 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.699 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.700 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.700 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.700 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.700 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.700 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.701 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.702 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.703 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-09-19T06:56:34.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.706 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.708 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-09-19T06:56:34.710 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-09-19T06:56:34.710 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-09-19T06:56:34.710 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.710 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-09-19T06:56:34.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-09-19T06:56:34.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-09-19T06:56:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.712 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.713 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.714 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.715 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.718 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.723 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.725 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.726 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-09-19T06:56:34.726 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-09-19T06:56:34.726 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.726 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.727 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-09-19T06:56:34.727 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.728 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-09-19T06:56:34.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-09-19T06:56:34.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.728 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.729 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.730 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.730 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.730 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-09-19T06:56:34.730 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-09-19T06:56:34.730 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.730 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.730 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-09-19T06:56:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-09-19T06:56:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.732 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:34.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:34.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-09-19T06:56:34.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.733 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T06:56:34.733 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.733 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.733 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.734 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-09-19T06:56:34.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.735 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.735 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.736 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-09-19T06:56:34.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-09-19T06:56:34.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.736 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-09-19T06:56:34.736 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:56:34.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.737 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.738 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.739 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.739 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.739 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.739 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.739 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.739 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.740 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.741 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.741 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.744 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.745 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.746 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.748 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.748 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.749 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.750 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.753 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.754 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.757 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-09-19T06:56:34.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T06:56:34.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.762 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.763 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.766 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.767 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.767 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.767 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.767 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.767 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.767 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.768 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.769 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.770 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.771 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.772 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.773 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.774 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.775 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.776 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.778 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.779 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.780 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.780 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.780 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.780 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.780 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.780 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-09-19T06:56:34.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.784 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-09-19T06:56:34.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.785 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-09-19T06:56:34.785 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.785 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.785 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.785 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-09-19T06:56:34.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T06:56:34.786 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.787 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.788 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-09-19T06:56:34.789 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.792 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T06:56:34.792 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T06:56:34.792 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.792 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.793 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.795 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.796 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.796 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.796 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.798 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.799 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.800 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.802 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.803 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.804 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.808 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.808 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.808 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.808 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.809 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.810 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.810 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.811 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.812 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.812 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.813 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.813 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-09-19T06:56:34.813 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.813 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.813 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.813 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.815 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.815 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.815 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.815 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.815 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.816 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.816 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.816 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.816 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.817 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.817 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.817 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.817 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.817 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.818 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.818 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.818 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.818 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.818 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.818 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.818 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.819 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.820 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.823 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-09-19T06:56:34.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.827 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.828 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.829 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.830 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.831 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.831 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.832 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.833 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.834 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-09-19T06:56:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-09-19T06:56:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.835 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-09-19T06:56:34.836 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-09-19T06:56:34.837 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.837 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-09-19T06:56:34.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.839 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.839 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.839 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.839 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.840 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.840 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.840 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.840 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.840 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.840 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.840 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.841 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.841 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.841 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.841 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.841 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.841 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.842 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.842 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.842 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.844 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.844 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.845 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.846 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.846 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.846 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.847 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.847 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.847 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.847 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.847 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.848 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.848 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.848 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.848 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.849 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.850 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.851 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.852 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.853 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.853 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.853 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.853 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.853 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.853 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.853 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.857 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.858 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.860 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-09-19T06:56:34.861 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.861 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.861 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.861 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.862 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.864 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.865 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.867 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.868 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.870 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.871 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:34.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:34.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.871 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.873 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-09-19T06:56:34.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-09-19T06:56:34.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.874 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.874 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.874 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-09-19T06:56:34.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-09-19T06:56:34.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.874 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-09-19T06:56:34.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T06:56:34.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.875 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.875 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.876 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.876 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.876 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-09-19T06:56:34.876 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.876 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.876 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.877 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-09-19T06:56:34.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-09-19T06:56:34.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.877 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.877 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T06:56:34.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.879 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.879 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-09-19T06:56:34.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-09-19T06:56:34.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.880 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.880 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.880 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.880 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.881 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.881 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.881 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.881 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.882 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.883 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.884 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.885 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.886 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.887 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.888 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.889 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.890 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-09-19T06:56:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-09-19T06:56:34.891 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.892 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-09-19T06:56:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.893 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.894 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-09-19T06:56:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-09-19T06:56:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.895 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.897 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.898 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.899 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.900 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.901 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.902 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.903 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.904 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.905 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.906 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.907 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.908 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.909 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T06:56:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T06:56:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-09-19T06:56:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.910 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.911 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.912 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:34.913 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T06:56:34.914 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.915 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-09-19T06:56:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T06:56:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.916 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.917 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.918 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.919 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.920 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-09-19T06:56:34.921 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.922 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.923 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.924 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.925 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.926 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.927 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.928 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.929 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.930 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.931 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.932 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.933 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.934 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.935 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-09-19T06:56:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-09-19T06:56:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.936 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.937 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-09-19T06:56:34.938 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.939 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-09-19T06:56:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-09-19T06:56:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.940 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.941 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.942 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.943 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.944 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.945 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.946 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.947 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.948 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.949 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.950 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.951 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.953 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.955 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.956 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.957 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.958 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.959 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.960 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.961 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.962 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.963 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.964 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T06:56:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.965 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.966 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.967 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.968 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.969 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.970 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.971 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.972 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.973 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.974 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.975 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.976 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:34.977 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.978 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.979 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:34.980 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.981 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.982 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:34.983 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.984 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:34.985 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.986 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.987 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-09-19T06:56:34.988 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.989 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-09-19T06:56:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-09-19T06:56:34.990 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.991 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.992 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.993 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.994 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.994 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-09-19T06:56:34.994 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-09-19T06:56:34.996 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:34.996 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.996 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.996 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.996 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.996 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-09-19T06:56:34.997 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:34.998 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:34.999 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.000 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.000 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.000 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.000 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.000 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.000 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.000 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.001 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.002 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.003 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.004 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.005 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.006 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.006 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.006 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.006 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.006 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.006 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-09-19T06:56:35.006 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.007 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.008 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.009 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.010 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.011 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-09-19T06:56:35.012 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.013 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.014 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.015 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.016 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.016 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.016 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.016 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.016 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.016 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.016 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.017 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.018 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.019 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.020 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.021 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.021 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.021 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.021 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.021 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.021 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.021 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.022 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-09-19T06:56:35.023 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.024 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.025 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.026 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.026 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.026 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.026 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.026 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.026 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.026 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.027 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.028 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.029 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.030 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.031 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.032 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.032 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.032 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.032 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.032 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.032 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.032 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.033 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.034 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.035 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.036 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.037 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.037 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.037 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.037 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.037 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.037 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.037 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.038 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.039 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.040 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.041 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.041 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.041 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.041 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.041 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.041 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.041 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.042 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-09-19T06:56:35.043 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-09-19T06:56:35.044 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.044 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.044 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.044 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T06:56:35.044 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.044 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.044 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.045 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.046 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.047 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.048 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.049 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.049 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.049 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.049 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.049 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.050 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.051 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.052 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.053 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.054 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.055 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.056 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.057 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.058 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.058 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.058 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T06:56:35.058 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.058 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.058 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.058 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.059 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-09-19T06:56:35.060 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.061 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.062 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.063 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.064 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.064 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.064 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-09-19T06:56:35.064 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-09-19T06:56:35.064 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.064 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.064 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.065 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.066 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.067 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.067 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.067 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.067 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.067 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.067 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.067 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.068 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-09-19T06:56:35.069 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.070 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.071 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.072 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.073 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.074 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.075 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-09-19T06:56:35.076 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.077 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.078 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.079 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.080 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.081 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.081 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.081 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.081 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.081 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.081 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.081 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.082 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.083 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.084 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.085 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.086 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.086 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.086 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.086 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.086 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.086 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.086 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.087 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.088 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.089 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-09-19T06:56:35.090 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.091 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.092 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.093 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.094 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.095 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.096 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.097 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.098 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.099 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.100 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.101 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.102 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.103 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.104 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.105 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.106 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.107 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.108 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.109 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.110 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.111 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.112 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.113 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.114 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.115 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.116 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.117 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.118 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.119 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-09-19T06:56:35.120 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.121 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.122 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.123 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:35.124 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.125 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.125 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.125 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.125 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.125 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.125 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.125 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.126 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.127 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.128 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.129 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.130 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.131 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.132 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.133 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.134 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.134 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.134 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.134 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.134 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-09-19T06:56:35.134 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-09-19T06:56:35.134 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.135 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.136 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.137 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.138 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-09-19T06:56:35.138 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-09-19T06:56:35.138 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.138 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.138 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.138 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T06:56:35.138 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.139 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.140 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.140 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.140 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.140 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.140 INFO:teuthology.orchestra.run.smithi037.stdout: "modules": [ 2024-09-19T06:56:35.140 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-09-19T06:56:35.140 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard", 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: "iostat", 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: "nfs", 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: "restful", 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: "snap_schedule" 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: "available_modules": [ 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.141 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "alerts", 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.142 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How frequently to reexamine health status", 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.143 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.144 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.144 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.144 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.144 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.144 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.144 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.144 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.145 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.145 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.145 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.145 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.145 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.145 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.145 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.146 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.147 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.148 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.149 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.149 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.149 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.149 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.149 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.149 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.150 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.150 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.150 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.150 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.150 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.150 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.150 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.151 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.152 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_destination": { 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_destination", 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.153 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email address to send alerts to", 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.154 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_from_name": { 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_from_name", 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "Ceph", 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.155 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Email From: name", 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_host": { 2024-09-19T06:56:35.156 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_host", 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.157 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP server", 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_password": { 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_password", 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.158 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Password to authenticate with", 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.159 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_port": { 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_port", 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.160 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "465", 2024-09-19T06:56:35.161 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.161 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.161 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.161 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP port", 2024-09-19T06:56:35.161 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.161 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.161 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_sender": { 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_sender", 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.162 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SMTP envelope sender", 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_ssl": { 2024-09-19T06:56:35.163 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_ssl", 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.164 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: "smtp_user": { 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "smtp_user", 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.165 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "User to authenticate as", 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.166 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "balancer", 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.167 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "active": { 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "active", 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.168 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically balance PGs across cluster", 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.169 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_time": { 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_time", 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0000", 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.170 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "beginning time of day to automatically balance", 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: "begin_weekday": { 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "begin_weekday", 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:35.171 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:56:35.172 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_max_iterations": { 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_max_iterations", 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.173 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "25", 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "250", 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.174 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_metrics": { 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_metrics", 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "pgs,objects,bytes", 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.175 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: "crush_compat_step": { 2024-09-19T06:56:35.176 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crush_compat_step", 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.5", 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0.001", 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "0.999", 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.177 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "aggressiveness of optimization", 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: "end_time": { 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_time", 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.178 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2359", 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ending time of day to automatically balance", 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.179 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: "end_weekday": { 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "end_weekday", 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:35.180 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "0", 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "6", 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.181 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.182 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.183 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.184 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.185 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.186 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.187 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.188 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.189 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: "min_score": { 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_score", 2024-09-19T06:56:35.190 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-19T06:56:35.191 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.192 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "upmap", 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "crush-compat", 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "none", 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "read", 2024-09-19T06:56:35.193 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap", 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap-read" 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Balancer mode", 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.194 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_ids": { 2024-09-19T06:56:35.195 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_ids", 2024-09-19T06:56:35.195 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.195 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.195 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.195 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.195 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.195 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:35.196 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:35.197 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.197 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.197 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.197 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:35.200 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.200 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.200 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.200 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-19T06:56:35.200 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.200 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.200 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_deviation": { 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_deviation", 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:35.201 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.202 INFO:teuthology.orchestra.run.smithi037.stdout: "upmap_max_optimizations": { 2024-09-19T06:56:35.203 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "upmap_max_optimizations", 2024-09-19T06:56:35.203 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:35.203 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.203 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.203 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.203 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.203 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.204 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm", 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_down_multiplier": { 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_down_multiplier", 2024-09-19T06:56:35.205 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-19T06:56:35.206 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_refresh_rate": { 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_refresh_rate", 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.207 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "20", 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.208 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: "agent_starting_port": { 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "agent_starting_port", 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4721", 2024-09-19T06:56:35.209 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.210 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_ptrace": { 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_ptrace", 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.211 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_interval": { 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_interval", 2024-09-19T06:56:35.212 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.213 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.213 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.213 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:35.213 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.213 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.213 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.213 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: "autotune_memory_target_ratio": { 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "autotune_memory_target_ratio", 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:35.214 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0.7", 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.215 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm_log_destination": { 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cephadm_log_destination", 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.216 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "file", 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "file,syslog", 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "syslog" 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-19T06:56:35.217 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "cgroups_split": { 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cgroups_split", 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.218 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.219 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "config_checks_enabled": { 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_checks_enabled", 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.220 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "config_dashboard": { 2024-09-19T06:56:35.221 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_dashboard", 2024-09-19T06:56:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.222 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_alertmanager": { 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_alertmanager", 2024-09-19T06:56:35.223 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:35.224 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_base": { 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_base", 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.225 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Container image name, without the tag", 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.226 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_elasticsearch": { 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_elasticsearch", 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.227 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "elasticsearch container image", 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_grafana": { 2024-09-19T06:56:35.228 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_grafana", 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.229 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_haproxy": { 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_haproxy", 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.230 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "HAproxy container image", 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.231 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_agent": { 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_agent", 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-19T06:56:35.232 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger agent container image", 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.233 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_collector": { 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_collector", 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.234 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger collector container image", 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_jaeger_query": { 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_jaeger_query", 2024-09-19T06:56:35.235 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Jaeger query container image", 2024-09-19T06:56:35.236 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_keepalived": { 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_keepalived", 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.237 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Keepalived container image", 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.238 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_loki": { 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_loki", 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.239 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Loki container image", 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_node_exporter": { 2024-09-19T06:56:35.240 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_node_exporter", 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.241 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_nvmeof": { 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_nvmeof", 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.242 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Nvme-of container image", 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.243 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_prometheus": { 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_prometheus", 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.244 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Prometheus container image", 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.245 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_promtail": { 2024-09-19T06:56:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_promtail", 2024-09-19T06:56:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-19T06:56:35.246 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Promtail container image", 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.247 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_samba": { 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_samba", 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.248 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Samba/SMB container image", 2024-09-19T06:56:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "container_image_snmp_gateway": { 2024-09-19T06:56:35.249 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_image_snmp_gateway", 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.250 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "SNMP Gateway container image", 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "container_init": { 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "container_init", 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.251 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Run podman/docker with `--init`", 2024-09-19T06:56:35.252 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "daemon_cache_timeout": { 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "daemon_cache_timeout", 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.253 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.254 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_cephadm_command_timeout": { 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_cephadm_command_timeout", 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "900", 2024-09-19T06:56:35.255 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.256 INFO:teuthology.orchestra.run.smithi037.stdout: "default_registry": { 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "default_registry", 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "docker.io", 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.257 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "device_cache_timeout": { 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_cache_timeout", 2024-09-19T06:56:35.258 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1800", 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache device inventory", 2024-09-19T06:56:35.259 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "device_enhanced_scan": { 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_enhanced_scan", 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.260 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-19T06:56:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.261 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "facts_cache_timeout": { 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "facts_cache_timeout", 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:35.262 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "seconds to cache host facts data", 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.263 INFO:teuthology.orchestra.run.smithi037.stdout: "grafana_dashboards_path": { 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "grafana_dashboards_path", 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.264 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "host_check_interval": { 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "host_check_interval", 2024-09-19T06:56:35.265 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to perform a host check", 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.266 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "hw_monitoring": { 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hw_monitoring", 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.267 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.268 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "inventory_list_all": { 2024-09-19T06:56:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "inventory_list_all", 2024-09-19T06:56:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.269 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.270 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.271 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.272 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "log_refresh_metadata": { 2024-09-19T06:56:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_refresh_metadata", 2024-09-19T06:56:35.273 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-19T06:56:35.274 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.275 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-19T06:56:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.276 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.277 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.278 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.279 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.280 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-19T06:56:35.281 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-19T06:56:35.282 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.283 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "*", 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.284 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max_count_per_host": { 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_count_per_host", 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.285 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of daemons per service per host", 2024-09-19T06:56:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.286 INFO:teuthology.orchestra.run.smithi037.stdout: "max_osd_draining_count": { 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_osd_draining_count", 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.287 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "migration_current": { 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "migration_current", 2024-09-19T06:56:35.288 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "internal - do not modify", 2024-09-19T06:56:35.289 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "mode": { 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mode", 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.290 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "root", 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm-package", 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "root" 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "mode for remote execution of cephadm", 2024-09-19T06:56:35.291 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "oob_default_addr": { 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "oob_default_addr", 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.292 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "169.254.1.1", 2024-09-19T06:56:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-19T06:56:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.293 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "prometheus_alerts_path": { 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus_alerts_path", 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.294 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.295 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_insecure": { 2024-09-19T06:56:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_insecure", 2024-09-19T06:56:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.296 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_password": { 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_password", 2024-09-19T06:56:35.297 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-19T06:56:35.298 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_url": { 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_url", 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.299 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-19T06:56:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.300 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "registry_username": { 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "registry_username", 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.301 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.302 INFO:teuthology.orchestra.run.smithi037.stdout: "secure_monitoring_stack": { 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secure_monitoring_stack", 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.303 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "service_discovery_port": { 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "service_discovery_port", 2024-09-19T06:56:35.304 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8765", 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "cephadm service discovery port", 2024-09-19T06:56:35.305 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "ssh_config_file": { 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssh_config_file", 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.306 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.307 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "use_agent": { 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_agent", 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.308 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.309 INFO:teuthology.orchestra.run.smithi037.stdout: "use_repo_digest": { 2024-09-19T06:56:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "use_repo_digest", 2024-09-19T06:56:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.310 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_failed_host_check": { 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_failed_host_check", 2024-09-19T06:56:35.311 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.312 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if the host check fails", 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_daemons": { 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_daemons", 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.313 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.314 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_on_stray_hosts": { 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_on_stray_hosts", 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.315 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-19T06:56:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.316 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crash", 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.317 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.318 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.319 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.320 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.321 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.322 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.323 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.324 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.325 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: "retain_interval": { 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retain_interval", 2024-09-19T06:56:35.326 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "31536000", 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.327 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_recent_interval": { 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_recent_interval", 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.328 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1209600", 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.329 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dashboard", 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.330 INFO:teuthology.orchestra.run.smithi037.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.331 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_HOST": { 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-19T06:56:35.332 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.333 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.334 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.335 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_ENABLED": { 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_ENABLED", 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.336 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.337 INFO:teuthology.orchestra.run.smithi037.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.338 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: "ENABLE_BROWSABLE_API": { 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-19T06:56:35.339 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.340 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.340 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.340 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.340 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.340 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.340 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.340 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.341 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.342 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.343 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.344 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.345 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.346 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.347 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-19T06:56:35.348 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.349 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.349 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.349 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.349 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.349 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.349 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.349 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.350 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.351 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.352 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.353 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.354 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_PASSWORD": { 2024-09-19T06:56:35.355 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.356 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.357 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.358 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_URL": { 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_URL", 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.359 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_API_USERNAME": { 2024-09-19T06:56:35.360 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.361 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.362 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.363 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.364 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.365 INFO:teuthology.orchestra.run.smithi037.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.366 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.367 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.368 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_HOST": { 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.369 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-19T06:56:35.370 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.371 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.372 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.373 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.374 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.375 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.376 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-19T06:56:35.377 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.378 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.379 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.380 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.381 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.382 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.383 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_ENABLED": { 2024-09-19T06:56:35.384 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.385 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.386 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.387 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.388 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.389 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8", 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.390 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-19T06:56:35.391 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "45", 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.392 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ACCESS_KEY": { 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.393 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.394 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "admin", 2024-09-19T06:56:35.395 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.396 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.396 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.396 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.396 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.396 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.396 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.396 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SECRET_KEY": { 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.397 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "RGW_API_SSL_VERIFY": { 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-19T06:56:35.398 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.399 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.400 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.401 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:35.401 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.401 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.401 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.401 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.404 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.404 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.404 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.404 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.404 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-19T06:56:35.404 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.405 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.406 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.407 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "cross_origin_url": { 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cross_origin_url", 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.408 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: "crt_file": { 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "crt_file", 2024-09-19T06:56:35.409 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.410 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.411 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.411 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.411 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.411 INFO:teuthology.orchestra.run.smithi037.stdout: "debug": { 2024-09-19T06:56:35.411 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "debug", 2024-09-19T06:56:35.411 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.411 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Enable/disable debug options", 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.412 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: "jwt_token_ttl": { 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "jwt_token_ttl", 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.413 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "28800", 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.414 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.415 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.416 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.417 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.418 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.419 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.420 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.421 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.422 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.423 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "motd": { 2024-09-19T06:56:35.424 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "motd", 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "The message of the day", 2024-09-19T06:56:35.425 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect_resolve_ip_addr": { 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "redirect_resolve_ip_addr", 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.426 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.427 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.428 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:35.429 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8080", 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.430 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.431 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.432 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl_server_port": { 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl_server_port", 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8443", 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.433 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-09-19T06:56:35.434 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "redirect", 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.435 INFO:teuthology.orchestra.run.smithi037.stdout: "redirect" 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-09-19T06:56:35.436 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.437 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "url_prefix": { 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url_prefix", 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.438 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.439 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-09-19T06:56:35.440 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_monitoring": { 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_monitoring", 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.441 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "monitor device health metrics", 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.442 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.443 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.444 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.445 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.446 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.447 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.448 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.449 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.450 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "mark_out_threshold": { 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mark_out_threshold", 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.451 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "2419200", 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.452 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "pool_name": { 2024-09-19T06:56:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pool_name", 2024-09-19T06:56:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "device_health_metrics", 2024-09-19T06:56:35.453 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.454 INFO:teuthology.orchestra.run.smithi037.stdout: "retention_period": { 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "retention_period", 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15552000", 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.455 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long to retain device health metrics", 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_frequency": { 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_frequency", 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.456 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.457 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "self_heal": { 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "self_heal", 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.458 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:35.459 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.460 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how frequently to wake up and check device health", 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "warn_threshold": { 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "warn_threshold", 2024-09-19T06:56:35.461 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7257600", 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.462 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "diskprediction_local", 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.463 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.464 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.465 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.466 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.467 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.468 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.469 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.470 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.471 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.472 INFO:teuthology.orchestra.run.smithi037.stdout: "predict_interval": { 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predict_interval", 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "86400", 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.473 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "predictor_model": { 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "predictor_model", 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.474 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "prophetstor", 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.475 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "600", 2024-09-19T06:56:35.476 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.477 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "influx", 2024-09-19T06:56:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": false, 2024-09-19T06:56:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "influxdb python module not found", 2024-09-19T06:56:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.478 INFO:teuthology.orchestra.run.smithi037.stdout: "batch_size": { 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "batch_size", 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5000", 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.479 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "database": { 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "database", 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.480 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "ceph", 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.481 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "hostname": { 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "hostname", 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.482 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server hostname", 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.483 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "30", 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "5", 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.484 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.485 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.486 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.487 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.488 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.489 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.490 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.491 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.492 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "password": { 2024-09-19T06:56:35.493 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "password", 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.494 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "password of InfluxDB server user", 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "port": { 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "port", 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.495 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "8086", 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "InfluxDB server port", 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.496 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "ssl": { 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ssl", 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "false", 2024-09-19T06:56:35.497 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.498 INFO:teuthology.orchestra.run.smithi037.stdout: "threads": { 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threads", 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "32", 2024-09-19T06:56:35.499 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "username": { 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "username", 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.500 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "username of InfluxDB server user", 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.501 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "verify_ssl": { 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "verify_ssl", 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.502 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "true", 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.503 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "insights", 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.504 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.505 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.506 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.507 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.508 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.509 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.510 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.511 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.512 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "iostat", 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.513 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.514 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.515 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.516 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.517 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.518 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.519 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.520 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.521 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "k8sevents", 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "ceph_event_retention_days": { 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "ceph_event_retention_days", 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.522 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "7", 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.523 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "config_check_secs": { 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "config_check_secs", 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.524 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "10", 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.525 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.526 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.527 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.528 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.529 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.530 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.531 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.532 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.533 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "localpool", 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "failure_domain": { 2024-09-19T06:56:35.534 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "failure_domain", 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "host", 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.535 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "failure domain for any created local pool", 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.536 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.537 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.538 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.539 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.540 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.541 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.542 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.543 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.544 INFO:teuthology.orchestra.run.smithi037.stdout: "min_size": { 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "min_size", 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.545 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default min_size for any created local pool", 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "num_rep": { 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "num_rep", 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.546 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3", 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default replica count for any created local pool", 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.547 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_num": { 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_num", 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "128", 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.548 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "default pg_num for any created local pool", 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "prefix": { 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prefix", 2024-09-19T06:56:35.549 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "name prefix for any created local pool", 2024-09-19T06:56:35.550 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "subtree": { 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "subtree", 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.551 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "rack", 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.552 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mds_autoscaler", 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.553 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.554 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.555 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.556 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.557 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.558 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.559 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.560 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.561 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirroring", 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.562 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.563 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.564 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.565 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.566 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.567 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.568 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.569 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.570 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "nfs", 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.571 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.572 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.573 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.574 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.575 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.576 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.577 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.578 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.579 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: "fail_fs": { 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "fail_fs", 2024-09-19T06:56:35.580 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.581 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.582 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.583 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.584 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.585 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.586 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.587 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.588 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.589 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator": { 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "orchestrator", 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.590 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: "cephadm", 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: "rook", 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: "test_orchestrator" 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.591 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Orchestrator backend", 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.592 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_perf_query", 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.593 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.594 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.595 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.596 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.597 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.598 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.599 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.600 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.601 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "osd_support", 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.602 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.603 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.604 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.605 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.606 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.607 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.608 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.609 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.612 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.612 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.612 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.612 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.612 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.612 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.613 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "pg_autoscaler", 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.614 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.615 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.616 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.617 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.618 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.619 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.620 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.621 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.622 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:35.623 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.624 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: "threshold": { 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "threshold", 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.625 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "3.0", 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1.0", 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "scaling threshold", 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.626 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "progress", 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.627 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_pg_recovery_event": { 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_pg_recovery_event", 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.628 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-09-19T06:56:35.629 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.630 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.631 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.632 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.632 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.632 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.632 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.632 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.632 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.632 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.633 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.634 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.635 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.636 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.637 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.638 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.639 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max_completed_events": { 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_completed_events", 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "50", 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.640 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "number of past completed events to remember", 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "sleep_interval": { 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "sleep_interval", 2024-09-19T06:56:35.641 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "how long the module is going to sleep", 2024-09-19T06:56:35.642 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "prometheus", 2024-09-19T06:56:35.643 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "cache": { 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "cache", 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.644 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.645 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "exclude_perf_counters": { 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "exclude_perf_counters", 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.646 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.647 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.648 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.649 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.650 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.651 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.652 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.653 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.654 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.655 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools": { 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools", 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.656 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.657 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "300", 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.658 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: "scrape_interval": { 2024-09-19T06:56:35.659 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "scrape_interval", 2024-09-19T06:56:35.660 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:35.660 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.660 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.660 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15.0", 2024-09-19T06:56:35.660 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.660 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.660 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.661 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "::", 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.662 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.663 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "9283", 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.664 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "stale_cache_strategy": { 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stale_cache_strategy", 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "log", 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.665 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_behaviour": { 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_behaviour", 2024-09-19T06:56:35.666 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.667 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.667 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "default", 2024-09-19T06:56:35.667 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.667 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.667 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.667 INFO:teuthology.orchestra.run.smithi037.stdout: "default", 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: "error" 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: "standby_error_status_code": { 2024-09-19T06:56:35.668 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "standby_error_status_code", 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "500", 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "400", 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "599", 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.669 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.670 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.671 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.672 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.673 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.674 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.675 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.676 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.677 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.678 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.679 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_snap_create": { 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_snap_create", 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10", 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.680 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: "mirror_snapshot_schedule": { 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "mirror_snapshot_schedule", 2024-09-19T06:56:35.681 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.682 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "trash_purge_schedule": { 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "trash_purge_schedule", 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.683 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.684 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "restful", 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.685 INFO:teuthology.orchestra.run.smithi037.stdout: "enable_auth": { 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enable_auth", 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.686 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "key_file": { 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "key_file", 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.687 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.688 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.689 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.690 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.691 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.692 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.693 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.694 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.695 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.696 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: "server_addr": { 2024-09-19T06:56:35.697 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_addr", 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.698 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: "server_port": { 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "server_port", 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.699 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.700 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rgw", 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.701 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.702 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.703 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.704 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.705 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.706 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.707 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.708 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.709 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: "secondary_zone_period_retry_limit": { 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.710 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "5", 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.711 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.712 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.712 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.712 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.712 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rook", 2024-09-19T06:56:35.712 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.712 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.713 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.714 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.715 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.716 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.716 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.716 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.716 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.716 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.716 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.716 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.717 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.718 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.719 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.720 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: "storage_class": { 2024-09-19T06:56:35.721 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "storage_class", 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "local", 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-19T06:56:35.722 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "selftest", 2024-09-19T06:56:35.723 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.724 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.725 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.726 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.727 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.728 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.729 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.730 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.731 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.732 INFO:teuthology.orchestra.run.smithi037.stdout: "roption1": { 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption1", 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.733 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: "roption2": { 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "roption2", 2024-09-19T06:56:35.734 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "xyz", 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.735 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.736 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.736 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.736 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption1": { 2024-09-19T06:56:35.736 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption1", 2024-09-19T06:56:35.736 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.736 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.736 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.737 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption2": { 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption2", 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.738 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption3": { 2024-09-19T06:56:35.739 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption3", 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "float", 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.740 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption4": { 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption4", 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.741 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.742 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption5": { 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption5", 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.743 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.744 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.744 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.744 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.744 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.744 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.744 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.744 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption6": { 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption6", 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.745 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "rwoption7": { 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rwoption7", 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.746 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "1", 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "42", 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.747 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: "testkey": { 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testkey", 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.748 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.749 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "testlkey": { 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testlkey", 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.750 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "testnewline": { 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "testnewline", 2024-09-19T06:56:35.751 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.752 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.752 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.752 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.752 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.752 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.752 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.752 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-09-19T06:56:35.753 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "allow_m_granularity": { 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "allow_m_granularity", 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.754 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "allow minute scheduled snapshots", 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.755 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "dump_on_update": { 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "dump_on_update", 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.756 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.757 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.757 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "dump database to debug log on update", 2024-09-19T06:56:35.757 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.757 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.757 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.757 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.757 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.758 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.759 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.760 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.761 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.762 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.763 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.764 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.765 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "stats", 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.766 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.767 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.768 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.769 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.770 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.771 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.772 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.773 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.774 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.775 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "status", 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.776 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.777 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.778 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.779 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.780 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.780 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.780 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.780 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.780 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.780 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.780 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.781 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.782 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.783 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.784 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.785 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.785 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.785 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.785 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telegraf", 2024-09-19T06:56:35.785 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.785 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "address": { 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "address", 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-19T06:56:35.786 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.787 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "15", 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.788 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.789 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.790 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.791 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.792 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.793 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.794 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.795 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.796 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.797 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "telemetry", 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.798 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_basic": { 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_basic", 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.799 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_crash": { 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_crash", 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.800 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.801 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_device": { 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_device", 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.802 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.803 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_ident": { 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_ident", 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.804 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "channel_perf": { 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "channel_perf", 2024-09-19T06:56:35.805 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-19T06:56:35.806 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "contact": { 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "contact", 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.807 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.808 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "description": { 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "description", 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.809 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "device_url": { 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "device_url", 2024-09-19T06:56:35.810 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.811 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "enabled": { 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "enabled", 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.812 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.813 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "24", 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "8", 2024-09-19T06:56:35.814 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "last_opt_revision": { 2024-09-19T06:56:35.815 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "last_opt_revision", 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "1", 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.816 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.817 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard": { 2024-09-19T06:56:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard", 2024-09-19T06:56:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.820 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.821 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "leaderboard_description": { 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "leaderboard_description", 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.822 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.823 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.824 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.825 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.826 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.827 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.828 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.829 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.830 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.831 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "organization": { 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "organization", 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.832 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.833 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "proxy": { 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "proxy", 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.834 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "url": { 2024-09-19T06:56:35.835 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "url", 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.836 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.837 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "test_orchestrator", 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.838 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.839 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.840 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.841 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.842 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.843 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.844 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.845 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.846 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.847 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.848 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.849 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.850 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.851 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.852 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.853 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.854 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.855 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "max_concurrent_clones": { 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "max_concurrent_clones", 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.856 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "4", 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.857 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "periodic_async_work": { 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "periodic_async_work", 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.858 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Periodically check for async work", 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_delay": { 2024-09-19T06:56:35.859 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_delay", 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "0", 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.860 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "snapshot_clone_no_wait": { 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snapshot_clone_no_wait", 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.861 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "True", 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.862 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix", 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "can_run": true, 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "error_string": "", 2024-09-19T06:56:35.863 INFO:teuthology.orchestra.run.smithi037.stdout: "module_options": { 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "discovery_interval": { 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "discovery_interval", 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "uint", 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "100", 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.864 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "identifier": { 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "identifier", 2024-09-19T06:56:35.865 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.866 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "interval": { 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "interval", 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "secs", 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.867 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "60", 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.868 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "log_level": { 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_level", 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.869 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.870 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster": { 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster", 2024-09-19T06:56:35.871 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.872 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_cluster_level": { 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_cluster_level", 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.873 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "info", 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [ 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "", 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "critical", 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "debug", 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "error", 2024-09-19T06:56:35.874 INFO:teuthology.orchestra.run.smithi037.stdout: "info", 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "warning" 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "log_to_file": { 2024-09-19T06:56:35.875 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "log_to_file", 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "bool", 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 1, 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "False", 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.876 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_host": { 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_host", 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.877 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "", 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.878 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_port": { 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_port", 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "int", 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "10051", 2024-09-19T06:56:35.879 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "zabbix_sender": { 2024-09-19T06:56:35.880 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "zabbix_sender", 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "str", 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "level": "advanced", 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "flags": 0, 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "min": "", 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "max": "", 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "enum_allowed": [], 2024-09-19T06:56:35.881 INFO:teuthology.orchestra.run.smithi037.stdout: "desc": "", 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "long_desc": "", 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "tags": [], 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "see_also": [] 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "services": { 2024-09-19T06:56:35.882 INFO:teuthology.orchestra.run.smithi037.stdout: "dashboard": "https://172.21.15.37:8443/" 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "always_on_modules": { 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "octopus": [ 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-09-19T06:56:35.883 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "pacific": [ 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-09-19T06:56:35.884 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-09-19T06:56:35.885 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "quincy": [ 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-09-19T06:56:35.886 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "reef": [ 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-09-19T06:56:35.887 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-09-19T06:56:35.888 INFO:teuthology.orchestra.run.smithi037.stdout: ], 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "squid": [ 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "balancer", 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "crash", 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "devicehealth", 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "orchestrator", 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "pg_autoscaler", 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "progress", 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "rbd_support", 2024-09-19T06:56:35.889 INFO:teuthology.orchestra.run.smithi037.stdout: "status", 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "telemetry", 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "volumes" 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "last_failure_osd_epoch": 84, 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "active_clients": [ 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.890 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "devicehealth", 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2467962751 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.891 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "libcephsqlite", 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 3827774948 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.892 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "rbd_support", 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 4180826040 2024-09-19T06:56:35.893 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-19T06:56:35.894 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1661977371 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "snap_schedule", 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-09-19T06:56:35.895 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-19T06:56:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 2207724725 2024-09-19T06:56:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "name": "volumes", 2024-09-19T06:56:35.896 INFO:teuthology.orchestra.run.smithi037.stdout: "addrvec": [ 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: { 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "type": "v2", 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "addr": "172.21.15.37:0", 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: "nonce": 1151273770 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout: ] 2024-09-19T06:56:35.897 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:56:35.899 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:34 smithi096 ceph-mon[35004]: pgmap v469: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 5.4 MiB/s wr, 181 op/s 2024-09-19T06:56:35.899 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:34 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/4064362234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:56:35.899 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:34 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1689363924' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T06:56:35.899 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:33 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1446005001' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:56:35.899 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:33 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3599524847' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-19T06:56:35.899 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:33 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/793191126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-19T06:56:35.900 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:34 smithi151 ceph-mon[35236]: pgmap v469: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 5.4 MiB/s wr, 181 op/s 2024-09-19T06:56:35.900 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:34 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/4064362234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:56:35.900 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:34 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1689363924' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T06:56:35.900 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d440a33c0 msgr2=0x7f7d440a5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d440a33c0 0x7f7d440a5880 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c1000c0 tx=0x7f7d5000b410 comp rx=0 tx=0).stop 2024-09-19T06:56:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100880 msgr2=0x7f7d5c198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100880 0x7f7d5c198390 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00e9f0 tx=0x7f7d4c00edf0 comp rx=0 tx=0).stop 2024-09-19T06:56:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 shutdown_connections 2024-09-19T06:56:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d5c19cc50 0x7f7d5c19d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:35.901 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7d5c1080d0 0x7f7d5c1988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:35.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f7d440a33c0 0x7f7d440a5880 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:35.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 --2- 172.21.15.37:0/1689363924 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f7d5c100880 0x7f7d5c198390 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:35.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 >> 172.21.15.37:0/1689363924 conn(0x7f7d5c075f60 msgr2=0x7f7d5c105aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:35.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 shutdown_connections 2024-09-19T06:56:35.902 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:33.574+0000 7f7d62f98640 1 -- 172.21.15.37:0/1689363924 wait complete. 2024-09-19T06:56:35.909 DEBUG:tasks.check_counter:Getting stats from a 2024-09-19T06:56:35.909 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-19T06:56:35.909 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:34 smithi037 ceph-mon[31211]: pgmap v469: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 5.4 MiB/s wr, 181 op/s 2024-09-19T06:56:35.909 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:34 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/4064362234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-19T06:56:35.910 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:34 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1689363924' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-19T06:56:36.177 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:36.455 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:36 smithi037 ceph-mon[31211]: pgmap v470: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-09-19T06:56:36.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:36 smithi096 ceph-mon[35004]: pgmap v470: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-09-19T06:56:36.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:36 smithi151 ceph-mon[35236]: pgmap v470: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 2.9 MiB/s wr, 103 op/s 2024-09-19T06:56:36.677 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:56:36.677 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:36.677 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-09-19T06:56:36.677 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001826661, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000325822, 2024-09-19T06:56:36.678 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:36.679 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 208, 2024-09-19T06:56:36.680 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-09-19T06:56:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 193968, 2024-09-19T06:56:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-09-19T06:56:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-09-19T06:56:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015130479, 2024-09-19T06:56:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.010765218, 2024-09-19T06:56:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001841298, 2024-09-19T06:56:36.681 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 205, 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014954905, 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000072950 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:36.682 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 193968 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 287, 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 274, 2024-09-19T06:56:36.683 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1607200, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 168192, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.048213893, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.016920744, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.044385799, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:36.684 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 274, 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.017837358, 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000065099 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:36.685 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1607200, 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 168192 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-09-19T06:56:36.686 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:36.687 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:36.688 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:36.688 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:36.688 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:36.688 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:36.688 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:36.688 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:36.688 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:36.689 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:36.690 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148994, 2024-09-19T06:56:36.691 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 31, 2024-09-19T06:56:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-09-19T06:56:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-09-19T06:56:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:36.692 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-09-19T06:56:36.693 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-09-19T06:56:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-09-19T06:56:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-09-19T06:56:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-09-19T06:56:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:36.694 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 285, 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1564175, 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 285, 2024-09-19T06:56:36.695 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 285, 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1564175, 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:36.696 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:36.697 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:36.697 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:56:37.737 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-19T06:56:37.738 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-19T06:56:37.738 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-19T06:56:37.738 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-19T06:56:37.738 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-19T06:56:37.738 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-19T06:56:37.738 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-19T06:56:37.738 DEBUG:tasks.check_counter:Getting stats from d 2024-09-19T06:56:37.738 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-19T06:56:37.993 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:38.371 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:38 smithi037 ceph-mon[31211]: pgmap v471: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.8 MiB/s wr, 101 op/s 2024-09-19T06:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:56:38.477 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001302145, 2024-09-19T06:56:38.478 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000247172, 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:38.479 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:38.480 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 288, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 277, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1605488, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 170944, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.046180536, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.017391722, 2024-09-19T06:56:38.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.041150323, 2024-09-19T06:56:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 276, 2024-09-19T06:56:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.017493730, 2024-09-19T06:56:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000063383 2024-09-19T06:56:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.482 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1605488, 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 170944 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.483 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 6, 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 215, 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3392, 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 195296, 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.018856116, 2024-09-19T06:56:38.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.012348602, 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.004047527, 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 213, 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.020124975, 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000094483 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.485 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3392, 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 195296 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.486 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:38.487 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:38.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:38.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:38.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:38.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 149057, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 32, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-09-19T06:56:38.492 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:38.493 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-09-19T06:56:38.494 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 290, 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1564825, 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:38.495 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 290, 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 290, 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1564825, 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:38.496 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:38.497 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:38.497 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:56:38.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:38 smithi096 ceph-mon[35004]: pgmap v471: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.8 MiB/s wr, 101 op/s 2024-09-19T06:56:38.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:38 smithi151 ceph-mon[35236]: pgmap v471: 97 pgs: 97 active+clean; 283 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 2.8 MiB/s wr, 101 op/s 2024-09-19T06:56:39.198 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-19T06:56:39.198 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-19T06:56:39.198 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-19T06:56:39.198 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-19T06:56:39.199 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-19T06:56:39.199 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-19T06:56:39.199 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-19T06:56:39.199 DEBUG:tasks.check_counter:Getting stats from g 2024-09-19T06:56:39.199 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-19T06:56:39.460 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:39.949 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:56:39.949 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:39.949 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 284, 2024-09-19T06:56:39.949 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 273, 2024-09-19T06:56:39.949 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1604400, 2024-09-19T06:56:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 168384, 2024-09-19T06:56:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-09-19T06:56:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-09-19T06:56:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.048727398, 2024-09-19T06:56:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.017702094, 2024-09-19T06:56:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.042734743, 2024-09-19T06:56:39.950 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 272, 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.013034594, 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000047921 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:39.951 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1604400, 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 168384 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 209, 2024-09-19T06:56:39.952 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 194144, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015254391, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.010731998, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.001862119, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:39.953 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 206, 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014897183, 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000072316 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:39.954 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 194144 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 4, 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 4, 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 3072, 2024-09-19T06:56:39.955 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 448, 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.002443613, 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000057083, 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002343144, 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:39.956 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 3, 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.004137551, 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.001379183 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:39.957 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.958 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:39.959 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:39.960 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:39.961 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:39.962 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148977, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:39.963 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-09-19T06:56:39.964 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:39.965 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:39.965 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:39.965 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-09-19T06:56:39.965 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-09-19T06:56:39.965 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-09-19T06:56:39.965 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:39.965 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 286, 2024-09-19T06:56:39.966 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1564305, 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 286, 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 286, 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1564305, 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-09-19T06:56:39.967 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:39.968 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:39.968 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:39.968 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:39.968 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:39.968 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:56:40.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:40 smithi096 ceph-mon[35004]: pgmap v472: 97 pgs: 97 active+clean; 298 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 4.6 MiB/s wr, 154 op/s 2024-09-19T06:56:40.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:40 smithi037 ceph-mon[31211]: pgmap v472: 97 pgs: 97 active+clean; 298 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 4.6 MiB/s wr, 154 op/s 2024-09-19T06:56:40.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:40 smithi151 ceph-mon[35236]: pgmap v472: 97 pgs: 97 active+clean; 298 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 300 KiB/s rd, 4.6 MiB/s wr, 154 op/s 2024-09-19T06:56:40.718 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-19T06:56:40.718 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-19T06:56:40.718 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-19T06:56:40.718 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-19T06:56:40.718 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-19T06:56:40.718 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-19T06:56:40.719 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-19T06:56:40.719 DEBUG:tasks.check_counter:Getting stats from j 2024-09-19T06:56:40.719 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-19T06:56:40.978 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:41.467 INFO:teuthology.orchestra.run.smithi037.stdout:{ 2024-09-19T06:56:41.467 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:41.467 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 0, 2024-09-19T06:56:41.468 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 2, 2024-09-19T06:56:41.468 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 0, 2024-09-19T06:56:41.468 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 0, 2024-09-19T06:56:41.468 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 4, 2024-09-19T06:56:41.469 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 0, 2024-09-19T06:56:41.469 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.001272107, 2024-09-19T06:56:41.469 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T06:56:41.469 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.000271782, 2024-09-19T06:56:41.470 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:41.470 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:41.470 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:41.470 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:41.471 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:41.471 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.471 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:41.471 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:41.472 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:41.472 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:41.472 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.473 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:56:41.473 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:56:41.473 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.473 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:41.474 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 295, 2024-09-19T06:56:41.474 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 282, 2024-09-19T06:56:41.474 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 1609440, 2024-09-19T06:56:41.474 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 172832, 2024-09-19T06:56:41.475 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 2, 2024-09-19T06:56:41.475 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 1, 2024-09-19T06:56:41.475 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.050691262, 2024-09-19T06:56:41.475 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.018163444, 2024-09-19T06:56:41.476 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.046238518, 2024-09-19T06:56:41.476 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:41.476 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:41.476 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 282, 2024-09-19T06:56:41.477 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.017803188, 2024-09-19T06:56:41.477 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000063131 2024-09-19T06:56:41.477 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.477 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:41.478 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:41.478 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:41.478 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:41.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.479 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 1609440, 2024-09-19T06:56:41.479 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 172832 2024-09-19T06:56:41.479 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.480 INFO:teuthology.orchestra.run.smithi037.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:41.480 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_messages": 2, 2024-09-19T06:56:41.480 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages": 213, 2024-09-19T06:56:41.480 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_bytes": 320, 2024-09-19T06:56:41.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_bytes": 194848, 2024-09-19T06:56:41.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_created_connections": 5, 2024-09-19T06:56:41.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_active_connections": 3, 2024-09-19T06:56:41.481 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_total_time": 0.015630316, 2024-09-19T06:56:41.482 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_send_time": 0.010737232, 2024-09-19T06:56:41.482 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_recv_time": 0.002424607, 2024-09-19T06:56:41.482 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:41.482 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:41.483 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 210, 2024-09-19T06:56:41.483 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.014366328, 2024-09-19T06:56:41.483 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000068411 2024-09-19T06:56:41.483 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.484 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:41.484 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:41.484 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:41.485 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:41.485 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.485 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T06:56:41.485 INFO:teuthology.orchestra.run.smithi037.stdout: "msgr_send_encrypted_bytes": 194848 2024-09-19T06:56:41.486 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.486 INFO:teuthology.orchestra.run.smithi037.stdout: "cct": { 2024-09-19T06:56:41.486 INFO:teuthology.orchestra.run.smithi037.stdout: "total_workers": 0, 2024-09-19T06:56:41.486 INFO:teuthology.orchestra.run.smithi037.stdout: "unhealthy_workers": 0 2024-09-19T06:56:41.487 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.487 INFO:teuthology.orchestra.run.smithi037.stdout: "mempool": { 2024-09-19T06:56:41.487 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:41.487 INFO:teuthology.orchestra.run.smithi037.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:41.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:41.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:41.488 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:41.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:41.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:41.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:41.489 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:41.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:41.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:41.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:41.490 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:41.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:41.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:41.491 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:41.492 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:41.492 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:41.492 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:41.492 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:41.493 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:41.493 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:41.493 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:41.493 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:41.493 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:41.494 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:41.494 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:41.494 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:41.494 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:41.495 INFO:teuthology.orchestra.run.smithi037.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:41.495 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:41.495 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_items": 0, 2024-09-19T06:56:41.495 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:41.495 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:41.495 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:41.496 INFO:teuthology.orchestra.run.smithi037.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:41.496 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_bytes": 148977, 2024-09-19T06:56:41.496 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_anon_items": 30, 2024-09-19T06:56:41.496 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:41.496 INFO:teuthology.orchestra.run.smithi037.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:41.496 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_bytes": 0, 2024-09-19T06:56:41.497 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_items": 0, 2024-09-19T06:56:41.497 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:41.497 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:41.497 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:41.497 INFO:teuthology.orchestra.run.smithi037.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:41.497 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_bytes": 0, 2024-09-19T06:56:41.497 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_items": 0, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "pgmap_items": 0, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_bytes": 928, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "mds_co_items": 2, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:41.498 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_1_items": 0, 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: "unittest_2_items": 0 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: }, 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: "val": 0, 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: "max": 104857600, 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: "get_started": 0, 2024-09-19T06:56:41.499 INFO:teuthology.orchestra.run.smithi037.stdout: "get": 293, 2024-09-19T06:56:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "get_sum": 1565279, 2024-09-19T06:56:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "get_or_fail_success": 293, 2024-09-19T06:56:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "take": 0, 2024-09-19T06:56:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "take_sum": 0, 2024-09-19T06:56:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "put": 293, 2024-09-19T06:56:41.500 INFO:teuthology.orchestra.run.smithi037.stdout: "put_sum": 1565279, 2024-09-19T06:56:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: "wait": { 2024-09-19T06:56:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: "avgcount": 0, 2024-09-19T06:56:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: "sum": 0.000000000, 2024-09-19T06:56:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: "avgtime": 0.000000000 2024-09-19T06:56:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:41.501 INFO:teuthology.orchestra.run.smithi037.stdout: } 2024-09-19T06:56:41.501 INFO:teuthology.orchestra.run.smithi037.stdout:} 2024-09-19T06:56:42.176 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-19T06:56:42.176 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-19T06:56:42.176 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-19T06:56:42.177 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-19T06:56:42.177 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-19T06:56:42.177 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-19T06:56:42.177 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-19T06:56:42.177 DEBUG:tasks.check_counter:Getting stats from b 2024-09-19T06:56:42.177 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-19T06:56:42.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:42 smithi096 ceph-mon[35004]: pgmap v473: 97 pgs: 97 active+clean; 298 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 3.7 MiB/s wr, 93 op/s 2024-09-19T06:56:42.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:42 smithi037 ceph-mon[31211]: pgmap v473: 97 pgs: 97 active+clean; 298 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 3.7 MiB/s wr, 93 op/s 2024-09-19T06:56:42.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:42 smithi151 ceph-mon[35236]: pgmap v473: 97 pgs: 97 active+clean; 298 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 3.7 MiB/s wr, 93 op/s 2024-09-19T06:56:42.674 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:56:43.393 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T06:56:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 160392, 2024-09-19T06:56:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 256896, 2024-09-19T06:56:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 33507299, 2024-09-19T06:56:43.394 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 449837842, 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 16, 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 13.985887768, 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 6.792623930, 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 6.124537792, 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 1.373001519, 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:43.395 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 256894, 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 24.840474394, 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000096695 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 308365, 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.062063939, 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000201 2024-09-19T06:56:43.396 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 513840, 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 148736 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 47794, 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 47795, 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 18712611, 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 257238501, 2024-09-19T06:56:43.397 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 354, 2024-09-19T06:56:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-09-19T06:56:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 3.600250240, 2024-09-19T06:56:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 1.246613652, 2024-09-19T06:56:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.854852792, 2024-09-19T06:56:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 1.216241905, 2024-09-19T06:56:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:43.398 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 47793, 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 22.666753397, 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000474269 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 625, 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000073047, 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000116 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.399 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1859904, 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 655328 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 550737, 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 823883, 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 177110721, 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 786370725, 2024-09-19T06:56:43.400 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 13, 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 34.633000364, 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 16.201880289, 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 21.966080189, 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 1.079550121, 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 823882, 2024-09-19T06:56:43.401 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 31.296484273, 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000037986 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1243283, 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.169350683, 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000136 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 150721968, 2024-09-19T06:56:43.402 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 535258176 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:43.403 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 124220, 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 111.626426943, 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000898618 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:43.404 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 64801, 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 7.976844832, 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000123097 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 312074, 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 311938, 2024-09-19T06:56:43.405 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T06:56:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 311938, 2024-09-19T06:56:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 854.556252092, 2024-09-19T06:56:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002739506 2024-09-19T06:56:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T06:56:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 136, 2024-09-19T06:56:43.406 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 2, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 1, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 3845, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 34, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 31, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 5289, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 1640, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 3486, 2024-09-19T06:56:43.407 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 163, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 2946, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 170, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 612, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 612, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 22, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 466461, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 294476, 2024-09-19T06:56:43.408 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 136, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 1, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 3463, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 9, 2024-09-19T06:56:43.409 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 132, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 2, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 58, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 2360, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 743, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 228507865, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-09-19T06:56:43.410 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 2360, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 500, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 21, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 479, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 1860, 2024-09-19T06:56:43.411 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 9, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 11031, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 139125, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 76841, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 32943, 2024-09-19T06:56:43.412 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 15655, 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 225757 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 229, 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 1, 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 516, 2024-09-19T06:56:43.413 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 40170, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 40170, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T06:56:43.414 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 12, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 948, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 21, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 9, 2024-09-19T06:56:43.415 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 65, 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 191331, 2024-09-19T06:56:43.416 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 186820, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 186820, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 4511, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 5273, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 299, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 292, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 292, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 7, 2024-09-19T06:56:43.417 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 1, 2024-09-19T06:56:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T06:56:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 7, 2024-09-19T06:56:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 704263327, 2024-09-19T06:56:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 714122351, 2024-09-19T06:56:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-19T06:56:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T06:56:43.418 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 63173, 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 195.301609684, 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003091536 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 2129, 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 42427, 2024-09-19T06:56:43.419 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 40298, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 196, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 437, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 241, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 5289, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 84714, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 79425, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 612, 2024-09-19T06:56:43.420 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 45136, 2024-09-19T06:56:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 44524, 2024-09-19T06:56:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 99048, 2024-09-19T06:56:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 231712 2024-09-19T06:56:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T06:56:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 431313, 2024-09-19T06:56:43.421 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 696, 2024-09-19T06:56:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 312074, 2024-09-19T06:56:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 5713, 2024-09-19T06:56:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 1646, 2024-09-19T06:56:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T06:56:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 40897, 2024-09-19T06:56:43.422 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 249.558993643, 2024-09-19T06:56:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.006102134 2024-09-19T06:56:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T06:56:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11259, 2024-09-19T06:56:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 43.587328257, 2024-09-19T06:56:43.423 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003871332 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T06:56:43.424 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T06:56:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 86619, 2024-09-19T06:56:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 110.303167889, 2024-09-19T06:56:43.425 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001273429 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T06:56:43.426 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.427 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T06:56:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.428 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T06:56:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.429 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 15, 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.008369910, 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000557994 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T06:56:43.430 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 119, 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 20.957319284, 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.176111926 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.431 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 15, 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.148632285, 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.009908819 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 10, 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.025657363, 2024-09-19T06:56:43.432 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002565736 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 123066, 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 236.901454233, 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001924995 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T06:56:43.433 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8458, 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 80.053467939, 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.009464822 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T06:56:43.434 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006537822, 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.006537822 2024-09-19T06:56:43.435 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 9, 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.065450471, 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.007272274 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.436 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 20, 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.155371501, 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.007768575 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.437 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1272, 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.352676103, 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001063424 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.438 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8, 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.019296754, 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002412094 2024-09-19T06:56:43.439 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T06:56:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T06:56:43.440 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 40170, 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 111.412528638, 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002773525 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.441 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 6, 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 4, 2024-09-19T06:56:43.442 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 43583, 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 21791, 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 889, 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T06:56:43.443 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:43.444 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:43.445 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:43.446 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T06:56:43.447 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 2591300, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 2072, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 264, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 3, 2024-09-19T06:56:43.448 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8320, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 183, 2024-09-19T06:56:43.449 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 10199106, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 129329, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T06:56:43.450 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T06:56:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T06:56:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T06:56:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T06:56:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 134505, 2024-09-19T06:56:43.451 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 746872584, 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 37, 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 134505, 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 134505, 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1998.573946251, 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.014858733 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.452 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 134505, 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 184598 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 134505, 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3465, 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 131040, 2024-09-19T06:56:43.453 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 3418, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 20251, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1081, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 64233, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 760, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T06:56:43.454 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 34, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 41223, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 2384, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 40502, 2024-09-19T06:56:43.455 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T06:56:43.456 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T06:56:43.457 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T06:56:43.458 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 107, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 26, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T06:56:43.459 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 8037, 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 6, 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 2669 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T06:56:43.460 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 972 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 10, 2024-09-19T06:56:43.461 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T06:56:43.462 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 10, 2024-09-19T06:56:43.462 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 40170, 2024-09-19T06:56:43.462 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 40170, 2024-09-19T06:56:43.462 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T06:56:43.462 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.462 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:43.462 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:43.463 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:43.463 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:43.463 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 758919, 2024-09-19T06:56:43.463 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 142222969, 2024-09-19T06:56:43.463 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:43.463 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 758919, 2024-09-19T06:56:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 758919, 2024-09-19T06:56:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 142222969, 2024-09-19T06:56:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.464 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T06:56:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:43.465 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:43.466 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 134505, 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 750936382, 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 134505, 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 750936382, 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.467 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:43.468 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 134505, 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 134505, 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 134505, 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 134505, 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.469 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:43.470 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 40170, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 4063834, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 40170, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1060, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 4063834, 2024-09-19T06:56:43.471 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:43.472 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.472 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.472 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.472 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.472 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:43.472 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55e4da8d8960": { 2024-09-19T06:56:43.472 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 191331, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 709928047, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 191331, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:43.473 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 63173, 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 709928047, 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.474 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:43.475 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T06:56:44.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:44 smithi096 ceph-mon[35004]: pgmap v474: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.0 MiB/s wr, 150 op/s 2024-09-19T06:56:44.485 INFO:tasks.check_counter:Daemon mds.b mds.exported=9 2024-09-19T06:56:44.486 INFO:tasks.check_counter:Daemon mds.b mds.imported=2 2024-09-19T06:56:44.486 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=229 2024-09-19T06:56:44.486 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-19T06:56:44.486 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-19T06:56:44.486 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=15 2024-09-19T06:56:44.486 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=9 2024-09-19T06:56:44.486 DEBUG:tasks.check_counter:Getting stats from e 2024-09-19T06:56:44.487 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-19T06:56:44.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:44 smithi037 ceph-mon[31211]: pgmap v474: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.0 MiB/s wr, 150 op/s 2024-09-19T06:56:44.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:44 smithi151 ceph-mon[35236]: pgmap v474: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 277 KiB/s rd, 5.0 MiB/s wr, 150 op/s 2024-09-19T06:56:44.744 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:56:45.583 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T06:56:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 238271, 2024-09-19T06:56:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 146274, 2024-09-19T06:56:45.583 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 94220937, 2024-09-19T06:56:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 33236007, 2024-09-19T06:56:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 10, 2024-09-19T06:56:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-09-19T06:56:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 11.231331859, 2024-09-19T06:56:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 5.037913401, 2024-09-19T06:56:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 8.376383914, 2024-09-19T06:56:45.584 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.031649510, 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 146275, 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.181164536, 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000014911 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 375507, 2024-09-19T06:56:45.585 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.048025101, 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000127 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 11773984, 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 11458432 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 402, 2024-09-19T06:56:45.586 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 435, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 139144, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 652143, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 38, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.039225291, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.011192602, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.022210324, 2024-09-19T06:56:45.587 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.005034606, 2024-09-19T06:56:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 432, 2024-09-19T06:56:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.024404085, 2024-09-19T06:56:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000056490 2024-09-19T06:56:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:45.588 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 338, 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000052257, 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000154 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 6192, 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 135280 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:45.589 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 590, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 786, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1812432, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 959340, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 31, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.077824470, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.028213249, 2024-09-19T06:56:45.590 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.063142648, 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.002904821, 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 783, 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.044234776, 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000056493 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:45.591 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 113, 2024-09-19T06:56:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000020083, 2024-09-19T06:56:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000177 2024-09-19T06:56:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1757952, 2024-09-19T06:56:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 629232 2024-09-19T06:56:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.592 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1755, 2024-09-19T06:56:45.593 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.678748221, 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000386751 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 356, 2024-09-19T06:56:45.594 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.016683406, 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000046863 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 26075, 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 1818, 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T06:56:45.595 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1818, 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 5.879772299, 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003234198 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 24257, 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-19T06:56:45.596 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 11, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 325, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 77, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 227, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 21, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 215, 2024-09-19T06:56:45.597 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 61, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 20, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 21, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 7, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 28794, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 2970, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 24253, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 115, 2024-09-19T06:56:45.598 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 59, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 2, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 58, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 7, 2024-09-19T06:56:45.599 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 126, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 89, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 601, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 4014409, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 89, 2024-09-19T06:56:45.600 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 89, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 6, 2024-09-19T06:56:45.601 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 6145, 2024-09-19T06:56:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 6453, 2024-09-19T06:56:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:56:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:56:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:56:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 8235, 2024-09-19T06:56:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 678, 2024-09-19T06:56:45.602 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 100, 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 994 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 52, 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 374, 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 242, 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 256, 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 60, 2024-09-19T06:56:45.603 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-19T06:56:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T06:56:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:56:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 176, 2024-09-19T06:56:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 176, 2024-09-19T06:56:45.604 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 12, 2024-09-19T06:56:45.605 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 235, 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 2, 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.606 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 1859, 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 1859, 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-19T06:56:45.607 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 3, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 3, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-19T06:56:45.608 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 7103699, 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 780, 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.046345863, 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002623520 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-19T06:56:45.609 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 268, 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 486, 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 218, 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 141, 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 178, 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 37, 2024-09-19T06:56:45.610 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 325, 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 656, 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 331, 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 21, 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 246, 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 225, 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 60596, 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223520 2024-09-19T06:56:45.611 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 27989, 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 254, 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 26075, 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 63, 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 930, 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T06:56:45.612 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 180, 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.258028421, 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.012544602 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 760, 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.141968093, 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001502589 2024-09-19T06:56:45.613 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.614 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 338, 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.308041965, 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000911366 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T06:56:45.615 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T06:56:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T06:56:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000111727, 2024-09-19T06:56:45.616 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000111727 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T06:56:45.617 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.618 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T06:56:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T06:56:45.619 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.620 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.621 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 288, 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.199867173, 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000693983 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.622 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 59, 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.692935305, 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.011744666 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.623 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.624 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T06:56:45.624 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.624 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.624 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.624 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.624 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.625 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.626 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.627 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T06:56:45.628 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 16, 2024-09-19T06:56:45.628 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.005317857, 2024-09-19T06:56:45.628 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000332366 2024-09-19T06:56:45.628 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.628 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T06:56:45.628 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.628 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.629 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 176, 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.273501758, 2024-09-19T06:56:45.630 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.007235805 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.631 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.632 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.632 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T06:56:45.632 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-09-19T06:56:45.632 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-09-19T06:56:45.632 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-09-19T06:56:45.632 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-09-19T06:56:45.632 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T06:56:45.633 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 880, 2024-09-19T06:56:45.633 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 440, 2024-09-19T06:56:45.633 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 877, 2024-09-19T06:56:45.633 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T06:56:45.633 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.636 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T06:56:45.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:45.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:45.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:45.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:45.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:45.636 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:45.637 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:45.638 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:45.639 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T06:56:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:45.640 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 493624, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 394, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:45.641 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8320, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 183, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T06:56:45.642 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 1128900, 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 12607, 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T06:56:45.643 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 1260, 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 2962881, 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 33, 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 1260, 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1260, 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.658921796, 2024-09-19T06:56:45.644 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002903906 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1260, 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1350 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 1260, 2024-09-19T06:56:45.645 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 114, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 1146, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 25, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 805, 2024-09-19T06:56:45.646 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 68, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 188, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T06:56:45.647 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T06:56:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 89, 2024-09-19T06:56:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-09-19T06:56:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:56:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T06:56:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:56:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T06:56:45.648 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T06:56:45.649 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T06:56:45.650 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 107, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T06:56:45.651 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 26, 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 117, 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 28 2024-09-19T06:56:45.652 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T06:56:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-19T06:56:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:56:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T06:56:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T06:56:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.653 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 247 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 10, 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 10, 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 176, 2024-09-19T06:56:45.654 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 176, 2024-09-19T06:56:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T06:56:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:45.655 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 239259, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 73094359, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 239259, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 239259, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 73094359, 2024-09-19T06:56:45.656 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:45.657 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1260, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2980945, 2024-09-19T06:56:45.658 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1260, 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2980945, 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.659 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:45.660 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 1260, 2024-09-19T06:56:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 1260, 2024-09-19T06:56:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1260, 2024-09-19T06:56:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 1260, 2024-09-19T06:56:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.661 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 176, 2024-09-19T06:56:45.662 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 18064, 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 176, 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 25, 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 18064, 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:45.663 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x563c8e5de960": { 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 11880, 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:45.664 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 1859, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2909395, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 1859, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 780, 2024-09-19T06:56:45.665 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2897515, 2024-09-19T06:56:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.666 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:45.666 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T06:56:46.463 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:46 smithi096 ceph-mon[35004]: pgmap v475: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 3.2 MiB/s wr, 113 op/s 2024-09-19T06:56:46.494 INFO:tasks.check_counter:Daemon mds.e mds.exported=2 2024-09-19T06:56:46.494 INFO:tasks.check_counter:Daemon mds.e mds.imported=7 2024-09-19T06:56:46.495 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=52 2024-09-19T06:56:46.495 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=374 2024-09-19T06:56:46.495 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-09-19T06:56:46.495 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:56:46.495 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:56:46.495 DEBUG:tasks.check_counter:Getting stats from h 2024-09-19T06:56:46.495 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-19T06:56:46.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:46 smithi037 ceph-mon[31211]: pgmap v475: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 3.2 MiB/s wr, 113 op/s 2024-09-19T06:56:46.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:46 smithi151 ceph-mon[35236]: pgmap v475: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 231 KiB/s rd, 3.2 MiB/s wr, 113 op/s 2024-09-19T06:56:46.755 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:56:47.317 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T06:56:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 25646, 2024-09-19T06:56:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 26476, 2024-09-19T06:56:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 8226256, 2024-09-19T06:56:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 6162448, 2024-09-19T06:56:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 16, 2024-09-19T06:56:47.318 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 1.574467624, 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.591307041, 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 1.324110859, 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.006330433, 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 26472, 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.389941192, 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000014730 2024-09-19T06:56:47.319 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 51286, 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006559430, 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000127 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 8166256, 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 5096608 2024-09-19T06:56:47.320 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 200305, 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 100926, 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 83296770, 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 17314861, 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 15, 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-09-19T06:56:47.321 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 8.982514070, 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 4.000386349, 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 6.846217575, 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.005417758, 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 100927, 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.657775399, 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000016425 2024-09-19T06:56:47.322 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 299692, 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.038889821, 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000129 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1750496, 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 182464 2024-09-19T06:56:47.323 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 285, 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 325, 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 135264, 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 707986, 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 50, 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-09-19T06:56:47.324 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.036919395, 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.009015334, 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.024695901, 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.002559689, 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 323, 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.026907048, 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000083303 2024-09-19T06:56:47.325 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.326 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:47.326 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 332, 2024-09-19T06:56:47.326 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000176469, 2024-09-19T06:56:47.326 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000531 2024-09-19T06:56:47.326 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.326 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 6800, 2024-09-19T06:56:47.326 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 132896 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:47.327 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:47.328 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 400, 2024-09-19T06:56:47.328 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.260763774, 2024-09-19T06:56:47.328 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000651909 2024-09-19T06:56:47.328 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.328 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.328 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T06:56:47.328 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 210, 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.010200535, 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000048573 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 25178, 2024-09-19T06:56:47.329 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 633, 2024-09-19T06:56:47.330 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T06:56:47.330 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 633, 2024-09-19T06:56:47.330 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3.013958608, 2024-09-19T06:56:47.330 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.004761388 2024-09-19T06:56:47.330 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 24545, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 11, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 162, 2024-09-19T06:56:47.331 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 67, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 95, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 65, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 21, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 21, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 4, 2024-09-19T06:56:47.332 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 26454, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 1019, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 24545, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 110, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 40, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T06:56:47.333 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 2, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 6, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 6, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 601, 2024-09-19T06:56:47.334 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 4014409, 2024-09-19T06:56:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-09-19T06:56:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 6, 2024-09-19T06:56:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T06:56:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:56:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:56:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:56:47.335 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 6, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 5, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 419, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:56:47.336 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 113, 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 84, 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 29, 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 626 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 8, 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 417, 2024-09-19T06:56:47.337 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 260, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 327, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 8, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 108, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 108, 2024-09-19T06:56:47.338 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 12, 2024-09-19T06:56:47.339 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 211, 2024-09-19T06:56:47.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:56:47.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-19T06:56:47.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T06:56:47.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:56:47.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T06:56:47.340 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T06:56:47.340 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 676, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 676, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-19T06:56:47.341 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 2, 2024-09-19T06:56:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-19T06:56:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-19T06:56:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-09-19T06:56:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-19T06:56:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T06:56:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-19T06:56:47.342 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 5106384, 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 174, 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.592065127, 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003402673 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-09-19T06:56:47.343 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 155, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 263, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 108, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 139, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 208, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 69, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 162, 2024-09-19T06:56:47.344 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 275, 2024-09-19T06:56:47.345 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 113, 2024-09-19T06:56:47.345 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 21, 2024-09-19T06:56:47.345 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 131, 2024-09-19T06:56:47.345 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 110, 2024-09-19T06:56:47.345 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 61204, 2024-09-19T06:56:47.345 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223520 2024-09-19T06:56:47.345 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 25857, 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 25178, 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 60, 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 274, 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 120, 2024-09-19T06:56:47.346 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.653838033, 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.013781983 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001439620, 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001439620 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.347 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.348 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 161, 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.180246195, 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001119541 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.349 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T06:56:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-09-19T06:56:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000219819, 2024-09-19T06:56:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000219819 2024-09-19T06:56:47.350 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T06:56:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T06:56:47.351 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.352 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T06:56:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T06:56:47.353 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.354 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.355 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 170, 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.093146444, 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000547920 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T06:56:47.356 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 40, 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.490399988, 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.012259999 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.357 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.358 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.359 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.360 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.361 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T06:56:47.361 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.361 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.361 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.361 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.362 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T06:56:47.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 32, 2024-09-19T06:56:47.362 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.012996962, 2024-09-19T06:56:47.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000406155 2024-09-19T06:56:47.362 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.362 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T06:56:47.362 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.363 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 108, 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.581671547, 2024-09-19T06:56:47.364 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005385847 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.365 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 3, 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 2, 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T06:56:47.366 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 551, 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 551, 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 901, 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:47.367 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:47.368 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:47.369 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:47.370 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:47.371 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 312131, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 233, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T06:56:47.372 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8320, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 183, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:47.373 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:47.374 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T06:56:47.374 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 675665, 2024-09-19T06:56:47.374 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 4730, 2024-09-19T06:56:47.374 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:47.374 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T06:56:47.374 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:47.374 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 395, 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 949672, 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 33, 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 395, 2024-09-19T06:56:47.375 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 395, 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.508967969, 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003820172 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 395, 2024-09-19T06:56:47.376 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 442 2024-09-19T06:56:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 395, 2024-09-19T06:56:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 21, 2024-09-19T06:56:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 374, 2024-09-19T06:56:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T06:56:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T06:56:47.377 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 15, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 189, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 21, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-19T06:56:47.378 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 120, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 6, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T06:56:47.379 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T06:56:47.380 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T06:56:47.381 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T06:56:47.382 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 107, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 26, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 16, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 4, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 49, 2024-09-19T06:56:47.383 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-09-19T06:56:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 12 2024-09-19T06:56:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T06:56:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-09-19T06:56:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:56:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T06:56:47.384 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 223 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 10, 2024-09-19T06:56:47.385 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 10, 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 108, 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 108, 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:47.386 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 226232, 2024-09-19T06:56:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 70212071, 2024-09-19T06:56:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 226232, 2024-09-19T06:56:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:47.387 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 226232, 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 70212071, 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.388 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:47.389 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 395, 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 960580, 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 395, 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 960580, 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.390 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T06:56:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T06:56:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:47.391 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 395, 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 395, 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 395, 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 395, 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:47.392 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:47.393 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 108, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 10908, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 108, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 15, 2024-09-19T06:56:47.394 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 10908, 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55f645b84960": { 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:47.395 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 676, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 912080, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 676, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 174, 2024-09-19T06:56:47.396 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 912080, 2024-09-19T06:56:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.397 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:47.397 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T06:56:48.170 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-19T06:56:48.170 INFO:tasks.check_counter:Daemon mds.h mds.imported=2 2024-09-19T06:56:48.170 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=8 2024-09-19T06:56:48.170 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=417 2024-09-19T06:56:48.170 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-19T06:56:48.170 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:56:48.170 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:56:48.170 DEBUG:tasks.check_counter:Getting stats from k 2024-09-19T06:56:48.171 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-19T06:56:48.421 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.b/config 2024-09-19T06:56:48.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:48 smithi096 ceph-mon[35004]: pgmap v476: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.0 MiB/s wr, 112 op/s 2024-09-19T06:56:48.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:48 smithi037 ceph-mon[31211]: pgmap v476: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.0 MiB/s wr, 112 op/s 2024-09-19T06:56:48.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:48 smithi151 ceph-mon[35236]: pgmap v476: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 221 KiB/s rd, 3.0 MiB/s wr, 112 op/s 2024-09-19T06:56:48.945 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-09-19T06:56:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 353, 2024-09-19T06:56:48.945 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 316, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1741202, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 179844, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 1, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.050510433, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.016830106, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.048512810, 2024-09-19T06:56:48.946 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000017729, 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 316, 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.018503845, 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000058556 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.947 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1740368, 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 179520 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 925, 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1139, 2024-09-19T06:56:48.948 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 227400, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 832271, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 7, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.085391462, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.029692967, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.018463037, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.027904983, 2024-09-19T06:56:48.949 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1136, 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.047199663, 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000041548 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 538752 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2246, 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2248, 2024-09-19T06:56:48.951 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 3568540, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 714286, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 4, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.179706198, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.062636152, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.046700810, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.054706130, 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:48.952 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2246, 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.072993976, 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000032499 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.953 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.954 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5328, 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.125740099, 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000023599 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.955 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.956 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.957 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 358, 2024-09-19T06:56:48.958 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 2, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 356, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 355, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 7, 2024-09-19T06:56:48.959 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-09-19T06:56:48.960 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-09-19T06:56:48.961 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:56:48.962 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:56:48.963 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:56:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-09-19T06:56:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:56:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-09-19T06:56:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-09-19T06:56:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-09-19T06:56:48.964 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:56:48.965 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-09-19T06:56:48.966 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-09-19T06:56:48.967 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 1859, 2024-09-19T06:56:48.968 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 3, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-09-19T06:56:48.969 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 7103699, 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 7103100, 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.970 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 1859 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 149, 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 395, 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 246, 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 26, 2024-09-19T06:56:48.971 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 40, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 14, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 358, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 428, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 70, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-09-19T06:56:48.972 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 51636, 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215328 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-09-19T06:56:48.973 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.974 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-09-19T06:56:48.975 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.976 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.977 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.978 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.979 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-09-19T06:56:48.980 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-09-19T06:56:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.981 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.982 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.983 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.984 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-09-19T06:56:48.985 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.986 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.987 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-09-19T06:56:48.988 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.989 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.989 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.989 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-09-19T06:56:48.989 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.990 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.991 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:48.992 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 4, 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-09-19T06:56:48.993 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 3486, 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:48.994 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:48.995 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:48.996 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:48.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:48.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:48.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:48.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:48.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:48.997 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:48.998 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 194610, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 192, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-09-19T06:56:48.999 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 8320, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 183, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:49.000 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 642250, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 4826, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:49.001 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 3170, 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-09-19T06:56:49.002 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 3170, 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3170, 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.011956518, 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000634686 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-09-19T06:56:49.003 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3170, 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3171 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 3170, 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 3170, 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-09-19T06:56:49.004 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 1926, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1242, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-09-19T06:56:49.005 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:56:49.006 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:56:49.007 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-09-19T06:56:49.008 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-09-19T06:56:49.009 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-09-19T06:56:49.009 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-09-19T06:56:49.009 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-09-19T06:56:49.009 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-09-19T06:56:49.009 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-09-19T06:56:49.009 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-09-19T06:56:49.009 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 107, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 26, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 5, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 6, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 1, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-09-19T06:56:49.010 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-09-19T06:56:49.011 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-09-19T06:56:49.011 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-09-19T06:56:49.011 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.011 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-09-19T06:56:49.011 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-09-19T06:56:49.011 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:56:49.011 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-09-19T06:56:49.012 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-09-19T06:56:49.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.012 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-09-19T06:56:49.012 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-09-19T06:56:49.012 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.012 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-09-19T06:56:49.012 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:49.013 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 3520, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 5199579, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 3520, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:49.014 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3520, 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 5199579, 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:49.015 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:49.016 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3170, 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2909400, 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3170, 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2909400, 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:49.017 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:49.018 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 3170, 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 3170, 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 3170, 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 3170, 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:49.019 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:49.020 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:49.021 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55cec18d8b40": { 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-09-19T06:56:49.022 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-09-19T06:56:49.023 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-09-19T06:56:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-09-19T06:56:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-09-19T06:56:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-09-19T06:56:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:49.024 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-09-19T06:56:49.024 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-09-19T06:56:49.764 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-19T06:56:49.764 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-19T06:56:49.764 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-19T06:56:49.764 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-19T06:56:49.764 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-09-19T06:56:49.764 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:56:49.764 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:56:49.765 DEBUG:tasks.check_counter:Getting stats from c 2024-09-19T06:56:49.765 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-19T06:56:50.026 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:56:50.419 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:50 smithi151 ceph-mon[35236]: pgmap v477: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 119 op/s 2024-09-19T06:56:50.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:50 smithi096 ceph-mon[35004]: pgmap v477: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 119 op/s 2024-09-19T06:56:50.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:50 smithi037 ceph-mon[31211]: pgmap v477: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 236 KiB/s rd, 3.0 MiB/s wr, 119 op/s 2024-09-19T06:56:50.558 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T06:56:50.558 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:50.558 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 2950, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2950, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1747233, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 937283, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.232232138, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.076088661, 2024-09-19T06:56:50.559 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.061133459, 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.077724634, 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2949, 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.107005413, 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000036285 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:50.560 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:50.561 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 347, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 312, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1737036, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 177850, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 3, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 2, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.050897470, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.018867866, 2024-09-19T06:56:50.562 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.046647380, 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000082706, 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 311, 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.018443788, 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000059304 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.563 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1736480, 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 177152 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-09-19T06:56:50.564 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 215, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 748, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 626942, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 8, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.013237930, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.008063287, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.002561580, 2024-09-19T06:56:50.565 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000031659, 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 212, 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.017827333, 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000084091 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.566 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 626624 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-19T06:56:50.567 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 4977, 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.122353854, 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000024583 2024-09-19T06:56:50.568 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.569 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.569 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-19T06:56:50.569 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T06:56:50.569 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T06:56:50.569 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.569 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.569 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.570 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-09-19T06:56:50.571 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 148, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 0, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 148, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 148, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-19T06:56:50.572 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-19T06:56:50.573 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-19T06:56:50.573 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 4, 2024-09-19T06:56:50.573 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-19T06:56:50.573 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-19T06:56:50.573 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-19T06:56:50.573 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-19T06:56:50.574 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 0, 2024-09-19T06:56:50.575 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:56:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-19T06:56:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:56:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:56:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:56:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:56:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:56:50.576 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-19T06:56:50.577 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-19T06:56:50.578 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-19T06:56:50.579 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:56:50.580 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-19T06:56:50.581 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 676, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-19T06:56:50.582 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 2, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 2, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 4194304, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 5106384, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 5105810, 2024-09-19T06:56:50.583 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 676 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-19T06:56:50.584 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 41, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 149, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 108, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 16, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 16, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 0, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 148, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 148, 2024-09-19T06:56:50.585 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 0, 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 50188, 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215328 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-19T06:56:50.586 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.587 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.588 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.589 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.590 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-19T06:56:50.591 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.592 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.593 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.594 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.595 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-19T06:56:50.596 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.597 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.598 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.599 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.600 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-19T06:56:50.601 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.602 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.603 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.604 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-19T06:56:50.605 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 1, 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 3, 2024-09-19T06:56:50.606 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 2, 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 3, 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 4701, 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.607 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:50.608 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:50.609 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:50.610 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T06:56:50.611 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 182523, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 79, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:50.612 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8320, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 183, 2024-09-19T06:56:50.613 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 224959, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 1502, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T06:56:50.614 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 2949, 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 21, 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 3, 2024-09-19T06:56:50.615 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 2949, 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2949, 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 1.664978785, 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000564590 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-19T06:56:50.616 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 2949, 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 2950 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 2949, 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 2949, 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-19T06:56:50.617 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 1912, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 1035, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-19T06:56:50.618 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 1, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-19T06:56:50.619 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-19T06:56:50.620 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-19T06:56:50.621 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 107, 2024-09-19T06:56:50.622 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 26, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 4, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 5, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 1, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-09-19T06:56:50.623 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.624 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-19T06:56:50.625 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 3297, 2024-09-19T06:56:50.626 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 3167637, 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 3297, 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 3297, 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 3167637, 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:50.627 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T06:56:50.628 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2949, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 912085, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2949, 2024-09-19T06:56:50.629 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 912085, 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-19T06:56:50.630 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 2949, 2024-09-19T06:56:50.631 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 2949, 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 2949, 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 2949, 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.632 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:50.633 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.634 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x559e89420b40": { 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:50.635 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:50.636 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:50.636 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:50.636 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:50.636 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T06:56:50.636 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T06:56:50.636 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:50.636 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:50.637 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:50.637 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:50.637 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.637 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:50.637 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T06:56:51.504 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-19T06:56:51.505 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-19T06:56:51.505 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-19T06:56:51.505 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-19T06:56:51.505 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-09-19T06:56:51.505 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:56:51.505 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:56:51.506 DEBUG:tasks.check_counter:Getting stats from f 2024-09-19T06:56:51.506 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-19T06:56:51.753 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:56:52.240 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T06:56:52.240 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:52.240 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 291, 2024-09-19T06:56:52.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 280, 2024-09-19T06:56:52.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1606368, 2024-09-19T06:56:52.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 172384, 2024-09-19T06:56:52.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-09-19T06:56:52.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-09-19T06:56:52.241 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.046845529, 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.017700474, 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.042401355, 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 279, 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.016250773, 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000058246 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.242 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1606368, 2024-09-19T06:56:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 172384 2024-09-19T06:56:52.243 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.244 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:52.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 3, 2024-09-19T06:56:52.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 215, 2024-09-19T06:56:52.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 480, 2024-09-19T06:56:52.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 274144, 2024-09-19T06:56:52.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-19T06:56:52.244 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.013636921, 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.009141802, 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001828748, 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 212, 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.020581438, 2024-09-19T06:56:52.245 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000097082 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T06:56:52.246 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 274144 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 4, 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 4, 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 3072, 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 448, 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-09-19T06:56:52.247 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.002270740, 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000039735, 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.002228183, 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 3, 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.004002362, 2024-09-19T06:56:52.248 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001334120 2024-09-19T06:56:52.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.249 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:52.249 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:52.249 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:52.249 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:52.249 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.249 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-19T06:56:52.250 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 448 2024-09-19T06:56:52.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.250 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T06:56:52.250 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-09-19T06:56:52.250 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T06:56:52.250 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.250 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T06:56:52.251 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:52.251 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:52.251 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:52.251 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:52.251 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:52.251 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:52.251 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:52.252 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:52.253 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:52.253 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:52.253 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:52.253 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:52.253 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:52.253 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:52.253 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:52.254 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 147930, 2024-09-19T06:56:52.255 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 28, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:52.256 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-09-19T06:56:52.257 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:52.258 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 294, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1565306, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 294, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:52.259 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 294, 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1565306, 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:52.260 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T06:56:52.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:52 smithi096 ceph-mon[35004]: pgmap v478: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 66 op/s 2024-09-19T06:56:52.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:52 smithi037 ceph-mon[31211]: pgmap v478: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 66 op/s 2024-09-19T06:56:52.562 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:52 smithi151 ceph-mon[35236]: pgmap v478: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 1.3 MiB/s wr, 66 op/s 2024-09-19T06:56:52.950 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-09-19T06:56:52.950 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-09-19T06:56:52.951 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-09-19T06:56:52.951 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-09-19T06:56:52.951 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-09-19T06:56:52.951 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-09-19T06:56:52.951 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-09-19T06:56:52.951 DEBUG:tasks.check_counter:Getting stats from i 2024-09-19T06:56:52.951 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-19T06:56:53.197 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:56:53.702 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T06:56:53.703 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:53.703 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 119, 2024-09-19T06:56:53.703 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 121, 2024-09-19T06:56:53.703 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 4237175, 2024-09-19T06:56:53.703 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 37861, 2024-09-19T06:56:53.703 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 8, 2024-09-19T06:56:53.704 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-19T06:56:53.704 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.017913772, 2024-09-19T06:56:53.704 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.003989128, 2024-09-19T06:56:53.704 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.007346369, 2024-09-19T06:56:53.704 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.004329475, 2024-09-19T06:56:53.704 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:53.704 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 119, 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.012132252, 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000101951 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.705 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.706 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:56:53.706 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:56:53.706 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.706 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:53.706 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 269, 2024-09-19T06:56:53.706 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 266, 2024-09-19T06:56:53.706 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 37997024, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 91225, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 5, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 4, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.041066316, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.007162035, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.030282453, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.004628572, 2024-09-19T06:56:53.707 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 265, 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.011848053, 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000044709 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.708 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 174624, 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 19936 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 70, 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 91, 2024-09-19T06:56:53.709 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 12603167, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 147151, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 5, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.016092492, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.002470962, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.011265096, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.001673904, 2024-09-19T06:56:53.710 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 90, 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.011216064, 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000124622 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.711 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.712 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-19T06:56:53.712 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 126480 2024-09-19T06:56:53.712 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.712 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 1, 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-MDSRank": { 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 658, 2024-09-19T06:56:53.713 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.042802529, 2024-09-19T06:56:53.714 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000065049 2024-09-19T06:56:53.714 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.714 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.714 INFO:teuthology.orchestra.run.smithi151.stdout: "finisher-PurgeQueue": { 2024-09-19T06:56:53.714 INFO:teuthology.orchestra.run.smithi151.stdout: "queue_len": 0, 2024-09-19T06:56:53.714 INFO:teuthology.orchestra.run.smithi151.stdout: "complete_latency": { 2024-09-19T06:56:53.714 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: "mds": { 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: "request": 0, 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: "reply": 0, 2024-09-19T06:56:53.715 INFO:teuthology.orchestra.run.smithi151.stdout: "reply_latency": { 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_reply": 0, 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: "forward": 0, 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_complete": 0, 2024-09-19T06:56:53.716 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_fetch_keys": 0, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_commit": 0, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_split": 0, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_merge": 0, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes": 6344, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_top": 2393, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_bottom": 3951, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pin_tail": 0, 2024-09-19T06:56:53.717 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_pinned": 2932, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_expired": 0, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "inodes_with_caps": 0, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "caps": 0, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "subtrees": 9, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse": 0, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_hit": 0, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_forward": 0, 2024-09-19T06:56:53.718 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_discover": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_dir_fetch": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_remote_ino": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "traverse_lock": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "load_cent": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "q": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "exported": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "exported_inodes": 0, 2024-09-19T06:56:53.719 INFO:teuthology.orchestra.run.smithi151.stdout: "imported": 0, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "imported_inodes": 0, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_dir_fetch": 0, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_backtrace_fetch": 0, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "openino_peer_discover": 0, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rfiles": 743, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rbytes": 231573210, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "root_rsnaps": 6, 2024-09-19T06:56:53.720 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_fetch": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_set_tag": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_backtrace_repaired": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_inotable_repaired": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_inodes": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dir_base_inodes": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_dirfrag_rstats": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "scrub_file_inodes": 0, 2024-09-19T06:56:53.721 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_inode_file_caps": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_revoke": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_grant": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_trunc": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_caps_dirty": 0, 2024-09-19T06:56:53.722 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_cap_release": 0, 2024-09-19T06:56:53.723 INFO:teuthology.orchestra.run.smithi151.stdout: "process_request_cap_release": 0 2024-09-19T06:56:53.723 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.723 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_cache": { 2024-09-19T06:56:53.723 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update": 0, 2024-09-19T06:56:53.723 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_update_receipt": 0, 2024-09-19T06:56:53.723 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_try_discover": 0, 2024-09-19T06:56:53.723 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_send_discover": 0, 2024-09-19T06:56:53.724 INFO:teuthology.orchestra.run.smithi151.stdout: "dir_handle_discover": 0, 2024-09-19T06:56:53.724 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays": 0, 2024-09-19T06:56:53.724 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_delayed": 0, 2024-09-19T06:56:53.724 INFO:teuthology.orchestra.run.smithi151.stdout: "num_strays_enqueuing": 0, 2024-09-19T06:56:53.724 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_created": 0, 2024-09-19T06:56:53.724 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_enqueued": 0, 2024-09-19T06:56:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_reintegrated": 0, 2024-09-19T06:56:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: "strays_migrated": 0, 2024-09-19T06:56:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_processing": 0, 2024-09-19T06:56:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_enqueued": 0, 2024-09-19T06:56:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: "num_recovering_prioritized": 0, 2024-09-19T06:56:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_started": 0, 2024-09-19T06:56:53.725 INFO:teuthology.orchestra.run.smithi151.stdout: "recovery_completed": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_path": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_quiesce_inode": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_enqueue_scrub": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_exportdir": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_flush": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragmentdir": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_fragstats": 0, 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: "ireq_inodestats": 0 2024-09-19T06:56:53.726 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.727 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_log": { 2024-09-19T06:56:53.727 INFO:teuthology.orchestra.run.smithi151.stdout: "evlrg": 0, 2024-09-19T06:56:53.727 INFO:teuthology.orchestra.run.smithi151.stdout: "evadd": 0, 2024-09-19T06:56:53.727 INFO:teuthology.orchestra.run.smithi151.stdout: "evex": 0, 2024-09-19T06:56:53.727 INFO:teuthology.orchestra.run.smithi151.stdout: "evtrm": 0, 2024-09-19T06:56:53.727 INFO:teuthology.orchestra.run.smithi151.stdout: "ev": 21132, 2024-09-19T06:56:53.727 INFO:teuthology.orchestra.run.smithi151.stdout: "evexg": 0, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "evexd": 0, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "segadd": 0, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "segex": 0, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "segtrm": 0, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "seg": 7, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "segmjr": 5, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "segexg": 0, 2024-09-19T06:56:53.728 INFO:teuthology.orchestra.run.smithi151.stdout: "segexd": 0, 2024-09-19T06:56:53.729 INFO:teuthology.orchestra.run.smithi151.stdout: "expos": 704263327, 2024-09-19T06:56:53.729 INFO:teuthology.orchestra.run.smithi151.stdout: "wrpos": 714201533, 2024-09-19T06:56:53.729 INFO:teuthology.orchestra.run.smithi151.stdout: "rdpos": 714188960, 2024-09-19T06:56:53.729 INFO:teuthology.orchestra.run.smithi151.stdout: "jlat": { 2024-09-19T06:56:53.729 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.729 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.729 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.730 INFO:teuthology.orchestra.run.smithi151.stdout: "replayed": 21132 2024-09-19T06:56:53.730 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.730 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_mem": { 2024-09-19T06:56:53.730 INFO:teuthology.orchestra.run.smithi151.stdout: "ino": 1441, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "ino+": 5649, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "ino-": 4208, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "dir": 185, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "dir+": 208, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "dir-": 23, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "dn": 6344, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "dn+": 6452, 2024-09-19T06:56:53.731 INFO:teuthology.orchestra.run.smithi151.stdout: "dn-": 108, 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: "cap": 0, 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: "cap+": 0, 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: "cap-": 0, 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: "rss": 60444, 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: "heap": 215328 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_server": { 2024-09-19T06:56:53.732 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_client_request": 0, 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "dispatch_server_request": 0, 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_request": 0, 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_client_session": 0, 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "handle_peer_request": 0, 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "req_create_latency": { 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.733 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.734 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getattr_latency": { 2024-09-19T06:56:53.734 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.734 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.734 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.734 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.734 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getfilelock_latency": { 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: "req_link_latency": { 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.735 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.736 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookup_latency": { 2024-09-19T06:56:53.736 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.736 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.736 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.736 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.736 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookuphash_latency": { 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupino_latency": { 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.737 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.738 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupname_latency": { 2024-09-19T06:56:53.738 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.738 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.738 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.738 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.738 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupparent_latency": { 2024-09-19T06:56:53.738 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lookupsnap_latency": { 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.739 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: "req_lssnap_latency": { 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mkdir_latency": { 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.740 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.741 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.741 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mknod_latency": { 2024-09-19T06:56:53.741 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.741 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.741 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.741 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "req_mksnap_latency": { 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "req_open_latency": { 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.742 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: "req_readdir_latency": { 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rename_latency": { 2024-09-19T06:56:53.743 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: "req_renamesnap_latency": { 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.744 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: "req_snapdiff_latency": { 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmdir_latency": { 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.745 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmsnap_latency": { 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.746 INFO:teuthology.orchestra.run.smithi151.stdout: "req_rmxattr_latency": { 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setattr_latency": { 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.747 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setdirlayout_latency": { 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setfilelock_latency": { 2024-09-19T06:56:53.748 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setlayout_latency": { 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.749 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: "req_setxattr_latency": { 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: "req_symlink_latency": { 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.750 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: "req_unlink_latency": { 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.751 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_revoke_eviction": 0, 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: "cap_acquisition_throttle": 0, 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: "req_getvxattr_latency": { 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.752 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_sessions": { 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "session_count": 2, 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "session_add": 2, 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "session_remove": 0, 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_open": 0, 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "sessions_stale": 0, 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "total_load": 0, 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "average_load": 0, 2024-09-19T06:56:53.753 INFO:teuthology.orchestra.run.smithi151.stdout: "avg_session_uptime": 2172, 2024-09-19T06:56:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: "mdthresh_evicted": 0 2024-09-19T06:56:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T06:56:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:53.754 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:53.755 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:53.756 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:53.757 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:53.758 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 288893, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 1688, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T06:56:53.759 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 8320, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 183, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:53.760 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 8255507, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 63033, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:53.761 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T06:56:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "objecter": { 2024-09-19T06:56:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "op_active": 0, 2024-09-19T06:56:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "op_laggy": 0, 2024-09-19T06:56:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send": 408, 2024-09-19T06:56:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "op_send_bytes": 26, 2024-09-19T06:56:53.762 INFO:teuthology.orchestra.run.smithi151.stdout: "op_resend": 4, 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "op_reply": 408, 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "op_latency": { 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 408, 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.423994639, 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.001039202 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "op_inflight": 0, 2024-09-19T06:56:53.763 INFO:teuthology.orchestra.run.smithi151.stdout: "oplen_avg": { 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 408, 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 409 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: "op": 408, 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: "op_r": 408, 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: "op_w": 0, 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: "op_rmw": 0, 2024-09-19T06:56:53.764 INFO:teuthology.orchestra.run.smithi151.stdout: "op_pg": 0, 2024-09-19T06:56:53.765 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_stat": 232, 2024-09-19T06:56:53.765 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_create": 0, 2024-09-19T06:56:53.765 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_read": 173, 2024-09-19T06:56:53.765 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_write": 0, 2024-09-19T06:56:53.765 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writefull": 0, 2024-09-19T06:56:53.765 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_writesame": 0, 2024-09-19T06:56:53.765 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_append": 0, 2024-09-19T06:56:53.766 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_zero": 0, 2024-09-19T06:56:53.766 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_truncate": 0, 2024-09-19T06:56:53.766 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_delete": 0, 2024-09-19T06:56:53.766 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_mapext": 0, 2024-09-19T06:56:53.766 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_sparse_read": 0, 2024-09-19T06:56:53.766 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_clonerange": 0, 2024-09-19T06:56:53.766 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_getxattr": 2, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_setxattr": 0, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_cmpxattr": 0, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_rmxattr": 0, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_resetxattrs": 0, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_call": 0, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_watch": 0, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_notify": 0, 2024-09-19T06:56:53.767 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_src_cmpxattr": 0, 2024-09-19T06:56:53.768 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls": 0, 2024-09-19T06:56:53.768 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_pgls_filter": 0, 2024-09-19T06:56:53.768 INFO:teuthology.orchestra.run.smithi151.stdout: "osdop_other": 0, 2024-09-19T06:56:53.768 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_active": 0, 2024-09-19T06:56:53.768 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_send": 0, 2024-09-19T06:56:53.768 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_resend": 0, 2024-09-19T06:56:53.768 INFO:teuthology.orchestra.run.smithi151.stdout: "linger_ping": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_active": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_send": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "poolop_resend": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_active": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_send": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "poolstat_resend": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_active": 0, 2024-09-19T06:56:53.769 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_send": 0, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "statfs_resend": 0, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "command_active": 0, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "command_send": 0, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "command_resend": 0, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "map_epoch": 107, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "map_full": 0, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "map_inc": 3, 2024-09-19T06:56:53.770 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_sessions": 10, 2024-09-19T06:56:53.771 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_open": 10, 2024-09-19T06:56:53.771 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_session_close": 0, 2024-09-19T06:56:53.771 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_laggy": 0, 2024-09-19T06:56:53.771 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_wr": 0, 2024-09-19T06:56:53.771 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_rd": 2, 2024-09-19T06:56:53.771 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_del": 0 2024-09-19T06:56:53.771 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: "oft": { 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_objs": 0, 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_kv_pairs": 0, 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_updates": 0, 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: "omap_total_removes": 0 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: "osd-slow-ops": { 2024-09-19T06:56:53.772 INFO:teuthology.orchestra.run.smithi151.stdout: "slow_ops_count": 0 2024-09-19T06:56:53.773 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.773 INFO:teuthology.orchestra.run.smithi151.stdout: "purge_queue": { 2024-09-19T06:56:53.773 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops": 0, 2024-09-19T06:56:53.773 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_ops_high_water": 0, 2024-09-19T06:56:53.773 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing": 0, 2024-09-19T06:56:53.773 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executing_high_water": 0, 2024-09-19T06:56:53.773 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed_ops": 0, 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_executed": 0, 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: "pq_item_in_journal": 0 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:53.774 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 454, 2024-09-19T06:56:53.775 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 54791172, 2024-09-19T06:56:53.775 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:53.775 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 454, 2024-09-19T06:56:53.775 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:53.775 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:53.775 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 454, 2024-09-19T06:56:53.775 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 54791172, 2024-09-19T06:56:53.776 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:53.776 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.776 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.776 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.776 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.776 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.777 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_bytes": { 2024-09-19T06:56:53.777 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:53.777 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T06:56:53.777 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:53.777 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:53.777 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:53.777 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 408, 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 54541219, 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 408, 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 54541219, 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.778 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-objecter_ops": { 2024-09-19T06:56:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 1024, 2024-09-19T06:56:53.779 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 408, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 408, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 408, 2024-09-19T06:56:53.780 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 408, 2024-09-19T06:56:53.781 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:53.781 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.781 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.781 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.781 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.781 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.781 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle": { 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:53.782 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.783 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-write_buf_throttle-0x558a47d346e0": { 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 3758096384, 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 0, 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 0, 2024-09-19T06:56:53.784 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 0, 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 0, 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 0, 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:53.785 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:53.786 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:53.786 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.786 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:53.786 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T06:56:54.469 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:54 smithi151 ceph-mon[35236]: pgmap v479: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-09-19T06:56:54.496 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-19T06:56:54.496 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-19T06:56:54.496 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-19T06:56:54.496 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-19T06:56:54.497 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-09-19T06:56:54.497 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-09-19T06:56:54.497 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-09-19T06:56:54.497 DEBUG:tasks.check_counter:Getting stats from l 2024-09-19T06:56:54.497 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-19T06:56:54.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:54 smithi096 ceph-mon[35004]: pgmap v479: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-09-19T06:56:54.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:54 smithi037 ceph-mon[31211]: pgmap v479: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 128 KiB/s rd, 1.3 MiB/s wr, 69 op/s 2024-09-19T06:56:54.745 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.c/config 2024-09-19T06:56:55.258 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-09-19T06:56:55.258 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-0": { 2024-09-19T06:56:55.258 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 0, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 2, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 0, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 0, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 4, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 0, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.001056010, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.000000000, 2024-09-19T06:56:55.259 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.000162132, 2024-09-19T06:56:55.260 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:55.260 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:55.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:55.260 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:55.260 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:55.260 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.260 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 0 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.261 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-1": { 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 300, 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 287, 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 1610800, 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 176032, 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 2, 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 1, 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.050462178, 2024-09-19T06:56:55.262 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.018366469, 2024-09-19T06:56:55.263 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.047373088, 2024-09-19T06:56:55.263 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:55.263 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:55.263 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 287, 2024-09-19T06:56:55.263 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.020856341, 2024-09-19T06:56:55.263 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000072670 2024-09-19T06:56:55.263 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 1610800, 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 176032 2024-09-19T06:56:55.264 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "AsyncMessenger::Worker-2": { 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_messages": 3, 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages": 219, 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_bytes": 480, 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_bytes": 274848, 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_created_connections": 6, 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_active_connections": 3, 2024-09-19T06:56:55.265 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_total_time": 0.014338366, 2024-09-19T06:56:55.266 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_send_time": 0.009774833, 2024-09-19T06:56:55.266 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_recv_time": 0.001811549, 2024-09-19T06:56:55.266 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-19T06:56:55.266 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_messages_queue_lat": { 2024-09-19T06:56:55.266 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 216, 2024-09-19T06:56:55.266 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.020893760, 2024-09-19T06:56:55.266 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000096730 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_handle_ack_lat": { 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_recv_encrypted_bytes": 480, 2024-09-19T06:56:55.267 INFO:teuthology.orchestra.run.smithi151.stdout: "msgr_send_encrypted_bytes": 274848 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: "cct": { 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: "total_workers": 0, 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: "unhealthy_workers": 0 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: "mempool": { 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_bytes": 0, 2024-09-19T06:56:55.268 INFO:teuthology.orchestra.run.smithi151.stdout: "bloom_filter_items": 0, 2024-09-19T06:56:55.269 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_bytes": 0, 2024-09-19T06:56:55.269 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_alloc_items": 0, 2024-09-19T06:56:55.269 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_bytes": 0, 2024-09-19T06:56:55.269 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_data_items": 0, 2024-09-19T06:56:55.269 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-19T06:56:55.269 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_onode_items": 0, 2024-09-19T06:56:55.269 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_meta_items": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_bytes": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_other_items": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_cache_buffer_items": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_bytes": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_extent_items": 0, 2024-09-19T06:56:55.270 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_bytes": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_blob_items": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_shared_blob_items": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_inline_bl_items": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_bytes": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_fsck_items": 0, 2024-09-19T06:56:55.271 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_bytes": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_txc_items": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_deferred_items": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_bytes": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluestore_writing_items": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_bytes": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_items": 0, 2024-09-19T06:56:55.272 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_bytes": 0, 2024-09-19T06:56:55.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_reader_items": 0, 2024-09-19T06:56:55.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_bytes": 0, 2024-09-19T06:56:55.273 INFO:teuthology.orchestra.run.smithi151.stdout: "bluefs_file_writer_items": 0, 2024-09-19T06:56:55.273 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_bytes": 147960, 2024-09-19T06:56:55.273 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_anon_items": 28, 2024-09-19T06:56:55.273 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_bytes": 0, 2024-09-19T06:56:55.273 INFO:teuthology.orchestra.run.smithi151.stdout: "buffer_meta_items": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_bytes": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_items": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_bytes": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_mapbl_items": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_bytes": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osd_pglog_items": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_bytes": 0, 2024-09-19T06:56:55.274 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_items": 0, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_bytes": 0, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "osdmap_mapping_items": 0, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_bytes": 0, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "pgmap_items": 0, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_bytes": 928, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "mds_co_items": 2, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_bytes": 0, 2024-09-19T06:56:55.275 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_1_items": 0, 2024-09-19T06:56:55.276 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_bytes": 0, 2024-09-19T06:56:55.276 INFO:teuthology.orchestra.run.smithi151.stdout: "unittest_2_items": 0 2024-09-19T06:56:55.276 INFO:teuthology.orchestra.run.smithi151.stdout: }, 2024-09-19T06:56:55.276 INFO:teuthology.orchestra.run.smithi151.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-19T06:56:55.276 INFO:teuthology.orchestra.run.smithi151.stdout: "val": 0, 2024-09-19T06:56:55.276 INFO:teuthology.orchestra.run.smithi151.stdout: "max": 104857600, 2024-09-19T06:56:55.277 INFO:teuthology.orchestra.run.smithi151.stdout: "get_started": 0, 2024-09-19T06:56:55.277 INFO:teuthology.orchestra.run.smithi151.stdout: "get": 299, 2024-09-19T06:56:55.277 INFO:teuthology.orchestra.run.smithi151.stdout: "get_sum": 1565956, 2024-09-19T06:56:55.277 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_fail": 0, 2024-09-19T06:56:55.277 INFO:teuthology.orchestra.run.smithi151.stdout: "get_or_fail_success": 299, 2024-09-19T06:56:55.277 INFO:teuthology.orchestra.run.smithi151.stdout: "take": 0, 2024-09-19T06:56:55.277 INFO:teuthology.orchestra.run.smithi151.stdout: "take_sum": 0, 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: "put": 299, 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: "put_sum": 1565956, 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: "wait": { 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: "avgcount": 0, 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: "sum": 0.000000000, 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: "avgtime": 0.000000000 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:55.278 INFO:teuthology.orchestra.run.smithi151.stdout: } 2024-09-19T06:56:55.279 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-09-19T06:56:56.146 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-19T06:56:56.146 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-19T06:56:56.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-19T06:56:56.146 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-19T06:56:56.146 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-19T06:56:56.146 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-19T06:56:56.146 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-19T06:56:56.147 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-19T06:56:56.157 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-19T06:56:56.157 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:56:56.157 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:56:56.179 DEBUG:teuthology.orchestra.run.smithi037:> set -ex 2024-09-19T06:56:56.179 DEBUG:teuthology.orchestra.run.smithi037:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-19T06:56:56.239 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-19T06:56:56.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:56 smithi096 ceph-mon[35004]: pgmap v480: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-09-19T06:56:56.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:56 smithi037 ceph-mon[31211]: pgmap v480: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-09-19T06:56:56.559 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:56.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:56 smithi151 ceph-mon[35236]: pgmap v480: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-09-19T06:56:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 -- 172.21.15.37:0/4084105640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 msgr2=0x7fcd2010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 --2- 172.21.15.37:0/4084105640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2010e610 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009ca0 tx=0x7fcd1802a950 comp rx=0 tx=0).stop 2024-09-19T06:56:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 -- 172.21.15.37:0/4084105640 shutdown_connections 2024-09-19T06:56:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 --2- 172.21.15.37:0/4084105640 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2010e610 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.995 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 --2- 172.21.15.37:0/4084105640 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd20103660 0x7fcd2010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 --2- 172.21.15.37:0/4084105640 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd20102d40 0x7fcd20103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 -- 172.21.15.37:0/4084105640 >> 172.21.15.37:0/4084105640 conn(0x7fcd200fc7a0 msgr2=0x7fcd200febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 -- 172.21.15.37:0/4084105640 shutdown_connections 2024-09-19T06:56:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 -- 172.21.15.37:0/4084105640 wait complete. 2024-09-19T06:56:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.993+0000 7fcd25892640 1 Processor -- start 2024-09-19T06:56:56.996 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd25892640 1 -- start start 2024-09-19T06:56:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd25892640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd20102d40 0x7fcd201984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd25892640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd20103660 0x7fcd20198a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd25892640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd25892640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20111fb0 con 0x7fcd2010c170 2024-09-19T06:56:56.998 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd25892640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20111e30 con 0x7fcd20102d40 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd25892640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20112130 con 0x7fcd20103660 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36838/0 (socket says 172.21.15.37:36838) 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 -- 172.21.15.37:0/3605826569 learned_addr learned my addr 172.21.15.37:0/3605826569 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1e7fc640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd20103660 0x7fcd20198a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1effd640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd20102d40 0x7fcd201984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 -- 172.21.15.37:0/3605826569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd20103660 msgr2=0x7fcd20198a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd20103660 0x7fcd20198a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:56.999 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 -- 172.21.15.37:0/3605826569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd20102d40 msgr2=0x7fcd201984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd20102d40 0x7fcd201984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1f7fe640 1 -- 172.21.15.37:0/3605826569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd2019d490 con 0x7fcd2010c170 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.994+0000 7fcd1effd640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd20102d40 0x7fcd201984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.995+0000 7fcd1f7fe640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2019cdb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcd18006010 tx=0x7fcd1802f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.995+0000 7fcd24890640 1 -- 172.21.15.37:0/3605826569 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd18034040 con 0x7fcd2010c170 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.995+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd2019d720 con 0x7fcd2010c170 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.995+0000 7fcd24890640 1 -- 172.21.15.37:0/3605826569 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcd18003e00 con 0x7fcd2010c170 2024-09-19T06:56:57.000 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.995+0000 7fcd24890640 1 -- 172.21.15.37:0/3605826569 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180456e0 con 0x7fcd2010c170 2024-09-19T06:56:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.995+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd201a5060 con 0x7fcd2010c170 2024-09-19T06:56:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.996+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd20103120 con 0x7fcd2010c170 2024-09-19T06:56:57.001 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.998+0000 7fcd24890640 1 -- 172.21.15.37:0/3605826569 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fcd18037020 con 0x7fcd2010c170 2024-09-19T06:56:57.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.999+0000 7fcd24890640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd080a33b0 0x7fcd080a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:57.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.999+0000 7fcd1effd640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd080a33b0 0x7fcd080a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:57.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.999+0000 7fcd24890640 1 -- 172.21.15.37:0/3605826569 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fcd180f0f10 con 0x7fcd2010c170 2024-09-19T06:56:57.002 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:56.999+0000 7fcd1effd640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd080a33b0 0x7fcd080a5870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009930 tx=0x7fcd10008040 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:57.003 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.002+0000 7fcd24890640 1 -- 172.21.15.37:0/3605826569 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fcd18081210 con 0x7fcd2010c170 2024-09-19T06:56:57.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.190+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcd20107270 con 0x7fcd2010c170 2024-09-19T06:56:57.192 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.190+0000 7fcd24890640 1 -- 172.21.15.37:0/3605826569 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v107) ==== 81+0+942 (secure 0 0 0) 0x7fcd180bd9d0 con 0x7fcd2010c170 2024-09-19T06:56:57.192 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3406854118 2024-09-20T06:41:46.581265+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1436144832 2024-09-20T06:41:46.581265+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/871019432 2024-09-20T06:41:46.581265+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2591230747 2024-09-20T06:32:32.108497+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3313349719 2024-09-20T06:32:32.108497+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/180594249 2024-09-20T06:32:32.108497+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/2641687433 2024-09-20T06:32:56.109611+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2601148313 2024-09-20T06:32:56.109611+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1459260343 2024-09-20T06:32:56.109611+0000 2024-09-19T06:56:57.193 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2963797247 2024-09-20T06:32:32.108497+0000 2024-09-19T06:56:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/3941836172 2024-09-20T06:32:56.109611+0000 2024-09-19T06:56:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/2963797247 2024-09-20T06:32:32.108497+0000 2024-09-19T06:56:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6801/2601148313 2024-09-20T06:32:56.109611+0000 2024-09-19T06:56:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/1644258429 2024-09-20T06:41:46.581265+0000 2024-09-19T06:56:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:0/711456420 2024-09-20T06:41:46.581265+0000 2024-09-19T06:56:57.194 INFO:teuthology.orchestra.run.smithi037.stdout:172.21.15.37:6800/871019432 2024-09-20T06:41:46.581265+0000 2024-09-19T06:56:57.195 INFO:teuthology.orchestra.run.smithi037.stderr:listed 16 entries 2024-09-19T06:56:57.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd080a33b0 msgr2=0x7fcd080a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:57.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd080a33b0 0x7fcd080a5870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fcd10009930 tx=0x7fcd10008040 comp rx=0 tx=0).stop 2024-09-19T06:56:57.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 msgr2=0x7fcd2019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:57.195 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2019cdb0 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fcd18006010 tx=0x7fcd1802f710 comp rx=0 tx=0).stop 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 shutdown_connections 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fcd080a33b0 0x7fcd080a5870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fcd2010c170 0x7fcd2019cdb0 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fcd20103660 0x7fcd20198a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 --2- 172.21.15.37:0/3605826569 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcd20102d40 0x7fcd201984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 >> 172.21.15.37:0/3605826569 conn(0x7fcd200fc7a0 msgr2=0x7fcd200fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 shutdown_connections 2024-09-19T06:56:57.196 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:57.192+0000 7fcd25892640 1 -- 172.21.15.37:0/3605826569 wait complete. 2024-09-19T06:56:57.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:57 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3605826569' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:56:57.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:57 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3605826569' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:56:57.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:57 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3605826569' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-19T06:56:58.011 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-19T06:56:58.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:56:58.012 DEBUG:teuthology.orchestra.run.smithi037:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-19T06:56:58.130 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T06:56:58.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:56:58.130 DEBUG:teuthology.orchestra.run.smithi037:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-19T06:56:58.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:56:58.202 DEBUG:teuthology.orchestra.run.smithi037:> ip netns list 2024-09-19T06:56:58.253 INFO:teuthology.orchestra.run.smithi037.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-19T06:56:58.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:56:58.254 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-19T06:56:58.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-19T06:56:58.317 DEBUG:teuthology.orchestra.run.smithi037:> sudo ip link delete ceph-brx 2024-09-19T06:56:58.417 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-19T06:56:58.428 INFO:tasks.cephadm:Teardown begin 2024-09-19T06:56:58.428 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-19T06:56:58.428 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:56:58.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:56:58 smithi096 ceph-mon[35004]: pgmap v481: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-19T06:56:58.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:56:58 smithi037 ceph-mon[31211]: pgmap v481: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-19T06:56:58.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:56:58 smithi151 ceph-mon[35236]: pgmap v481: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 19 KiB/s wr, 10 op/s 2024-09-19T06:56:58.710 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 -- 172.21.15.37:0/272823486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8102790 msgr2=0x7f10e81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 --2- 172.21.15.37:0/272823486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8102790 0x7f10e81067e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f10d8009900 tx=0x7f10d802a410 comp rx=0 tx=0).stop 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 -- 172.21.15.37:0/272823486 shutdown_connections 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 --2- 172.21.15.37:0/272823486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e8106f10 0x7f10e810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.208 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 --2- 172.21.15.37:0/272823486 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8102790 0x7f10e81067e0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 --2- 172.21.15.37:0/272823486 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8101de0 0x7f10e81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 -- 172.21.15.37:0/272823486 >> 172.21.15.37:0/272823486 conn(0x7f10e80fd8c0 msgr2=0x7f10e80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 -- 172.21.15.37:0/272823486 shutdown_connections 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 -- 172.21.15.37:0/272823486 wait complete. 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 Processor -- start 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.182+0000 7f10eff5f640 1 -- start start 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10eff5f640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8101de0 0x7f10e80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.209 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10eff5f640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8102790 0x7f10e8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10eff5f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e8106f10 0x7f10e8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10eff5f640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8111fc0 con 0x7f10e8101de0 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10eff5f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8111e40 con 0x7f10e8106f10 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10eff5f640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f10e8112140 con 0x7f10e8102790 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8101de0 0x7f10e80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8101de0 0x7f10e80744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:36862/0 (socket says 172.21.15.37:36862) 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 -- 172.21.15.37:0/1138577573 learned_addr learned my addr 172.21.15.37:0/1138577573 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:56:59.210 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10ed4fa640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8102790 0x7f10e8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 -- 172.21.15.37:0/1138577573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8102790 msgr2=0x7f10e8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8102790 0x7f10e8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 -- 172.21.15.37:0/1138577573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e8106f10 msgr2=0x7f10e8074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e8106f10 0x7f10e8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 -- 172.21.15.37:0/1138577573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10e81a6900 con 0x7f10e8101de0 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10ed4fa640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8102790 0x7f10e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10edcfb640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8101de0 0x7f10e80744f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f10e400d950 tx=0x7f10e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.211 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.183+0000 7f10deffd640 1 -- 172.21.15.37:0/1138577573 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e4017070 con 0x7f10e8101de0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.184+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10e81a6bf0 con 0x7f10e8101de0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.184+0000 7f10deffd640 1 -- 172.21.15.37:0/1138577573 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f10e4009c30 con 0x7f10e8101de0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.184+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10e81a7150 con 0x7f10e8101de0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.184+0000 7f10deffd640 1 -- 172.21.15.37:0/1138577573 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10e400f6b0 con 0x7f10e8101de0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.184+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10e81021c0 con 0x7f10e8101de0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.187+0000 7f10deffd640 1 -- 172.21.15.37:0/1138577573 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f10e40109d0 con 0x7f10e8101de0 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.187+0000 7f10deffd640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f10d40a33b0 0x7f10d40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:56:59.212 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.187+0000 7f10deffd640 1 -- 172.21.15.37:0/1138577573 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f10e40cfda0 con 0x7f10e8101de0 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.187+0000 7f10ed4fa640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f10d40a33b0 0x7f10d40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.188+0000 7f10ed4fa640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f10d40a33b0 0x7f10d40a5870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10d8000c00 tx=0x7f10d8005a40 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:56:59.213 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.190+0000 7f10deffd640 1 -- 172.21.15.37:0/1138577573 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f10e4009de0 con 0x7f10e8101de0 2024-09-19T06:56:59.409 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.408+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10e8075a20 con 0x7f10e8101de0 2024-09-19T06:56:59.410 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.408+0000 7f10deffd640 1 -- 172.21.15.37:0/1138577573 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11907 (secure 0 0 0) 0x7f10e8075a20 con 0x7f10e8101de0 2024-09-19T06:56:59.410 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:56:59.410 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"btime":"2024-09-19T06:55:38:930626+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:55:38.046955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28132":{"gid":28132,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2835776386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2835776386},{"type":"v1","addr":"172.21.15.151:6839","nonce":2835776386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:56:59.411 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 47 2024-09-19T06:56:59.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f10d40a33b0 msgr2=0x7f10d40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.412 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f10d40a33b0 0x7f10d40a5870 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f10d8000c00 tx=0x7f10d8005a40 comp rx=0 tx=0).stop 2024-09-19T06:56:59.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8101de0 msgr2=0x7f10e80744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:56:59.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8101de0 0x7f10e80744f0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f10e400d950 tx=0x7f10e400de10 comp rx=0 tx=0).stop 2024-09-19T06:56:59.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 shutdown_connections 2024-09-19T06:56:59.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10e8106f10 0x7f10e8074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f10e8102790 0x7f10e8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f10d40a33b0 0x7f10d40a5870 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.413 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 --2- 172.21.15.37:0/1138577573 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f10e8101de0 0x7f10e80744f0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:56:59.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 >> 172.21.15.37:0/1138577573 conn(0x7f10e80fd8c0 msgr2=0x7f10e80fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:56:59.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 shutdown_connections 2024-09-19T06:56:59.414 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:56:59.411+0000 7f10eff5f640 1 -- 172.21.15.37:0/1138577573 wait complete. 2024-09-19T06:57:00.034 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-19T06:57:00.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:00 smithi037 ceph-mon[31211]: pgmap v482: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-19T06:57:00.275 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:00 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1138577573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:00.290 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:00.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:00 smithi096 ceph-mon[35004]: pgmap v482: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-19T06:57:00.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:00 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1138577573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:00.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:00 smithi151 ceph-mon[35236]: pgmap v482: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 12 op/s 2024-09-19T06:57:00.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:00 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1138577573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:00.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.748+0000 7fc7bfec9640 1 -- 172.21.15.37:0/2753972670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b810e480 msgr2=0x7fc7b8110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.875 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.748+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/2753972670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b810e480 0x7fc7b8110870 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fc7b0009ca0 tx=0x7fc7b002a8c0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 -- 172.21.15.37:0/2753972670 shutdown_connections 2024-09-19T06:57:00.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/2753972670 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b810e480 0x7fc7b8110870 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/2753972670 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7b8101280 0x7fc7b810df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/2753972670 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7b81008d0 0x7fc7b8100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.876 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 -- 172.21.15.37:0/2753972670 >> 172.21.15.37:0/2753972670 conn(0x7fc7b80fc7a0 msgr2=0x7fc7b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 -- 172.21.15.37:0/2753972670 shutdown_connections 2024-09-19T06:57:00.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 -- 172.21.15.37:0/2753972670 wait complete. 2024-09-19T06:57:00.877 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 Processor -- start 2024-09-19T06:57:00.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 -- start start 2024-09-19T06:57:00.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b81008d0 0x7fc7b819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7b8101280 0x7fc7b819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.878 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.749+0000 7fc7bfec9640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7b810e480 0x7fc7b819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bfec9640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8114160 con 0x7fc7b81008d0 2024-09-19T06:57:00.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bfec9640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b8113fe0 con 0x7fc7b8101280 2024-09-19T06:57:00.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bfec9640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b81142e0 con 0x7fc7b810e480 2024-09-19T06:57:00.879 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b81008d0 0x7fc7b819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b81008d0 0x7fc7b819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44906/0 (socket says 172.21.15.37:44906) 2024-09-19T06:57:00.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 -- 172.21.15.37:0/687210615 learned_addr learned my addr 172.21.15.37:0/687210615 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:57:00.880 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bd464640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7b8101280 0x7fc7b819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 -- 172.21.15.37:0/687210615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7b810e480 msgr2=0x7fc7b819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7b810e480 0x7fc7b819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 -- 172.21.15.37:0/687210615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7b8101280 msgr2=0x7fc7b819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.881 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7b8101280 0x7fc7b819ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 -- 172.21.15.37:0/687210615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b819f5f0 con 0x7fc7b81008d0 2024-09-19T06:57:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bdc65640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b81008d0 0x7fc7b819a640 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400d950 tx=0x7fc7b400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7aeffd640 1 -- 172.21.15.37:0/687210615 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b4017030 con 0x7fc7b81008d0 2024-09-19T06:57:00.882 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b819f8e0 con 0x7fc7b81008d0 2024-09-19T06:57:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7aeffd640 1 -- 172.21.15.37:0/687210615 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc7b4004020 con 0x7fc7b81008d0 2024-09-19T06:57:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7aeffd640 1 -- 172.21.15.37:0/687210615 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b40047a0 con 0x7fc7b81008d0 2024-09-19T06:57:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.750+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b81a71c0 con 0x7fc7b81008d0 2024-09-19T06:57:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.751+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7b8100cb0 con 0x7fc7b81008d0 2024-09-19T06:57:00.883 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.753+0000 7fc7aeffd640 1 -- 172.21.15.37:0/687210615 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7fc7b40041d0 con 0x7fc7b81008d0 2024-09-19T06:57:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.753+0000 7fc7aeffd640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc7a40a33b0 0x7fc7a40a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.754+0000 7fc7bd464640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc7a40a33b0 0x7fc7a40a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:00.884 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.754+0000 7fc7aeffd640 1 -- 172.21.15.37:0/687210615 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7fc7b40cfe60 con 0x7fc7b81008d0 2024-09-19T06:57:00.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.754+0000 7fc7bd464640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc7a40a33b0 0x7fc7a40a5870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7b810ed80 tx=0x7fc7a8005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:00.885 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.757+0000 7fc7aeffd640 1 -- 172.21.15.37:0/687210615 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7fc7b409c810 con 0x7fc7b81008d0 2024-09-19T06:57:00.972 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.971+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc7b819b9a0 con 0x7fc7b81008d0 2024-09-19T06:57:00.973 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.971+0000 7fc7aeffd640 1 -- 172.21.15.37:0/687210615 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+11907 (secure 0 0 0) 0x7fc7b409c110 con 0x7fc7b81008d0 2024-09-19T06:57:00.973 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:57:00.974 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":47,"btime":"2024-09-19T06:55:38:930626+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6835/146682084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6834","nonce":146682084},{"type":"v1","addr":"172.21.15.37:6835","nonce":146682084}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6837/773501481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6836","nonce":773501481},{"type":"v1","addr":"172.21.15.37:6837","nonce":773501481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6839/2512161212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6838","nonce":2512161212},{"type":"v1","addr":"172.21.15.37:6839","nonce":2512161212}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.37:6841/2995019412","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6840","nonce":2995019412},{"type":"v1","addr":"172.21.15.37:6841","nonce":2995019412}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6833/3007825106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6832","nonce":3007825106},{"type":"v1","addr":"172.21.15.151:6833","nonce":3007825106}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24424,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.151:6835/2848315285","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6834","nonce":2848315285},{"type":"v1","addr":"172.21.15.151:6835","nonce":2848315285}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":47,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-19T06:39:53.022194+0000","modified":"2024-09-19T06:55:38.046955+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24497,"mds_1":24470,"mds_2":24449},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24428":{"gid":24428,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/672218237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":672218237},{"type":"v1","addr":"172.21.15.96:6833","nonce":672218237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6837/3741792814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6836","nonce":3741792814},{"type":"v1","addr":"172.21.15.151:6837","nonce":3741792814}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24449":{"gid":24449,"name":"e","rank":2,"incarnation":21,"state":"up:active","state_seq":11,"addr":"172.21.15.96:6835/946199433","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":946199433},{"type":"v1","addr":"172.21.15.96:6835","nonce":946199433}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24470":{"gid":24470,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":9,"addr":"172.21.15.96:6837/3655112770","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":3655112770},{"type":"v1","addr":"172.21.15.96:6837","nonce":3655112770}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24497":{"gid":24497,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.96:6839/1066552330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":1066552330},{"type":"v1","addr":"172.21.15.96:6839","nonce":1066552330}]},"join_fscid":-1,"export_targets":[1,2],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_28132":{"gid":28132,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.151:6839/2835776386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6838","nonce":2835776386},{"type":"v1","addr":"172.21.15.151:6839","nonce":2835776386}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24497,"qdb_cluster":[24497,24470,24449]},"id":1}]} 2024-09-19T06:57:00.974 INFO:teuthology.orchestra.run.smithi037.stderr:dumped fsmap epoch 47 2024-09-19T06:57:00.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc7a40a33b0 msgr2=0x7fc7a40a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.975 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc7a40a33b0 0x7fc7a40a5870 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fc7b810ed80 tx=0x7fc7a8005f90 comp rx=0 tx=0).stop 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b81008d0 msgr2=0x7fc7b819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b81008d0 0x7fc7b819a640 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fc7b400d950 tx=0x7fc7b400de10 comp rx=0 tx=0).stop 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 shutdown_connections 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc7b810e480 0x7fc7b819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7b8101280 0x7fc7b819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7fc7a40a33b0 0x7fc7a40a5870 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 --2- 172.21.15.37:0/687210615 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7fc7b81008d0 0x7fc7b819a640 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:00.976 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 >> 172.21.15.37:0/687210615 conn(0x7fc7b80fc7a0 msgr2=0x7fc7b80feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 shutdown_connections 2024-09-19T06:57:00.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:00.974+0000 7fc7bfec9640 1 -- 172.21.15.37:0/687210615 wait complete. 2024-09-19T06:57:01.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:01 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/687210615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:01.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:01 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/687210615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:01.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:01 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/687210615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-19T06:57:01.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-19T06:57:02.024 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:02.581 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:02 smithi096 ceph-mon[35004]: pgmap v483: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-19T06:57:02.581 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:02 smithi037 ceph-mon[31211]: pgmap v483: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-19T06:57:02.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 -- 172.21.15.37:0/2986467219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b8103660 msgr2=0x7f44b810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/2986467219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b8103660 0x7f44b810bb50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009900 tx=0x7f44ac02a480 comp rx=0 tx=0).stop 2024-09-19T06:57:02.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 -- 172.21.15.37:0/2986467219 shutdown_connections 2024-09-19T06:57:02.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/2986467219 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44b810c170 0x7f44b810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/2986467219 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b8103660 0x7f44b810bb50 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/2986467219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8102d40 0x7f44b8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.582 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 -- 172.21.15.37:0/2986467219 >> 172.21.15.37:0/2986467219 conn(0x7f44b80fc7a0 msgr2=0x7f44b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 -- 172.21.15.37:0/2986467219 shutdown_connections 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.465+0000 7f44bcd8b640 1 -- 172.21.15.37:0/2986467219 wait complete. 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.466+0000 7f44bcd8b640 1 Processor -- start 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.466+0000 7f44bcd8b640 1 -- start start 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44bcd8b640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44b8102d40 0x7f44b81984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44bcd8b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8103660 0x7f44b81989e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44bcd8b640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b810c170 0x7f44b819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44bcd8b640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8111fb0 con 0x7f44b810c170 2024-09-19T06:57:02.583 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44bcd8b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8111e30 con 0x7f44b8103660 2024-09-19T06:57:02.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44bcd8b640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8112130 con 0x7f44b8102d40 2024-09-19T06:57:02.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b810c170 0x7f44b819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b810c170 0x7f44b819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44926/0 (socket says 172.21.15.37:44926) 2024-09-19T06:57:02.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 -- 172.21.15.37:0/3492873297 learned_addr learned my addr 172.21.15.37:0/3492873297 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:57:02.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b5d9b640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8103660 0x7f44b81989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b659c640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44b8102d40 0x7f44b81984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.584 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 -- 172.21.15.37:0/3492873297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44b8102d40 msgr2=0x7f44b81984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44b8102d40 0x7f44b81984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 -- 172.21.15.37:0/3492873297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8103660 msgr2=0x7f44b81989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8103660 0x7f44b81989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 -- 172.21.15.37:0/3492873297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b819d450 con 0x7f44b810c170 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b659c640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44b8102d40 0x7f44b81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.467+0000 7f44b6d9d640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b810c170 0x7f44b819cd70 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009fd0 tx=0x7f44b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.468+0000 7f44a77fe640 1 -- 172.21.15.37:0/3492873297 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0011830 con 0x7f44b810c170 2024-09-19T06:57:02.585 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.468+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b819d740 con 0x7f44b810c170 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.468+0000 7f44a77fe640 1 -- 172.21.15.37:0/3492873297 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f44b0016070 con 0x7f44b810c170 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.468+0000 7f44a77fe640 1 -- 172.21.15.37:0/3492873297 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000f2e0 con 0x7f44b810c170 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.468+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b81a5020 con 0x7f44b810c170 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.469+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44b806b070 con 0x7f44b810c170 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.470+0000 7f44a77fe640 1 -- 172.21.15.37:0/3492873297 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f44b00119d0 con 0x7f44b810c170 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.471+0000 7f44a77fe640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44a00a33b0 0x7f44a00a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.471+0000 7f44b659c640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44a00a33b0 0x7f44a00a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:02.586 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.471+0000 7f44a77fe640 1 -- 172.21.15.37:0/3492873297 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f44b00cf290 con 0x7f44b810c170 2024-09-19T06:57:02.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.472+0000 7f44b659c640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44a00a33b0 0x7f44a00a5870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005fd0 tx=0x7f44a8009520 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:02.587 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.475+0000 7f44a77fe640 1 -- 172.21.15.37:0/3492873297 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f44b009bd50 con 0x7f44b810c170 2024-09-19T06:57:02.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:02 smithi151 ceph-mon[35236]: pgmap v483: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 4.7 KiB/s wr, 6 op/s 2024-09-19T06:57:02.656 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.654+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f44b81071e0 con 0x7f44b810c170 2024-09-19T06:57:02.657 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.656+0000 7f44a77fe640 1 -- 172.21.15.37:0/3492873297 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) ==== 74+0+23264 (secure 0 0 0) 0x7f44b009b650 con 0x7f44b810c170 2024-09-19T06:57:02.658 INFO:teuthology.orchestra.run.smithi037.stdout: 2024-09-19T06:57:02.658 INFO:teuthology.orchestra.run.smithi037.stdout:{"epoch":107,"fsid":"b715dee2-7650-11ef-bceb-c7b262605968","created":"2024-09-19T06:32:04.919737+0000","modified":"2024-09-19T06:56:09.906178+0000","last_up_change":"2024-09-19T06:39:01.122972+0000","last_in_change":"2024-09-19T06:38:37.756931+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-19T06:35:14.622171+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-19T06:39:49.247361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-19T06:39:51.369694+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":25,"snap_epoch":106,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"49e46443-fcf6-403e-95af-871ca077aee4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6802","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6803","nonce":2764918113}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6804","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6805","nonce":2764918113}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6808","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6809","nonce":2764918113}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6806","nonce":2764918113},{"type":"v1","addr":"172.21.15.37:6807","nonce":2764918113}]},"public_addr":"172.21.15.37:6803/2764918113","cluster_addr":"172.21.15.37:6805/2764918113","heartbeat_back_addr":"172.21.15.37:6809/2764918113","heartbeat_front_addr":"172.21.15.37:6807/2764918113","state":["exists","up"]},{"osd":1,"uuid":"971dd8c5-2680-4d3a-812a-a7e391ff5560","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6801","nonce":515532313}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6803","nonce":515532313}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6807","nonce":515532313}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":515532313},{"type":"v1","addr":"172.21.15.96:6805","nonce":515532313}]},"public_addr":"172.21.15.96:6801/515532313","cluster_addr":"172.21.15.96:6803/515532313","heartbeat_back_addr":"172.21.15.96:6807/515532313","heartbeat_front_addr":"172.21.15.96:6805/515532313","state":["exists","up"]},{"osd":2,"uuid":"fd59ee43-9b0c-4440-a57c-7f944560b77c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6801","nonce":1198314257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6803","nonce":1198314257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6807","nonce":1198314257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1198314257},{"type":"v1","addr":"172.21.15.151:6805","nonce":1198314257}]},"public_addr":"172.21.15.151:6801/1198314257","cluster_addr":"172.21.15.151:6803/1198314257","heartbeat_back_addr":"172.21.15.151:6807/1198314257","heartbeat_front_addr":"172.21.15.151:6805/1198314257","state":["exists","up"]},{"osd":3,"uuid":"77fbc8c9-cfb7-49d4-a395-c747b5061988","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6810","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6811","nonce":1041463019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6812","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6813","nonce":1041463019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6816","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6817","nonce":1041463019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6814","nonce":1041463019},{"type":"v1","addr":"172.21.15.37:6815","nonce":1041463019}]},"public_addr":"172.21.15.37:6811/1041463019","cluster_addr":"172.21.15.37:6813/1041463019","heartbeat_back_addr":"172.21.15.37:6817/1041463019","heartbeat_front_addr":"172.21.15.37:6815/1041463019","state":["exists","up"]},{"osd":4,"uuid":"c58779a9-0c41-444b-b1c1-93e50dbca0eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6809","nonce":3832301595}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6811","nonce":3832301595}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6815","nonce":3832301595}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":3832301595},{"type":"v1","addr":"172.21.15.96:6813","nonce":3832301595}]},"public_addr":"172.21.15.96:6809/3832301595","cluster_addr":"172.21.15.96:6811/3832301595","heartbeat_back_addr":"172.21.15.96:6815/3832301595","heartbeat_front_addr":"172.21.15.96:6813/3832301595","state":["exists","up"]},{"osd":5,"uuid":"9318d75e-c472-4c7e-b758-53c7f9cb12e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6809","nonce":4035837098}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6811","nonce":4035837098}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6815","nonce":4035837098}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":4035837098},{"type":"v1","addr":"172.21.15.151:6813","nonce":4035837098}]},"public_addr":"172.21.15.151:6809/4035837098","cluster_addr":"172.21.15.151:6811/4035837098","heartbeat_back_addr":"172.21.15.151:6815/4035837098","heartbeat_front_addr":"172.21.15.151:6813/4035837098","state":["exists","up"]},{"osd":6,"uuid":"b76fc6b5-0f24-447c-9541-f19b2dcf0066","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":79,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6818","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6819","nonce":3128436148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6820","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6821","nonce":3128436148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6824","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6825","nonce":3128436148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6822","nonce":3128436148},{"type":"v1","addr":"172.21.15.37:6823","nonce":3128436148}]},"public_addr":"172.21.15.37:6819/3128436148","cluster_addr":"172.21.15.37:6821/3128436148","heartbeat_back_addr":"172.21.15.37:6825/3128436148","heartbeat_front_addr":"172.21.15.37:6823/3128436148","state":["exists","up"]},{"osd":7,"uuid":"57af0c79-adbc-4e9c-a848-4586e582c9b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6817","nonce":4105644959}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6819","nonce":4105644959}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6823","nonce":4105644959}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":4105644959},{"type":"v1","addr":"172.21.15.96:6821","nonce":4105644959}]},"public_addr":"172.21.15.96:6817/4105644959","cluster_addr":"172.21.15.96:6819/4105644959","heartbeat_back_addr":"172.21.15.96:6823/4105644959","heartbeat_front_addr":"172.21.15.96:6821/4105644959","state":["exists","up"]},{"osd":8,"uuid":"f560456d-ef7e-4b12-8bcf-be9bab3c3def","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":56,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6817","nonce":2450296865}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6819","nonce":2450296865}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6823","nonce":2450296865}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":2450296865},{"type":"v1","addr":"172.21.15.151:6821","nonce":2450296865}]},"public_addr":"172.21.15.151:6817/2450296865","cluster_addr":"172.21.15.151:6819/2450296865","heartbeat_back_addr":"172.21.15.151:6823/2450296865","heartbeat_front_addr":"172.21.15.151:6821/2450296865","state":["exists","up"]},{"osd":9,"uuid":"ab6dfa58-1497-4cee-9145-64b049b10e93","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":63,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6826","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6827","nonce":3305545849}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6828","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6829","nonce":3305545849}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6832","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6833","nonce":3305545849}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.37:6830","nonce":3305545849},{"type":"v1","addr":"172.21.15.37:6831","nonce":3305545849}]},"public_addr":"172.21.15.37:6827/3305545849","cluster_addr":"172.21.15.37:6829/3305545849","heartbeat_back_addr":"172.21.15.37:6833/3305545849","heartbeat_front_addr":"172.21.15.37:6831/3305545849","state":["exists","up"]},{"osd":10,"uuid":"8817f0ca-2ce3-483e-a70c-68d96cacdbd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6825","nonce":2313306979}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6827","nonce":2313306979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6831","nonce":2313306979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":2313306979},{"type":"v1","addr":"172.21.15.96:6829","nonce":2313306979}]},"public_addr":"172.21.15.96:6825/2313306979","cluster_addr":"172.21.15.96:6827/2313306979","heartbeat_back_addr":"172.21.15.96:6831/2313306979","heartbeat_front_addr":"172.21.15.96:6829/2313306979","state":["exists","up"]},{"osd":11,"uuid":"d075a370-835b-443b-a582-a19ac7a79167","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6825","nonce":3357779771}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6827","nonce":3357779771}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6831","nonce":3357779771}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3357779771},{"type":"v1","addr":"172.21.15.151:6829","nonce":3357779771}]},"public_addr":"172.21.15.151:6825/3357779771","cluster_addr":"172.21.15.151:6827/3357779771","heartbeat_back_addr":"172.21.15.151:6831/3357779771","heartbeat_front_addr":"172.21.15.151:6829/3357779771","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:20.789802+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:34:46.625811+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:10.401514+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:35:36.175807+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:01.157493+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:26.361796+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:36:51.732157+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:17.624109+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:37:43.101365+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:08.098576+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:34.114176+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-19T06:38:59.495318+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":4}]},{"pgid":"2.12","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":1}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.11","mappings":[{"from":4,"to":8}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.27","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.37:0/3406854118":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/1436144832":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6801/871019432":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/2591230747":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3313349719":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/180594249":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/2641687433":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1459260343":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6801/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:0/3941836172":"2024-09-20T06:32:56.109611+0000","172.21.15.37:6800/2963797247":"2024-09-20T06:32:32.108497+0000","172.21.15.37:6801/2601148313":"2024-09-20T06:32:56.109611+0000","172.21.15.37:0/1644258429":"2024-09-20T06:41:46.581265+0000","172.21.15.37:0/711456420":"2024-09-20T06:41:46.581265+0000","172.21.15.37:6800/871019432":"2024-09-20T06:41:46.581265+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":12,"length":1},{"begin":25,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-19T06:57:02.659 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44a00a33b0 msgr2=0x7f44a00a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44a00a33b0 0x7f44a00a5870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f44a8005fd0 tx=0x7f44a8009520 comp rx=0 tx=0).stop 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b810c170 msgr2=0x7f44b819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b810c170 0x7f44b819cd70 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009fd0 tx=0x7f44b000b8e0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 shutdown_connections 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f44a00a33b0 0x7f44a00a5870 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f44b810c170 0x7f44b819cd70 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8103660 0x7f44b81989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.660 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 --2- 172.21.15.37:0/3492873297 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f44b8102d40 0x7f44b81984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:02.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 >> 172.21.15.37:0/3492873297 conn(0x7f44b80fc7a0 msgr2=0x7f44b80fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:02.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 shutdown_connections 2024-09-19T06:57:02.661 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:02.658+0000 7f44bcd8b640 1 -- 172.21.15.37:0/3492873297 wait complete. 2024-09-19T06:57:03.282 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-19T06:57:03.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:03 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3492873297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:57:03.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:03 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3492873297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:57:03.541 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:03.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:03 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3492873297' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-19T06:57:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 -- 172.21.15.37:0/2766357621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68102790 msgr2=0x7f1a681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/2766357621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68102790 0x7f1a681067e0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009900 tx=0x7f1a5802a410 comp rx=0 tx=0).stop 2024-09-19T06:57:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 -- 172.21.15.37:0/2766357621 shutdown_connections 2024-09-19T06:57:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/2766357621 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a68106f10 0x7f1a6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/2766357621 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68102790 0x7f1a681067e0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.028 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/2766357621 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a68101de0 0x7f1a681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 -- 172.21.15.37:0/2766357621 >> 172.21.15.37:0/2766357621 conn(0x7f1a680fd8c0 msgr2=0x7f1a680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 -- 172.21.15.37:0/2766357621 shutdown_connections 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 -- 172.21.15.37:0/2766357621 wait complete. 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 Processor -- start 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.008+0000 7f1a6ed11640 1 -- start start 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6ed11640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68101de0 0x7f1a6819a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6ed11640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a68102790 0x7f1a6819ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:04.029 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6ed11640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a68106f10 0x7f1a6819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6ed11640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68111f70 con 0x7f1a68101de0 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6ed11640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1a68111df0 con 0x7f1a68106f10 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6ed11640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f1a681120f0 con 0x7f1a68102790 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68101de0 0x7f1a6819a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68101de0 0x7f1a6819a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44956/0 (socket says 172.21.15.37:44956) 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 -- 172.21.15.37:0/548842387 learned_addr learned my addr 172.21.15.37:0/548842387 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6d2ae640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a68106f10 0x7f1a6819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:04.030 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a67fff640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a68102790 0x7f1a6819ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 -- 172.21.15.37:0/548842387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a68102790 msgr2=0x7f1a6819ab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a68102790 0x7f1a6819ab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 -- 172.21.15.37:0/548842387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a68106f10 msgr2=0x7f1a6819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a68106f10 0x7f1a6819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 -- 172.21.15.37:0/548842387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a6819f5f0 con 0x7f1a68101de0 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.009+0000 7f1a6caad640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68101de0 0x7f1a6819a640 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000d950 tx=0x7f1a5000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.010+0000 7f1a65ffb640 1 -- 172.21.15.37:0/548842387 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a50017070 con 0x7f1a68101de0 2024-09-19T06:57:04.031 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.010+0000 7f1a65ffb640 1 -- 172.21.15.37:0/548842387 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a50009c30 con 0x7f1a68101de0 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.010+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a6819f8e0 con 0x7f1a68101de0 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.010+0000 7f1a65ffb640 1 -- 172.21.15.37:0/548842387 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a5000f6b0 con 0x7f1a68101de0 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.010+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a681a71c0 con 0x7f1a68101de0 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.011+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a681021c0 con 0x7f1a68101de0 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.012+0000 7f1a65ffb640 1 -- 172.21.15.37:0/548842387 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f1a500109d0 con 0x7f1a68101de0 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.013+0000 7f1a65ffb640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1a540a33b0 0x7f1a540a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.013+0000 7f1a67fff640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1a540a33b0 0x7f1a540a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:04.032 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.013+0000 7f1a65ffb640 1 -- 172.21.15.37:0/548842387 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 8571+0+0 (secure 0 0 0) 0x7f1a500cf990 con 0x7f1a68101de0 2024-09-19T06:57:04.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.013+0000 7f1a67fff640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1a540a33b0 0x7f1a540a5870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009230 tx=0x7f1a580058d0 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:04.033 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.017+0000 7f1a65ffb640 1 -- 172.21.15.37:0/548842387 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f1a50009de0 con 0x7f1a68101de0 2024-09-19T06:57:04.228 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.227+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f1a681a74b0 con 0x7f1a68101de0 2024-09-19T06:57:04.230 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.229+0000 7f1a65ffb640 1 -- 172.21.15.37:0/548842387 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f1a5009bc90 con 0x7f1a68101de0 2024-09-19T06:57:04.230 INFO:teuthology.orchestra.run.smithi037.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-19T06:57:04.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1a540a33b0 msgr2=0x7f1a540a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:04.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1a540a33b0 0x7f1a540a5870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f1a58009230 tx=0x7f1a580058d0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68101de0 msgr2=0x7f1a6819a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:04.234 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68101de0 0x7f1a6819a640 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f1a5000d950 tx=0x7f1a5000de10 comp rx=0 tx=0).stop 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 shutdown_connections 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1a68106f10 0x7f1a6819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1a68102790 0x7f1a6819ab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f1a540a33b0 0x7f1a540a5870 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 --2- 172.21.15.37:0/548842387 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f1a68101de0 0x7f1a6819a640 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.232+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 >> 172.21.15.37:0/548842387 conn(0x7f1a680fd8c0 msgr2=0x7f1a680feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.233+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 shutdown_connections 2024-09-19T06:57:04.235 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:04.233+0000 7f1a6ed11640 1 -- 172.21.15.37:0/548842387 wait complete. 2024-09-19T06:57:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mon[35004]: pgmap v484: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 75 KiB/s wr, 27 op/s 2024-09-19T06:57:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/548842387' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T06:57:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mon[35004]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:57:04.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mon[35004]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:57:04.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/548842387' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T06:57:04.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mon[35004]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T06:57:04.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mon[35004]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T06:57:04.526 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k[64088]: ignoring --setuser ceph since I am not root 2024-09-19T06:57:04.526 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k[64088]: ignoring --setgroup ceph since I am not root 2024-09-19T06:57:04.526 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mds[64092]: -- 172.21.15.96:0/1246069304 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f4101b7860 con 0x55f410195400 2024-09-19T06:57:04.526 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k[64088]: starting mds.k at 2024-09-19T06:57:04.526 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e[64404]: ignoring --setuser ceph since I am not root 2024-09-19T06:57:04.527 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e[64404]: ignoring --setgroup ceph since I am not root 2024-09-19T06:57:04.527 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mds[64408]: -- 172.21.15.96:0/1034633990 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561343099860 con 0x561343076800 2024-09-19T06:57:04.527 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e[64404]: starting mds.e at 2024-09-19T06:57:04.527 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h[64706]: ignoring --setuser ceph since I am not root 2024-09-19T06:57:04.527 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h[64706]: ignoring --setgroup ceph since I am not root 2024-09-19T06:57:04.527 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mds[64710]: -- 172.21.15.96:0/2630883880 <== mon.0 v2:172.21.15.37:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560ea9429860 con 0x560ea9406800 2024-09-19T06:57:04.528 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h[64706]: starting mds.h at 2024-09-19T06:57:04.528 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: ignoring --setuser ceph since I am not root 2024-09-19T06:57:04.528 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: ignoring --setgroup ceph since I am not root 2024-09-19T06:57:04.528 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-mds[65027]: -- 172.21.15.96:0/2331664040 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f24e41bc20 con 0x55f24e3f9400 2024-09-19T06:57:04.528 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:57:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: starting mds.b at 2024-09-19T06:57:04.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a[31207]: 2024-09-19T06:57:04.227+0000 7f46d9475640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:57:04.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-mon[31211]: pgmap v484: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 75 KiB/s wr, 27 op/s 2024-09-19T06:57:04.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/548842387' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T06:57:04.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-mon[31211]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:57:04.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-mon[31211]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:57:04.529 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/548842387' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T06:57:04.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-mon[31211]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T06:57:04.530 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:04 smithi037 ceph-mon[31211]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T06:57:04.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mon[35236]: pgmap v484: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 75 KiB/s wr, 27 op/s 2024-09-19T06:57:04.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/548842387' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-19T06:57:04.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mon[35236]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-19T06:57:04.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mon[35236]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-19T06:57:04.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/548842387' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-19T06:57:04.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mon[35236]: osdmap e108: 12 total, 12 up, 12 in 2024-09-19T06:57:04.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mon[35236]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-19T06:57:04.614 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c[65967]: ignoring --setuser ceph since I am not root 2024-09-19T06:57:04.614 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c[65967]: ignoring --setgroup ceph since I am not root 2024-09-19T06:57:04.614 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c[65967]: starting mds.c at 2024-09-19T06:57:04.614 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setuser ceph since I am not root 2024-09-19T06:57:04.614 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: ignoring --setgroup ceph since I am not root 2024-09-19T06:57:04.615 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-mds[66270]: -- 172.21.15.151:0/2566405738 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5584b97e5860 con 0x5584b97c2800 2024-09-19T06:57:04.615 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:57:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: starting mds.i at 2024-09-19T06:57:05.015 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-19T06:57:05.275 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:05.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 -- 172.21.15.37:0/2708613243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f228410c180 msgr2=0x7f228410e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 --2- 172.21.15.37:0/2708613243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f228410c180 0x7f228410e600 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f227c009ca0 tx=0x7f227c02a950 comp rx=0 tx=0).stop 2024-09-19T06:57:05.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 -- 172.21.15.37:0/2708613243 shutdown_connections 2024-09-19T06:57:05.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 --2- 172.21.15.37:0/2708613243 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f228410c180 0x7f228410e600 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 --2- 172.21.15.37:0/2708613243 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f22841036c0 0x7f228410bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.938 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 --2- 172.21.15.37:0/2708613243 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2284102da0 0x7f2284103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 -- 172.21.15.37:0/2708613243 >> 172.21.15.37:0/2708613243 conn(0x7f22840fc7a0 msgr2=0x7f22840febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 -- 172.21.15.37:0/2708613243 shutdown_connections 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.887+0000 7f228a266640 1 -- 172.21.15.37:0/2708613243 wait complete. 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 Processor -- start 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 -- start start 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2284102da0 0x7f2284100a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22841036c0 0x7f2284100f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.939 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f228410c180 0x7f2284101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f2284111f40 con 0x7f2284102da0 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2284111dc0 con 0x7f22841036c0 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f228a266640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f22841120c0 con 0x7f228410c180 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f2283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2284102da0 0x7f2284100a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f2283fff640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2284102da0 0x7f2284100a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44976/0 (socket says 172.21.15.37:44976) 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f2283fff640 1 -- 172.21.15.37:0/3961396511 learned_addr learned my addr 172.21.15.37:0/3961396511 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f22837fe640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22841036c0 0x7f2284100f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.940 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.888+0000 7f2288803640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f228410c180 0x7f2284101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f2283fff640 1 -- 172.21.15.37:0/3961396511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f228410c180 msgr2=0x7f2284101480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f2283fff640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f228410c180 0x7f2284101480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f2283fff640 1 -- 172.21.15.37:0/3961396511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22841036c0 msgr2=0x7f2284100f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f2283fff640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22841036c0 0x7f2284100f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f2283fff640 1 -- 172.21.15.37:0/3961396511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22841a6890 con 0x7f2284102da0 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f2283fff640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2284102da0 0x7f2284100a00 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f227400d950 tx=0x7f227400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f22817fa640 1 -- 172.21.15.37:0/3961396511 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2274017070 con 0x7f2284102da0 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22841a6b80 con 0x7f2284102da0 2024-09-19T06:57:05.941 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f22817fa640 1 -- 172.21.15.37:0/3961396511 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2274009c30 con 0x7f2284102da0 2024-09-19T06:57:05.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f22817fa640 1 -- 172.21.15.37:0/3961396511 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227400f6b0 con 0x7f2284102da0 2024-09-19T06:57:05.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.889+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22841a70c0 con 0x7f2284102da0 2024-09-19T06:57:05.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.890+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2284103180 con 0x7f2284102da0 2024-09-19T06:57:05.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.891+0000 7f22817fa640 1 -- 172.21.15.37:0/3961396511 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f22740109d0 con 0x7f2284102da0 2024-09-19T06:57:05.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.892+0000 7f22817fa640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f226c0a33b0 0x7f226c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:05.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.892+0000 7f22837fe640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f226c0a33b0 0x7f226c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:05.942 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.892+0000 7f22817fa640 1 -- 172.21.15.37:0/3961396511 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8785+0+0 (secure 0 0 0) 0x7f22740cf9f0 con 0x7f2284102da0 2024-09-19T06:57:05.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.892+0000 7f22837fe640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f226c0a33b0 0x7f226c0a5870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2284102490 tx=0x7f2278005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:05.943 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:05.896+0000 7f22817fa640 1 -- 172.21.15.37:0/3961396511 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f2274009de0 con 0x7f2284102da0 2024-09-19T06:57:06.108 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.106+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f2284106c90 con 0x7f2284102da0 2024-09-19T06:57:06.260 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.258+0000 7f22817fa640 1 -- 172.21.15.37:0/3961396511 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v50) ==== 104+0+0 (secure 0 0 0) 0x7f227409bc10 con 0x7f2284102da0 2024-09-19T06:57:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f226c0a33b0 msgr2=0x7f226c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f226c0a33b0 0x7f226c0a5870 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f2284102490 tx=0x7f2278005f90 comp rx=0 tx=0).stop 2024-09-19T06:57:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2284102da0 msgr2=0x7f2284100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2284102da0 0x7f2284100a00 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f227400d950 tx=0x7f227400de10 comp rx=0 tx=0).stop 2024-09-19T06:57:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 shutdown_connections 2024-09-19T06:57:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f228410c180 0x7f2284101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:06.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f22841036c0 0x7f2284100f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:06.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f226c0a33b0 0x7f226c0a5870 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:06.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 --2- 172.21.15.37:0/3961396511 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f2284102da0 0x7f2284100a00 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:06.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 >> 172.21.15.37:0/3961396511 conn(0x7f22840fc7a0 msgr2=0x7f22840fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:06.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 shutdown_connections 2024-09-19T06:57:06.264 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:06.261+0000 7f228a266640 1 -- 172.21.15.37:0/3961396511 wait complete. 2024-09-19T06:57:06.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: pgmap v486: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 84 KiB/s wr, 29 op/s 2024-09-19T06:57:06.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6836/1900131203,v1:172.21.15.151:6838/1900131203] up:boot 2024-09-19T06:57:06.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.151:6837/2744175012,v1:172.21.15.151:6839/2744175012] up:boot 2024-09-19T06:57:06.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6832/2513963514,v1:172.21.15.96:6834/2513963514] up:boot 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6835/2220074939,v1:172.21.15.96:6837/2220074939] up:boot 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6833/2682348622,v1:172.21.15.96:6836/2682348622] up:boot 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: mds.? [v2:172.21.15.96:6838/827699148,v1:172.21.15.96:6839/827699148] up:boot 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:57:06.525 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:57:06.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:57:06.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:57:06.526 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:06 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3961396511' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:57:06.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: pgmap v486: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 84 KiB/s wr, 29 op/s 2024-09-19T06:57:06.526 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6836/1900131203,v1:172.21.15.151:6838/1900131203] up:boot 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.151:6837/2744175012,v1:172.21.15.151:6839/2744175012] up:boot 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6832/2513963514,v1:172.21.15.96:6834/2513963514] up:boot 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6835/2220074939,v1:172.21.15.96:6837/2220074939] up:boot 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6833/2682348622,v1:172.21.15.96:6836/2682348622] up:boot 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: mds.? [v2:172.21.15.96:6838/827699148,v1:172.21.15.96:6839/827699148] up:boot 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:57:06.527 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:57:06.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:57:06.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:57:06.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:57:06.528 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:06 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3961396511' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:57:06.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: pgmap v486: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 84 KiB/s wr, 29 op/s 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6836/1900131203,v1:172.21.15.151:6838/1900131203] up:boot 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.151:6837/2744175012,v1:172.21.15.151:6839/2744175012] up:boot 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6832/2513963514,v1:172.21.15.96:6834/2513963514] up:boot 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6835/2220074939,v1:172.21.15.96:6837/2220074939] up:boot 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6833/2682348622,v1:172.21.15.96:6836/2682348622] up:boot 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: mds.? [v2:172.21.15.96:6838/827699148,v1:172.21.15.96:6839/827699148] up:boot 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-19T06:57:06.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-19T06:57:06.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-19T06:57:06.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-19T06:57:06.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-19T06:57:06.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-19T06:57:06.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: from='mgr.15156 172.21.15.37:0/707499691' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-19T06:57:06.614 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:06 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3961396511' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-19T06:57:07.080 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-19T06:57:07.344 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:07.465 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:07 smithi037 ceph-mon[31211]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:57:07.466 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:07 smithi037 ceph-mon[31211]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:57:07.466 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:07 smithi037 ceph-mon[31211]: Cluster is now healthy 2024-09-19T06:57:07.466 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:07 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/3961396511' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:57:07.466 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:07 smithi037 ceph-mon[31211]: fsmap 12 up:standby 2024-09-19T06:57:07.523 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:07 smithi096 ceph-mon[35004]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:57:07.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:07 smithi096 ceph-mon[35004]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:57:07.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:07 smithi096 ceph-mon[35004]: Cluster is now healthy 2024-09-19T06:57:07.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:07 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/3961396511' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:57:07.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:07 smithi096 ceph-mon[35004]: fsmap 12 up:standby 2024-09-19T06:57:07.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:07 smithi151 ceph-mon[35236]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-19T06:57:07.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:07 smithi151 ceph-mon[35236]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-19T06:57:07.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:07 smithi151 ceph-mon[35236]: Cluster is now healthy 2024-09-19T06:57:07.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:07 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/3961396511' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-19T06:57:07.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:07 smithi151 ceph-mon[35236]: fsmap 12 up:standby 2024-09-19T06:57:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 -- 172.21.15.37:0/2175700292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 msgr2=0x7f49841021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 --2- 172.21.15.37:0/2175700292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49841021c0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f4974009900 tx=0x7f497402a410 comp rx=0 tx=0).stop 2024-09-19T06:57:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 -- 172.21.15.37:0/2175700292 shutdown_connections 2024-09-19T06:57:07.807 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 --2- 172.21.15.37:0/2175700292 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4984106f10 0x7f498410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 --2- 172.21.15.37:0/2175700292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4984102790 0x7f49841067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 --2- 172.21.15.37:0/2175700292 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49841021c0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 -- 172.21.15.37:0/2175700292 >> 172.21.15.37:0/2175700292 conn(0x7f49840fd8c0 msgr2=0x7f49840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:07.808 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 -- 172.21.15.37:0/2175700292 shutdown_connections 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.792+0000 7f4989b97640 1 -- 172.21.15.37:0/2175700292 wait complete. 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 Processor -- start 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 -- start start 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4984102790 0x7f4984074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4984106f10 0x7f4984074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f4984111f70 con 0x7f4984101de0 2024-09-19T06:57:07.809 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4984111df0 con 0x7f4984106f10 2024-09-19T06:57:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.793+0000 7f4989b97640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f49841120f0 con 0x7f4984102790 2024-09-19T06:57:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49840744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:44994/0 (socket says 172.21.15.37:44994) 2024-09-19T06:57:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 -- 172.21.15.37:0/1723158152 learned_addr learned my addr 172.21.15.37:0/1723158152 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:57:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4983fff640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4984106f10 0x7f4984074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4982ffd640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4984102790 0x7f4984074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:07.810 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 -- 172.21.15.37:0/1723158152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4984102790 msgr2=0x7f4984074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4984102790 0x7f4984074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 -- 172.21.15.37:0/1723158152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4984106f10 msgr2=0x7f4984074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4984106f10 0x7f4984074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 -- 172.21.15.37:0/1723158152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49841a6970 con 0x7f4984101de0 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4982ffd640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4984102790 0x7f4984074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f49837fe640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49840744f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f4974006010 tx=0x7f49740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4980ff9640 1 -- 172.21.15.37:0/1723158152 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497403f030 con 0x7f4984101de0 2024-09-19T06:57:07.811 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49841a6c00 con 0x7f4984101de0 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4980ff9640 1 -- 172.21.15.37:0/1723158152 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4974003ce0 con 0x7f4984101de0 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4980ff9640 1 -- 172.21.15.37:0/1723158152 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4974035070 con 0x7f4984101de0 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.794+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49841a7130 con 0x7f4984101de0 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.796+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49841021c0 con 0x7f4984101de0 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.797+0000 7f4980ff9640 1 -- 172.21.15.37:0/1723158152 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f4974042030 con 0x7f4984101de0 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.797+0000 7f4980ff9640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f496c0a33b0 0x7f496c0a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.797+0000 7f4982ffd640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f496c0a33b0 0x7f496c0a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:07.812 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.797+0000 7f4980ff9640 1 -- 172.21.15.37:0/1723158152 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8785+0+0 (secure 0 0 0) 0x7f49740f0440 con 0x7f4984101de0 2024-09-19T06:57:07.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.798+0000 7f4982ffd640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f496c0a33b0 0x7f496c0a5870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4984075f80 tx=0x7f4978005f90 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:07.813 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.802+0000 7f4980ff9640 1 -- 172.21.15.37:0/1723158152 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f49740bcdd0 con 0x7f4984101de0 2024-09-19T06:57:07.977 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:07.976+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f498410a0f0 con 0x7f4984101de0 2024-09-19T06:57:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.261+0000 7f4980ff9640 1 -- 172.21.15.37:0/1723158152 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) ==== 181+0+0 (secure 0 0 0) 0x7f49740bc6d0 con 0x7f4984101de0 2024-09-19T06:57:08.263 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_metadata' removed 2024-09-19T06:57:08.266 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f496c0a33b0 msgr2=0x7f496c0a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f496c0a33b0 0x7f496c0a5870 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f4984075f80 tx=0x7f4978005f90 comp rx=0 tx=0).stop 2024-09-19T06:57:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 msgr2=0x7f49840744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49840744f0 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f4974006010 tx=0x7f49740042f0 comp rx=0 tx=0).stop 2024-09-19T06:57:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 shutdown_connections 2024-09-19T06:57:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4984106f10 0x7f4984074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4984102790 0x7f4984074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:08.267 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f496c0a33b0 0x7f496c0a5870 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 --2- 172.21.15.37:0/1723158152 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f4984101de0 0x7f49840744f0 secure :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f4974006010 tx=0x7f49740042f0 comp rx=0 tx=0).stop 2024-09-19T06:57:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.264+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 >> 172.21.15.37:0/1723158152 conn(0x7f49840fd8c0 msgr2=0x7f49840fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.266+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 shutdown_connections 2024-09-19T06:57:08.268 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:08.266+0000 7f4989b97640 1 -- 172.21.15.37:0/1723158152 wait complete. 2024-09-19T06:57:08.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:08 smithi096 ceph-mon[35004]: pgmap v487: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 84 KiB/s wr, 29 op/s 2024-09-19T06:57:08.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:08 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1723158152' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:57:08.524 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:08 smithi037 ceph-mon[31211]: pgmap v487: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 84 KiB/s wr, 29 op/s 2024-09-19T06:57:08.525 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:08 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1723158152' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:57:08.528 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:08 smithi151 ceph-mon[35236]: pgmap v487: 97 pgs: 97 active+clean; 303 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 84 KiB/s wr, 29 op/s 2024-09-19T06:57:08.528 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:08 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1723158152' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:57:09.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961 shell --fsid b715dee2-7650-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-19T06:57:09.284 INFO:teuthology.orchestra.run.smithi037.stderr:Inferring config /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/mon.a/config 2024-09-19T06:57:09.544 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:09 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1723158152' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:57:09.545 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:09 smithi037 ceph-mon[31211]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T06:57:09.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:09 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1723158152' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:57:09.613 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:09 smithi151 ceph-mon[35236]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T06:57:09.656 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:09 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1723158152' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:57:09.656 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:09 smithi096 ceph-mon[35004]: osdmap e109: 12 total, 12 up, 12 in 2024-09-19T06:57:09.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 -- 172.21.15.37:0/3148610370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 msgr2=0x7f17bc1031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:09.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 --2- 172.21.15.37:0/3148610370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1031b0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009cd0 tx=0x7f17ac02a8d0 comp rx=0 tx=0).stop 2024-09-19T06:57:09.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 -- 172.21.15.37:0/3148610370 shutdown_connections 2024-09-19T06:57:09.767 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 --2- 172.21.15.37:0/3148610370 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17bc108300 0x7f17bc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 --2- 172.21.15.37:0/3148610370 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17bc103780 0x7f17bc107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 --2- 172.21.15.37:0/3148610370 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1031b0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 -- 172.21.15.37:0/3148610370 >> 172.21.15.37:0/3148610370 conn(0x7f17bc0fc7d0 msgr2=0x7f17bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 -- 172.21.15.37:0/3148610370 shutdown_connections 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.759+0000 7f17c09ee640 1 -- 172.21.15.37:0/3148610370 wait complete. 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 Processor -- start 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 -- start start 2024-09-19T06:57:09.768 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17bc103780 0x7f17bc198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17bc108300 0x7f17bc19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 -- --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc112130 con 0x7f17bc102dd0 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc111fb0 con 0x7f17bc103780 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17c09ee640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 -- 0x7f17bc1122b0 con 0x7f17bc108300 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 --2- >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.37:3300/0 says I am v2:172.21.15.37:56328/0 (socket says 172.21.15.37:56328) 2024-09-19T06:57:09.769 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 -- 172.21.15.37:0/1691429 learned_addr learned my addr 172.21.15.37:0/1691429 (peer_addr_for_me v2:172.21.15.37:0/0) 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17b9d9b640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17bc103780 0x7f17bc198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17bad9d640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17bc108300 0x7f17bc19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 -- 172.21.15.37:0/1691429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17bc108300 msgr2=0x7f17bc19cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17bc108300 0x7f17bc19cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 -- 172.21.15.37:0/1691429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17bc103780 msgr2=0x7f17bc198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17bc103780 0x7f17bc198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17ba59c640 1 -- 172.21.15.37:0/1691429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17bc19d6a0 con 0x7f17bc102dd0 2024-09-19T06:57:09.770 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.760+0000 7f17b9d9b640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17bc103780 0x7f17bc198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.761+0000 7f17ba59c640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1986f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009df0 tx=0x7f17ac031420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.761+0000 7f17b37fe640 1 -- 172.21.15.37:0/1691429 <== mon.0 v2:172.21.15.37:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac031840 con 0x7f17bc102dd0 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.761+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17bc19d930 con 0x7f17bc102dd0 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.761+0000 7f17b37fe640 1 -- 172.21.15.37:0/1691429 <== mon.0 v2:172.21.15.37:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f17ac003e50 con 0x7f17bc102dd0 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.761+0000 7f17b37fe640 1 -- 172.21.15.37:0/1691429 <== mon.0 v2:172.21.15.37:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac0307f0 con 0x7f17bc102dd0 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.761+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17bc1a5270 con 0x7f17bc102dd0 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.762+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f17bc1031b0 con 0x7f17bc102dd0 2024-09-19T06:57:09.771 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.763+0000 7f17b37fe640 1 -- 172.21.15.37:0/1691429 <== mon.0 v2:172.21.15.37:3300/0 4 ==== mgrmap(e 26) ==== 138486+0+0 (secure 0 0 0) 0x7f17ac004340 con 0x7f17bc102dd0 2024-09-19T06:57:09.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.764+0000 7f17b37fe640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f17a80a33b0 0x7f17a80a5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-19T06:57:09.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.764+0000 7f17b9d9b640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f17a80a33b0 0x7f17a80a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-19T06:57:09.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.764+0000 7f17b37fe640 1 -- 172.21.15.37:0/1691429 <== mon.0 v2:172.21.15.37:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8202+0+0 (secure 0 0 0) 0x7f17ac0f0950 con 0x7f17bc102dd0 2024-09-19T06:57:09.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.765+0000 7f17b9d9b640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f17a80a33b0 0x7f17a80a5870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0fee90 tx=0x7f17a400b390 comp rx=0 tx=0).ready entity=mgr.15156 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-19T06:57:09.772 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.768+0000 7f17b37fe640 1 -- 172.21.15.37:0/1691429 <== mon.0 v2:172.21.15.37:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188913 (secure 0 0 0) 0x7f17ac0bd580 con 0x7f17bc102dd0 2024-09-19T06:57:09.947 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:09.945+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 --> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f17bc10a290 con 0x7f17bc102dd0 2024-09-19T06:57:10.280 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.279+0000 7f17b37fe640 1 -- 172.21.15.37:0/1691429 <== mon.0 v2:172.21.15.37:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) ==== 169+0+0 (secure 0 0 0) 0x7f17ac0bce80 con 0x7f17bc102dd0 2024-09-19T06:57:10.281 INFO:teuthology.orchestra.run.smithi037.stderr:pool 'cephfs_data' removed 2024-09-19T06:57:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f17a80a33b0 msgr2=0x7f17a80a5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f17a80a33b0 0x7f17a80a5870 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0fee90 tx=0x7f17a400b390 comp rx=0 tx=0).stop 2024-09-19T06:57:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 msgr2=0x7f17bc1986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-19T06:57:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1986f0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009df0 tx=0x7f17ac031420 comp rx=0 tx=0).stop 2024-09-19T06:57:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 shutdown_connections 2024-09-19T06:57:10.283 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f17bc108300 0x7f17bc19cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f17bc103780 0x7f17bc198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:6800/3483159058,v1:172.21.15.37:6801/3483159058] conn(0x7f17a80a33b0 0x7f17a80a5870 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 --2- 172.21.15.37:0/1691429 >> [v2:172.21.15.37:3300/0,v1:172.21.15.37:6789/0] conn(0x7f17bc102dd0 0x7f17bc1986f0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-19T06:57:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 >> 172.21.15.37:0/1691429 conn(0x7f17bc0fc7d0 msgr2=0x7f17bc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-19T06:57:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 shutdown_connections 2024-09-19T06:57:10.284 INFO:teuthology.orchestra.run.smithi037.stderr:2024-09-19T06:57:10.281+0000 7f17c09ee640 1 -- 172.21.15.37:0/1691429 wait complete. 2024-09-19T06:57:10.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:10 smithi037 ceph-mon[31211]: pgmap v489: 65 pgs: 65 active+clean; 282 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:57:10.605 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:10 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1691429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:57:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:10 smithi151 ceph-mon[35236]: pgmap v489: 65 pgs: 65 active+clean; 282 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:57:10.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:10 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1691429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:57:10.773 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:10 smithi096 ceph-mon[35004]: pgmap v489: 65 pgs: 65 active+clean; 282 MiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail 2024-09-19T06:57:10.774 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:10 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1691429' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-19T06:57:11.036 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:57:11.064 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:57:11.093 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:57:11.121 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-19T06:57:11.122 DEBUG:teuthology.orchestra.run.smithi037:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:57:11.138 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:57:11.154 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-19T06:57:11.177 INFO:tasks.cephadm:Stopping all daemons... 2024-09-19T06:57:11.177 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-19T06:57:11.178 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a 2024-09-19T06:57:11.486 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:11 smithi037 systemd[1]: Stopping Ceph mon.a for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:11.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:11 smithi037 ceph-mon[31211]: from='client.? 172.21.15.37:0/1691429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:57:11.487 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:11 smithi037 ceph-mon[31211]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T06:57:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:11 smithi096 ceph-mon[35004]: from='client.? 172.21.15.37:0/1691429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:57:11.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:11 smithi096 ceph-mon[35004]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T06:57:11.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:11 smithi151 ceph-mon[35236]: from='client.? 172.21.15.37:0/1691429' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-19T06:57:11.612 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:11 smithi151 ceph-mon[35236]: osdmap e110: 12 total, 12 up, 12 in 2024-09-19T06:57:11.738 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a[31207]: 2024-09-19T06:57:11.484+0000 7f46dec80640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:11.738 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a[31207]: 2024-09-19T06:57:11.484+0000 7f46dec80640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-19T06:57:11.738 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:11 smithi037 podman[119506]: 2024-09-19 06:57:11.719179407 +0000 UTC m=+0.346180992 container died 98ade29977a1e0ece0b32d797a5175e96eaf2a9230222862612c0cb0f52b0a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:57:12.438 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:12 smithi037 podman[119506]: 2024-09-19 06:57:12.235998916 +0000 UTC m=+0.863000494 container cleanup 98ade29977a1e0ece0b32d797a5175e96eaf2a9230222862612c0cb0f52b0a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:12.439 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:12 smithi037 bash[119506]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a 2024-09-19T06:57:12.671 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service' 2024-09-19T06:57:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:12 smithi037 podman[119519]: 2024-09-19 06:57:12.438041587 +0000 UTC m=+0.722010505 container remove 98ade29977a1e0ece0b32d797a5175e96eaf2a9230222862612c0cb0f52b0a0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-a, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T06:57:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:12 smithi037 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-19T06:57:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:12 smithi037 systemd[1]: Stopped Ceph mon.a for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:57:12.694 INFO:journalctl@ceph.mon.a.smithi037.stdout:Sep 19 06:57:12 smithi037 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.a.service: Consumed 23.759s CPU time. 2024-09-19T06:57:12.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:12.729 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-19T06:57:12.729 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-19T06:57:12.729 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.b 2024-09-19T06:57:13.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:12 smithi096 systemd[1]: Stopping Ceph mon.b for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-b[35000]: 2024-09-19T06:57:13.094+0000 7fb8501ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-b[35000]: 2024-09-19T06:57:13.094+0000 7fb8501ea640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-19T06:57:13.524 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:13 smithi096 podman[71058]: 2024-09-19 06:57:13.227654682 +0000 UTC m=+0.260902691 container died fbdc43f0f542a7d8849b61ed6162e7bde5ae5896cbe7c9a01760285533c2e6ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-19T06:57:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:13 smithi096 podman[71058]: 2024-09-19 06:57:13.637297353 +0000 UTC m=+0.670545359 container cleanup fbdc43f0f542a7d8849b61ed6162e7bde5ae5896cbe7c9a01760285533c2e6ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, io.buildah.version=1.37.2) 2024-09-19T06:57:14.024 INFO:journalctl@ceph.mon.b.smithi096.stdout:Sep 19 06:57:13 smithi096 bash[71058]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-b 2024-09-19T06:57:14.187 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.b.service' 2024-09-19T06:57:14.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:14.236 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-19T06:57:14.236 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-19T06:57:14.237 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.c 2024-09-19T06:57:14.551 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:14 smithi151 systemd[1]: Stopping Ceph mon.c for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:14.862 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-c[35230]: 2024-09-19T06:57:14.549+0000 7fd6dcac1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:14.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-c[35230]: 2024-09-19T06:57:14.549+0000 7fd6dcac1640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-19T06:57:14.863 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:14 smithi151 podman[72898]: 2024-09-19 06:57:14.601521578 +0000 UTC m=+0.163848684 container died 50dfa29c6126b85ac32b8926a64e8d9ca79c7b22c5a238b531ff714fdd2eae82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-c, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:15.246 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:14 smithi151 podman[72898]: 2024-09-19 06:57:14.993530114 +0000 UTC m=+0.555857221 container cleanup 50dfa29c6126b85ac32b8926a64e8d9ca79c7b22c5a238b531ff714fdd2eae82 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T06:57:15.246 INFO:journalctl@ceph.mon.c.smithi151.stdout:Sep 19 06:57:14 smithi151 bash[72898]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mon-c 2024-09-19T06:57:15.376 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mon.c.service' 2024-09-19T06:57:15.418 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:15.419 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-19T06:57:15.419 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-19T06:57:15.419 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x 2024-09-19T06:57:15.775 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:15 smithi037 systemd[1]: Stopping Ceph mgr.x for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:16.068 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:15 smithi037 podman[119629]: 2024-09-19 06:57:15.781430234 +0000 UTC m=+0.154235046 container died 0d11568288e00937fa6984046f34a92244bc472c99ed39bf59c050ce1d88751d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:57:16.330 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:16 smithi037 podman[119629]: 2024-09-19 06:57:16.301424435 +0000 UTC m=+0.674229235 container cleanup 0d11568288e00937fa6984046f34a92244bc472c99ed39bf59c050ce1d88751d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:16.330 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:16 smithi037 bash[119629]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x 2024-09-19T06:57:16.759 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-19T06:57:16.783 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:16 smithi037 podman[119641]: 2024-09-19 06:57:16.528581832 +0000 UTC m=+0.738801851 container remove 0d11568288e00937fa6984046f34a92244bc472c99ed39bf59c050ce1d88751d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2f599a46ad10e37c929e5c43da2f6907ddcd1961, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-x, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T06:57:16.783 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:16 smithi037 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x.service: Deactivated successfully. 2024-09-19T06:57:16.783 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:16 smithi037 systemd[1]: Stopped Ceph mgr.x for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:57:16.783 INFO:journalctl@ceph.mgr.x.smithi037.stdout:Sep 19 06:57:16 smithi037 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.x.service: Consumed 56.673s CPU time. 2024-09-19T06:57:16.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:16.797 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-19T06:57:16.798 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-19T06:57:16.798 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.y 2024-09-19T06:57:17.122 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:57:16 smithi096 systemd[1]: Stopping Ceph mgr.y for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:17.419 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:57:17 smithi096 podman[71182]: 2024-09-19 06:57:17.121813054 +0000 UTC m=+0.140598949 container died 3b0d4123ccc87faca78e307bdc443f4b78f2ea0d34e3661af7e32485a9ffc8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T06:57:17.745 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:57:17 smithi096 podman[71182]: 2024-09-19 06:57:17.526711498 +0000 UTC m=+0.545497393 container cleanup 3b0d4123ccc87faca78e307bdc443f4b78f2ea0d34e3661af7e32485a9ffc8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T06:57:17.745 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:57:17 smithi096 bash[71182]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y 2024-09-19T06:57:17.998 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:57:17 smithi096 podman[71194]: 2024-09-19 06:57:17.745035096 +0000 UTC m=+0.617207151 container remove 3b0d4123ccc87faca78e307bdc443f4b78f2ea0d34e3661af7e32485a9ffc8c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T06:57:17.998 INFO:journalctl@ceph.mgr.y.smithi096.stdout:Sep 19 06:57:17 smithi096 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-19T06:57:18.002 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-19T06:57:18.036 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:18.036 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-19T06:57:18.037 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-19T06:57:18.037 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.z 2024-09-19T06:57:18.362 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:57:18 smithi151 systemd[1]: Stopping Ceph mgr.z for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:18.676 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:57:18 smithi151 podman[73023]: 2024-09-19 06:57:18.370050089 +0000 UTC m=+0.133387391 container died ef213e9b1b0853b529be0e7340f0c885ede1ecd87ca08c2daba71df91169f351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:57:19.028 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:57:18 smithi151 podman[73023]: 2024-09-19 06:57:18.767465484 +0000 UTC m=+0.530802794 container cleanup ef213e9b1b0853b529be0e7340f0c885ede1ecd87ca08c2daba71df91169f351 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:19.029 INFO:journalctl@ceph.mgr.z.smithi151.stdout:Sep 19 06:57:18 smithi151 bash[73023]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mgr-z 2024-09-19T06:57:19.170 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-19T06:57:19.205 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:19.205 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-19T06:57:19.206 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-19T06:57:19.206 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.0 2024-09-19T06:57:19.529 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:19 smithi037 systemd[1]: Stopping Ceph osd.0 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:20.025 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0[45094]: 2024-09-19T06:57:19.531+0000 7fe605314640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:20.025 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0[45094]: 2024-09-19T06:57:19.531+0000 7fe605314640 -1 osd.0 110 *** Got signal Terminated *** 2024-09-19T06:57:20.026 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0[45094]: 2024-09-19T06:57:19.531+0000 7fe605314640 -1 osd.0 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:57:25.259 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:24 smithi037 podman[119753]: 2024-09-19 06:57:24.953888705 +0000 UTC m=+5.546152376 container died 538a9aac3e85d5e55347a8143f04e3a5f1c20e349a28e6b096a561e3676a6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:57:25.525 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:25 smithi037 podman[119753]: 2024-09-19 06:57:25.466085564 +0000 UTC m=+6.058349242 container cleanup 538a9aac3e85d5e55347a8143f04e3a5f1c20e349a28e6b096a561e3676a6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T06:57:25.525 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:25 smithi037 bash[119753]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0 2024-09-19T06:57:26.025 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:25 smithi037 podman[119768]: 2024-09-19 06:57:25.676119516 +0000 UTC m=+0.718853818 container remove 538a9aac3e85d5e55347a8143f04e3a5f1c20e349a28e6b096a561e3676a6933 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T06:57:26.489 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:26 smithi037 podman[119856]: 2024-09-19 06:57:25.994994258 +0000 UTC m=+0.016539491 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:57:26.489 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:26 smithi037 podman[119856]: 2024-09-19 06:57:26.118859449 +0000 UTC m=+0.140404673 container create fb7c5ef21601774b399366b724007f0c5c4eb4142a1a06f79e0bbb339379b983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T06:57:26.490 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:26 smithi037 podman[119856]: 2024-09-19 06:57:26.488721481 +0000 UTC m=+0.510266728 container init fb7c5ef21601774b399366b724007f0c5c4eb4142a1a06f79e0bbb339379b983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:57:26.775 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:26 smithi037 podman[119856]: 2024-09-19 06:57:26.492926541 +0000 UTC m=+0.514471765 container start fb7c5ef21601774b399366b724007f0c5c4eb4142a1a06f79e0bbb339379b983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T06:57:26.775 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:26 smithi037 podman[119856]: 2024-09-19 06:57:26.596899774 +0000 UTC m=+0.618445015 container attach fb7c5ef21601774b399366b724007f0c5c4eb4142a1a06f79e0bbb339379b983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:57:26.776 INFO:journalctl@ceph.osd.0.smithi037.stdout:Sep 19 06:57:26 smithi037 podman[119856]: 2024-09-19 06:57:26.68764326 +0000 UTC m=+0.709188491 container died fb7c5ef21601774b399366b724007f0c5c4eb4142a1a06f79e0bbb339379b983 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T06:57:27.514 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.0.service' 2024-09-19T06:57:27.562 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:27.563 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-19T06:57:27.563 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-19T06:57:27.563 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.1 2024-09-19T06:57:27.881 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:27 smithi096 systemd[1]: Stopping Ceph osd.1 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:28.274 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1[39479]: 2024-09-19T06:57:27.880+0000 7fa06b177640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:28.274 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1[39479]: 2024-09-19T06:57:27.880+0000 7fa06b177640 -1 osd.1 110 *** Got signal Terminated *** 2024-09-19T06:57:28.274 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1[39479]: 2024-09-19T06:57:27.880+0000 7fa06b177640 -1 osd.1 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:57:33.748 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:33 smithi096 podman[71299]: 2024-09-19 06:57:33.452323881 +0000 UTC m=+5.693354378 container died 213bf026dadcc39a93e520757bfa378073840280bf2c0b7c2723aaaef5e7d51b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-19T06:57:34.023 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:33 smithi096 podman[71299]: 2024-09-19 06:57:33.847212359 +0000 UTC m=+6.088242856 container cleanup 213bf026dadcc39a93e520757bfa378073840280bf2c0b7c2723aaaef5e7d51b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-19T06:57:34.024 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:33 smithi096 bash[71299]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1 2024-09-19T06:57:34.439 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:34 smithi096 podman[71317]: 2024-09-19 06:57:34.132278219 +0000 UTC m=+0.679950356 container remove 213bf026dadcc39a93e520757bfa378073840280bf2c0b7c2723aaaef5e7d51b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-19T06:57:34.736 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:34 smithi096 podman[71400]: 2024-09-19 06:57:34.341832182 +0000 UTC m=+0.015693565 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:57:34.736 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:34 smithi096 podman[71400]: 2024-09-19 06:57:34.466615752 +0000 UTC m=+0.140477165 container create fe01eae99851a5f5fbda313fa08443c7b5f12a8f3ff7b81d9d0f01dbd920018b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-19T06:57:34.737 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:34 smithi096 podman[71400]: 2024-09-19 06:57:34.735863081 +0000 UTC m=+0.409724452 container init fe01eae99851a5f5fbda313fa08443c7b5f12a8f3ff7b81d9d0f01dbd920018b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2) 2024-09-19T06:57:35.024 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:34 smithi096 podman[71400]: 2024-09-19 06:57:34.739406713 +0000 UTC m=+0.413268088 container start fe01eae99851a5f5fbda313fa08443c7b5f12a8f3ff7b81d9d0f01dbd920018b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True) 2024-09-19T06:57:35.024 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:34 smithi096 podman[71400]: 2024-09-19 06:57:34.886125941 +0000 UTC m=+0.559987313 container attach fe01eae99851a5f5fbda313fa08443c7b5f12a8f3ff7b81d9d0f01dbd920018b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731) 2024-09-19T06:57:35.323 INFO:journalctl@ceph.osd.1.smithi096.stdout:Sep 19 06:57:35 smithi096 podman[71400]: 2024-09-19 06:57:35.042641718 +0000 UTC m=+0.716503093 container died fe01eae99851a5f5fbda313fa08443c7b5f12a8f3ff7b81d9d0f01dbd920018b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T06:57:35.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.1.service' 2024-09-19T06:57:35.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:35.810 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-19T06:57:35.810 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-19T06:57:35.810 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.2 2024-09-19T06:57:36.112 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:35 smithi151 systemd[1]: Stopping Ceph osd.2 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:36.612 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2[40199]: 2024-09-19T06:57:36.231+0000 7f72a59f7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:36.612 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2[40199]: 2024-09-19T06:57:36.231+0000 7f72a59f7640 -1 osd.2 110 *** Got signal Terminated *** 2024-09-19T06:57:36.613 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2[40199]: 2024-09-19T06:57:36.231+0000 7f72a59f7640 -1 osd.2 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:57:41.892 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:41 smithi151 podman[73147]: 2024-09-19 06:57:41.612363988 +0000 UTC m=+5.607831270 container died 13bf3e81dc718e9a9bd085a2ccbba62a551b1c6c34058e184bc5e1dd556af196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:57:42.310 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:41 smithi151 podman[73147]: 2024-09-19 06:57:41.974685883 +0000 UTC m=+5.970153131 container cleanup 13bf3e81dc718e9a9bd085a2ccbba62a551b1c6c34058e184bc5e1dd556af196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:42.311 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:41 smithi151 bash[73147]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2 2024-09-19T06:57:42.612 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:42 smithi151 podman[73166]: 2024-09-19 06:57:42.310188891 +0000 UTC m=+0.696624093 container remove 13bf3e81dc718e9a9bd085a2ccbba62a551b1c6c34058e184bc5e1dd556af196 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-09-19T06:57:42.898 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:42 smithi151 podman[73250]: 2024-09-19 06:57:42.539384392 +0000 UTC m=+0.016072630 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:57:42.899 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:42 smithi151 podman[73250]: 2024-09-19 06:57:42.670118722 +0000 UTC m=+0.146806946 container create 8963ad59aafe95dc6f4ec2fd4efb9c6455b976b1ddea0529b12e58bcfe82610f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:57:42.899 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:42 smithi151 podman[73250]: 2024-09-19 06:57:42.898304983 +0000 UTC m=+0.374993212 container init 8963ad59aafe95dc6f4ec2fd4efb9c6455b976b1ddea0529b12e58bcfe82610f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:43.354 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:42 smithi151 podman[73250]: 2024-09-19 06:57:42.902157071 +0000 UTC m=+0.378845314 container start 8963ad59aafe95dc6f4ec2fd4efb9c6455b976b1ddea0529b12e58bcfe82610f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-19T06:57:43.355 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:42 smithi151 podman[73250]: 2024-09-19 06:57:42.973484609 +0000 UTC m=+0.450172846 container attach 8963ad59aafe95dc6f4ec2fd4efb9c6455b976b1ddea0529b12e58bcfe82610f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T06:57:43.355 INFO:journalctl@ceph.osd.2.smithi151.stdout:Sep 19 06:57:43 smithi151 podman[73250]: 2024-09-19 06:57:43.087916087 +0000 UTC m=+0.564604323 container died 8963ad59aafe95dc6f4ec2fd4efb9c6455b976b1ddea0529b12e58bcfe82610f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True) 2024-09-19T06:57:43.819 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.2.service' 2024-09-19T06:57:43.858 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:43.858 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-19T06:57:43.858 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-19T06:57:43.859 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.3 2024-09-19T06:57:44.181 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:43 smithi037 systemd[1]: Stopping Ceph osd.3 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:44.525 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:44 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:57:44.180+0000 7f0381b2d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:44.525 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:44 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:57:44.180+0000 7f0381b2d640 -1 osd.3 110 *** Got signal Terminated *** 2024-09-19T06:57:44.526 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:44 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:57:44.180+0000 7f0381b2d640 -1 osd.3 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:57:46.775 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:46 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:57:46.495+0000 7f037a13b640 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:22.175308+0000 front 2024-09-19T06:57:22.175076+0000 (oldest deadline 2024-09-19T06:57:45.675329+0000) 2024-09-19T06:57:47.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:46.959+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:47.745 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:47 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:57:47.451+0000 7f037a13b640 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:22.175308+0000 front 2024-09-19T06:57:22.175076+0000 (oldest deadline 2024-09-19T06:57:45.675329+0000) 2024-09-19T06:57:47.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:47 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:47.502+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:48.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:47 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:47.745+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:48.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:48.003+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:48.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:48 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:48.465+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:48.775 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3[51545]: 2024-09-19T06:57:48.493+0000 7f037a13b640 -1 osd.3 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:22.175308+0000 front 2024-09-19T06:57:22.175076+0000 (oldest deadline 2024-09-19T06:57:45.675329+0000) 2024-09-19T06:57:48.775 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:48.399+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:48.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:48.525+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:49.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:48 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:48.781+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:49.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:49.017+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:49.626 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:49 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:49.373+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:49.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:49 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:49.495+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:49.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:49.543+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:49.932 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:49 smithi037 podman[119960]: 2024-09-19 06:57:49.626186559 +0000 UTC m=+5.563067059 container died 8fd8a5b539eda02d71d280d54ce03f712bc08d544ae60b59e0e7df0f86a4314e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T06:57:49.932 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:49 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:49.740+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:50.275 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:50 smithi037 podman[119960]: 2024-09-19 06:57:50.10457113 +0000 UTC m=+6.041451617 container cleanup 8fd8a5b539eda02d71d280d54ce03f712bc08d544ae60b59e0e7df0f86a4314e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-19T06:57:50.275 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:50 smithi037 bash[119960]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3 2024-09-19T06:57:50.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:50.031+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:50.475 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:50 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:50.169+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:50.709 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:50 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:50.328+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:50.709 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:50 smithi037 podman[119989]: 2024-09-19 06:57:50.52314352 +0000 UTC m=+0.890233067 container remove 8fd8a5b539eda02d71d280d54ce03f712bc08d544ae60b59e0e7df0f86a4314e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:57:50.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:50 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:50.475+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:50.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:50.495+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:50.980 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:50 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:50.708+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:50.981 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:50 smithi037 podman[120078]: 2024-09-19 06:57:50.747329168 +0000 UTC m=+0.016648437 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:57:50.981 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:50 smithi037 podman[120078]: 2024-09-19 06:57:50.865792992 +0000 UTC m=+0.135112287 container create 67ce9d32732792edd301133e9a4102484ef63857e1a799ddf66ff0ce96eb1560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T06:57:51.275 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:51 smithi037 podman[120078]: 2024-09-19 06:57:51.101976659 +0000 UTC m=+0.371295918 container init 67ce9d32732792edd301133e9a4102484ef63857e1a799ddf66ff0ce96eb1560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3-deactivate, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-19T06:57:51.275 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:51 smithi037 podman[120078]: 2024-09-19 06:57:51.105325709 +0000 UTC m=+0.374644961 container start 67ce9d32732792edd301133e9a4102484ef63857e1a799ddf66ff0ce96eb1560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-19T06:57:51.275 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:51 smithi037 podman[120078]: 2024-09-19 06:57:51.201808697 +0000 UTC m=+0.471127947 container attach 67ce9d32732792edd301133e9a4102484ef63857e1a799ddf66ff0ce96eb1560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:51.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:51.050+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:51.524 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:51 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:51.218+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:51.524 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:51 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:51.434+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:51.606 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:51.360+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:51.606 INFO:journalctl@ceph.osd.3.smithi037.stdout:Sep 19 06:57:51 smithi037 podman[120078]: 2024-09-19 06:57:51.297616995 +0000 UTC m=+0.566936263 container died 67ce9d32732792edd301133e9a4102484ef63857e1a799ddf66ff0ce96eb1560 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:57:51.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:51.481+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:51.923 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:51 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:51.708+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:51.945 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.3.service' 2024-09-19T06:57:51.991 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:57:51.991 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-19T06:57:51.992 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-19T06:57:51.992 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.4 2024-09-19T06:57:52.274 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:52 smithi096 systemd[1]: Stopping Ceph osd.4 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:57:52.274 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:52 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:52.187+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:52.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:52 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:52.069+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:52.729 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:52 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:52.379+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:52.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:52 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:52.469+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:52.774 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:52 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:52.317+0000 7f7f11f3e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:57:52.774 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:52 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:52.317+0000 7f7f11f3e640 -1 osd.4 110 *** Got signal Terminated *** 2024-09-19T06:57:52.774 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:52 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:52.317+0000 7f7f11f3e640 -1 osd.4 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:57:52.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:52 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:52.445+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:53.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:52 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:52.728+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:53.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:53.060+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:53.508 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:53 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:53.173+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:53.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:53 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:53.508+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:53.774 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:53 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:53.357+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:53.827 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:53 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:53.487+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:54.086 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:57:54 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:57:54.025+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:57:54.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:53 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:53.775+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:54.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:54 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:54.085+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:54.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:54 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:54.085+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:57:54.477 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:54 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:54.148+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:54.730 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:54.325+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:54.731 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:54.325+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:57:54.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:54 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:54.476+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:54.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:54 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:54.522+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:54.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:54 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:54.522+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:57:55.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:54 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:54.730+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:55.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:57:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:57:55.046+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:57:55.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:55.038+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:55.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:55.038+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:57:55.499 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:55.099+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:55.499 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:57:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:57:55.278+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:57:55.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:55 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:55.497+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:55.774 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:55.349+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:55.775 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:55.349+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:57:55.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:55.480+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:55.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:55 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:55.480+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:57:56.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:55.777+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:56.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:57:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:57:56.047+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:57:56.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:56.057+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:56.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:56.057+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:57:56.499 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:56 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:56.054+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:56.500 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:57:56 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:57:56.233+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:57:56.729 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:56 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:56.302+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:56.729 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:56 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:56.302+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:57:56.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:56 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:56.498+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:56.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:56.485+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:56.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:56 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:56.485+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:57:57.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:56 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:56.728+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:57.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:56 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:56.728+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:57:57.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:57:57 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:57:57.007+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:57:57.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:57 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:57.015+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:57.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:57 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:57.015+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:57:57.477 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:57 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4[45924]: 2024-09-19T06:57:57.054+0000 7f7f0e554640 -1 osd.4 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.348821+0000 front 2024-09-19T06:57:23.348769+0000 (oldest deadline 2024-09-19T06:57:49.248674+0000) 2024-09-19T06:57:57.477 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:57:57 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:57:57.243+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:57:57.698 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:57 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:57.315+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:57.698 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:57 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:57.315+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:57:57.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:57 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:57.475+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:57.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:57 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:57.489+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:57.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:57 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:57.489+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:57:58.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:57 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:57.697+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:58.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:57 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:57.697+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:57:58.078 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:57 smithi096 podman[71516]: 2024-09-19 06:57:57.814318034 +0000 UTC m=+5.622271113 container died 1fa96ec4c71d11ba8912d200d6369e869ae6486bdc01ab5f3eff73f80b69cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T06:57:58.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:57:58 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:57:58.036+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:57:58.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:58 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:58.036+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:58.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:58 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:58.036+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:57:58.502 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:57:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:57:58.241+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:57:58.503 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:58 smithi096 podman[71516]: 2024-09-19 06:57:58.175432849 +0000 UTC m=+5.983385926 container cleanup 1fa96ec4c71d11ba8912d200d6369e869ae6486bdc01ab5f3eff73f80b69cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:57:58.503 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:58 smithi096 bash[71516]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4 2024-09-19T06:57:58.663 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:58.357+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:58.664 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:58.357+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:57:58.774 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:58 smithi096 podman[71531]: 2024-09-19 06:57:58.502153712 +0000 UTC m=+0.687523228 container remove 1fa96ec4c71d11ba8912d200d6369e869ae6486bdc01ab5f3eff73f80b69cac6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:57:58.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:58.507+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:58.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:58 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:58.507+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:57:58.864 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:58 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:58.515+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:58.864 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:58 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:58.515+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:57:59.027 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:58.662+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:57:59.027 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:58 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:58.662+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:57:59.131 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:58 smithi096 podman[71622]: 2024-09-19 06:57:58.738435915 +0000 UTC m=+0.015622688 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:57:59.132 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:58 smithi096 podman[71622]: 2024-09-19 06:57:58.853004684 +0000 UTC m=+0.130191463 container create e7a8b27c66ffda99f23937f9d204e26e399bd4b331212c694905b907a68fcc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4-deactivate, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-19T06:57:59.132 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:59 smithi096 podman[71622]: 2024-09-19 06:57:59.130706526 +0000 UTC m=+0.407893299 container init e7a8b27c66ffda99f23937f9d204e26e399bd4b331212c694905b907a68fcc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-09-19T06:57:59.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:57:59 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:57:59.083+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:57:59.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:59 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:59.048+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:57:59.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:57:59 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:57:59.048+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:57:59.490 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:57:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:57:59.280+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:57:59.491 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:59 smithi096 podman[71622]: 2024-09-19 06:57:59.134216806 +0000 UTC m=+0.411403579 container start e7a8b27c66ffda99f23937f9d204e26e399bd4b331212c694905b907a68fcc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-19T06:57:59.491 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:59 smithi096 podman[71622]: 2024-09-19 06:57:59.239150019 +0000 UTC m=+0.516336808 container attach e7a8b27c66ffda99f23937f9d204e26e399bd4b331212c694905b907a68fcc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T06:57:59.491 INFO:journalctl@ceph.osd.4.smithi096.stdout:Sep 19 06:57:59 smithi096 podman[71622]: 2024-09-19 06:57:59.327417422 +0000 UTC m=+0.604604200 container died e7a8b27c66ffda99f23937f9d204e26e399bd4b331212c694905b907a68fcc33 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_CLEAN=True) 2024-09-19T06:57:59.681 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:59 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:59.313+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:57:59.682 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:57:59 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:57:59.313+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:57:59.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:59.489+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:57:59.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:57:59 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:57:59.489+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:57:59.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:59 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:59.516+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:57:59.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:57:59 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:57:59.516+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:57:59.970 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.4.service' 2024-09-19T06:58:00.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:00.013 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-19T06:58:00.013 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-19T06:58:00.013 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.5 2024-09-19T06:58:00.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:59 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:59.680+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:00.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:57:59 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:57:59.680+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:00.336 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:00 smithi151 systemd[1]: Stopping Ceph osd.5 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:00.337 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:00.044+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:00.337 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:00.044+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:00.337 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:00.044+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:00.612 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:00.332+0000 7f85e666f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:00.613 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:00.332+0000 7f85e666f640 -1 osd.5 110 *** Got signal Terminated *** 2024-09-19T06:58:00.613 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:00.332+0000 7f85e666f640 -1 osd.5 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:58:00.613 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:00.541+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:58:00.613 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:00 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:00.541+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:58:00.703 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:00.293+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:00.703 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:00.293+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:00.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:00 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:00.287+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:00.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:00 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:00.521+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:00.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:00 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:00.521+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:01.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:00.703+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:01.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:00.703+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:01.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:01 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:01.080+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:01.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:01 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:01.011+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:01.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:01 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:01.011+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:01.550 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:01 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:01.273+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:01.672 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:01 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:01.330+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:01.672 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:01 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:01.330+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:01.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:01 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:01.562+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:58:01.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:01 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:01.562+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:58:02.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:01 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:01.549+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:02.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:01 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:01.549+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:02.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:01 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:01.671+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:02.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:01 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:01.671+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:02.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:01 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:01.671+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:02.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:02 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:02.075+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:02.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:02 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:02.032+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:02.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:02 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:02.032+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:02.523 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:02 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:02.260+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:02.717 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:02 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:02.313+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:02.718 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:02 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:02.313+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:02.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:02 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:02.550+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:58:02.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:02 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:02.550+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:58:03.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:02 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:02.565+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:03.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:02 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:02.565+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:03.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:02 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:02.565+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:03.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:02 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:02.717+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:03.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:02 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:02.717+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:03.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:02 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:02.717+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:03.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:03 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:03.086+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:03.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:03 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:03.053+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:03.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:03 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:03.053+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:03.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:03 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:03.053+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:03.520 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:03 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:03.233+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:03.521 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:03 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:03.233+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:03.705 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:03 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:03.315+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:03.706 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:03 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:03.315+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:03.706 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:03 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:03.315+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:03.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:03 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:03.516+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:03.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:03 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:03.516+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:03.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:03 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:03.516+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:03.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:03 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:03.502+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:58:03.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:03 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:03.502+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:58:03.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:03 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:03.502+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.799397+0000 front 2024-09-19T06:57:38.799432+0000 (oldest deadline 2024-09-19T06:58:02.899521+0000) 2024-09-19T06:58:04.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:03 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:03.704+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:04.257 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:03 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:03.704+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:04.257 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:03 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:03.704+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:04.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:04.084+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:04.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:04.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:04.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:04.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:04.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:04.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:04.476 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:04.216+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:04.476 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:04.216+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:04.674 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:04 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:04.276+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:04.674 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:04 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:04.276+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:04.674 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:04 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:04.276+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:04.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:04.475+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:04.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:04.475+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:04.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:04 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:04.475+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:04.862 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:04.484+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:21.396554+0000 front 2024-09-19T06:57:21.396564+0000 (oldest deadline 2024-09-19T06:57:47.796946+0000) 2024-09-19T06:58:04.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:04.484+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.398400+0000 front 2024-09-19T06:57:32.398383+0000 (oldest deadline 2024-09-19T06:57:53.498520+0000) 2024-09-19T06:58:04.863 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:04 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5[46946]: 2024-09-19T06:58:04.484+0000 7f85e2c85640 -1 osd.5 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.799397+0000 front 2024-09-19T06:57:38.799432+0000 (oldest deadline 2024-09-19T06:58:02.899521+0000) 2024-09-19T06:58:05.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:04 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:04.673+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:05.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:04 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:04.673+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:05.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:04 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:04.673+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:05.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:05 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:05.112+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:05.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:05 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:05.112+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:05.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:05 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:05.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:05.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:05 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:05.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:05.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:05 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:05.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:05.462 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:05 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:05.199+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:05.462 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:05 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:05.199+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:05.655 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:05.292+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:05.656 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:05.292+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:05.656 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:05.292+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:05.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:05 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:05.461+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:05.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:05 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:05.461+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:05.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:05 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:05.461+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:06.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:05.655+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:06.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:05.655+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:06.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:05.655+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:06.160 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:05 smithi151 podman[73365]: 2024-09-19 06:58:05.877740372 +0000 UTC m=+5.665607414 container died 22f12d51ec75191c0f023c4c326c6e90baba7485a2acca5b0d7cb4852a716532 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-19T06:58:06.160 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:06 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:06.108+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:06.161 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:06 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:06.108+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:06.161 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:06 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:06.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:06.161 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:06 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:06.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:06.161 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:06 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:06.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:06.507 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:06 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:06.227+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:06.507 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:06 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:06.227+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:06.544 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:06 smithi151 podman[73365]: 2024-09-19 06:58:06.258992076 +0000 UTC m=+6.046859116 container cleanup 22f12d51ec75191c0f023c4c326c6e90baba7485a2acca5b0d7cb4852a716532 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:58:06.544 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:06 smithi151 bash[73365]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5 2024-09-19T06:58:06.702 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:06.297+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:06.702 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:06.297+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:06.703 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:06.297+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:06.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:06 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:06.506+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:06.773 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:06 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:06.506+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:06.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:06 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:06.506+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:06.856 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:06 smithi151 podman[73380]: 2024-09-19 06:58:06.544135056 +0000 UTC m=+0.668079966 container remove 22f12d51ec75191c0f023c4c326c6e90baba7485a2acca5b0d7cb4852a716532 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-19T06:58:07.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:06.702+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:07.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:06.702+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:07.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:06 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:06.702+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:07.107 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:06 smithi151 podman[73470]: 2024-09-19 06:58:06.758910053 +0000 UTC m=+0.015985147 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:58:07.107 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:06 smithi151 podman[73470]: 2024-09-19 06:58:06.870632333 +0000 UTC m=+0.127707457 container create ca4e395f6f6f191711e2c58b8aa54f48761d1e6dfb46c70bd9d62a2fe27a07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:58:07.108 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:07 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:07.062+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:07.108 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:07 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:07.062+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:07.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:07 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:07.127+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:07.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:07 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:07.127+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:07.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:07 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:07.127+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:07.363 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:07 smithi151 podman[73470]: 2024-09-19 06:58:07.106793676 +0000 UTC m=+0.363868755 container init ca4e395f6f6f191711e2c58b8aa54f48761d1e6dfb46c70bd9d62a2fe27a07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-19T06:58:07.363 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:07 smithi151 podman[73470]: 2024-09-19 06:58:07.110459534 +0000 UTC m=+0.367534616 container start ca4e395f6f6f191711e2c58b8aa54f48761d1e6dfb46c70bd9d62a2fe27a07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-19T06:58:07.363 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:07 smithi151 podman[73470]: 2024-09-19 06:58:07.207107945 +0000 UTC m=+0.464183024 container attach ca4e395f6f6f191711e2c58b8aa54f48761d1e6dfb46c70bd9d62a2fe27a07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-19T06:58:07.364 INFO:journalctl@ceph.osd.5.smithi151.stdout:Sep 19 06:58:07 smithi151 podman[73470]: 2024-09-19 06:58:07.298029863 +0000 UTC m=+0.555104952 container died ca4e395f6f6f191711e2c58b8aa54f48761d1e6dfb46c70bd9d62a2fe27a07c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-19T06:58:07.523 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:07 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:07.244+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:07.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:07 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:07.244+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:07.697 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:07 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:07.341+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:07.698 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:07 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:07.341+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:07.698 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:07 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:07.341+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:07.981 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.5.service' 2024-09-19T06:58:08.022 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:08.022 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-19T06:58:08.023 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-19T06:58:08.023 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.6 2024-09-19T06:58:08.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:07 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:07.522+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:08.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:07 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:07.522+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:08.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:07 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:07.522+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:08.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:07 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:07.697+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:08.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:07 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:07.697+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:08.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:07 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:07.697+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:08.307 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:08 smithi037 systemd[1]: Stopping Ceph osd.6 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:08.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:08 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:08.092+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:08.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:08 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:08.092+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:08.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:08 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:08.088+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:08.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:08 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:08.088+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:08.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:08 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:08.088+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:08.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:08 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:08.272+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:08.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:08 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:08.272+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:08.652 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:08.306+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:08.652 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:08.306+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:08.652 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:08.306+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:08.652 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:08.342+0000 7fa7fc5ea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:08.653 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:08.342+0000 7fa7fc5ea640 -1 osd.6 110 *** Got signal Terminated *** 2024-09-19T06:58:08.653 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:08.342+0000 7fa7fc5ea640 -1 osd.6 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:58:09.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:08 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:08.529+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:09.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:08 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:08.529+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:09.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:08 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:08.529+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:09.024 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:08.650+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:09.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:08.650+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:09.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:08 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:08.650+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:09.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:09.265+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:09.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:09.265+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:09.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:09.265+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:09.612 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:09 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:09.141+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:09.613 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:09 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:09.141+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:09.613 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:09 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:09.124+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:09.613 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:09 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:09.124+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:09.613 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:09 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:09.124+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:09.773 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:09 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:09.313+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:09.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:09 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:09.313+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:09.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:09 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:09.511+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:09.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:09 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:09.511+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:09.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:09 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:09.511+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:10.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:09.693+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:10.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:09.693+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:10.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:09.693+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:10.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:10 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:10.108+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:10.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:10 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:10.108+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:10.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:10 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:10.139+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:10.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:10 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:10.139+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:10.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:10 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:10.139+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:10.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:10 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:10.264+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:10.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:10 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:10.264+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:10.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:10 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:10.264+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:10.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:10 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:10.264+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:44.784256+0000 front 2024-09-19T06:57:44.784284+0000 (oldest deadline 2024-09-19T06:58:10.084521+0000) 2024-09-19T06:58:10.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:10 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:10.287+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:10.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:10 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:10.287+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:10.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:10 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:10.476+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:10.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:10 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:10.476+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:10.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:10 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:10.476+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:11.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:10 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:10.661+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:11.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:10 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:10.661+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:11.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:10 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:10.661+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:11.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:11.064+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:11.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:11.064+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:11.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:11.118+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:11.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:11.118+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:11.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:11.118+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:11.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:11 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:11.118+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:11.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:11.269+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:11.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:11.269+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:11.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:11.269+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:11.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:11.269+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:44.784256+0000 front 2024-09-19T06:57:44.784284+0000 (oldest deadline 2024-09-19T06:58:10.084521+0000) 2024-09-19T06:58:11.773 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:11 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:11.296+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:11.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:11 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:11.296+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:11.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:11 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:11.503+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:11.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:11 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:11.503+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:11.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:11 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:11.503+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:12.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:11.701+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:12.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:11.701+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:12.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:11.701+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:12.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:11 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:11.701+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:12.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:12.098+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:12.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:12.098+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:12.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:12.114+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:12.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:12.114+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:12.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:12.114+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:12.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:12 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:12.114+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:12.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:12.252+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:12.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:12.252+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:12.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:12.252+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:12.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:12.252+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:44.784256+0000 front 2024-09-19T06:57:44.784284+0000 (oldest deadline 2024-09-19T06:58:10.084521+0000) 2024-09-19T06:58:12.773 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:12 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:12.294+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:12.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:12 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:12.294+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:12.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:12 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:12.517+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:12.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:12 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:12.517+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:12.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:12 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:12.517+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:12.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:12 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:12.517+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:13.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:12.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:13.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:12.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:13.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:12.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:13.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:12 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:12.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:13.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:13 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:13.115+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:13.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:13 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:13.115+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:13.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:13 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:13.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:13.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:13 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:13.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:13.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:13 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:13.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:13.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:13 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:13.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:13.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:13.250+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:24.478928+0000 front 2024-09-19T06:57:24.479070+0000 (oldest deadline 2024-09-19T06:57:47.979134+0000) 2024-09-19T06:58:13.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:13.250+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.480723+0000 front 2024-09-19T06:57:31.480568+0000 (oldest deadline 2024-09-19T06:57:53.780698+0000) 2024-09-19T06:58:13.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:13.250+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:40.183023+0000 front 2024-09-19T06:57:40.182860+0000 (oldest deadline 2024-09-19T06:58:03.083054+0000) 2024-09-19T06:58:13.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6[58160]: 2024-09-19T06:58:13.250+0000 7fa7f4bf8640 -1 osd.6 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:44.784256+0000 front 2024-09-19T06:57:44.784284+0000 (oldest deadline 2024-09-19T06:58:10.084521+0000) 2024-09-19T06:58:13.553 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:13.277+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:13.554 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:13.277+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:13.554 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:13.277+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:14.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:13.552+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:14.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:13.552+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:14.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:13.552+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:14.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:13.552+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:14.025 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:13 smithi037 podman[120179]: 2024-09-19 06:58:13.851144741 +0000 UTC m=+5.622058096 container died 5dfff3f4ce32b6ebf4dfee8cb0b96603171450ea99a3a0869b237b2f8ec85f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-19T06:58:14.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:13.652+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:14.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:13.652+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:14.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:13.652+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:14.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:13 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:13.652+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:14.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:14.078+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:14.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:14.078+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:14.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:14.078+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:14.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:14.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:14.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:14.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:14.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:14.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:14.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:14 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:14.077+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:14.523 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:14 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:14.272+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:14.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:14 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:14.272+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:14.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:14 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:14.272+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:14.524 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:14 smithi037 podman[120179]: 2024-09-19 06:58:14.308852838 +0000 UTC m=+6.079766183 container cleanup 5dfff3f4ce32b6ebf4dfee8cb0b96603171450ea99a3a0869b237b2f8ec85f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T06:58:14.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:14 smithi037 bash[120179]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6 2024-09-19T06:58:14.882 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:14 smithi037 podman[120198]: 2024-09-19 06:58:14.560318598 +0000 UTC m=+0.709077945 container remove 5dfff3f4ce32b6ebf4dfee8cb0b96603171450ea99a3a0869b237b2f8ec85f6d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:58:14.882 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:14 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:14.659+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:14.882 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:14 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:14.659+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:14.882 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:14 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:14.659+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:14.883 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:14 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:14.659+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:15.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:14 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:14.595+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:15.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:14 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:14.595+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:15.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:14 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:14.595+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:15.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:14 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:14.595+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:15.155 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:14 smithi037 podman[120281]: 2024-09-19 06:58:14.784617104 +0000 UTC m=+0.016192741 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:58:15.155 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:14 smithi037 podman[120281]: 2024-09-19 06:58:14.919500335 +0000 UTC m=+0.151075952 container create 4bc5702adb2b572b85636c0728ab3afed59ec1d57fdda550db8c9d191f68eef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T06:58:15.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:15 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:15.049+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:15.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:15 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:15.049+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:15.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:15 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:15.049+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:15.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:15 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:15.066+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:15.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:15 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:15.066+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:15.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:15 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:15.066+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:15.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:15 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:15.066+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:15.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:15 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:15.229+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:15.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:15 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:15.229+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:15.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:15 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:15.229+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:15.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:15 smithi037 podman[120281]: 2024-09-19 06:58:15.155266767 +0000 UTC m=+0.386842387 container init 4bc5702adb2b572b85636c0728ab3afed59ec1d57fdda550db8c9d191f68eef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T06:58:15.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:15 smithi037 podman[120281]: 2024-09-19 06:58:15.158765688 +0000 UTC m=+0.390341305 container start 4bc5702adb2b572b85636c0728ab3afed59ec1d57fdda550db8c9d191f68eef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T06:58:15.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:15 smithi037 podman[120281]: 2024-09-19 06:58:15.213533438 +0000 UTC m=+0.445109064 container attach 4bc5702adb2b572b85636c0728ab3afed59ec1d57fdda550db8c9d191f68eef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4) 2024-09-19T06:58:15.525 INFO:journalctl@ceph.osd.6.smithi037.stdout:Sep 19 06:58:15 smithi037 podman[120281]: 2024-09-19 06:58:15.350688864 +0000 UTC m=+0.582264489 container died 4bc5702adb2b572b85636c0728ab3afed59ec1d57fdda550db8c9d191f68eef1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-6-deactivate, RELEASE=squid-2f599a4, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-19T06:58:15.984 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:15 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:15.630+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:15.985 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:15 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:15.630+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:15.985 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:15 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:15.630+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:15.985 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:15 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:15.630+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:16.011 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.6.service' 2024-09-19T06:58:16.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:15 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:15.576+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:16.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:15 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:15.576+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:16.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:15 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:15.576+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:16.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:15 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:15.576+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:16.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:16.052 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-19T06:58:16.052 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-19T06:58:16.052 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.7 2024-09-19T06:58:16.338 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:16 smithi096 systemd[1]: Stopping Ceph osd.7 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:16.339 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:16.250+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:16.339 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:16.250+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:16.339 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:16.250+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:16.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:16 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:16.053+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:16.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:16 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:16.053+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:16.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:16 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:16.053+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:16.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:16 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:16.104+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:16.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:16 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:16.104+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:16.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:16 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:16.104+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:16.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:16 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:16.104+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:16.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:16.403+0000 7fa6a0148640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:16.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:16.403+0000 7fa6a0148640 -1 osd.7 110 *** Got signal Terminated *** 2024-09-19T06:58:16.774 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:16.403+0000 7fa6a0148640 -1 osd.7 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:58:16.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:16.577+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:16.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:16.577+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:16.775 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:16.577+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:16.775 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:16 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:16.577+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:17.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:16 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:16.622+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:17.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:16 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:16.622+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:17.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:16 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:16.622+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:17.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:16 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:16.622+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:17.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:17 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:17.038+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:17.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:17 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:17.038+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:17.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:17 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:17.038+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:17.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:17 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:17.072+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:17.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:17 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:17.072+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:17.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:17 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:17.072+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:17.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:17 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:17.072+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:17.523 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:17.251+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:17.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:17.251+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:17.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:17.251+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:18.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:17.585+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:18.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:17.586+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:18.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:17.586+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:18.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:17.586+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:18.024 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:17 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:17.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:18.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:17 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:17.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:18.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:17 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:17.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:18.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:17 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:17.670+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:18.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:18.007+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:18.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:18.007+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:18.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:18.007+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:18.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:18.111+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:18.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:18.111+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:18.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:18.111+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:18.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:18.111+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:18.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:18.203+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:18.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:18.203+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:18.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:18.203+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:19.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:18.599+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:19.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:18.599+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:19.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:18.599+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:19.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:18 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:18.599+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:19.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:18.706+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:19.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:18.706+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:19.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:18.706+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:19.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:18 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:18.706+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:19.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:18.979+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:19.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:18.979+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:19.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:18.979+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:19.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:18 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:18.979+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:19.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:19.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:19.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:19.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:19.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:19.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:19.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:19.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:19.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:19.067+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:19.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:19.158+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:19.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:19.158+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:19.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:19.158+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:20.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:19.609+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:20.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:19.609+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:20.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:19.609+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:20.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:19.609+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:20.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:19 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:19.609+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:20.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:19.741+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:20.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:19.741+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:20.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:19.741+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:20.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:19 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:19.741+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:20.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:19.981+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:20.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:19.981+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:20.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:19.981+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:20.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:19 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:19.981+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:20.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:20.071+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:20.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:20.071+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:20.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:20.071+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:20.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:20.071+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:20.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:20.071+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:20.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:20.137+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:20.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:20.137+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:20.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:20.137+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:21.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:20.590+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:21.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:20.590+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:21.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:20.590+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:21.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:20.590+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:21.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:20 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:20.590+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:21.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:20.721+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:21.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:20.721+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:21.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:20.721+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:21.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:20.721+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:21.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:20 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:20.721+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:21.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:20.993+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:21.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:20.993+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:21.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:20.993+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:21.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:20 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:20.993+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:21.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:21.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:21.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:21.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:21.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:21.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:21.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:21.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:21.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:21.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:21.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:21.178+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.309168+0000 front 2024-09-19T06:57:32.309193+0000 (oldest deadline 2024-09-19T06:57:55.209306+0000) 2024-09-19T06:58:21.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:21.178+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.110366+0000 front 2024-09-19T06:57:38.110282+0000 (oldest deadline 2024-09-19T06:58:02.810463+0000) 2024-09-19T06:58:21.524 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7[52554]: 2024-09-19T06:58:21.178+0000 7fa69c75e640 -1 osd.7 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.612326+0000 front 2024-09-19T06:57:48.612390+0000 (oldest deadline 2024-09-19T06:58:12.712368+0000) 2024-09-19T06:58:21.849 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:21.580+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:21.849 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:21.580+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:21.850 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:21.580+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:21.850 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:21.580+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:21.850 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:21.580+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:22.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:21.740+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:22.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:21.740+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:22.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:21.740+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:22.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:21.740+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:22.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:21 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:21.740+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:22.180 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:21 smithi096 podman[71723]: 2024-09-19 06:58:21.848658487 +0000 UTC m=+5.567756011 container stop 50a23e586100aa931fd5b5d366eb53a0349e72907ea6455fb31ab17bef5058f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4) 2024-09-19T06:58:22.180 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:21 smithi096 podman[71723]: 2024-09-19 06:58:21.874464788 +0000 UTC m=+5.593562359 container died 50a23e586100aa931fd5b5d366eb53a0349e72907ea6455fb31ab17bef5058f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T06:58:22.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:21.964+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:22.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:21.964+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:22.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:21.964+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:22.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:21 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:21.964+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:22.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:22.022+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:22.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:22.022+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:22.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:22.022+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:22.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:22.022+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:22.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:22.022+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:22.513 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:22 smithi096 podman[71723]: 2024-09-19 06:58:22.277893794 +0000 UTC m=+5.996991319 container cleanup 50a23e586100aa931fd5b5d366eb53a0349e72907ea6455fb31ab17bef5058f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-19T06:58:22.513 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:22 smithi096 bash[71723]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7 2024-09-19T06:58:22.773 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:22 smithi096 podman[71749]: 2024-09-19 06:58:22.512615846 +0000 UTC m=+0.656792284 container remove 50a23e586100aa931fd5b5d366eb53a0349e72907ea6455fb31ab17bef5058f6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-19T06:58:22.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:22 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:22.610+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:22.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:22 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:22.610+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:22.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:22 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:22.610+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:22.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:22 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:22.610+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:22.774 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:22 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:22.610+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:23.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:22 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:22.749+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:23.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:22 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:22.749+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:23.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:22 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:22.749+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:23.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:22 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:22.749+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:23.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:22 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:22.749+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:23.235 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:22 smithi096 podman[71839]: 2024-09-19 06:58:22.814434316 +0000 UTC m=+0.015691281 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:58:23.235 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:22 smithi096 podman[71839]: 2024-09-19 06:58:22.915676967 +0000 UTC m=+0.116933919 container create 31fa9c40466aba266bd5e7b1d83e6da3f73a1914eeb8a58aebd7cee250c1c65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4) 2024-09-19T06:58:23.235 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:23 smithi096 podman[71839]: 2024-09-19 06:58:23.151932145 +0000 UTC m=+0.353189103 container init 31fa9c40466aba266bd5e7b1d83e6da3f73a1914eeb8a58aebd7cee250c1c65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T06:58:23.235 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:23 smithi096 podman[71839]: 2024-09-19 06:58:23.155579103 +0000 UTC m=+0.356836064 container start 31fa9c40466aba266bd5e7b1d83e6da3f73a1914eeb8a58aebd7cee250c1c65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-19T06:58:23.236 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:23 smithi096 podman[71839]: 2024-09-19 06:58:23.234923323 +0000 UTC m=+0.436180284 container attach 31fa9c40466aba266bd5e7b1d83e6da3f73a1914eeb8a58aebd7cee250c1c65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-19T06:58:23.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:23.058+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:23.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:23.058+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:23.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:23.058+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:23.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:23.058+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:23.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:23.058+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:23.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:22.922+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:23.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:22.922+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:23.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:22.922+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:23.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:22 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:22.922+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:23.523 INFO:journalctl@ceph.osd.7.smithi096.stdout:Sep 19 06:58:23 smithi096 podman[71839]: 2024-09-19 06:58:23.348050757 +0000 UTC m=+0.549307720 container died 31fa9c40466aba266bd5e7b1d83e6da3f73a1914eeb8a58aebd7cee250c1c65a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-7-deactivate, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:58:24.006 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:23 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:23.562+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:24.006 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:23 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:23.562+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:24.006 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:23 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:23.562+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:24.006 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:23 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:23.562+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:24.007 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:23 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:23.562+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:24.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:23 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:23.716+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:24.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:23 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:23.716+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:24.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:23 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:23.716+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:24.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:23 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:23.716+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:24.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:23 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:23.716+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:24.036 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.7.service' 2024-09-19T06:58:24.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:24.079 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-19T06:58:24.080 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-19T06:58:24.080 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.8 2024-09-19T06:58:24.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:23.892+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:24.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:23.892+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:24.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:23.892+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:24.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:23 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:23.892+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:24.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 systemd[1]: Stopping Ceph osd.8 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:24.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:24.084+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:24.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:24.084+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:24.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:24.084+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:24.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:24.084+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:24.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:24.084+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:24.862 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:24.389+0000 7f9b925b3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:24.862 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:24.389+0000 7f9b925b3640 -1 osd.8 110 *** Got signal Terminated *** 2024-09-19T06:58:24.863 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:24.389+0000 7f9b925b3640 -1 osd.8 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:58:25.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:24 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:24.568+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:25.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:24 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:24.568+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:25.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:24 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:24.568+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:25.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:24 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:24.568+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:25.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:24 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:24.568+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:25.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:24 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:24.744+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:25.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:24 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:24.744+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:25.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:24 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:24.744+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:25.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:24 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:24.744+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:25.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:24 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:24.744+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:25.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:25.102+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:25.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:25.102+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:25.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:25.102+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:25.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:25.102+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:25.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:25.102+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:25.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:24.941+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:25.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:24.941+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:25.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:24.941+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:25.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:24 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:24.941+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:26.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:25 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:25.584+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:26.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:25 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:25.584+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:26.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:25 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:25.584+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:26.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:25 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:25.584+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:26.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:25 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:25.584+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:26.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:25 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:25.779+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:26.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:25 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:25.779+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:26.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:25 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:25.779+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:26.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:25 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:25.779+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:26.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:25 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:25.779+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:26.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:25.929+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:26.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:25.929+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:26.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:25.929+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:26.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:25 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:25.929+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:26.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:26.069+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:26.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:26.069+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:26.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:26.069+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:26.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:26.069+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:26.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:26.069+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:27.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:26 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:26.583+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:27.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:26 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:26.583+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:27.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:26 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:26.583+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:27.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:26 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:26.583+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:27.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:26 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:26.583+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:27.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:26 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:26.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:27.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:26 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:26.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:27.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:26 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:26.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:27.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:26 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:26.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:27.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:26 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:26.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:27.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:26.950+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:27.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:26.950+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:27.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:26.950+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:27.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:26 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:26.950+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:27.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:27.078+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:27.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:27.078+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:27.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:27.078+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:27.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:27.078+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:27.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:27.078+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:28.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:27.573+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:28.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:27.573+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:28.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:27.573+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:28.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:27.573+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:28.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:27.573+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:28.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:27 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:27.573+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:28.070 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:27.775+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:28.070 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:27.775+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:28.070 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:27.775+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:28.070 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:27.775+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:28.070 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:27.775+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:28.070 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:27 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:27.775+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:28.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:27.987+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:28.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:27.987+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:28.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:27.987+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:28.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:27.987+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:28.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:27 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:27.987+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.218825+0000 front 2024-09-19T06:58:03.218777+0000 (oldest deadline 2024-09-19T06:58:27.219306+0000) 2024-09-19T06:58:28.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:28.038+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:28.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:28.038+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:28.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:28.038+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:28.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:28.038+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:28.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:28.038+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:29.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:28 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:28.615+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:29.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:28 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:28.615+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:29.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:28 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:28.615+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:29.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:28 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:28.615+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:29.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:28 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:28.615+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:29.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:28 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:28.615+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:29.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:28.791+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:29.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:28.791+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:29.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:28.791+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:29.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:28.791+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:29.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:28.791+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:29.026 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:28 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:28.791+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:29.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:28.938+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.013991+0000 front 2024-09-19T06:57:31.014052+0000 (oldest deadline 2024-09-19T06:57:53.914014+0000) 2024-09-19T06:58:29.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:28.938+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:38.615057+0000 front 2024-09-19T06:57:38.615190+0000 (oldest deadline 2024-09-19T06:58:04.515258+0000) 2024-09-19T06:58:29.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:28.938+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:47.416301+0000 front 2024-09-19T06:57:47.416224+0000 (oldest deadline 2024-09-19T06:58:13.316380+0000) 2024-09-19T06:58:29.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:28.938+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.817922+0000 front 2024-09-19T06:57:56.817941+0000 (oldest deadline 2024-09-19T06:58:18.517787+0000) 2024-09-19T06:58:29.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:28 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8[53849]: 2024-09-19T06:58:28.938+0000 7f9b8abc1640 -1 osd.8 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.218825+0000 front 2024-09-19T06:58:03.218777+0000 (oldest deadline 2024-09-19T06:58:27.219306+0000) 2024-09-19T06:58:29.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:29.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:29.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:29.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:29.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:29.364 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.030+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:30.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:29 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:29.603+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:30.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:29 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:29.603+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:30.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:29 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:29.603+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:30.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:29 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:29.603+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:30.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:29 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:29.603+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:30.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:29 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:29.603+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:30.092 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:29 smithi151 podman[73586]: 2024-09-19 06:58:29.80915528 +0000 UTC m=+5.535948902 container died 95a3611889c8e46df322c6037bbd49bba6617f799423949b71b9ed018ff6ad28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:58:30.092 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.988+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:30.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.988+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:30.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.988+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:30.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.988+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:30.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.988+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:30.093 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:29.988+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:30.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:29.828+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:30.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:29.828+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:30.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:29.828+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:30.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:29.828+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:30.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:29.828+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:30.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:29 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:29.828+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:30.362 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:30 smithi151 podman[73586]: 2024-09-19 06:58:30.190870525 +0000 UTC m=+5.917664152 container cleanup 95a3611889c8e46df322c6037bbd49bba6617f799423949b71b9ed018ff6ad28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2) 2024-09-19T06:58:30.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:30 smithi151 bash[73586]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8 2024-09-19T06:58:30.704 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:30 smithi151 podman[73602]: 2024-09-19 06:58:30.383963358 +0000 UTC m=+0.577714852 container remove 95a3611889c8e46df322c6037bbd49bba6617f799423949b71b9ed018ff6ad28 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T06:58:30.966 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:30 smithi151 podman[73692]: 2024-09-19 06:58:30.606623783 +0000 UTC m=+0.015716691 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:58:30.967 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:30 smithi151 podman[73692]: 2024-09-19 06:58:30.726704857 +0000 UTC m=+0.135797768 container create 1a226c1af58e9d35e217a2412667a965b1d387575eec2e28a84682bc03ee6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:58:31.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:30 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:30.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:31.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:30 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:30.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:31.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:30 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:30.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:31.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:30 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:30.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:31.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:30 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:30.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:31.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:30 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:30.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:31.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:30.832+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:31.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:30.832+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:31.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:30.832+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:31.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:30.832+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:31.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:30.832+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:31.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:30 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:30.832+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:31.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:30 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:30.965+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:31.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:30 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:30.965+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:31.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:30 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:30.965+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:31.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:30 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:30.965+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:31.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:30 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:30.965+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:31.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:30 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:30.965+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:31.363 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:31 smithi151 podman[73692]: 2024-09-19 06:58:31.005330798 +0000 UTC m=+0.414423707 container init 1a226c1af58e9d35e217a2412667a965b1d387575eec2e28a84682bc03ee6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8-deactivate, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T06:58:31.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:31 smithi151 podman[73692]: 2024-09-19 06:58:31.008660309 +0000 UTC m=+0.417753221 container start 1a226c1af58e9d35e217a2412667a965b1d387575eec2e28a84682bc03ee6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-19T06:58:31.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:31 smithi151 podman[73692]: 2024-09-19 06:58:31.096938561 +0000 UTC m=+0.506031472 container attach 1a226c1af58e9d35e217a2412667a965b1d387575eec2e28a84682bc03ee6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8-deactivate, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-19T06:58:31.364 INFO:journalctl@ceph.osd.8.smithi151.stdout:Sep 19 06:58:31 smithi151 podman[73692]: 2024-09-19 06:58:31.193302103 +0000 UTC m=+0.602395014 container died 1a226c1af58e9d35e217a2412667a965b1d387575eec2e28a84682bc03ee6c81 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:58:31.769 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.8.service' 2024-09-19T06:58:31.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:31.809 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-19T06:58:31.810 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-19T06:58:31.810 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.9 2024-09-19T06:58:32.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:31 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:31.622+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:32.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:31 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:31.622+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:32.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:31 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:31.622+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:32.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:31 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:31.622+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:32.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:31 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:31.622+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:32.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:31 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:31.622+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:32.138 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:31.842+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:32.138 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:31.842+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:32.138 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:31.842+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:32.138 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:31.842+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:32.138 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:31.842+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:32.139 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:31 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:31.842+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:32.139 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:31 smithi037 systemd[1]: Stopping Ceph osd.9 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:32.139 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.136+0000 7ff103688640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:32.139 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.136+0000 7ff103688640 -1 osd.9 110 *** Got signal Terminated *** 2024-09-19T06:58:32.139 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.136+0000 7ff103688640 -1 osd.9 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:58:32.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:31 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:31.993+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:32.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:31 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:31.993+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:32.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:31 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:31.993+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:32.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:31 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:31.993+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:32.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:31 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:31.993+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:32.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:31 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:31.993+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:33.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:32 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:32.667+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:33.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:32 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:32.667+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:33.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:32 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:32.667+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:33.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:32 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:32.667+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:33.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:32 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:32.667+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:33.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:32 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:32.667+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:33.124 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:32 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:32.997+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:33.124 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:32 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:32.997+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:33.124 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:32 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:32.997+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:33.124 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:32 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:32.997+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:33.124 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:32 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:32.997+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:33.124 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:32 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:32.997+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:33.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.823+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:33.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.823+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:33.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.823+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:33.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.823+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:33.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.823+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:33.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:32 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:32.823+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:34.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:33 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:33.671+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:34.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:33 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:33.671+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:34.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:33 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:33.671+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:34.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:33 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:33.671+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:34.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:33 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:33.671+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:34.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:33 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:33.671+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:34.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:33 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:33.786+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:34.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:33 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:33.786+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:34.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:33 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:33.786+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:34.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:33 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:33.786+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:34.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:33 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:33.786+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:34.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:33 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:33.786+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:34.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:33.956+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:34.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:33.956+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:34.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:33.956+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:34.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:33.956+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:34.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:33.956+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:34.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:33.956+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:35.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:34 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:34.695+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:35.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:34 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:34.695+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:35.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:34 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:34.695+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:35.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:34 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:34.695+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:35.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:34 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:34.695+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:35.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:34 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:34.695+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:35.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:34 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:34.695+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:35.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:34.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:35.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:34.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:35.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:34.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:35.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:34.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:35.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:34.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:35.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:34.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:35.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:34 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:34.783+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:08.543250+0000 front 2024-09-19T06:58:08.543219+0000 (oldest deadline 2024-09-19T06:58:34.443278+0000) 2024-09-19T06:58:35.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:34 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:34.944+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:35.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:34 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:34.944+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:35.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:34 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:34.944+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:35.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:34 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:34.944+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:35.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:34 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:34.944+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:35.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:34 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:34.944+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:36.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:35.661+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:36.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:35.661+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:36.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:35.661+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:36.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:35.661+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:36.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:35.661+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:36.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:35.661+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:36.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:35 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:35.661+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:36.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:35.809+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:36.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:35.809+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:36.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:35.809+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:36.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:35.809+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:36.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:35.809+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:36.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:35.809+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:36.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:35 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:35.809+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:08.543250+0000 front 2024-09-19T06:58:08.543219+0000 (oldest deadline 2024-09-19T06:58:34.443278+0000) 2024-09-19T06:58:36.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:35.908+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:36.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:35.908+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:36.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:35.908+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:36.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:35.908+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:36.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:35.908+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:36.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:35 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:35.908+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:37.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:36 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:36.686+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:37.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:36 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:36.686+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:37.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:36 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:36.686+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:37.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:36 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:36.686+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:37.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:36 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:36.686+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:37.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:36 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:36.686+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:37.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:36 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:36.686+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:37.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:36.773+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.035806+0000 front 2024-09-19T06:57:23.035869+0000 (oldest deadline 2024-09-19T06:57:47.136085+0000) 2024-09-19T06:58:37.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:36.773+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:32.437496+0000 front 2024-09-19T06:57:32.437343+0000 (oldest deadline 2024-09-19T06:57:56.537318+0000) 2024-09-19T06:58:37.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:36.773+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.139088+0000 front 2024-09-19T06:57:41.138954+0000 (oldest deadline 2024-09-19T06:58:01.639078+0000) 2024-09-19T06:58:37.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:36.773+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:46.340210+0000 front 2024-09-19T06:57:46.340191+0000 (oldest deadline 2024-09-19T06:58:11.640355+0000) 2024-09-19T06:58:37.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:36.773+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:51.640953+0000 front 2024-09-19T06:57:51.641099+0000 (oldest deadline 2024-09-19T06:58:20.441325+0000) 2024-09-19T06:58:37.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:36.773+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:00.441904+0000 front 2024-09-19T06:58:00.441889+0000 (oldest deadline 2024-09-19T06:58:27.442307+0000) 2024-09-19T06:58:37.276 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:36 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9[64685]: 2024-09-19T06:58:36.773+0000 7ff0fbc96640 -1 osd.9 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:08.543250+0000 front 2024-09-19T06:58:08.543219+0000 (oldest deadline 2024-09-19T06:58:34.443278+0000) 2024-09-19T06:58:37.362 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:36.877+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:37.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:36.877+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:37.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:36.877+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:37.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:36.877+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:37.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:36.877+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:37.363 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:36 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:36.877+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:37.775 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:37 smithi037 podman[120397]: 2024-09-19 06:58:37.499459696 +0000 UTC m=+5.486316096 container died 369466b1a8893d51aec687df6924a558a45d2a0e1890a9064d7b4ae8b728b1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:58:38.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:37 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:37.687+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:38.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:37 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:37.687+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:38.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:37 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:37.687+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:38.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:37 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:37.687+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:38.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:37 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:37.687+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:38.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:37 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:37.687+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:38.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:37 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:37.687+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:38.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:37 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:37.853+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:38.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:37 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:37.853+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:38.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:37 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:37.853+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:38.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:37 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:37.853+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:38.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:37 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:37.853+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:38.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:37 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:37.853+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:38.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:37 smithi037 podman[120397]: 2024-09-19 06:58:37.90295166 +0000 UTC m=+5.889808058 container cleanup 369466b1a8893d51aec687df6924a558a45d2a0e1890a9064d7b4ae8b728b1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:58:38.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:37 smithi037 bash[120397]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9 2024-09-19T06:58:38.275 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:38 smithi037 podman[120416]: 2024-09-19 06:58:38.113023317 +0000 UTC m=+0.615081843 container remove 369466b1a8893d51aec687df6924a558a45d2a0e1890a9064d7b4ae8b728b1a5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:58:38.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:38 smithi037 podman[120503]: 2024-09-19 06:58:38.330042042 +0000 UTC m=+0.016250631 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:58:38.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:38 smithi037 podman[120503]: 2024-09-19 06:58:38.430450298 +0000 UTC m=+0.116658931 container create 4cb493fd2f5edddb7f31ae35a75f0e9196805154dc32ab1d7c6525eaecc19005 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-19T06:58:38.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:38 smithi037 podman[120503]: 2024-09-19 06:58:38.641440567 +0000 UTC m=+0.327649161 container init 4cb493fd2f5edddb7f31ae35a75f0e9196805154dc32ab1d7c6525eaecc19005 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-19T06:58:38.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:38 smithi037 podman[120503]: 2024-09-19 06:58:38.645083863 +0000 UTC m=+0.331292457 container start 4cb493fd2f5edddb7f31ae35a75f0e9196805154dc32ab1d7c6525eaecc19005 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9-deactivate, ceph=True, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:58:38.725 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:38 smithi037 podman[120503]: 2024-09-19 06:58:38.724233614 +0000 UTC m=+0.410442202 container attach 4cb493fd2f5edddb7f31ae35a75f0e9196805154dc32ab1d7c6525eaecc19005 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:58:39.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:38 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:38.683+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:39.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:38 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:38.683+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:39.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:38 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:38.683+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:39.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:38 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:38.683+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:39.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:38 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:38.683+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:39.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:38 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:38.683+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:39.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:38 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:38.683+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:39.025 INFO:journalctl@ceph.osd.9.smithi037.stdout:Sep 19 06:58:38 smithi037 podman[120503]: 2024-09-19 06:58:38.84698683 +0000 UTC m=+0.533195418 container died 4cb493fd2f5edddb7f31ae35a75f0e9196805154dc32ab1d7c6525eaecc19005 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:58:39.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:38.858+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:39.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:38.858+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:39.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:38.858+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:39.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:38.858+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:39.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:38.858+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:39.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:38.858+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:39.470 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.9.service' 2024-09-19T06:58:39.515 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:39.516 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-19T06:58:39.516 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-19T06:58:39.516 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.10 2024-09-19T06:58:39.810 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 systemd[1]: Stopping Ceph osd.10 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:39.810 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.660+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:39.810 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.660+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:39.810 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.660+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:39.810 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.660+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:39.810 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.660+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:39.811 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.660+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:39.811 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.660+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:39.811 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.806+0000 7feaa7558640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:39.811 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.806+0000 7feaa7558640 -1 osd.10 110 *** Got signal Terminated *** 2024-09-19T06:58:39.811 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:39 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:39.806+0000 7feaa7558640 -1 osd.10 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:58:40.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:39 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:39.809+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:40.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:39 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:39.809+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:40.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:39 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:39.809+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:40.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:39 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:39.809+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:40.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:39 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:39.809+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:40.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:39 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:39.809+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:41.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:40 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:40.700+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:41.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:40 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:40.700+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:41.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:40 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:40.700+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:41.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:40 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:40.700+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:41.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:40 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:40.700+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:41.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:40 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:40.700+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:41.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:40 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:40.700+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:41.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:40 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:40.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:41.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:40 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:40.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:41.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:40 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:40.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:41.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:40 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:40.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:41.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:40 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:40.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:41.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:40 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:40.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:42.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:41.672+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:42.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:41.672+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:42.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:41.672+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:42.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:41.672+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:42.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:41.672+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:42.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:41.672+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:42.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:41 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:41.672+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:42.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:41.836+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:42.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:41.836+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:42.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:41.836+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:42.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:41.836+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:42.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:41.836+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:42.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:41.836+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:43.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:42.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:43.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:42.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:43.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:42.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:43.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:42.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:43.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:42.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:43.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:42.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:43.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:42 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:42.640+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:43.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:42.845+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:43.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:42.845+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:43.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:42.845+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:43.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:42.845+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:43.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:42.845+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:43.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:42.845+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:43.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:42 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:42.845+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:44.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:43.637+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:44.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:43.637+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:44.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:43.637+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:44.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:43.637+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:44.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:43.637+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:44.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:43.637+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:44.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:43 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:43.637+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:44.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:43.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:44.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:43.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:44.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:43.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:44.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:43.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:44.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:43.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:44.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:43.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:44.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:43 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:43.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:45.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:44.663+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:23.213370+0000 front 2024-09-19T06:57:23.213408+0000 (oldest deadline 2024-09-19T06:57:46.713442+0000) 2024-09-19T06:58:45.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:44.663+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.915117+0000 front 2024-09-19T06:57:31.915144+0000 (oldest deadline 2024-09-19T06:57:57.815313+0000) 2024-09-19T06:58:45.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:44.663+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:39.516684+0000 front 2024-09-19T06:57:39.516605+0000 (oldest deadline 2024-09-19T06:58:01.816651+0000) 2024-09-19T06:58:45.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:44.663+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:45.917824+0000 front 2024-09-19T06:57:45.917843+0000 (oldest deadline 2024-09-19T06:58:11.817754+0000) 2024-09-19T06:58:45.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:44.663+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:52.918804+0000 front 2024-09-19T06:57:52.918762+0000 (oldest deadline 2024-09-19T06:58:18.818931+0000) 2024-09-19T06:58:45.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:44.663+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:04.720124+0000 front 2024-09-19T06:58:04.720077+0000 (oldest deadline 2024-09-19T06:58:27.019977+0000) 2024-09-19T06:58:45.025 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:44 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10[59513]: 2024-09-19T06:58:44.663+0000 7feaa3b6e640 -1 osd.10 110 heartbeat_check: no reply from 172.21.15.37:6822 osd.6 since back 2024-09-19T06:58:12.321208+0000 front 2024-09-19T06:58:12.321234+0000 (oldest deadline 2024-09-19T06:58:34.621158+0000) 2024-09-19T06:58:45.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:44.772+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:45.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:44.772+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:45.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:44.772+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:45.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:44.772+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:45.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:44.772+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:45.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:44.772+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:45.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:44 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:44.772+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:45.616 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:45 smithi096 podman[71943]: 2024-09-19 06:58:45.341717274 +0000 UTC m=+5.636811584 container died d81e1b69ffc5a2c70bf473e8670d924d3ba701bf513201cc3b10dbcf5d9256d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:58:45.966 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:45 smithi096 podman[71943]: 2024-09-19 06:58:45.714620356 +0000 UTC m=+6.009714656 container cleanup d81e1b69ffc5a2c70bf473e8670d924d3ba701bf513201cc3b10dbcf5d9256d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T06:58:45.966 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:45 smithi096 bash[71943]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10 2024-09-19T06:58:46.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:45.807+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:46.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:45.807+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:46.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:45.807+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:46.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:45.807+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:46.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:45.807+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:46.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:45.807+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:46.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:45 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:45.807+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:46.273 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:45 smithi096 podman[71958]: 2024-09-19 06:58:45.965831502 +0000 UTC m=+0.614173717 container remove d81e1b69ffc5a2c70bf473e8670d924d3ba701bf513201cc3b10dbcf5d9256d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-19T06:58:46.536 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:46 smithi096 podman[72043]: 2024-09-19 06:58:46.182909268 +0000 UTC m=+0.015434072 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:58:46.537 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:46 smithi096 podman[72043]: 2024-09-19 06:58:46.283348266 +0000 UTC m=+0.115873069 container create 9c6ca35e863a3da76f547c2b796a27e1f10fb684e34a09ab0f9d63da946f0319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T06:58:46.537 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:46 smithi096 podman[72043]: 2024-09-19 06:58:46.53626427 +0000 UTC m=+0.368789074 container init 9c6ca35e863a3da76f547c2b796a27e1f10fb684e34a09ab0f9d63da946f0319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-2f599a4, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-19T06:58:47.023 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:46 smithi096 podman[72043]: 2024-09-19 06:58:46.540072172 +0000 UTC m=+0.372596976 container start 9c6ca35e863a3da76f547c2b796a27e1f10fb684e34a09ab0f9d63da946f0319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=) 2024-09-19T06:58:47.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:46 smithi096 podman[72043]: 2024-09-19 06:58:46.636381125 +0000 UTC m=+0.468905946 container attach 9c6ca35e863a3da76f547c2b796a27e1f10fb684e34a09ab0f9d63da946f0319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T06:58:47.024 INFO:journalctl@ceph.osd.10.smithi096.stdout:Sep 19 06:58:46 smithi096 podman[72043]: 2024-09-19 06:58:46.725933161 +0000 UTC m=+0.558457965 container died 9c6ca35e863a3da76f547c2b796a27e1f10fb684e34a09ab0f9d63da946f0319 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-19T06:58:47.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:46.826+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:47.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:46.826+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:47.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:46.826+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:47.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:46.826+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:47.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:46.826+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:47.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:46.826+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:47.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:46 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:46.826+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:47.304 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.10.service' 2024-09-19T06:58:47.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:47.342 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-19T06:58:47.343 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-19T06:58:47.343 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.11 2024-09-19T06:58:47.638 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 systemd[1]: Stopping Ceph osd.11 for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:48.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.637+0000 7f8f0638a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:48.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.637+0000 7f8f0638a640 -1 osd.11 110 *** Got signal Terminated *** 2024-09-19T06:58:48.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.637+0000 7f8f0638a640 -1 osd.11 110 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-19T06:58:48.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.792+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:48.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.792+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:48.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.792+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:48.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.792+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:48.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.792+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:48.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.792+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:48.114 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:47 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:47.792+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:49.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:48.800+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:49.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:48.800+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:49.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:48.800+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:49.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:48.800+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:49.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:48.800+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:49.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:48.800+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:49.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:48 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:48.800+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:50.112 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:49.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:50.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:49.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:50.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:49.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:50.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:49.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:50.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:49.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:50.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:49.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:50.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:49 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:49.797+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:51.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:50.764+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:51.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:50.764+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:51.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:50.764+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:51.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:50.764+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:51.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:50.764+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:51.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:50.764+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:51.113 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:50 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:50.764+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:52.071 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:51.720+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6806 osd.0 since back 2024-09-19T06:57:20.705889+0000 front 2024-09-19T06:57:20.705862+0000 (oldest deadline 2024-09-19T06:57:46.006025+0000) 2024-09-19T06:58:52.072 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:51.720+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-09-19T06:57:31.208081+0000 front 2024-09-19T06:57:31.207949+0000 (oldest deadline 2024-09-19T06:57:53.508125+0000) 2024-09-19T06:58:52.072 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:51.720+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6804 osd.2 since back 2024-09-19T06:57:41.010417+0000 front 2024-09-19T06:57:41.010386+0000 (oldest deadline 2024-09-19T06:58:02.710542+0000) 2024-09-19T06:58:52.072 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:51.720+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.37:6814 osd.3 since back 2024-09-19T06:57:48.011586+0000 front 2024-09-19T06:57:48.011520+0000 (oldest deadline 2024-09-19T06:58:10.911700+0000) 2024-09-19T06:58:52.072 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:51.720+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-09-19T06:57:56.414968+0000 front 2024-09-19T06:57:56.415109+0000 (oldest deadline 2024-09-19T06:58:18.115010+0000) 2024-09-19T06:58:52.072 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:51.720+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.151:6812 osd.5 since back 2024-09-19T06:58:03.816860+0000 front 2024-09-19T06:58:03.816761+0000 (oldest deadline 2024-09-19T06:58:29.017190+0000) 2024-09-19T06:58:52.072 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:51 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11[60700]: 2024-09-19T06:58:51.720+0000 7f8efe998640 -1 osd.11 110 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-09-19T06:58:20.719334+0000 front 2024-09-19T06:58:20.719417+0000 (oldest deadline 2024-09-19T06:58:42.419094+0000) 2024-09-19T06:58:53.264 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:53 smithi151 podman[73808]: 2024-09-19 06:58:53.012795931 +0000 UTC m=+5.472083205 container died 39f747294b5d359665ca8397feb7a7fe4ef4e28d21efb0fcdf7d54a67ddf0904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-19T06:58:53.540 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:53 smithi151 podman[73808]: 2024-09-19 06:58:53.337856525 +0000 UTC m=+5.797143793 container cleanup 39f747294b5d359665ca8397feb7a7fe4ef4e28d21efb0fcdf7d54a67ddf0904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-19T06:58:53.540 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:53 smithi151 bash[73808]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11 2024-09-19T06:58:53.862 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:53 smithi151 podman[73822]: 2024-09-19 06:58:53.539542534 +0000 UTC m=+0.521095196 container remove 39f747294b5d359665ca8397feb7a7fe4ef4e28d21efb0fcdf7d54a67ddf0904 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-2f599a4, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-19T06:58:54.260 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:53 smithi151 podman[73911]: 2024-09-19 06:58:53.849286611 +0000 UTC m=+0.016064422 image pull d2baa855c067ff6a7470a2dcec37621dc44864d3a0d23b0aa8bba9bd386dfe3d quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e 2024-09-19T06:58:54.261 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:53 smithi151 podman[73911]: 2024-09-19 06:58:53.97391143 +0000 UTC m=+0.140689231 container create 23c2ac6d7b0b3859a60f995496765ce0731c072b22cd6c559bd4c8e2fd6857db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-19T06:58:54.261 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 podman[73911]: 2024-09-19 06:58:54.176842698 +0000 UTC m=+0.343620499 container init 23c2ac6d7b0b3859a60f995496765ce0731c072b22cd6c559bd4c8e2fd6857db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=) 2024-09-19T06:58:54.261 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 podman[73911]: 2024-09-19 06:58:54.180445172 +0000 UTC m=+0.347222982 container start 23c2ac6d7b0b3859a60f995496765ce0731c072b22cd6c559bd4c8e2fd6857db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4) 2024-09-19T06:58:54.261 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 podman[73911]: 2024-09-19 06:58:54.260009974 +0000 UTC m=+0.426787775 container attach 23c2ac6d7b0b3859a60f995496765ce0731c072b22cd6c559bd4c8e2fd6857db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-19T06:58:54.612 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 podman[73911]: 2024-09-19 06:58:54.3701581 +0000 UTC m=+0.536935906 container died 23c2ac6d7b0b3859a60f995496765ce0731c072b22cd6c559bd4c8e2fd6857db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-19T06:58:54.902 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.11.service' 2024-09-19T06:58:54.925 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 podman[73934]: 2024-09-19 06:58:54.880816664 +0000 UTC m=+0.505138652 container remove 23c2ac6d7b0b3859a60f995496765ce0731c072b22cd6c559bd4c8e2fd6857db (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-19T06:58:54.925 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.11.service: Deactivated successfully. 2024-09-19T06:58:54.925 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 systemd[1]: Stopped Ceph osd.11 for b715dee2-7650-11ef-bceb-c7b262605968. 2024-09-19T06:58:54.925 INFO:journalctl@ceph.osd.11.smithi151.stdout:Sep 19 06:58:54 smithi151 systemd[1]: ceph-b715dee2-7650-11ef-bceb-c7b262605968@osd.11.service: Consumed 1min 23.333s CPU time. 2024-09-19T06:58:54.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:54.940 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-19T06:58:54.940 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-19T06:58:54.940 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.a 2024-09-19T06:58:55.248 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:58:54 smithi037 systemd[1]: Stopping Ceph mds.a for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:58:55.525 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:58:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a[71410]: 2024-09-19T06:58:55.247+0000 7f7ac0ae6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:58:55.525 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:58:55 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a[71410]: 2024-09-19T06:58:55.247+0000 7f7ac0ae6640 -1 mds.a *** got signal Terminated *** 2024-09-19T06:58:59.275 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:58:59 smithi037 podman[120617]: 2024-09-19 06:58:59.023090168 +0000 UTC m=+3.878405540 container died df1b81e71043325d19cedf1f3563481210d455512a5b62f121e7ced855e95b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, RELEASE=squid-2f599a4, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:58:59.632 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:58:59 smithi037 podman[120617]: 2024-09-19 06:58:59.455190291 +0000 UTC m=+4.310505704 container cleanup df1b81e71043325d19cedf1f3563481210d455512a5b62f121e7ced855e95b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0) 2024-09-19T06:58:59.632 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:58:59 smithi037 bash[120617]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a 2024-09-19T06:58:59.888 INFO:journalctl@ceph.mds.a.smithi037.stdout:Sep 19 06:58:59 smithi037 podman[120628]: 2024-09-19 06:58:59.63161955 +0000 UTC m=+0.603868746 container remove df1b81e71043325d19cedf1f3563481210d455512a5b62f121e7ced855e95b94 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-19T06:58:59.893 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.a.service' 2024-09-19T06:58:59.927 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:58:59.928 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-19T06:58:59.928 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-19T06:58:59.928 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.d 2024-09-19T06:59:00.251 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:59:00 smithi037 systemd[1]: Stopping Ceph mds.d for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:00.525 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:59:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d[70222]: 2024-09-19T06:59:00.250+0000 7f633c035640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:00.525 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:59:00 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d[70222]: 2024-09-19T06:59:00.250+0000 7f633c035640 -1 mds.d *** got signal Terminated *** 2024-09-19T06:59:04.518 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:59:04 smithi037 podman[120740]: 2024-09-19 06:59:04.264197578 +0000 UTC m=+4.114952548 container died 9059d2ccd3581c2b84c98b59ff5ee711e90b97dd7962dc07c72b59fe6193f509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-19T06:59:04.775 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:59:04 smithi037 podman[120740]: 2024-09-19 06:59:04.600396969 +0000 UTC m=+4.451151937 container cleanup 9059d2ccd3581c2b84c98b59ff5ee711e90b97dd7962dc07c72b59fe6193f509 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-19T06:59:04.775 INFO:journalctl@ceph.mds.d.smithi037.stdout:Sep 19 06:59:04 smithi037 bash[120740]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-d 2024-09-19T06:59:05.035 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.d.service' 2024-09-19T06:59:05.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:05.069 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-19T06:59:05.070 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-19T06:59:05.070 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.g 2024-09-19T06:59:05.401 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:59:05 smithi037 systemd[1]: Stopping Ceph mds.g for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:05.775 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:59:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g[71024]: 2024-09-19T06:59:05.396+0000 7f4618fb4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:05.775 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:59:05 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g[71024]: 2024-09-19T06:59:05.396+0000 7f4618fb4640 -1 mds.g *** got signal Terminated *** 2024-09-19T06:59:08.275 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:59:07 smithi037 podman[120862]: 2024-09-19 06:59:07.992525235 +0000 UTC m=+2.696430818 container died 5bf2c8397de1a3cd63c0bb3e71360400edb254bb9e3c2129ffffa3cf27b798e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-2f599a4) 2024-09-19T06:59:08.739 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:59:08 smithi037 podman[120862]: 2024-09-19 06:59:08.321201291 +0000 UTC m=+3.025106861 container cleanup 5bf2c8397de1a3cd63c0bb3e71360400edb254bb9e3c2129ffffa3cf27b798e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-19T06:59:08.739 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:59:08 smithi037 bash[120862]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g 2024-09-19T06:59:08.739 INFO:journalctl@ceph.mds.g.smithi037.stdout:Sep 19 06:59:08 smithi037 podman[120875]: 2024-09-19 06:59:08.505693251 +0000 UTC m=+0.513985311 container remove 5bf2c8397de1a3cd63c0bb3e71360400edb254bb9e3c2129ffffa3cf27b798e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:59:08.761 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.g.service' 2024-09-19T06:59:08.795 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:08.795 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-19T06:59:08.795 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-19T06:59:08.795 DEBUG:teuthology.orchestra.run.smithi037:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.j 2024-09-19T06:59:09.025 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:59:08 smithi037 systemd[1]: Stopping Ceph mds.j for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:09.525 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:59:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j[69807]: 2024-09-19T06:59:09.099+0000 7ff5132d4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:09.525 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:59:09 smithi037 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j[69807]: 2024-09-19T06:59:09.100+0000 7ff5132d4640 -1 mds.j *** got signal Terminated *** 2024-09-19T06:59:13.017 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:59:12 smithi037 podman[120981]: 2024-09-19 06:59:12.712076954 +0000 UTC m=+3.695336238 container died 6512a1479098082483b74ac49b4cecd81bfd3ef6729a14e32a60f85226da85e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-2f599a4) 2024-09-19T06:59:13.017 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:59:13 smithi037 podman[120981]: 2024-09-19 06:59:13.016456108 +0000 UTC m=+3.999715381 container cleanup 6512a1479098082483b74ac49b4cecd81bfd3ef6729a14e32a60f85226da85e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux ) 2024-09-19T06:59:13.275 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:59:13 smithi037 bash[120981]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j 2024-09-19T06:59:13.275 INFO:journalctl@ceph.mds.j.smithi037.stdout:Sep 19 06:59:13 smithi037 podman[120994]: 2024-09-19 06:59:13.242892428 +0000 UTC m=+0.523549015 container remove 6512a1479098082483b74ac49b4cecd81bfd3ef6729a14e32a60f85226da85e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=squid-2f599a4, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:59:13.372 DEBUG:teuthology.orchestra.run.smithi037:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.j.service' 2024-09-19T06:59:13.405 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:13.406 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-19T06:59:13.406 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-19T06:59:13.406 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.b 2024-09-19T06:59:13.706 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:59:13 smithi096 systemd[1]: Stopping Ceph mds.b for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:14.023 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:59:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: 2024-09-19T06:59:13.705+0000 7f69e68d0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:14.024 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:59:13 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b[65023]: 2024-09-19T06:59:13.705+0000 7f69e68d0640 -1 mds.b *** got signal Terminated *** 2024-09-19T06:59:16.625 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:59:16 smithi096 podman[72158]: 2024-09-19 06:59:16.319700801 +0000 UTC m=+2.709720905 container died a47f4a9cf11ade7e02038632d804f2635382e584001edf63fae0e0ba0b38e4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-19T06:59:16.981 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:59:16 smithi096 podman[72158]: 2024-09-19 06:59:16.713729529 +0000 UTC m=+3.103749636 container cleanup a47f4a9cf11ade7e02038632d804f2635382e584001edf63fae0e0ba0b38e4ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b, RELEASE=squid-2f599a4, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-19T06:59:16.982 INFO:journalctl@ceph.mds.b.smithi096.stdout:Sep 19 06:59:16 smithi096 bash[72158]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-b 2024-09-19T06:59:17.123 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.b.service' 2024-09-19T06:59:17.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:17.158 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-19T06:59:17.159 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-19T06:59:17.159 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.e 2024-09-19T06:59:17.523 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:59:17 smithi096 systemd[1]: Stopping Ceph mds.e for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:17.524 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:59:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e[64404]: 2024-09-19T06:59:17.475+0000 7feca51d7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:17.524 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:59:17 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e[64404]: 2024-09-19T06:59:17.475+0000 7feca51d7640 -1 mds.e *** got signal Terminated *** 2024-09-19T06:59:20.636 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:59:20 smithi096 podman[72280]: 2024-09-19 06:59:20.30370365 +0000 UTC m=+2.922582003 container died 1c294244b3eef1c105e4cfdb1184095cdaf7504a65b7e64dc1d3f7857ea8e295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-19T06:59:20.935 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:59:20 smithi096 podman[72280]: 2024-09-19 06:59:20.734202816 +0000 UTC m=+3.353081170 container cleanup 1c294244b3eef1c105e4cfdb1184095cdaf7504a65b7e64dc1d3f7857ea8e295 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T06:59:20.936 INFO:journalctl@ceph.mds.e.smithi096.stdout:Sep 19 06:59:20 smithi096 bash[72280]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-e 2024-09-19T06:59:21.158 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.e.service' 2024-09-19T06:59:21.192 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:21.193 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-19T06:59:21.193 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-19T06:59:21.193 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.h 2024-09-19T06:59:21.522 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:59:21 smithi096 systemd[1]: Stopping Ceph mds.h for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:21.773 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:59:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h[64706]: 2024-09-19T06:59:21.521+0000 7f6a12e1b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:21.773 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:59:21 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h[64706]: 2024-09-19T06:59:21.521+0000 7f6a12e1b640 -1 mds.h *** got signal Terminated *** 2024-09-19T06:59:24.614 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:59:24 smithi096 podman[72412]: 2024-09-19 06:59:24.304349153 +0000 UTC m=+2.886788042 container died 1cfd244352b2bb005166f81a181d922563ad6ce6e0b026e1c536e30167794b1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, ceph=True) 2024-09-19T06:59:24.939 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:59:24 smithi096 podman[72412]: 2024-09-19 06:59:24.696392911 +0000 UTC m=+3.278831797 container cleanup 1cfd244352b2bb005166f81a181d922563ad6ce6e0b026e1c536e30167794b1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-19T06:59:24.940 INFO:journalctl@ceph.mds.h.smithi096.stdout:Sep 19 06:59:24 smithi096 bash[72412]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-h 2024-09-19T06:59:25.074 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.h.service' 2024-09-19T06:59:25.108 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:25.108 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-19T06:59:25.108 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-19T06:59:25.108 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.k 2024-09-19T06:59:25.523 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:59:25 smithi096 systemd[1]: Stopping Ceph mds.k for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:25.524 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:59:25 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k[64088]: 2024-09-19T06:59:25.424+0000 7ff6f57f2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:25.524 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:59:25 smithi096 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k[64088]: 2024-09-19T06:59:25.424+0000 7ff6f57f2640 -1 mds.k *** got signal Terminated *** 2024-09-19T06:59:28.642 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:59:28 smithi096 podman[72533]: 2024-09-19 06:59:28.31520836 +0000 UTC m=+2.981430922 container died 89ff57701125d1a9035d4c7878d9c2d4efaedc85cfdf4244230e87d085898dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-2f599a4, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:59:28.642 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:59:28 smithi096 podman[72533]: 2024-09-19 06:59:28.641698431 +0000 UTC m=+3.307920990 container cleanup 89ff57701125d1a9035d4c7878d9c2d4efaedc85cfdf4244230e87d085898dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-19T06:59:28.960 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:59:28 smithi096 bash[72533]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k 2024-09-19T06:59:28.961 INFO:journalctl@ceph.mds.k.smithi096.stdout:Sep 19 06:59:28 smithi096 podman[72545]: 2024-09-19 06:59:28.859438904 +0000 UTC m=+0.542311384 container remove 89ff57701125d1a9035d4c7878d9c2d4efaedc85cfdf4244230e87d085898dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-2f599a4, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-19T06:59:29.088 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.k.service' 2024-09-19T06:59:29.120 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:29.120 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-19T06:59:29.121 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-19T06:59:29.121 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.c 2024-09-19T06:59:29.465 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:59:29 smithi151 systemd[1]: Stopping Ceph mds.c for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:29.862 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:59:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c[65967]: 2024-09-19T06:59:29.463+0000 7fece78ce640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:29.862 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:59:29 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c[65967]: 2024-09-19T06:59:29.463+0000 7fece78ce640 -1 mds.c *** got signal Terminated *** 2024-09-19T06:59:32.566 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:59:32 smithi151 podman[74014]: 2024-09-19 06:59:32.29332188 +0000 UTC m=+2.970557546 container died 90aecbb0c8243b678f9d94666ef9758f500b30a1b1816bd6f7ddab01eff1a0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-19T06:59:32.862 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:59:32 smithi151 podman[74014]: 2024-09-19 06:59:32.656200535 +0000 UTC m=+3.333436202 container cleanup 90aecbb0c8243b678f9d94666ef9758f500b30a1b1816bd6f7ddab01eff1a0d4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731) 2024-09-19T06:59:32.862 INFO:journalctl@ceph.mds.c.smithi151.stdout:Sep 19 06:59:32 smithi151 bash[74014]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-c 2024-09-19T06:59:33.093 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.c.service' 2024-09-19T06:59:33.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:33.128 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-19T06:59:33.128 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-19T06:59:33.128 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.f 2024-09-19T06:59:33.445 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:59:33 smithi151 systemd[1]: Stopping Ceph mds.f for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:33.862 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:59:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-f[65665]: 2024-09-19T06:59:33.443+0000 7f1e1badf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:33.862 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:59:33 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-f[65665]: 2024-09-19T06:59:33.443+0000 7f1e1badf640 -1 mds.f *** got signal Terminated *** 2024-09-19T06:59:37.829 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:59:37 smithi151 podman[74134]: 2024-09-19 06:59:37.450197678 +0000 UTC m=+4.094572795 container died f12ce8dc46298eb7c905ac3dbc4379a396fcab58779621a0d4a6f08fd516ca88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-2f599a4, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-19T06:59:38.112 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:59:37 smithi151 podman[74134]: 2024-09-19 06:59:37.902451388 +0000 UTC m=+4.546826509 container cleanup f12ce8dc46298eb7c905ac3dbc4379a396fcab58779621a0d4a6f08fd516ca88 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-f, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T06:59:38.112 INFO:journalctl@ceph.mds.f.smithi151.stdout:Sep 19 06:59:37 smithi151 bash[74134]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-f 2024-09-19T06:59:38.338 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.f.service' 2024-09-19T06:59:38.371 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:38.371 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-19T06:59:38.372 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-19T06:59:38.372 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.i 2024-09-19T06:59:38.707 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:59:38 smithi151 systemd[1]: Stopping Ceph mds.i for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:39.112 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:59:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: 2024-09-19T06:59:38.706+0000 7f8946832640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:39.112 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:59:38 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i[66266]: 2024-09-19T06:59:38.706+0000 7f8946832640 -1 mds.i *** got signal Terminated *** 2024-09-19T06:59:40.612 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:59:40 smithi151 podman[74255]: 2024-09-19 06:59:40.294560613 +0000 UTC m=+1.704819856 container died b5479bcc48463c4a5ffb7905b3b86934b507b9cc444cedc6b87fe678b730c089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-19T06:59:41.112 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:59:40 smithi151 podman[74255]: 2024-09-19 06:59:40.832540852 +0000 UTC m=+2.242800099 container cleanup b5479bcc48463c4a5ffb7905b3b86934b507b9cc444cedc6b87fe678b730c089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-2f599a4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-19T06:59:41.112 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:59:40 smithi151 bash[74255]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i 2024-09-19T06:59:41.113 INFO:journalctl@ceph.mds.i.smithi151.stdout:Sep 19 06:59:40 smithi151 podman[74268]: 2024-09-19 06:59:40.9840879 +0000 UTC m=+0.686355433 container remove b5479bcc48463c4a5ffb7905b3b86934b507b9cc444cedc6b87fe678b730c089 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-2f599a4, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-19T06:59:41.214 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.i.service' 2024-09-19T06:59:41.246 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:41.247 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-19T06:59:41.247 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-19T06:59:41.247 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.l 2024-09-19T06:59:41.537 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:59:41 smithi151 systemd[1]: Stopping Ceph mds.l for b715dee2-7650-11ef-bceb-c7b262605968... 2024-09-19T06:59:41.862 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:59:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l[65363]: 2024-09-19T06:59:41.536+0000 7fd798006640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-19T06:59:41.862 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:59:41 smithi151 ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l[65363]: 2024-09-19T06:59:41.536+0000 7fd798006640 -1 mds.l *** got signal Terminated *** 2024-09-19T06:59:43.362 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:59:43 smithi151 podman[74376]: 2024-09-19 06:59:43.012217465 +0000 UTC m=+1.544665874 container died 1e9f1562a3dba6521593fcf04d4f98ed507e58f9c081c0d53e851e633922f851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-09-19T06:59:43.722 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:59:43 smithi151 podman[74376]: 2024-09-19 06:59:43.554172106 +0000 UTC m=+2.086620516 container cleanup 1e9f1562a3dba6521593fcf04d4f98ed507e58f9c081c0d53e851e633922f851 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ea3a79400324e9054be6f7f19ded2753beeca1d15cbe0b9cab2e816ae9b5fd3e, name=ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=squid-2f599a4, org.label-schema.license=GPLv2) 2024-09-19T06:59:43.722 INFO:journalctl@ceph.mds.l.smithi151.stdout:Sep 19 06:59:43 smithi151 bash[74376]: ceph-b715dee2-7650-11ef-bceb-c7b262605968-mds-l 2024-09-19T06:59:43.926 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-b715dee2-7650-11ef-bceb-c7b262605968@mds.l.service' 2024-09-19T06:59:43.957 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-19T06:59:43.957 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-19T06:59:43.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b715dee2-7650-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:59:44.167 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:59:46.869 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b715dee2-7650-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:59:47.069 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:59:49.588 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b715dee2-7650-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-19T06:59:49.789 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T06:59:52.110 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:59:52.145 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:59:52.178 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-19T06:59:52.205 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-19T06:59:52.208 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi037/crash 2024-09-19T06:59:52.208 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:59:52.335 INFO:teuthology.orchestra.run.smithi037.stderr:tar: /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:59:52.336 INFO:teuthology.orchestra.run.smithi037.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:59:52.338 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi096/crash 2024-09-19T06:59:52.339 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:59:52.458 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:59:52.458 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:59:52.460 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi151/crash 2024-09-19T06:59:52.461 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash -- . 2024-09-19T06:59:52.530 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/b715dee2-7650-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-19T06:59:52.530 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-09-19T06:59:52.531 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-19T06:59:52.532 DEBUG:teuthology.orchestra.run.smithi037:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-19T06:59:52.576 INFO:teuthology.orchestra.run.smithi037.stdout:2024-09-19T06:54:24.318304+0000 mds.b (mds.0) 94 : cluster [WRN] Scrub error on inode 0x10000000240 (/volumes/qa/sv_1/025bcbbb-0f23-46cb-9b33-b3db5fcbc0d4/client.0/tmp/clients/client0/~dmtmp/SEED) see mds.b log and `damage ls` output for details 2024-09-19T06:59:52.576 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-19T06:59:52.576 INFO:tasks.cephadm:Compressing logs... 2024-09-19T06:59:52.577 DEBUG:teuthology.orchestra.run.smithi037:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:59:52.619 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:59:52.622 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-19T06:59:52.648 INFO:teuthology.orchestra.run.smithi037.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-19T06:59:52.649 INFO:teuthology.orchestra.run.smithi037.stderr:: No such file or directory 2024-09-19T06:59:52.649 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-19T06:59:52.649 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:59:52.650 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.0.log: /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:59:52.650 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-19T06:59:52.650 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:59:52.650 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-19T06:59:52.650 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-19T06:59:52.651 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.a.log: 84.7% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:59:52.652 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-19T06:59:52.652 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:59:52.652 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.g.log: /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.6.log: 90.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:59:52.667 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-19T06:59:52.667 INFO:teuthology.orchestra.run.smithi037.stderr: 94.6% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-19T06:59:52.667 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-19T06:59:52.667 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-19T06:59:52.667 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-19T06:59:52.668 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.x.log: 87.1% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:59:52.668 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-19T06:59:52.668 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.j.log: 94.5% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-19T06:59:52.668 INFO:teuthology.orchestra.run.smithi037.stderr: 90.9% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:59:52.668 INFO:teuthology.orchestra.run.smithi037.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:59:52.668 INFO:teuthology.orchestra.run.smithi037.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.6% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-19T06:59:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-19T06:59:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-19T06:59:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-19T06:59:52.669 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-19T06:59:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.10.log: /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:59:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-19T06:59:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-19T06:59:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-19T06:59:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:59:52.670 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:59:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-19T06:59:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:59:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-19T06:59:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-19T06:59:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-19T06:59:52.671 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:59:52.672 INFO:teuthology.orchestra.run.smithi151.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-19T06:59:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:: No such file or directory 2024-09-19T06:59:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-19T06:59:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-19T06:59:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-19T06:59:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log: /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-19T06:59:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-19T06:59:52.673 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip 82.2% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr: -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log: 90.4% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr: 87.1% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-19T06:59:52.674 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-19T06:59:52.675 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.8.log: 94.5% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-19T06:59:52.675 INFO:teuthology.orchestra.run.smithi151.stderr: 90.9% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:59:52.675 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-19T06:59:52.675 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-19T06:59:52.676 INFO:teuthology.orchestra.run.smithi037.stderr: 94.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:59:52.679 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.c.log: 92.2% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-19T06:59:52.683 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-19T06:59:52.689 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.f.log: 94.6% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-19T06:59:52.690 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:59:52.800 INFO:teuthology.orchestra.run.smithi151.stderr: 94.5% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-19T06:59:52.802 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.e.log: /var/log/ceph/cephadm.log: 82.2% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-19T06:59:52.836 INFO:teuthology.orchestra.run.smithi096.stderr: 92.1% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-19T06:59:52.915 INFO:teuthology.orchestra.run.smithi096.stderr: 90.9% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-19T06:59:52.968 INFO:teuthology.orchestra.run.smithi096.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-19T06:59:52.970 INFO:teuthology.orchestra.run.smithi096.stderr: 87.1% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-19T06:59:52.994 INFO:teuthology.orchestra.run.smithi096.stderr: 90.4% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-19T06:59:53.111 INFO:teuthology.orchestra.run.smithi037.stderr: 90.2% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-19T06:59:53.208 INFO:teuthology.orchestra.run.smithi151.stderr: 91.7% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-19T06:59:53.314 INFO:teuthology.orchestra.run.smithi096.stderr: 94.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-19T06:59:54.194 INFO:teuthology.orchestra.run.smithi037.stderr: 90.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-19T07:00:01.412 INFO:teuthology.orchestra.run.smithi096.stderr: 92.9% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-19T07:00:01.897 INFO:teuthology.orchestra.run.smithi151.stderr: 95.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-19T07:00:05.981 INFO:teuthology.orchestra.run.smithi037.stderr: 91.6% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-19T07:00:08.541 INFO:teuthology.orchestra.run.smithi151.stderr: 91.7% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-19T07:00:09.026 INFO:teuthology.orchestra.run.smithi037.stderr: 91.7% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-19T07:00:09.063 INFO:teuthology.orchestra.run.smithi037.stderr: 91.4% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-19T07:00:09.638 INFO:teuthology.orchestra.run.smithi151.stderr: 91.5% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-19T07:00:10.349 INFO:teuthology.orchestra.run.smithi151.stderr: 91.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-19T07:00:11.015 INFO:teuthology.orchestra.run.smithi037.stderr: 91.5% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-19T07:00:11.018 INFO:teuthology.orchestra.run.smithi037.stderr: 2024-09-19T07:00:11.018 INFO:teuthology.orchestra.run.smithi037.stderr:real 0m18.384s 2024-09-19T07:00:11.018 INFO:teuthology.orchestra.run.smithi037.stderr:user 1m4.404s 2024-09-19T07:00:11.018 INFO:teuthology.orchestra.run.smithi037.stderr:sys 0m2.042s 2024-09-19T07:00:14.361 INFO:teuthology.orchestra.run.smithi151.stderr: 91.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-19T07:00:14.364 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-09-19T07:00:14.364 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m21.727s 2024-09-19T07:00:14.364 INFO:teuthology.orchestra.run.smithi151.stderr:user 1m19.620s 2024-09-19T07:00:14.364 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m2.500s 2024-09-19T07:00:38.499 INFO:teuthology.orchestra.run.smithi096.stderr: 91.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-19T07:00:43.975 INFO:teuthology.orchestra.run.smithi096.stderr: 95.6% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-19T07:01:11.945 INFO:teuthology.orchestra.run.smithi096.stderr: 91.3% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-19T07:01:12.316 INFO:teuthology.orchestra.run.smithi096.stderr: 96.0% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-19T07:01:25.931 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-19T07:02:00.001 INFO:teuthology.orchestra.run.smithi096.stderr: 91.4% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-19T07:05:24.435 INFO:teuthology.orchestra.run.smithi096.stderr: 93.7% -- replaced with /var/log/ceph/b715dee2-7650-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-19T07:05:24.437 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-09-19T07:05:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:real 5m31.799s 2024-09-19T07:05:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:user 4m51.621s 2024-09-19T07:05:24.437 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m10.849s 2024-09-19T07:05:24.438 INFO:tasks.cephadm:Archiving logs... 2024-09-19T07:05:24.438 DEBUG:teuthology.misc:Transferring archived files from smithi037:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi037/log 2024-09-19T07:05:24.439 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:05:34.855 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi096/log 2024-09-19T07:05:34.856 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:06:22.766 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi151/log 2024-09-19T07:06:22.767 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-19T07:06:34.923 INFO:tasks.cephadm:Removing cluster... 2024-09-19T07:06:34.924 DEBUG:teuthology.orchestra.run.smithi037:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b715dee2-7650-11ef-bceb-c7b262605968 --force 2024-09-19T07:06:35.136 INFO:teuthology.orchestra.run.smithi037.stdout:Deleting cluster with fsid: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T07:06:35.500 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b715dee2-7650-11ef-bceb-c7b262605968 --force 2024-09-19T07:06:35.701 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T07:06:36.262 DEBUG:teuthology.orchestra.run.smithi151:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid b715dee2-7650-11ef-bceb-c7b262605968 --force 2024-09-19T07:06:36.466 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: b715dee2-7650-11ef-bceb-c7b262605968 2024-09-19T07:06:36.833 INFO:tasks.cephadm:Removing cephadm ... 2024-09-19T07:06:36.833 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:06:36.855 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:06:36.874 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-19T07:06:36.891 INFO:tasks.cephadm:Teardown complete 2024-09-19T07:06:36.891 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-19T07:06:36.910 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-19T07:06:36.910 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:06:36.913 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:06:36.916 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-19T07:06:37.032 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-19T07:06:37.046 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-19T07:06:37.047 DEBUG:teuthology.orchestra.run.smithi037:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:06:37.050 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:06:37.053 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-19T07:06:37.065 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:06:37.065 INFO:teuthology.orchestra.run.smithi037.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:06:37.067 INFO:teuthology.orchestra.run.smithi151.stderr:bash: line 1: ntpq: command not found 2024-09-19T07:06:37.072 INFO:teuthology.orchestra.run.smithi151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:06:37.072 INFO:teuthology.orchestra.run.smithi151.stdout:=============================================================================== 2024-09-19T07:06:37.072 INFO:teuthology.orchestra.run.smithi151.stdout:^* hv01.front.sepia.ceph.com 3 6 377 62 +512us[ +538us] +/- 48ms 2024-09-19T07:06:37.072 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -1250us[-1250us] +/- 75ms 2024-09-19T07:06:37.073 INFO:teuthology.orchestra.run.smithi151.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +335us[ +360us] +/- 72ms 2024-09-19T07:06:37.073 INFO:teuthology.orchestra.run.smithi151.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:06:37.073 INFO:teuthology.orchestra.run.smithi037.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:06:37.073 INFO:teuthology.orchestra.run.smithi037.stdout:=============================================================================== 2024-09-19T07:06:37.073 INFO:teuthology.orchestra.run.smithi037.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 +558us[ +586us] +/- 48ms 2024-09-19T07:06:37.073 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -1233us[-1233us] +/- 75ms 2024-09-19T07:06:37.073 INFO:teuthology.orchestra.run.smithi037.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +378us[ +406us] +/- 73ms 2024-09-19T07:06:37.074 INFO:teuthology.orchestra.run.smithi037.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:06:37.088 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-19T07:06:37.089 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-09-19T07:06:37.089 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 6 377 64 +495us[ +485us] +/- 47ms 2024-09-19T07:06:37.089 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -1296us[-1296us] +/- 75ms 2024-09-19T07:06:37.089 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 64 +306us[ +296us] +/- 72ms 2024-09-19T07:06:37.089 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-19T07:06:37.090 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-19T07:06:37.103 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-19T07:06:37.104 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-19T07:06:37.120 DEBUG:teuthology.orchestra.run.smithi037:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:06:37.213 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:06:37.343 DEBUG:teuthology.orchestra.run.smithi151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-19T07:06:37.410 DEBUG:teuthology.orchestra.run.smithi037:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:06:37.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:06:37.444 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:06:37.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:06:37.472 DEBUG:teuthology.orchestra.run.smithi151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-19T07:06:37.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:06:37.500 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-19T07:06:37.513 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-19T07:06:37.529 INFO:teuthology.task.internal:Duration was 2516.069874 seconds 2024-09-19T07:06:37.529 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-19T07:06:37.547 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-19T07:06:37.547 DEBUG:teuthology.orchestra.run.smithi037:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:06:37.550 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:06:37.553 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-19T07:06:37.591 INFO:teuthology.orchestra.run.smithi037.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:06:37.593 INFO:teuthology.orchestra.run.smithi151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:06:37.608 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-19T07:06:38.066 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-19T07:06:38.067 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi037.front.sepia.ceph.com 2024-09-19T07:06:38.067 DEBUG:teuthology.orchestra.run.smithi037:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:06:38.091 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-09-19T07:06:38.092 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:06:38.122 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-09-19T07:06:38.123 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-19T07:06:38.150 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-19T07:06:38.151 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:06:38.153 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:06:38.165 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-19T07:06:38.217 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-19T07:06:38.217 DEBUG:teuthology.orchestra.run.smithi037:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:06:38.225 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:06:38.232 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-19T07:06:39.640 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-19T07:06:39.652 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-19T07:06:39.653 DEBUG:teuthology.orchestra.run.smithi037:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:06:39.679 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:06:39.711 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-19T07:06:39.742 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-19T07:06:39.757 DEBUG:teuthology.orchestra.run.smithi037:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:06:39.760 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:06:39.762 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:06:39.781 INFO:teuthology.orchestra.run.smithi037.stdout:kernel.core_pattern = core 2024-09-19T07:06:39.787 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-09-19T07:06:39.810 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-09-19T07:06:39.882 DEBUG:teuthology.orchestra.run.smithi037:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:06:39.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:06:39.898 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:06:39.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:06:39.914 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-19T07:06:39.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-19T07:06:39.942 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-19T07:06:39.956 INFO:teuthology.task.internal:Transferring archived files... 2024-09-19T07:06:39.958 DEBUG:teuthology.misc:Transferring archived files from smithi037:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi037 2024-09-19T07:06:39.958 DEBUG:teuthology.orchestra.run.smithi037:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:06:40.034 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi096 2024-09-19T07:06:40.034 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:06:40.074 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-18_21:24:17-fs-squid-distro-default-smithi/7911743/remote/smithi151 2024-09-19T07:06:40.074 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-19T07:06:40.114 INFO:teuthology.task.internal:Removing archive directory... 2024-09-19T07:06:40.114 DEBUG:teuthology.orchestra.run.smithi037:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:06:40.116 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:06:40.119 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-19T07:06:40.166 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-19T07:06:40.179 INFO:teuthology.task.internal:Not uploading archives. 2024-09-19T07:06:40.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-19T07:06:40.196 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-19T07:06:40.197 DEBUG:teuthology.orchestra.run.smithi037:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:06:40.199 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:06:40.202 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-19T07:06:40.212 INFO:teuthology.orchestra.run.smithi037.stdout: 265336 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:06 /home/ubuntu/cephtest 2024-09-19T07:06:40.216 INFO:teuthology.orchestra.run.smithi096.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:06 /home/ubuntu/cephtest 2024-09-19T07:06:40.223 INFO:teuthology.orchestra.run.smithi151.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 19 07:06 /home/ubuntu/cephtest 2024-09-19T07:06:40.225 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-19T07:06:40.240 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-19T07:06:40.278 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/dbench}} duration: 2516.069873571396 failure_reason: 'Command failed (workunit test suites/dbench.sh) on smithi037 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6fad5dd73c93545e52f57f681e140a9c3529c11e TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/dbench.sh''' flavor: default owner: scheduled_teuthology@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=87213e1fdee9413aa0945c20cfb19ca6 status: fail success: false 2024-09-19T07:06:40.278 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-19T07:06:40.366 INFO:teuthology.run:FAIL